crossvalscore

For the given estimator and dataset, it automatically applies cross validation and returns the cross validation score and corresponding model parameters.

Syntax

scores = crossvalscore(fit_func,pred_func,X,y,options)

[scores, model_parameters] = crossvalscore(...)

Inputs

fit_func
Fit method of the estimator.
Type: function handle
pred_func
Prediction method of the estimator.
Type: function handle
X
Input dataset which need to be splitted.
Type: double
Dimension: vector | matrix
y
Target labels.
Type: double
Dimension: vector
options
Type: struct
scorer
Which metric should be used for scoring. If nothing is specified, the scorer function is considered from the model's parameter output (parameters.scorer).
Type: function handle
cv_options.numfolds
Number of folds for splitting dataset (default: 4).
Type: integer
Dimension: scalar
cv_options.shuffle
Flag to shuffle the dataset or not (default: false).
Type: Boolean
Dimension: logical
cv_options.seed
Random seed used for shuffling.
Type: integer
Dimension: scalar
fit_options
Arguments to be passed to the fit method. If nothing is specified, default options of the chosen fit function is considered.
Type: struct
return_models
Flag to return model parameters trained for each fold or not (default: false).
Type: Boolean
Dimension: logical

Outputs

scores
Cross validation scores.
Type: double
Dimension: scalar
model_parameters
Each trained model's parameters corresponding to each score, if return_models is set to true.
Type: cell

Example

Usage of crossvalscore

data = dlmread('data_banknote_authentication.txt', ',');
X = data(:, 1:2);
y = data(:, end);

options = struct;
options.scorer = @accuracy;
options.return_models = true;

[scores, model_params] = crossvalscore(@logisticfit, @logisticpredict, X, y, options);
printf('Validation Scores (4) Accuracy: '); printf('%f ', scores);
Validation Scores (4) Accuracy: 0.883721 0.860465 0.903509 0.888889

Comments

If return_models is set to true in options, then multiple model parameters are returned. Among those the best model parameters can be chosen using it's corresponding score. Such chosen parameters can be passed to predict function.