Inmachine learning (ML), alearning curve (ortraining curve) is agraphical representation that shows how a model's performance on atraining set (and usually a validation set) changes with the number of training iterations (epochs) or the amount of training data.^[1]Typically, the number of training epochs or training set size is plotted on thex-axis, and the value of theloss function (and possibly some other metric such as thecross-validation score) on they-axis.

Synonyms includeerror curve,experience curve,improvement curve andgeneralization curve.^[2]

More abstractly, learning curves plot the difference between learning effort and predictive performance, where "learning effort" usually means the number of training samples, and "predictive performance" means accuracy on testing samples.^[3]

Learning curves have many useful purposes in ML, including:^[4][5][6]

• choosing model parameters during design,
• adjusting optimization to improve convergence,
• and diagnosing problems such asoverfitting (or underfitting).

Learning curves can also be tools for determining how much a model benefits from adding more training data, and whether the model suffers more from avariance error or a bias error. If both the validation score and the training score converge to a certain value, then the model will no longer significantly benefit from more training data.^[7]

When creating a function to approximate the distribution of some data, it is necessary to define a loss function${\displaystyle L(f_{\theta }(X),Y)}$ to measure how good the model output is (e.g., accuracy for classification tasks ormean squared error for regression). We then define an optimization process which finds model parameters${\displaystyle \theta }$ such that${\displaystyle L(f_{\theta }(X),Y)}$ is minimized, referred to as${\displaystyle {\theta }^{\ast }}$.

If the training data is

${\displaystyle \{x_1,x_2,\dots ,x_n\},\{y_1,y_2,\dots y_n\}}$

and the validation data is

${\displaystyle \{x_1^{\prime },x_2^{\prime },\dots x_m^{\prime }\},\{y_1^{\prime },y_2^{\prime },\dots y_m^{\prime }\}}$,

a learning curve is the plot of the two curves

1. ${\displaystyle i\mapsto L(f_{{\theta }^{\ast }(X_i,Y_i)}(X_i),Y_i)}$
2. ${\displaystyle i\mapsto L(f_{{\theta }^{\ast }(X_i,Y_i)}(X_i^{\prime }),Y_i^{\prime })}$

where${\displaystyle X_i=\{x_1,x_2,\dots x_i\}}$

Many optimizationalgorithms are iterative, repeating the same step (such asbackpropagation) until the processconverges to an optimal value.Gradient descent is one such algorithm. If${\displaystyle {\theta }_i^{\ast }}$ is the approximation of the optimal${\displaystyle \theta }$ after${\displaystyle i}$ steps, a learning curve is the plot of

1. ${\displaystyle i\mapsto L(f_{{\theta }_i^{\ast }(X,Y)}(X),Y)}$
2. ${\displaystyle i\mapsto L(f_{{\theta }_i^{\ast }(X,Y)}(X^{\prime }),Y^{\prime })}$

• Overfitting
• Bias–variance tradeoff
• Model selection
• Cross-validation (statistics)
• Validity (statistics)
• Verification and validation
• Double descent

• Model selection
• Machine learning

• Articles with short description
• Short description is different from Wikidata