Complexity of models

The pictures below show least squares fits (found usingFit in Mathematica) to polynomials with progressively higher degrees and therefore progressively more parameters. Which fit should be considered best in any particular case must ultimately depend on external considerations. But since the 1980s there have been attempts to find general criteria, typically based on maximizing quantities such as-Log[p] - d (the Akaike information criterion), wherep is the probability that the observed data would be generated from a given model (-Log[p] is proportional to variance in a least squares fit), andd is the number of parameters in the model.