Theparameter space is thespace of all possible parameter values that define a particularmathematical model. It is also sometimes calledweight space, and is often asubset of finite-dimensionalEuclidean space.
Instatistics, parameter spaces are particularly useful for describingparametric families ofprobability distributions. They also form the background forparameter estimation. In the case ofextremum estimators forparametric models, a certainobjective function is maximized or minimized over the parameter space.[1] Theorems ofexistence andconsistency of such estimators require some assumptions about thetopology of the parameter space. For instance,compactness of the parameter space, together withcontinuity of the objective function, suffices for the existence of an extremum estimator.[1]
Sometimes, parameters are analyzed to view how they affect their statistical model. In that context, they can be viewed as inputs of afunction, in which case the technical term for the parameter space isdomain of a function. The ranges of values of the parameters may form the axes of aplot, and particular outcomes of the model may be plotted against these axes to illustrate how different regions of the parameter space produce different types of behavior in the model.
Examples
edit- A simple model of health deterioration after developinglung cancer could include the two parameters gender[2] and smoker/non-smoker, in which case the parameter space is the following set of four possibilities:{(Male, Smoker), (Male, Non-smoker), (Female, Smoker), (Female, Non-smoker)}.
- Thelogistic map has one parameter,r, which can take any positive value. The parameter space is thereforepositive real numbers.
- For some values ofr, this function ends up cycling around a few values or becomes fixed on one value. These long-term values can be plotted againstr in abifurcation diagram to show the different behaviours of the function for different values ofr.
- In asine wave model the parameters areamplitudeA > 0,angular frequency ω > 0, andphase φ ∈ S1. Thus the parameter space is
- Incomplex dynamics, the parameter space is thecomplex planeC = {z =x +y i : x, y ∈R }, where i2 = −1.
- The famousMandelbrot set is asubset of this parameter space, consisting of the points in the complex plane which give abounded set of numbers when a particulariterated function is repeatedly applied from that starting point. The remaining points, which are not in the set, give an unbounded set of numbers (they tend to infinity) when this function is repeatedly applied from that starting point.
- Inmachine learning,hyperparameters are used to describe models. Indeep learning, the parameters of a deep network are calledweights. Due to the layered structure of deep networks, their weight space has a complex structure and geometry.[3][4] For example, inmultilayer perceptrons, the same function is preserved whenpermuting the nodes of a hidden layer, amounting to permuting weight matrices of the network. This property is known asequivariance to permutation ofdeep weight spaces.[3] The study seekshyperparameter optimization.
History
editParameter space contributed to the liberation ofgeometry from the confines ofthree-dimensional space. For instance, the parameter space ofspheres in three dimensions, has four dimensions—three for the sphere center and another for the radius. According toDirk Struik, it was the bookNeue Geometrie des Raumes (1849) byJulius Plücker that showed
- ...geometry need not solely be based on points as basic elements. Lines, planes, circles, spheres can all be used as the elements (Raumelemente) on which a geometry can be based. This fertile conception threw new light on both synthetic and algebraic geometry and created new forms of duality. The number of dimensions of a particular form of geometry could now be any positive number, depending on the number of parameters necessary to define the "element".[5]: 165
The requirement for higher dimensions is illustrated byPlücker's line geometry. Struik writes
- [Plücker's] geometry of lines in three-space could be considered as a four-dimensional geometry, or, asKlein has stressed, as the geometry of a four-dimensionalquadric in a five-dimensional space.[5]: 168
Thus theKlein quadric describes the parameters of lines in space.
See also
editReferences
edit- ^abHayashi, Fumio (2000).Econometrics. Princeton University Press. p. 446.ISBN 0-691-01018-8.
- ^Gasperino, J.; Rom, W. N. (2004). "Gender and lung cancer".Clinical Lung Cancer.5 (6):353–359.doi:10.3816/CLC.2004.n.013.PMID 15217534.
- ^abNavon, Aviv; Shamsian, Aviv; Achituve, Idan; Fetaya, Ethan; Chechik, Gal; Maron, Haggai (2023-07-03)."Equivariant Architectures for Learning in Deep Weight Spaces".Proceedings of the 40th International Conference on Machine Learning. PMLR:25790–25816.arXiv:2301.12780.
- ^Hecht-Nielsen, Robert (1990-01-01), Eckmiller, Rolf (ed.),"ON THE ALGEBRAIC STRUCTURE OF FEEDFORWARD NETWORK WEIGHT SPACES",Advanced Neural Computers, Amsterdam: North-Holland, pp. 129–135,ISBN 978-0-444-88400-8, retrieved2023-12-01
- ^abDirk Struik (1967)A Concise History of Mathematics, 3rd edition,Dover Books