Simultaneous observation and analysis of more than one outcome variable
"Multivariate analysis" redirects here. For the usage in mathematics, seeMultivariable calculus.
Multivariate statistics is a subdivision ofstatistics encompassing the simultaneous observation and analysis of more than oneoutcome variable, i.e.,multivariate random variables. Multivariate statistics concerns understanding the different aims and background of each of the different forms of multivariate analysis, and how they relate to each other. The practical application of multivariate statistics to a particular problem may involve several types of univariate and multivariate analyses in order to understand the relationships between variables and their relevance to the problem being studied.
In addition, multivariate statistics is concerned with multivariateprobability distributions, in terms of both
how these can be used to represent the distributions of observed data;
how they can be used as part ofstatistical inference, particularly where several different quantities are of interest to the same analysis.
Certain types of problems involving multivariate data, for examplesimple linear regression andmultiple regression, arenot usually considered to be special cases of multivariate statistics because the analysis is dealt with by considering the (univariate) conditional distribution of a single outcome variable given the other variables.
Multivariate analysis (MVA) is based on the principles of multivariate statistics. Typically, MVA is used to address situations where multiple measurements are made on each experimental unit and the relations among these measurements and their structures are important.[1] A modern, overlapping categorization of MVA includes:[1]
Multivariate analysis can be complicated by the desire to include physics-based analysis to calculate the effects of variables for a hierarchical "system-of-systems". Often, studies that wish to use multivariate analysis are stalled by the dimensionality of the problem. These concerns are often eased through the use ofsurrogate models, highly accurate approximations of the physics-based code. Since surrogate models take the form of an equation, they can be evaluated very quickly. This becomes an enabler for large-scale MVA studies: while aMonte Carlo simulation across the design space is difficult with physics-based codes, it becomes trivial when evaluating surrogate models, which often take the form ofresponse-surface equations.
Multivariate regression attempts to determine a formula that can describe how elements in avector of variables respond simultaneously to changes in others. For linear relations, regression analyses here are based on forms of thegeneral linear model. Some suggest that multivariate regression is distinct from multivariable regression, however, that is debated and not consistently true across scientific fields.[2]
Principal components analysis (PCA) creates a new set oforthogonal variables that contain the same information as the original set. It rotates the axes of variation to give a new set of orthogonal axes, ordered so that they summarize decreasing proportions of the variation.
Factor analysis is similar to PCA but allows the user to extract a specified number of synthetic variables, fewer than the original set, leaving the remaining unexplained variation as error. The extracted variables are known as latent variables or factors; each one may be supposed to account for covariation in a group of observed variables.
Canonical correlation analysis finds linear relationships among two sets of variables; it is the generalised (i.e. canonical) version of bivariate[3] correlation.
Redundancy analysis[4] (RDA) is similar to canonical correlation analysis but allows the user to derive a specified number of synthetic variables from one set of (independent) variables that explain as much variance as possible in another (independent) set. It is a multivariate analogue ofregression.[5]
Correspondence analysis (CA), or reciprocal averaging, finds (like PCA) a set of synthetic variables that summarise the original set. The underlying model assumes chi-squared dissimilarities among records (cases).
Canonical (or "constrained") correspondence analysis (CCA) for summarising the joint variation in two sets of variables (like redundancy analysis); combination of correspondence analysis and multivariate regression analysis. The underlying model assumes chi-squared dissimilarities among records (cases).
Multidimensional scaling comprises various algorithms to determine a set of synthetic variables that best represent the pairwise distances between records. The original method isprincipal coordinates analysis (PCoA; based on PCA).
Discriminant analysis, or canonical variate analysis, attempts to establish whether a set of variables can be used to distinguish between two or more groups of cases.
Linear discriminant analysis (LDA) computes a linear predictor from two sets of normally distributed data to allow for classification of new observations.
Clustering systems assign objects into groups (called clusters) so that objects (cases) from the same cluster are more similar to each other than objects from different clusters.
Recursive partitioning creates a decision tree that attempts to correctly classify members of the population based on a dichotomous dependent variable.
Simultaneous equations models involve more than one regression equation, with different dependent variables, estimated together.
Vector autoregression involves simultaneous regressions of varioustime series variables on their own and each other's lagged values.
Principal response curves analysis (PRC) is a method based on RDA that allows the user to focus on treatment effects over time by correcting for changes in control treatments over time.[6]
Iconography of correlations consists in replacing a correlation matrix by a diagram where the "remarkable" correlations are represented by a solid line (positive correlation), or a dotted line (negative correlation).
It is very common that in an experimentally acquired set of data the values of some components of a given data point aremissing. Rather than discarding the whole data point, it is common to "fill in" values for the missing components, a process called "imputation".[7]
There is a set ofprobability distributions used in multivariate analyses that play a similar role to the corresponding set of distributions that are used inunivariate analysis when thenormal distribution is appropriate to a dataset. These multivariate distributions are:
C.R. Rao made significant contributions to multivariate statistical theory throughout his career, particularly in the mid-20th century. One of his key works is the book titled "Advanced Statistical Methods in Biometric Research," published in 1952. This work laid the foundation for many concepts in multivariate statistics.[8]Anderson's 1958 textbook, An Introduction to Multivariate Statistical Analysis,[9] educated a generation of theorists and applied statisticians; Anderson's book emphasizeshypothesis testing vialikelihood ratio tests and the properties ofpower functions:admissibility,unbiasedness andmonotonicity.[10][11]
MVA was formerly discussed solely in the context of statistical theories, due to the size and complexity of underlying datasets and its high computational consumption. With the dramatic growth of computational power, MVA now plays an increasingly important role in data analysis and has wide application inOmics fields.
^abOlkin, I.; Sampson, A. R. (2001-01-01),"Multivariate Analysis: Overview", in Smelser, Neil J.; Baltes, Paul B. (eds.),International Encyclopedia of the Social & Behavioral Sciences, Pergamon, pp. 10240–10247,ISBN978-0-08-043076-8, retrieved2019-09-02
^Unsophisticated analysts of bivariate Gaussian problems may find useful a crude but accuratemethod of accurately gauging probability by simply taking the sumS of theN residuals' squares, subtracting the sumSm at minimum, dividing this difference bySm, multiplying the result by (N - 2) and taking the inverse anti-ln of half that product.
^Van Den Wollenberg, Arnold L. (1977). "Redundancy analysis an alternative for canonical correlation analysis".Psychometrika.42 (2):207–219.doi:10.1007/BF02294050.
^ter Braak, Cajo J.F. & Šmilauer, Petr (2012).Canoco reference manual and user's guide: software for ordination (version 5.0), p292. Microcomputer Power, Ithaca, NY.
^J.L. Schafer (1997).Analysis of Incomplete Multivariate Data. Chapman & Hall/CRC.ISBN978-1-4398-2186-2.
Malakooti, B. (2013). Operations and Production Systems with Multiple Objectives. John Wiley & Sons.
T. W. Anderson,An Introduction to Multivariate Statistical Analysis, Wiley, New York, 1958.
KV Mardia; JT Kent & JM Bibby (1979).Multivariate Analysis. Academic Press.ISBN978-0-12-471252-2. (M.A. level "likelihood" approach)
Feinstein, A. R. (1996)Multivariable Analysis. New Haven, CT: Yale University Press.
Hair, J. F. Jr. (1995)Multivariate Data Analysis with Readings, 4th ed. Prentice-Hall.
Schafer, J. L. (1997)Analysis of Incomplete Multivariate Data. CRC Press. (Advanced)
Sharma, S. (1996)Applied Multivariate Techniques. Wiley. (Informal, applied)
Izenman, Alan J. (2008). Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. Springer Texts in Statistics. New York: Springer-Verlag.ISBN9780387781884.
