 | This article includes a list ofgeneral references, butit lacks sufficient correspondinginline citations. Please help toimprove this article byintroducing more precise citations.(July 2024) (Learn how and when to remove this message) |
Instatistics, acategorical variable (also calledqualitative variable) is avariable that can take on one of a limited, and usually fixed, number of possible values, assigning each individual or other unit of observation to a particular group ornominal category on the basis of somequalitative property.[1] In computer science and some branches of mathematics, categorical variables are referred to asenumerations orenumerated types. Commonly (though not in this article), each of the possible values of a categorical variable is referred to as alevel. Theprobability distribution associated with arandom categorical variable is called acategorical distribution.
Categorical data is thestatistical data type consisting of categorical variables or of data that has been converted into that form, for example asgrouped data. More specifically, categorical data may derive from observations made ofqualitative data that are summarised as counts orcross tabulations, or from observations ofquantitative data grouped within given intervals. Often, purely categorical data are summarised in the form of acontingency table. However, particularly when considering data analysis, it is common to use the term "categorical data" to apply to data sets that, while containing some categorical variables, may also contain non-categorical variables.Ordinal variables have a meaningful ordering, whilenominal variables have no meaningful ordering.
A categorical variable that can take on exactly two values is termed abinary variable or adichotomous variable; an important special case is theBernoulli variable. Categorical variables with more than two possible values are calledpolytomous variables; categorical variables are often assumed to be polytomous unless otherwise specified.Discretization is treatingcontinuous data as if it were categorical.Dichotomization is treating continuous data or polytomous variables as if they were binary variables.Regression analysis often treats category membership with one or more quantitativedummy variables.
Examples of values that might be represented in a categorical variable:
For ease in statistical processing, categorical variables may be assigned numeric indices, e.g. 1 throughK for aK-way categorical variable (i.e. a variable that can express exactlyK possible values). In general, however, the numbers are arbitrary, and have no significance beyond simply providing a convenient label for a particular value. In other words, the values in a categorical variable exist on anominal scale: they each represent a logically separate concept, cannot necessarily be meaningfullyordered, and cannot be otherwise manipulated as numbers could be. Instead, valid operations areequivalence,set membership, and other set-related operations.
As a result, thecentral tendency of a set of categorical variables is given by itsmode; neither themean nor themedian can be defined. As an example, given a set of people, we can consider the set of categorical variables corresponding to their last names. We can consider operations such as equivalence (whether two people have the same last name), set membership (whether a person has a name in a given list), counting (how many people have a given last name), or finding the mode (which name occurs most often). However, we cannot meaningfully compute the "sum" of Smith + Johnson, or ask whether Smith is "less than" or "greater than" Johnson. As a result, we cannot meaningfully ask what the "average name" (the mean) or the "middle-most name" (the median) is in a set of names.
This ignores the concept ofalphabetical order, which is a property that is not inherent in the names themselves, but in the way we construct the labels. For example, if we write the names inCyrillic and consider the Cyrillic ordering of letters, we might get a different result of evaluating "Smith < Johnson" than if we write the names in the standardLatin alphabet; and if we write the names inChinese characters, we cannot meaningfully evaluate "Smith < Johnson" at all, because no consistent ordering is defined for such characters. However, if we do consider the names as written, e.g., in the Latin alphabet, and define an ordering corresponding to standard alphabetical order, then we have effectively converted them intoordinal variables defined on anordinal scale.
Categoricalrandom variables are normally described statistically by acategorical distribution, which allows an arbitraryK-way categorical variable to be expressed with separate probabilities specified for each of theK possible outcomes. Such multiple-category categorical variables are often analyzed using amultinomial distribution, which counts the frequency of each possible combination of numbers of occurrences of the various categories.Regression analysis on categorical outcomes is accomplished throughmultinomial logistic regression,multinomial probit or a related type ofdiscrete choice model.
Categorical variables that have only two possible outcomes (e.g., "yes" vs. "no" or "success" vs. "failure") are known asbinary variables (orBernoulli variables). Because of their importance, these variables are often considered a separate category, with a separate distribution (theBernoulli distribution) and separate regression models (logistic regression,probit regression, etc.). As a result, the term "categorical variable" is often reserved for cases with 3 or more outcomes, sometimes termed amulti-way variable in opposition to a binary variable.
It is also possible to consider categorical variables where the number of categories is not fixed in advance. As an example, for a categorical variable describing a particular word, we might not know in advance the size of the vocabulary, and we would like to allow for the possibility of encountering words that we have not already seen. Standard statistical models, such as those involving thecategorical distribution andmultinomial logistic regression, assume that the number of categories is known in advance, and changing the number of categories on the fly is tricky. In such cases, more advanced techniques must be used. An example is theDirichlet process, which falls in the realm ofnonparametric statistics. In such a case, it is logically assumed that an infinite number of categories exist, but at any one time most of them (in fact, all but a finite number) have never been seen. All formulas are phrased in terms of the number of categories actually seen so far rather than the (infinite) total number of potential categories in existence, and methods are created for incremental updating of statistical distributions, including adding "new" categories.
Categorical variables represent aqualitative method of scoring data (i.e. represents categories or group membership). These can be included asindependent variables in aregression analysis or as dependent variables inlogistic regression orprobit regression, but must be converted toquantitative data in order to be able to analyze the data. One does so through the use of coding systems. Analyses are conducted such that onlyg -1 (g being the number of groups) are coded. This minimizes redundancy while still representing the complete data set as no additional information would be gained from coding the totalg groups: for example, when coding gender (whereg = 2: male and female), if we only code females everyone left over would necessarily be males. In general, the group that one does not code for is the group of least interest.[2]
There are three main coding systems typically used in the analysis of categorical variables in regression: dummy coding, effects coding, and contrast coding. The regression equation takes the form ofY = bX + a, whereb is the slope and gives the weight empirically assigned to an explanator,X is the explanatory variable, anda is theY-intercept, and these values take on different meanings based on the coding system used. The choice of coding system does not affect theF orR2 statistics. However, one chooses a coding system based on the comparison of interest since the interpretation ofb values will vary.[2]
Dummy coding is used when there is acontrol or comparison group in mind. One is therefore analyzing the data of one group in relation to the comparison group:a represents the mean of the control group andb is the difference between the mean of theexperimental group and the mean of the control group. It is suggested that three criteria be met for specifying a suitable control group: the group should be a well-established group (e.g. should not be an "other" category), there should be a logical reason for selecting this group as a comparison (e.g. the group is anticipated to score highest on the dependent variable), and finally, the group's sample size should be substantive and not small compared to the other groups.[3]
In dummy coding, the reference group is assigned a value of 0 for each code variable, the group of interest for comparison to the reference group is assigned a value of 1 for its specified code variable, while all other groups are assigned 0 for that particular code variable.[2]
Theb values should be interpreted such that the experimental group is being compared against the control group. Therefore, yielding a negative b value would entail the experimental group have scored less than the control group on thedependent variable. To illustrate this, suppose that we are measuring optimism among several nationalities and we have decided that French people would serve as a useful control. If we are comparing them against Italians, and we observe a negativeb value, this would suggest Italians obtain lower optimism scores on average.
The following table is an example of dummy coding withFrench as the control group and C1, C2, and C3 respectively being the codes forItalian,German, andOther (neither French nor Italian nor German):
| Nationality | C1 | C2 | C3 |
| French | 0 | 0 | 0 |
| Italian | 1 | 0 | 0 |
| German | 0 | 1 | 0 |
| Other | 0 | 0 | 1 |
In the effects coding system, data are analyzed through comparing one group to all other groups. Unlike dummy coding, there is no control group. Rather, the comparison is being made at the mean of all groups combined (a is now thegrand mean). Therefore, one is not looking for data in relation to another group but rather, one is seeking data in relation to the grand mean.[2]
Effects coding can either be weighted or unweighted. Weighted effects coding is simply calculating a weighted grand mean, thus taking into account the sample size in each variable. This is most appropriate in situations where the sample is representative of the population in question. Unweighted effects coding is most appropriate in situations where differences in sample size are the result of incidental factors. The interpretation ofb is different for each: in unweighted effects codingb is the difference between the mean of the experimental group and the grand mean, whereas in the weighted situation it is the mean of the experimental group minus the weighted grand mean.[2]
In effects coding, we code the group of interest with a 1, just as we would for dummy coding. The principal difference is that we code −1 for the group we are least interested in. Since we continue to use ag - 1 coding scheme, it is in fact the −1 coded group that will not produce data, hence the fact that we are least interested in that group. A code of 0 is assigned to all other groups.
Theb values should be interpreted such that the experimental group is being compared against the mean of all groups combined (or weighted grand mean in the case of weighted effects coding). Therefore, yielding a negativeb value would entail the coded group as having scored less than the mean of all groups on the dependent variable. Using our previous example of optimism scores among nationalities, if the group of interest is Italians, observing a negativeb value suggest they obtain a lower optimism score.
The following table is an example of effects coding withOther as the group of least interest.
| Nationality | C1 | C2 | C3 |
| French | 0 | 0 | 1 |
| Italian | 1 | 0 | 0 |
| German | 0 | 1 | 0 |
| Other | −1 | −1 | −1 |
The contrast coding system allows a researcher to directly ask specific questions. Rather than having the coding system dictate the comparison being made (i.e., against a control group as in dummy coding, or against all groups as in effects coding) one can design a unique comparison catering to one's specific research question. This tailored hypothesis is generally based on previous theory and/or research. The hypotheses proposed are generally as follows: first, there is the central hypothesis which postulates a large difference between two sets of groups; the second hypothesis suggests that within each set, the differences among the groups are small. Through itsa priori focused hypotheses, contrast coding may yield an increase inpower of thestatistical test when compared with the less directed previous coding systems.[2]
Certain differences emerge when we compare our a priori coefficients betweenANOVA and regression. Unlike when used in ANOVA, where it is at the researcher's discretion whether they choose coefficient values that are eitherorthogonal or non-orthogonal, in regression, it is essential that the coefficient values assigned in contrast coding be orthogonal. Furthermore, in regression, coefficient values must be either in fractional or decimal form. They cannot take on interval values.
The construction of contrast codes is restricted by three rules:
Violating rule 2 produces accurateR2 andF values, indicating that we would reach the same conclusions about whether or not there is a significant difference; however, we can no longer interpret theb values as a mean difference.
To illustrate the construction of contrast codes consider the following table. Coefficients were chosen to illustrate our a priori hypotheses: Hypothesis 1: French and Italian persons will score higher on optimism than Germans (French = +0.33, Italian = +0.33, German = −0.66). This is illustrated through assigning the same coefficient to the French and Italian categories and a different one to the Germans. The signs assigned indicate the direction of the relationship (hence giving Germans a negative sign is indicative of their lower hypothesized optimism scores). Hypothesis 2: French and Italians are expected to differ on their optimism scores (French = +0.50, Italian = −0.50, German = 0). Here, assigning a zero value to Germans demonstrates their non-inclusion in the analysis of this hypothesis. Again, the signs assigned are indicative of the proposed relationship.
| Nationality | C1 | C2 |
| French | +0.33 | +0.50 |
| Italian | +0.33 | −0.50 |
| German | −0.66 | 0 |
Nonsense coding occurs when one uses arbitrary values in place of the designated "0"s "1"s and "-1"s seen in the previous coding systems. Although it produces correct mean values for the variables, the use of nonsense coding is not recommended as it will lead to uninterpretable statistical results.[2]
Embeddings are codings of categorical values into low-dimensionalreal-valued (sometimescomplex-valued) vector spaces, usually in such a way that ‘similar’ values are assigned ‘similar’ vectors, or with respect to some other kind of criterion making the vectors useful for the respective application. A common special case areword embeddings, where the possible values of the categorical variable are thewords in alanguage and words with similar meanings are to be assigned similar vectors.
Aninteraction may arise when considering the relationship among three or more variables, and describes a situation in which the simultaneous influence of two variables on a third is not additive. Interactions may arise with categorical variables in two ways: either categorical by categorical variable interactions, or categorical by continuous variable interactions.
This type of interaction arises when we have two categorical variables. In order to probe this type of interaction, one would code using the system that addresses the researcher's hypothesis most appropriately. The product of the codes yields the interaction. One may then calculate theb value and determine whether the interaction is significant.[2]
Simple slopes analysis is a commonpost hoc test used in regression which is similar to the simple effects analysis in ANOVA, used to analyze interactions. In this test, we are examining the simple slopes of one independent variable at specific values of the other independent variable. Such a test is not limited to use with continuous variables, but may also be employed when the independent variable is categorical. We cannot simply choose values to probe the interaction as we would in the continuous variable case because of the nominal nature of the data (i.e., in the continuous case, one could analyze the data at high, moderate, and low levels assigning 1 standard deviation above the mean, at the mean, and at one standard deviation below the mean respectively). In our categorical case we would use a simple regression equation for each group to investigate the simple slopes. It is common practice tostandardize or center variables to make the data more interpretable in simple slopes analysis; however, categorical variables should never be standardized or centered. This test can be used with all coding systems.[2]
