 | This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Feature" machine learning – news ·newspapers ·books ·scholar ·JSTOR(December 2014) (Learn how and when to remove this message) |
| Part of a series on |
| Machine learning anddata mining |
|---|
Learning with humans |
Model diagnostics |
Inmachine learning andpattern recognition, afeature is an individual measurable property or characteristic of a data set.[1] Choosing informative, discriminating, and independent features is crucial to producing effectivealgorithms forpattern recognition,classification, andregression tasks. Features are usually numeric, but other types such asstrings andgraphs are used insyntactic pattern recognition, after some pre-processing step such asone-hot encoding. The concept of "features" is related to that ofexplanatory variables used in statistical techniques such aslinear regression.
In feature engineering, two types of features are commonly used: numerical and categorical.
Numerical features are continuous values that can be measured on a scale. Examples of numerical features include age, height, weight, and income. Numerical features can be used in machine learning algorithms directly.[citation needed]
Categorical features are discrete values that can be grouped into categories. Examples of categorical features include gender, color, and zip code. Categorical features typically need to be converted to numerical features before they can be used in machine learning algorithms. This can be done using a variety of techniques, such as one-hot encoding, label encoding, and ordinal encoding.
The type of feature that is used in feature engineering depends on the specific machine learning algorithm that is being used. Some machine learning algorithms, such as decision trees, can handle both numerical and categorical features. Other machine learning algorithms, such as linear regression, can only handle numerical features.
A numeric feature can be conveniently described by a feature vector. One way to achievebinary classification is using alinear predictor function (related to theperceptron) with a feature vector as input. The method consists of calculating thescalar product between the feature vector and a vector of weights, qualifying those observations whose result exceeds a threshold.
Algorithms for classification from a feature vector includenearest neighbor classification,neural networks, andstatistical techniques such asBayesian approaches.
Incharacter recognition, features may includehistograms counting the number of black pixels along horizontal and vertical directions, number of internal holes, stroke detection and many others.
Inspeech recognition, features for recognizingphonemes can include noise ratios, length of sounds, relative power, filter matches and many others.
Inspam detection algorithms, features may include the presence or absence of certain email headers, the email structure, the language, the frequency of specific terms, the grammatical correctness of the text.
Incomputer vision, there are a large number of possiblefeatures, such as edges and objects.
Inpattern recognition andmachine learning, afeature vector is an n-dimensionalvector of numerical features that represent some object. Manyalgorithms in machine learning require a numerical representation of objects, since such representations facilitate processing and statistical analysis. When representing images, the feature values might correspond to the pixels of an image, while when representing texts the features might be the frequencies of occurrence of textual terms. Feature vectors are equivalent to the vectors ofexplanatory variables used instatistical procedures such aslinear regression. Feature vectors are often combined with weights using adot product in order to construct alinear predictor function that is used to determine a score for making a prediction.
Thevector space associated with these vectors is often called thefeature space. In order to reduce the dimensionality of the feature space, a number ofdimensionality reduction techniques can be employed.
Higher-level features can be obtained from already available features and added to the feature vector; for example, for the study of diseases the feature 'Age' is useful and is defined asAge = 'Year of death' minus 'Year of birth'. This process is referred to asfeature construction.[2][3] Feature construction is the application of a set of constructive operators to a set of existing features resulting in construction of new features. Examples of such constructive operators include checking for the equality conditions {=, ≠}, the arithmetic operators {+,−,×, /}, the array operators {max(S), min(S), average(S)} as well as other more sophisticated operators, for example count(S,C)[4] that counts the number of features in the feature vector S satisfying some condition C or, for example, distances to other recognition classes generalized by some accepting device. Feature construction has long been considered a powerful tool for increasing both accuracy and understanding of structure, particularly in high-dimensional problems.[5] Applications include studies of disease andemotion recognition from speech.[6]
 | This section'stone or style may not reflect theencyclopedic tone used on Wikipedia. See Wikipedia'sguide to writing better articles for suggestions.(August 2025) (Learn how and when to remove this message) |
The initial set of raw features can be redundant and large enough that estimation and optimization is made difficult or ineffective. Therefore, a preliminary step in many applications ofmachine learning andpattern recognition consists ofselecting a subset of features, orconstructing a new and reduced set of features to facilitate learning, and to improve generalization and interpretability.[7]
Extracting or selecting features is a combination of art and science; developing systems to do so is known asfeature engineering. It requires the experimentation of multiple possibilities and the combination of automated techniques with the intuition and knowledge of thedomain expert. Automating this process isfeature learning, where a machine not only uses features for learning, but learns the features itself.