Automatic feature preprocessing

BigQuery ML performs automatic preprocessing during training by using theCREATE MODEL statement.Automatic preprocessing consists ofmissing value imputationandfeature transformations.

For information about feature preprocessing support in BigQuery ML,seeFeature preprocessing overview.

Missing data imputation

In statistics, imputation is used to replace missing data with substitutedvalues. When you train a model in BigQuery ML,NULL values aretreated as missing data. When you predict outcomes in BigQuery ML,missing values can occur when BigQuery ML encounters aNULLvalue or a previously unseen value. BigQuery ML handles missingdata differently, based on the type of data in the column.

Feature transformations

By default, BigQuery ML transforms input features as follows:

TIMESTAMP feature transformation

The following table shows the components extracted fromTIMESTAMP columns andthe corresponding transformation method.

Category feature encoding

For features that are one-hot encoded, you can specify a different defaultencoding method by using the model optionCATEGORY_ENCODING_METHOD. Forgeneralized linear models (GLM) models, you can setCATEGORY_ENCODING_METHODto one of the following values:

• ONE_HOT_ENCODING
• DUMMY_ENCODING
• LABEL_ENCODING
• TARGET_ENCODING

One-hot encoding

One-hot encoding maps each category that a feature has to its own binaryfeature, where0 represents the absence of the feature and1 represents thepresence (known as adummy variable). This mapping createsN new featurecolumns, whereN is the number of unique categories for the feature acrossthe training table.

For example, suppose your training table has a feature column that's calledfruit with the categoriesApple,Banana, andCranberry, such as thefollowing:

In this case, theCATEGORY_ENCODING_METHOD='ONE_HOT_ENCODING' optiontransforms the table to the following internal representation:

One-hot encoding is supported bylinear and logistic regressionandboosted treemodels.

Dummy encoding

Dummy encoding issimilar to one-hot encoding, where a categorical feature is transformed into aset of placeholder variables. Dummy encoding usesN-1 placeholder variablesinstead ofN placeholder variables to representN categories for a feature.For example, if you setCATEGORY_ENCODING_METHOD to'DUMMY_ENCODING' forthe samefruit feature column shown in the preceding one-hot encoding example,then the table is transformed to the following internal representation:

The category with the most occurrences inthe training dataset is dropped. When multiple categories have themost occurrences, a random category within that set is dropped.

The final set of weights fromML.WEIGHTSstill includes the dropped category, but its weight is always0.0. ForML.ADVANCED_WEIGHTS,the standard error and p-value for the dropped variable isNaN.

Ifwarm_start is used on a model that was initially trained with'DUMMY_ENCODING', the same placeholder variable is dropped from the firsttraining run. Models cannot change encoding methods between training runs.

Dummy encoding is supported bylinear and logistic regression models.

Label encoding

Label encoding transforms the value of a categorical feature to anINT64 valuein[0, <number of categories>].

For example, if you had a book dataset like the following:

The label encoded values might look similar to the following:

The encoding vocabulary is sorted alphabetically.NULL values and categoriesthat aren't in the vocabulary are encoded to0.

Label encoding is supported byboosted tree models.

Target encoding

Target encoding replaces the categorical feature value with the probability ofthe target for classification models, or with the expected value of the targetfor regression models.

Features that have been target encoded might look similar to the followingexample:

# Classification model+------------------------+----------------------+| original value         | target encoded value |+------------------------+----------------------+| (category_1, target_1) |     0.5              || (category_1, target_2) |     0.5              || (category_2, target_1) |     0.0              |+------------------------+----------------------+# Regression model+------------------------+----------------------+| original value         | target encoded value |+------------------------+----------------------+| (category_1, 2)        |     2.5              || (category_1, 3)        |     2.5              || (category_2, 1)        |     1.5              || (category_2, 2)        |     1.5              |+------------------------+----------------------+

Target encoding is supported byboosted tree models.

What's next

For more information about supported SQL statements and functions for models thatsupport automatic feature preprocessing, see the following documents:

• End-to-end user journeys for ML models
• End-to-end user journeys for time series forecasting models