The ML.TRAINING_INFO function

This document describes theML.TRAINING_INFO function, which lets you seeinformation about the training iterations of a model.

You can runML.TRAINING_INFO while theCREATE MODELstatement for the target model is running, or you can wait until after theCREATE MODEL statement completes. If you runML.TRAINING_INFO before thefirst training iteration of theCREATE MODEL statement completes, the queryreturns aNot found error.

Syntax

ML.TRAINING_INFO(  MODEL `PROJECT_ID.DATASET.MODEL_NAME`,)

Arguments

ML.TRAINING_INFO takes the following arguments:

• PROJECT_ID: your project ID.
• DATASET: the BigQuery dataset that containsthe model.
• MODEL_NAME: the name of the model.

Output

ML.TRAINING_INFO returns the following columns:

• training_run: anINT64 value that contains the training run identifierfor the model. The value in this column is0 for a newly created model. Ifyou retrain the model using thewarm_start argument of theCREATE MODELstatement, this value is incremented.
• iteration: anINT64 value that contains the iteration number of thetraining run. The value for the first iteration is0. This value isincremented for each additional training run.

• loss: aFLOAT64 value that contains the loss metric calculated afteran iteration on the training data:

  • For logistic regression models, this islog loss.
  • For linear regression models, this ismean squared error.
  • For multiclass logistic regressions, this iscross-entropy log loss.
  • For explicit matrix factorization models this is mean squared errorcalculated over the seen input ratings.
  • For implicit matrix factorization models, the loss is calculated using thefollowing formula:
  $$ Loss = \sum_{u, i} c_{ui}(p_{ui} - x^T_uy_i)^2 +\lambda(\sum_u||x_u||^2 +\sum_i||y_i||^2) $$

  For more information about what the variables mean, seeFeedback types.

• eval_loss: aFLOAT64 value that contains the loss metric calculated on theholdout data. For k-means models,ML.TRAINING_INFO doesn't return aneval_loss column. If theDATA_SPLIT_METHODargument isNO_SPLIT, then all entries in theeval_loss column areNULL.

• learning_rate: aFLOAT64 value that contains thelearning ratein this iteration.

• duration_ms: anINT64 value that contains how long the iteration took,in milliseconds.

• cluster_info: anARRAY<STRUCT> value that contains thefieldscentroid_id,cluster_radius, andcluster_size.ML.TRAINING_INFO computescluster_radius andcluster_size withstandardized features. Only returned for k-means models.

Permissions

You must have thebigquery.models.create andbigquery.models.getDataIdentity and Access Management (IAM) permissionsin order to runML.TRAINING_INFO.

Limitations

ML.TRAINING_INFO is subject to the following limitations:

• ML.TRAINING_INFO doesn't supportimported TensorFlow models.
• Fortime series models,ML.TRAINING_INFO only returns three columns:training_run,iteration, andduration_ms. It doesn't expose the training information per iteration, or pertime series if multiple time series are forecasted at once. Theduration_ms isthe total time cost for the entire process.

Example

The following example retrieves training information from the modelmydataset.mymodel in your default project:

SELECT*FROMML.TRAINING_INFO(MODEL`mydataset.mymodel`)

What's next

• For more information about model evaluation, seeBigQuery ML model evaluation overview.
• For more information about supported SQL statements and functions for MLmodels, seeEnd-to-end user journeys for ML models.