For AutoML tabular models, AutoML image models,and custom-trained models, you can enable or disable inference logs duringmodel deployment or endpointcreation. This page explains the different types of inferencelogs available, and how to enable or disable these logs.

Types of inference logs

There are several types of inference logs that you can use to get informationfrom your inference nodes:

• Container logging, which logs thestdout andstderr streams from yourinference nodes toCloud Logging. These logsare required for debugging.

  • On thev1 service endpoint, container logging is enabled by default.You can disable it when you deploy a model. You can also disable orenable logging when youmutatethe deployed model.

  • On thev1beta1 service endpoint, container logging is disabled bydefault. You can enable it when you deploy a model. You can also disableor enable logging when youmutatethe deployed model.

  Note: The default logging behavior in Python sends outputs tostderr, whichwill appear at theERROR level in Cloud Logging. If you'd like forcontainer logs to appear at theINFO level, configure your containerlogging to send outputs tostdout. For more information, see the PythonLogging handlerstutorials and the PythonLogging Cookbook.

• Access logging, which logs information like timestamp and latency foreach request to Cloud Logging.

  On both thev1 andv1beta1 service endpoints, access logging is disabledby default. You can enable access logging when you deploy a model to anendpoint.

• Request-response logging, which logs a sample of online inferencerequests and responses to a BigQuery table.

  You can enable request-response logging by creating or patching theinference endpoint.

You can enable or disable each type of log independently.

Inference log settings

You can enable or disable online inference logs when you create an endpoint,deploy a model to the endpoint, or mutate a deployed model.

To update the settings for access logs, you mustundeploy your model,and then redeploy the model with your new settings. You can update the settingsfor container logs without re-deploying your model.

Online inference at a high rate of queries per second (QPS) can produce asubstantial number of logs, which are subject toCloud Logging pricing. To estimate the pricing foryour online inference logs, seeEstimating your bills for logging. To reducethis cost, you can disable inference logging.

Enable and disable inference logs

The following examples highlight where to modify the default log settings:

gcloudaiendpointsdeploy-modelENDPOINT_ID\--region=LOCATION\--model=MODEL_ID\--display-name=DEPLOYED_MODEL_NAME\--machine-type=MACHINE_TYPE\--accelerator=count=2,type=nvidia-tesla-t4\--disable-container-logging\--enable-access-logging

gcloudbetaaiendpointsdeploy-modelENDPOINT_ID\--region=LOCATION\--model=MODEL_ID\--display-name=DEPLOYED_MODEL_NAME\--machine-type=MACHINE_TYPE\--accelerator=count=2,type=nvidia-tesla-t4\--enable-access-logging\--enable-container-logging

{   "predict_request_response_logging_config": {     "enabled": true,     "sampling_rate": 0.5,     "bigquery_destination": {       "output_uri": "bq://PROJECT_ID.DATASET_NAME.TABLE_NAME"     }   }}

Inference request-response logging for dedicated endpoints and Private Service Connect endpoints

Fordedicated endpointsandPrivate Service Connectendpoints, you can use request-response logging to record requestand response payloads under 10 MB (larger payloads are skipped automatically)for TensorFlow, PyTorch, sklearn, and XGBoost models.

Request-response logging is available only for thepredictandrawPredictmethods.

To enable request-response logging, populate thepredictRequestResponseLoggingConfigfield of theEndpoint resource with the following entries:

• enabled: set toTrue to enable request-response logging.

• samplingRate: the fraction of requests and responses to log. Set to a numberthat is greater than 0 and less than or equal to 1. For example, set thisvalue to1 in order to log all requests or to0.1 to log 10% of requests.

• BigQueryDestination: the BigQuery location for theoutput content, as a URI to a project or table.

The following is an example configuration for creating a dedicated endpoint withrequest-response logging enabled:

curl-XPOST\-H"Content-Type: application/json"\-H"Authorization: Bearer `gcloud auth print-access-token`"https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/endpoints\-d'{displayName: "ENDPOINT_NAME", \     dedicatedEndpointEnabled: true, \     predictRequestResponseLoggingConfig: { \       enabled: true, \       samplingRate: 1.0, \       bigqueryDestination: { \          outputUri:"bq://PROJECT_ID" \       } \     } \   }'

Replace the following:

• LOCATION_ID: The region where you are using Vertex AI.
• PROJECT_NUMBER: The project number for your Google Cloudproject.
• ENDPOINT_NAME: The display name for the endpoint.
• PROJECT_ID: The project ID for your Google Cloud project.

The following is an example configuration for creating aPrivate Service Connect endpoint with request-response logging enabled:

curl-XPOST\-H"Content-Type: application/json"\-H"Authorization: Bearer `gcloud auth print-access-token`"https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/endpoints\-d'{displayName: "ENDPOINT_NAME", \     privateServiceConnectConfig: { \       enablePrivateServiceConnect: true, \       projectAllowlist: ["ALLOWED_PROJECTS"] \     }, \     predictRequestResponseLoggingConfig: { \       enabled: true, \       samplingRate: 1.0, \       bigqueryDestination: { \          outputUri:"bq://PROJECT_ID" \       } \     } \   }'

Replace the following:

• ALLOWED_PROJECTS: a comma-separated list of Google Cloudproject IDs, each enclosed in quotation marks. For example,["PROJECTID1", "PROJECTID2"].If a project isn't contained in this list,you won't be able to send inference requests to theVertex AI endpoint from it. Make sure to includeVERTEX_AI_PROJECT_ID in this list so that you can call theendpoint from the same project it's in.

Request-response logging and Model Monitoring v1

Request-response logging andModel Monitoring v1 usethe same BigQuery table on the backend to log incoming requests.To prevent unexpected changes to this BigQuery table, thefollowing limitations are enforced when using both features at the same time:

• If an endpoint has Model Monitoring enabled, you can'tenable request-response logging for the same endpoint.

• If you enable request-response logging and thenModel Monitoring on the same endpoint, you won't be ableto change the request-response logging configuration.

What's next

• Estimate pricing for online inferencelogging.
• Deploy a modelusing the Google Cloud console orusing the Vertex AI API.
• Learnhow to create a BigQuery table.