Make predictions with remote models on Vertex AI

In this tutorial, you register a Vertex AI endpoint as a remote modelin BigQuery. Then, you use theML.PREDICT function to makepredictions using the remote model.

You can use remote models when a model is too large to import intoBigQuery. They are also useful when you want to have a singlepoint of inference for online, batch, and micro-batch use cases.

Note: For a version of this tutorial that uses Python in a BigQuerynotebook, see theBQML Remote Model Tutorial in GitHub.

Objectives

  • Import a pretrained TensorFlow model into the Vertex AIModel Registry.
  • Deploy the model to a Vertex AI endpoint.
  • Create a Cloud resource connection.
  • Use theCREATE MODEL statement to create a remote model inBigQuery.
  • Use theML.PREDICT function to make predictions with the remote model.

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use thepricing calculator.

New Google Cloud users might be eligible for afree trial.

When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, seeClean up.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.
    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    Go to project selector

  3. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Roles required to select or create a project

    • Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
    • Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.
    Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

    Go to project selector

  4. Verify that billing is enabled for your Google Cloud project.

  5. Enable the BigQuery, Vertex AI, Cloud Storage, and BigQuery Connection APIs.

    Roles required to enable APIs

    To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

    Enable the APIs

  6. Ensure that you have thenecessary permissions to perform the tasks in this document.

Required roles

If you create a new project, you are the project owner, and you are granted allof the required IAM permissions that you need to complete thistutorial.

If you are using an existing project do the following.

Make sure that you have the following role or roles on the project:

Check for the roles

  1. In the Google Cloud console, go to theIAM page.

    Go to IAM
  2. Select the project.

  3. In thePrincipal column, find all rows that identify you or a group that you're included in. To learn which groups you're included in, contact your administrator.

  4. For all rows that specify or include you, check theRole column to see whether the list of roles includes the required roles.

Grant the roles

  1. In the Google Cloud console, go to theIAM page.

    Go to IAM
  2. Select the project.
  3. ClickGrant access.

  4. In theNew principals field, enter your user identifier. This is typically the email address for a Google Account.

  5. In theSelect a role list, select a role.
  6. To grant additional roles, clickAdd another role and add each additional role.
  7. ClickSave.

For more information about IAM permissions in BigQuery,seeBigQuery permissions.

Import the model to the Vertex AI Model Registry

In this tutorial you use a pretrained TensorFlow model that isavailable in Cloud Storage atgs://cloud-samples-data/bigquery/ml/remote_model_tutorial/. TheCloud Storage bucket is in theUS multi-region location.

The model is a TensorFlow model that's namedsaved_model.pb. It is acustomized sentiment analysis model that was created by fine-tuning a BERT modelwith plain text IMDB movie reviews. The model uses text input from the moviereviews and returns sentiment scores between zero and one. When you import themodel into the Model Registry, you use a prebuilt TensorFlowcontainer.

Note: For a tutorial on creating the sample model, seeClassify text withBERT in the TensorFlow documentation.

Follow these steps to import the model.

  1. In the Google Cloud console, go to the Vertex AIModelRegistry page.

    Go to Model Registry

  2. ClickImport.

  3. ForStep one: Name and region, do the following:

    1. SelectImport as new model.

    2. ForName, enterbert_sentiment.

    3. ForDescription, enterBQML tutorial model.

    4. ForRegion, selectus-central1. You must choose a US-based regionbecause the Cloud Storage bucket is in theUS multi-regionlocation.

    5. ClickContinue.

  4. ForStep two: Model settings, do the following:

    1. SelectImport model artifacts into a new prebuilt container.

    2. In thePrebuilt container settings section, do the following:

      1. ForModel framework, chooseTensorFlow.

      2. ForModel framework version, choose2.15.

      3. ForAccelerator type, chooseGPU.

      4. ForModel artifact location, entergs://cloud-samples-data/bigquery/ml/remote_model_tutorial/.

      5. Leave the default values for all remaining options and clickImport.

After the import is complete, your model appears on theModel Registry page.

Deploy the model to a Vertex AI endpoint

Follow these steps to deploy the model to an endpoint.

  1. In the Google Cloud console go to the Vertex AIModelRegistry page.

    Go to Model Registry

  2. In theName column, clickbert_sentiment.

  3. Click theDeploy & Test tab.

  4. ClickDeploy to endpoint.

  5. For step one,Define your endpoint, do the following:

    1. ClickCreate new endpoint.

    2. ForEndpoint name, enterbert sentiment endpoint.

    3. Leave the remaining default values and clickContinue.

  6. For step two,Model settings, do the following:

    1. In theCompute settings section, forMinimum number of computenodes, enter1. This is the number of nodes that need to beavailable to the model at all times.

      Note: In production, you should set the maximum number of compute nodes.This option turns on the autoscaling capability in Vertex AI,and it allows the endpoint to process more requests when yourBigQuery table has a large number of rows.

    2. In theAdvanced scaling options section, forMachine type,chooseStandard (n1-standard-2). Because you chose GPU as theaccelerator type when you imported the model, after you choose themachine type, the accelerator type and accelerator count are setautomatically.

    3. Leave the remaining default values and clickDeploy.

      When the model is deployed to the endpoint, the status changes toActive.

    4. Copy the numeric endpoint ID in theID column and the value in theRegion column. You'll need them later.

Create a dataset

Create a BigQuery dataset to store your ML model.

Console

  1. In the Google Cloud console, go to theBigQuery page.

    Go to the BigQuery page

  2. In theExplorer pane, click your project name.

  3. ClickView actions > Create dataset

  4. On theCreate dataset page, do the following:

    • ForDataset ID, enterbqml_tutorial.

    • ForLocation type, selectMulti-region, and then selectUS (multiple regions in United States).

    • Leave the remaining default settings as they are, and clickCreate dataset.

bq

To create a new dataset, use thebq mk commandwith the--location flag. For a full list of possible parameters, see thebq mk --dataset commandreference.

  1. Create a dataset namedbqml_tutorial with the data location set toUSand a description ofBigQuery ML tutorial dataset:

    bq --location=US mk -d \ --description "BigQuery ML tutorial dataset." \ bqml_tutorial

    Instead of using the--dataset flag, the command uses the-d shortcut.If you omit-d and--dataset, the command defaults to creating adataset.

  2. Confirm that the dataset was created:

    bqls

API

Call thedatasets.insertmethod with a defineddataset resource.

{"datasetReference":{"datasetId":"bqml_tutorial"}}

BigQuery DataFrames

Before trying this sample, follow the BigQuery DataFrames setup instructions in theBigQuery quickstart using BigQuery DataFrames. For more information, see theBigQuery DataFrames reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up ADC for a local development environment.

importgoogle.cloud.bigquerybqclient=google.cloud.bigquery.Client()bqclient.create_dataset("bqml_tutorial",exists_ok=True)

Create a BigQuery Cloud resource connection

You must have a Cloud resource connection to connect to a Vertex AIendpoint.

Console

  1. Go to theBigQuery page.

    Go to BigQuery

  2. In the left pane, clickExplorer:

    Highlighted button for the Explorer pane.

    If you don't see the left pane, clickExpand left pane to open the pane.

  3. In theExplorer pane, clickAdd data.

    TheAdd data dialog opens.

  4. In theFilter By pane, in theData Source Type section, selectDatabases.

    Alternatively, in theSearch for data sources field, you can enterVertex AI.

  5. In theFeatured data sources section, clickVertex AI.

  6. Click theVertex AI Models: BigQuery Federation solution card.

  7. In theConnection type list, selectVertex AI remote models,remote functions and BigLake (Cloud Resource).

  8. In theConnection ID field, enterbqml_tutorial.

  9. Verify thatMulti-region—US is selected.

  10. ClickCreate connection.

  11. At the bottom of the window, clickGo to connection. Alternatively, intheExplorer pane, clickConnections, and then clickus.bqml_tutorial.

  12. In theConnection info pane, copy the Service account ID. You needthis ID when you configure permissions for the connection. When you createa connection resource, BigQuery creates a unique systemservice account and associates it with the connection.

bq

  1. Create a connection:

    bqmk--connection--location=US--project_id=PROJECT_ID\--connection_type=CLOUD_RESOURCEbqml_tutorial

    ReplacePROJECT_ID with yourGoogle Cloud project ID. The--project_id parameter overrides thedefault project.

    When you create a connection resource, BigQuery creates aunique system service account and associates it with the connection.

    Troubleshooting: If you get the following connection error,update the Google Cloud SDK:

    Flags parsing error: flag --connection_type=CLOUD_RESOURCE: value should be one of...

  2. Retrieve and copy the service account ID for use in a laterstep:

    bqshow--connectionPROJECT_ID.us.bqml_tutorial

    The output is similar to the following:

    name                          properties1234.REGION.CONNECTION_ID {"serviceAccountId": "connection-1234-9u56h9@gcp-sa-bigquery-condel.iam.gserviceaccount.com"}

Set up connection access

Grant the Vertex AI User role to the Cloud resource connection's serviceaccount. You must grant this role in the same project where you created theremote model endpoint.

Note: If the connection is in a different project, this error is returned:bqcx-1234567890-xxxx@gcp-sa-bigquery-condel.iam.gserviceaccount.com does not have the permission to accessresource.

To grant the role, follow these steps:

  1. Go to theIAM & Admin page.

    Go to IAM & Admin

  2. ClickGrant Access.

  3. In theNew principals field, enter the Cloud resource connection'sservice account ID that you copied previously.

  4. In theSelect a role field, chooseVertex AI, and then selectVertex AI User.

  5. ClickSave.

Create a BigQuery ML remote model

You create a BigQuery ML remote model by using theCREATE MODELstatement with theREMOTE WITH CONNECTION clause. For more information ontheCREATE MODEL statement, seeThe CREATE MODEL statement for remote modelsover custom models.

You create your model in theUS multi-region location. In aBigQuery multi-region (US,EU) dataset, you can only create aremote model that connects to an endpoint deployed in a region within the samemulti-region location (US,EU).

When you create the remote model, you need the endpoint ID that was generatedwhen youdeployed the model to Vertex AI. Also, the input andoutput field names and types need to be exactly same as the Vertex AImodel's input and output. In this example, the input is a textSTRING, and theoutput is anARRAY of typeFLOAT64.

Console

  1. In the Google Cloud console, go to theBigQuery page.

    Go to BigQuery

  2. ForCreate new, clickSQL query.

  3. In the query editor, enter thisCREATE MODEL statement, and then clickRun:

    CREATEORREPLACEMODEL`PROJECT_ID.bqml_tutorial.bert_sentiment`INPUT(textSTRING)OUTPUT(scoresARRAY<FLOAT64>)REMOTEWITHCONNECTION`PROJECT_ID.us.bqml_tutorial`OPTIONS(ENDPOINT='https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID')

    Replace the following:

    • PROJECT_ID: your project name.
    • ENDPOINT_ID: the endpoint ID that you copied previously.

    When the operation is complete, you see a message similar toSuccessfully created model named bert_sentiment.

    Your new model appears in theResources panel. Models areindicated by the model icon:modelicon.

    If you select the new model in theResources panel, informationabout the model appears below theQuery editor.

bq

  1. Create the remote model by entering the followingCREATE MODELstatement:

    bqquery--use_legacy_sql=false\"CREATEORREPLACEMODEL`PROJECT_ID.bqml_tutorial.bert_sentiment`INPUT(textSTRING)OUTPUT(scoresARRAY<FLOAT64>)REMOTEWITHCONNECTION`PROJECT_ID.us.bqml_tutorial`OPTIONS(ENDPOINT='https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID')"

    Replace the following:

    • PROJECT_ID: your project name.
    • ENDPOINT_ID: the endpoint ID that you copied previously.

  2. After you create the model, verify that the model appears in thedataset:

    bq ls -m bqml_tutorial

    The output is similar to the following:

    IdModelTypeLabelsCreationTime-----------------------------------------------------bert_sentiment28Jan17:39:43

Get predictions usingML.PREDICT

You use theML.PREDICT function to get sentiment predictions from the remotemodel. The input is a text column (review) that contains reviews of moviesfrom thebigquery-public-data.imdb.reviews table.

In this example, 10,000 records are selected and sent for prediction. The remotemodel defaults to a batch size of 128 instances for requests.

Console

  1. In the Google Cloud console, go to theBigQuery page.

    Go to BigQuery

  2. In theCreate new section, clickSQL query.

  3. In the query editor, enter this query that uses theML.PREDICTfunction, and then clickRun.

    SELECT*FROMML.PREDICT(MODEL`PROJECT_ID.bqml_tutorial.bert_sentiment`,(SELECTreviewastextFROM`bigquery-public-data.imdb.reviews`LIMIT10000))

    The query results should look similar to the following:

    Query results

bq

Enter this command to run the query that usesML.PREDICT.

bq query --use_legacy_sql=false \'SELECT *FROM ML.PREDICT (MODEL`PROJECT_ID.bqml_tutorial.bert_sentiment`,  (    SELECT review as text    FROM`bigquery-public-data.imdb.reviews`    LIMIT 10000  ))'

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

Delete the project

Console

    Caution: Deleting a project has the following effects:
    • Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
    • Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as anappspot.com URL, delete selected resources inside the project instead of deleting the whole project.

    If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

  1. In the Google Cloud console, go to theManage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then clickDelete.
  3. In the dialog, type the project ID, and then clickShut down to delete the project.

gcloud

    Caution: Deleting a project has the following effects:
    • Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
    • Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as anappspot.com URL, delete selected resources inside the project instead of deleting the whole project.

    If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

  1. In the Google Cloud console, go to theManage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then clickDelete.
  3. In the dialog, type the project ID, and then clickShut down to delete the project.

Delete individual resources

Alternatively, to remove the individual resources used in this tutorial:

  1. Delete the model.

  2. Optional:Delete the dataset.

  3. Undeploy the model and delete the endpoint.

  4. Delete the model from the Model Registry.

  5. Delete the Cloud resource connection.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.