This page provides an overview of Spanner Vertex AI integration. Spanner Vertex AI integration works with bothGoogleSQL and PostgreSQL databases.

Spanner Vertex AI integration helps you to access classifier and regression ML models hosted onVertex AI through the GoogleSQL andPostgreSQL interface. Thishelps to seamlessly integrate ML predictions serving functionality with generalSpanner data access operations performed using DQL/DML queries.

Benefits of Spanner Vertex AI integration

Generating ML predictions using Spanner Vertex AI integration provides multiple benefits comparedto the approach where Spanner data access and access to the Vertex AIprediction endpoint are performed separately:

• Performance:
  • Better latency: Spanner Vertex AI integration talking to the Vertex AI service directlyeliminates additional round-trips between a compute node running a Spanner'sclient and the Vertex AI service.
  • Better throughput/parallelism: Spanner Vertex AI integration runs on top of Spanner'sdistributed query processing infrastructure, which supports highlyparallelizable query execution.
• User experience:
  • Ability to use a single, simple, coherent, and familiar SQL interface tofacilitate both data transformation and ML serving scenarios on Spannerlevel of scale lowers the ML entry barrier and allows for a much smoother userexperience.
• Costs:
  • Spanner Vertex AI integration uses Spannercompute capacity tomerge the results of ML computations and SQL query execution, whicheliminates the need to provision an additional compute (for example, inCompute Engine or Google Kubernetes Engine) for that.

How does Spanner Vertex AI integration work?

Spanner Vertex AI integration doesn't host ML models, but relies on the Vertex AI serviceinfrastructure instead. You don't need to train a model using Vertex AIto use it with Spanner Vertex AI integration, but you must deploy it to a Vertex AI endpoint.

To train models on data stored in Spanner, you can use the following:

• BigQuery Federated queries togetherwithBigQuery ML.

• Dataflow to export data from Spanner into CSV formatand import theCSV data source into Vertex AI.

Spanner Vertex AI integration extends the following functions for using ML models:

• Generate ML predictions by calling a model usingSQL on your Spanner data. You can use a model from theVertex AI Model Gardenor a model deployed to yourVertex AI endpoint.

• Generate text embeddings to have an LLM translatetext prompts into numbers. Tolearn more about embeddings, seeGet text embeddings.

Using Spanner Vertex AI integration functions

A model in Spanner Vertex AI integration can be used to generate predictions or textembeddings in your SQL code using the ML Predict functions. These functionsare as follows:

To use the functions,you can select a model from theVertex AI Model Gardenor use a model that you've deployed to Vertex AI.

For more information on how to deploy a model to an endpointin Vertex AI, seeDeploy a model to an endpoint.

For more informationon how to use these functions to generate an ML prediction,seeGenerate ML predictions using SQL.

For more information on how to use these functions to generate text embeddings,seeGet text embeddings.

Pricing

There are no additional charges from Spanner when you use it with Spanner Vertex AI integration.However, there are other potential charges associated with this feature:

• You pay thestandard rates for Vertex AIonline prediction. The total charge depends on the model type you use. Some modeltypes have a flat per hour rate, depending on the machine type and number ofnodes that you use. Some model types have per call rates. We recommend youdeploy the latter in a dedicated project where you have set explicit predictionquotas.

• You pay thestandard rates for data transferbetween Spanner and Vertex AI. The total charge depends onthe region hosting the server that executes the query and the region hosting thecalled endpoint. To minimize charges, deploy your Vertex AI endpointsin the same region as your Spanner instance. When using multi-regionalinstance configurations or multiple Vertex AI endpoints, deploy yourendpoints on the same continent.

SLA

Due toVertex AI online prediction availability beinglower, you must properly configure Spanner ML models to maintainSpanner's high availability while using Spanner Vertex AI integration:

1. Spanner ML models must use multiple Vertex AIendpoints on the backend to enable failover.
2. Vertex AI endpoints must conform to theVertex AI SLA.
3. Vertex AI endpoints must provision enough capacity to handleincoming traffic.
4. Vertex AI endpoints must use separate regions close to theSpanner database to avoid regional outages.
5. Vertex AI endpoints should use separate projects to avoidissues with per-project prediction quotas.

The number of redundant Vertex AI endpoints depends on their SLA, andthe number of rows in Spanner queries:

Vertex AI endpoints don't need to host exactly the same model. Werecommend that you configure the Spanner ML model to have a primary,complex and compute intensive model as its first endpoint. Subsequent failoverendpoints can point to simplified models that are less compute intensive,scale better and can absorb traffic spikes.

Limitations

• Model input and output must be a JSON object.

Compliance

Assured Workloads don't support theVertex AI Prediction API. Enabling arestrict resource usage constraintdisables the Vertex AI API and effectively the Spanner Vertex AI integration feature.

Additionally, we recommend that you create aVPC Service Controls perimeter to ensure yourproduction databases cannot connect to Vertex AI endpoints in yournon-production projects that might not have the proper compliance configuration.