Preview

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

This page describes model endpoint management. Model endpoint management lets you experiment with registering an AI model endpoint and invoking predictions. To use AI models inproduction environments, seeBuild generative AI applications usingCloud SQL andInvoke online predictions from Cloud SQL instances.

After the model endpoints are added and registered in model endpoint management, you canreference them using the model ID to invoke predictions.

Before you begin

Make sure that you complete the following actions:

• Register your model endpoint with model endpoint management. Formore information, seeRegister and call remote AI models using model endpoint management.
• Create or update your Cloud SQL instance so that the instance can integrate with Vertex AI. For more information, seeEnable database integration with Vertex AI.

Invoke predictions for generic models

Use thegoogle_ml.predict_row() SQL function to call a registered generic model endpoint to invokepredictions. You can usegoogle_ml.predict_row() function with any model type.

SELECTgoogle_ml.predict_row(model_id=>'MODEL_ID',request_body=>'REQUEST_BODY');

Replace the following:

• MODEL_ID: the model ID you defined when registering the model endpoint
• REQUEST_BODY: the parameters to the prediction function, in JSON format

Examples

To generate predictions for a registeredgemini-pro model endpoint, run the following statement:

SELECTjson_array_elements(google_ml.predict_row(model_id=>'gemini-pro',request_body=>'{        "contents": [            {                "role": "user",                "parts": [                    {                        "text": "For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all supplier names which are located in the India nation."                    }                ]            }        ]        }'))->'candidates'->0->'content'->'parts'->0->'text';