Preview

This product is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

This page describes how to invoke predictions or generate embeddings using a model, and then register the model endpoint with model endpoint management.

For more information about thegoogle_ml.create_model() function, seeModel endpoint management reference.

Before you begin

• If your model endpoint requires authentication, thenenable thegoogle_ml_integration extension.

• Based on the model provider, set up authentication.

• Make sure that you use thepostgres default username to access your database.

Enable the extension

1. Set thegoogle_ml_integration.enable_model_support database flag toon for your instance. For more information about setting database flags, seeConfigure database flags.

2. Connect to your primary instance using either apsql client or Cloud SQL Studio.

3. Run the following command to ensure that thegoogle_ml_integration extension is updated to version 1.4.2:

   ALTEREXTENSIONgoogle_ml_integrationUPDATETO'1.4.2'

4. Add thegoogle_ml_integration version 1.4.2 extension usingpsql:

   CREATEEXTENSIONgoogle_ml_integrationVERSION'1.4.2';

5. Optional: Grant permission to a non-super PostgreSQL user to manage model metadata:

   GRANTSELECT,INSERT,UPDATE,DELETEONALLTABLESINSCHEMAgoogle_mlTONON_SUPER_USER;

   ReplaceNON_SUPER_USER with the non-super PostgreSQL username.

Set up authentication

The following sections show how to set up authentication before adding a Vertex AI model endpoint or model endpoints hosted within Google Cloud.

Set up authentication for Vertex AI

To use the Google Vertex AI model endpoints, you must add Vertex AI permissions to the IAM-based Cloud SQL service account you use to connect to the database. For more information about integrating with Vertex AI, seeIntegrate Cloud SQL with Vertex AI.

Set up authentication for custom-hosted models

This section explains how to set up authentication if you're using Secret Manager. For all models except Vertex AI model endpoints, you can store your API keys or bearer tokens in Secret Manager.

If your model endpoint doesn't handle authentication through Secret Manager, then this section is optional. For example, if your model endpoint uses HTTP headers to pass authentication information or doesn't use authentication at all, then don't complete the steps in this section.

To create and use an API key or a bearer token, complete the following steps:

1. Create a secret in Secret Manager. For more information, seeCreate a secret and access a secret version.

   The secret name and the secret path are used in thegoogle_ml.create_sm_secret() SQL function.

2. Grant permissions to the Cloud SQL instance to access the secret.

     gcloud secrets add-iam-policy-bindingSECRET_ID \      --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \      --role="roles/secretmanager.secretAccessor"

   Replace the following:

   • SECRET_ID: the secret ID in Secret Manager.
   • SERVICE_ACCOUNT_EMAIL: the email address of the IAM-based Cloud SQL service account. To find this email address, use thegcloud sql instances describeINSTANCE_NAME command and replaceINSTANCE_NAME with the name of the instance. The value that appears next to theserviceAccountEmailAddress parameter is the email address.

Text embedding models with built-in support

This section shows how to register model endpoints for model endpoint management.

Vertex AI embedding models

Model endpoint management provides built-in support for all versions of thetext-embedding-gecko model by Vertex AI. Use the qualified name to set the model version to eithertextembedding-gecko@001 ortextembedding-gecko@002.

Because both thetextembedding-gecko andtextembedding-gecko@001 model endpoint IDsare pre-registered with model endpoint management, you can use either of them directly asthe model ID. For these models, the extension automatically sets up defaulttransform functions.

Ensure that both the Cloud SQL instance and the Vertex AI model that you're querying are in the same region.

To register thetextembedding-gecko@002 model endpoint, call thecreate_model function:

CALLgoogle_ml.create_model(model_id=>'textembedding-gecko@002',model_provider=>'google',model_qualified_name=>'textembedding-gecko@002',model_type=>'text_embedding',model_auth_type=>'cloudsql_service_agent_iam');

Custom-hosted text embedding models

This section shows how to register custom model endpoints hosted in networks within Google Cloud.

Adding custom-hosted text embedding model endpoints involves creating transform functions, and optionally, custom HTTP headers. On the other hand, adding custom-hosted generic model endpoints involves optionally generating custom HTTP headers and setting the model request URL.

The following example adds thecustom-embedding-model text embedding model endpoint hosted byCymbal, which is hosted within Google Cloud. Thecymbal_text_input_transform andcymbal_text_output_transformtransform functions are used to transform the input and output format of themodel to the input and output format of the prediction function.

To register custom-hosted text embedding model endpoints, complete the following steps:

1. Call the secret stored in the Secret Manager:

   CALLgoogle_ml.create_sm_secret(secret_id=>'SECRET_ID',secret_path=>'projects/project-id/secrets/SECRET_MANAGER_SECRET_ID/versions/VERSION_NUMBER');

   Replace the following:

   • SECRET_ID: the secret ID that you set and is subsequently used when registering a model endpoint—for example,key1.
   • SECRET_MANAGER_SECRET_ID: the secret ID set in Secret Manager when you created the secret.
   • PROJECT_ID: the ID of your Google Cloud project.
   • VERSION_NUMBER: the version number of the secret ID.
   Note: Secret Manager generates anAuthorization: BearerSECRET_VALUE_FROM_SECRET_MANAGER header for authentication by default. If this format matches your model endpoint's authorization bearer token format, then you don't have to generate auth headers using the header generation function.

2. Create the input and output transform functions based on the following signature for the prediction function for text embedding model endpoints. For more information about how to create transform functions, seeTransform functions example.

   The following are example transform functions that are specific to thecustom-embedding-model text embedding model endpoint:

   -- Input Transform Function corresponding to the custom model endpointCREATEORREPLACEFUNCTIONcymbal_text_input_transform(model_idVARCHAR(100),input_textTEXT)RETURNSJSONLANGUAGEplpgsqlAS$$DECLAREtransformed_inputJSON;model_qualified_nameTEXT;BEGINSELECTjson_build_object('prompt',json_build_array(input_text))::JSONINTOtransformed_input;RETURNtransformed_input;END;$$;-- Output Transform Function corresponding to the custom model endpointCREATEORREPLACEFUNCTIONcymbal_text_output_transform(model_idVARCHAR(100),response_jsonJSON)RETURNSREAL[]LANGUAGEplpgsqlAS$$DECLAREtransformed_outputREAL[];BEGINSELECTARRAY(SELECTjson_array_elements_text(response_json->0))INTOtransformed_output;RETURNtransformed_output;END;$$;

3. Call the create model function to register the custom embedding model endpoint:

   CALLgoogle_ml.create_model(model_id=>'MODEL_ID',model_request_url=>'REQUEST_URL',model_provider=>'custom',model_type=>'text_embedding',model_auth_type=>'secret_manager',model_auth_id=>'SECRET_ID',model_qualified_name=>'MODEL_QUALIFIED_NAME',model_in_transform_fn=>'cymbal_text_input_transform',model_out_transform_fn=>'cymbal_text_output_transform');

   Replace the following:

   • MODEL_ID: required. A unique ID for the model endpoint that you define (for example,custom-embedding-model). This model ID is referenced for metadata that the model endpoint needs to generate embeddings or invoke predictions.
   • REQUEST_URL: required. The model-specific endpoint when adding custom text embedding and generic model endpoints—for example,https://cymbal.com/models/text/embeddings/v1. Ensure that the model endpoint is accessible through an internal IP address. Model endpoint management doesn't support external IP addresses.
   • MODEL_QUALIFIED_NAME: required if your model endpoint uses a qualified name. The fully qualified name in case the model endpoint has multiple versions.
   • SECRET_ID: the secret ID you used earlier in thegoogle_ml.create_sm_secret() procedure.

Generic models

This section shows how to register a genericgemini-pro model endpoint from Vertex AIModel Garden, which doesn't have built-in support. You can register anygeneric model endpoint that is hosted within Google Cloud.

Cloud SQL only supports model endpoints that are available through Vertex AIModel Garden and model endpoints hosted in networks within Google Cloud.

Gemini model

The following example adds thegemini-1.0-pro model endpoint from the Vertex AI Model Garden.

To register thegemini-1.0-pro model endpoint, call thecreate model function:

```sqlCALLgoogle_ml.create_model(model_id=>'MODEL_ID',model_request_url=>'https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/gemini-1.0-pro:streamGenerateContent',model_provider=>'google',model_auth_type=>'cloudsql_service_agent_iam');```Replacethefollowing:*<code><var>MODEL_ID</var></code>:auniqueIDforthemodelendpointthatyoudefine(forexample,<br>`gemini-1`).ThismodelIDisreferencedformetadatathatthemodelendpointneedstogenerateembeddingsorinvokepredictions.*<code><var>PROJECT_ID</var></code>:theIDofyour Google Cloudproject.

For more information, see how toinvoke predictions for generic model endpoints.

What's next

• Learn about themodel endpoint management reference.