The AI.EMBED function

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

Note: To provide feedback or request support for this feature during thepreview, contactbqml-feedback@google.com.

This document describes theAI.EMBED function, which lets you createembeddings from text or image data in BigQuery.For example, the following query creates an embedding for a piece of text:

SELECTAI.EMBED("Some text to embed!",endpoint=>'text-embedding-005');

The function works by sending a request to astable Vertex AI embedding model,and then returning that model's response. The function incurs charges inVertex AI each time it's called.

Embeddings

Embeddings are high-dimensional numerical vectors that represent a given entity.Machine learning (ML) models use embeddings to encode semantics about entitiesto make it easier to reason about and compare them. If two entities aresemantically similar, then their respective embeddings are located near eachother in the embedding vector space.

Embeddings help you perform the following tasks:

  • Semantic search: search entities ranked by semantic similarity.
  • Recommendation: return entities with attributes similar to a givenentity.
  • Classification: return the class of entities whose attributes aresimilar to the given entity.
  • Clustering: cluster entities whose attributes are similar to a givenentity.
  • Outlier detection: return entities whose attributes are least related tothe given entity.

Input

Using theAI.EMBED function, you can use the following typesof input:

When you analyze image data, the content must be in one of the supported imageformats that are described in the Gemini API modelmimeType parameter.

Syntax

Text embedding

AI.EMBED([content=>]'content',endpoint=>'endpoint'[,task_type=>'task_type'][,title=>'title'][,model_params=>model_params][,connection_id=>'connection'])

Arguments

AI.EMBED takes the following arguments:

  • content: aSTRING value that provides the text to embed.

    Thecontent value can be a string literal, the name of a table column, or the output of an expression that evaluates to a string.

  • endpoint: aSTRING value that specifies a supported Vertex AItext embedding model endpoint to use for the text embedding model. The endpoint value that you specify must include the model version, for exampletext-embedding-005. If you specify the model name rather than a URL, BigQuery ML automatically identifies the model and uses the model's full endpoint.
  • task_type: aSTRING literal that specifies the intended downstream application to help the model produce better quality embeddings. Thetask_type argument accepts the following values:
    • RETRIEVAL_QUERY: specifies that the given text is a query in a search or retrieval setting.
    • RETRIEVAL_DOCUMENT: specifies that the given text is a document in a search or retrieval setting.
    • SEMANTIC_SIMILARITY: specifies that the given text will be used for Semantic Textual Similarity (STS).
    • CLASSIFICATION: specifies that the embeddings will be used for classification.
    • CLUSTERING: specifies that the embeddings will be used for clustering.
    • QUESTION_ANSWERING: specifies that the embeddings will be used for question answering.
    • FACT_VERIFICATION: specifies that the embeddings will be used for fact verification.
    • CODE_RETRIEVAL_QUERY: specifies that the embeddings will be used for code retrieval.
  • title: ASTRING value that specifies the document title, which the model uses to improve embedding quality. You can only use this parameter if you specifyRETRIEVAL_DOCUMENT for thetask_type value.
  • model_params: aJSON literal that provides additional parameters to the model. You can use any of theparameters object fields. One of these fields,outputDimensionality, lets you specify the number of dimensions to use when generating embeddings. For example, if you specify256 for theoutputDimensionality field , then the model returns 256 embeddings for each input value.
  • connection_id: aSTRING value specifying the connection to use to communicate with the model, in the formatPROJECT_ID.LOCATION.CONNECTION_ID. For example,myproject.us.myconnection.

    For user-initiated queries, theCONNECTION argument is optional. When a user initiates a query, BigQuery ML uses the credentials of the user who submitted the query to run it.

    If your query job is expected to run for 48 hours or longer, you should use theCONNECTION argument to run the query using a service account.

    Replace the following:

    • PROJECT_ID: the project ID of the project that contains the connection.
    • LOCATION: thelocation used by the connection. The connection must be in the same region in which the query is run.
    • CONNECTION_ID: the connection ID—for example,myconnection.

      You can get this value byviewing the connection details in the Google Cloud console and copying the value in the last section of the fully qualified connection ID that is shown inConnection ID. For example,projects/myproject/locations/connection_location/connections/myconnection.

    You need to grant theVertex AI User role to the connection's service account in the project where you run the function.

Multimodal embedding

AI.EMBED([content=>]'content',connection_id=>'connection',endpoint=>'endpoint'[,model_params=>model_params])

Arguments

AI.EMBED takes the following arguments:

  • content: aSTRING,ObjectRef, orObjectRefRuntimevalue that provides the data to embed.
    • For text embeddings, you can specify one of the following:
      • A string literal.
      • The name of aSTRING column.
      • The output of an expression that evaluates to a string.
    • For image content embeddings, you can specify one of the following:
      • The name of anObjectRef column.
      • AnObjectRef value generated by a combination of theOBJ.FETCH_METADATA andOBJ.MAKE_REF functions. For exampleSELECT OBJ.FETCH_METADATA(OBJ.MAKE_REF("gs://mybucket/path/to/file.jpg", "us.connection1"));.
      • AnObjectRefRuntime value generated by theOBJ.GET_ACCESS_URL function.

    ObjectRef values must have thedetails.gcs_metadata.content_type elements of the JSON value populated.

    ObjectRefRuntime values must have theaccess_url.read_url anddetails.gcs_metadata.content_type elements of the JSON value populated.

  • connection_id: aSTRING value specifying the connection to use to communicate with the model, in the formatPROJECT_ID.LOCATION.CONNECTION_ID. For example,myproject.us.myconnection.

    Replace the following:

    • PROJECT_ID: the project ID of the project that contains the connection.
    • LOCATION: thelocation used by the connection. The connection must be in the same region in which the query is run.
    • CONNECTION_ID: the connection ID—for example,myconnection.

      You can get this value byviewing the connection details in the Google Cloud console and copying the value in the last section of the fully qualified connection ID that is shown inConnection ID. For example,projects/myproject/locations/connection_location/connections/myconnection.

    You need to grant theVertex AI User role to the connection's service account in the project where you run the function.

  • endpoint: aSTRING value that specifies a supported Vertex AImultimodal embedding model endpoint to use for the multimodal embedding model. The endpoint value that you specify must include the model version, for examplemultimodalembedding@001. If you specify the model name rather than a URL, BigQuery ML automatically identifies the model and uses the model's full endpoint.
  • model_params: aJSON literal that provides additional parameters to the model. Only thedimension field is supported. You can use thedimension field to specify the number of dimensions to use when generating embeddings. For example, if you specify256 for thedimension field , then the model returns 256 embeddings for each input value.

Output

AI.EMBED returns aSTRUCT value for each row in the table. The structcontains the following fields:

  • result: anARRAY<FLOAT64> value containing the generated embeddings.
  • status: aSTRING value that contains the API responsestatus for the corresponding row. This value is empty if the operation wassuccessful.

Examples

text embedding

The following example shows how to embed a string literal:

SELECTAI.EMBED('A piece of text to embed',endpoint=>'text-embedding-005').resultASembedding;

If you need to reuse embeddings of the same data across many queries, youshould save the results to table. The following example generates768-dimensional embeddings for publiclyavailable BBC news articles and writes the results to a table:

CREATEORREPLACETABLEmydataset.bbc_news_embeddingsASSELECTtitle,body,AI.EMBED(body,endpoint=>'text-embedding-005',model_params=>JSON'{"outputDimensionality": 768}').resultASembeddingFROM`bigquery-public-data.bbc_news.fulltext`;

The following example queries the table that you just created for thefive articles that are most related to the topic "latest news in tech".It calls theVECTOR_SEARCH function and usesAI.EMBED to createan embedding to pass to the function as the search query.

SELECTbase.title,base.bodyFROMVECTOR_SEARCH(TABLEmydataset.bbc_news_embeddings,# The name of the column that contains the embedding'embedding',# The embedding to search(SELECTAI.EMBED('latest news in tech',endpoint=>'text-embedding-005').result),top_k=>5);

multimodal embedding

The following query creates an external table from images of pet productsstored in a publicly available Cloud Storage bucket. Then, it generatesembeddings for two of the images:

# Create a datasetCREATESCHEMAIFNOTEXISTScymbal_pets;# Create an object tableCREATEORREPLACEEXTERNALTABLEcymbal_pets.product_imagesWITHCONNECTIONDEFAULTOPTIONS(object_metadata='SIMPLE',uris=['gs://cloud-samples-data/bigquery/tutorials/cymbal-pets/images/*.png']);SELECTref.uri,STRING(OBJ.GET_ACCESS_URL(ref,'r').access_urls.read_url)ASsigned_url,AI.EMBED((OBJ.GET_ACCESS_URL(ref,'r')),connection_id=>'us.example_connection',endpoint=>'multimodalembedding@001')ASembeddingFROM`cymbal_pets.product_images`LIMIT2;

Locations

You can runAI.EMBED in all of thelocationsthat support Vertex AI embedding models, and also in theUSandEU multi-regions.

Quotas

SeeVertex AI and Cloud AI service functions quotas and limits.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.