gemini-embedding-001

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([FLATTEN_JSON_OUTPUTASflatten_json_output][,TASK_TYPEAStask_type][,OUTPUT_DIMENSIONALITYASoutput_dimensionality]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa remote model over a supported open model.

  You can confirm what LLM is used by the remote model by opening the Google Cloud console and looking at theRemote endpoint field in the model details page.

• QUERY_STATEMENT: a query whose result contains aSTRING column that's namedcontent. For information about thesupported SQL syntax of theQUERY_STATEMENT clause, seeGoogleSQL querysyntax.

• FLATTEN_JSON_OUTPUT: aBOOL value thatdetermines whether theJSON content returned by the function is parsedinto separate columns. The default isTRUE.

• TASK_TYPE: aSTRING literal that specifies theintended downstream application to help the model produce better qualityembeddings. TheTASK_TYPE argument accepts the following values:

  • RETRIEVAL_QUERY: specifies that the given text is a query in asearch or retrieval setting.

  • RETRIEVAL_DOCUMENT: specifies that the given text is a document in asearch or retrieval setting.

    When using this task type, it is helpful to include the document titlein the query statement in order to improve embedding quality.The document title must be in a column either namedtitle oraliased astitle, for example:

    SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.embedding_model`,(SELECTabstractascontent,headerastitle,publication_numberFROM`mydataset.publications`),STRUCT(TRUEASflatten_json_output,'RETRIEVAL_DOCUMENT'astask_type));

    Specifying the title column in the input query populates thetitle fieldof the request body sent to the model.If you specify atitle value when using any other task type, thatinput is ignored and has no effect on the embedding results.

  • SEMANTIC_SIMILARITY: specifies that the given text will be used forSemantic Textual Similarity (STS).

  • CLASSIFICATION: specifies that the embeddings will be used forclassification.

  • CLUSTERING: specifies that the embeddings will be used forclustering.

  • QUESTION_ANSWERING: specifies that the embeddings will be used for question answering.

  • FACT_VERIFICATION: specifies that the embeddings will be used for fact verification.

  • CODE_RETRIEVAL_QUERY: specifies that the embeddings will be used for code retrieval.

• OUTPUT_DIMENSIONALITY: anINT64 value in the range[1, 3072]that specifies the number of dimensions to use when generatingembeddings. For example, if you specify256 AS output_dimensionality,then theml_generate_embedding_result output column contains 256embeddings for each input value.The default value is3072.

Details

The model and input table must be in the same region.

text-embedding

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([FLATTEN_JSON_OUTPUTASflatten_json_output][,TASK_TYPEAStask_type][,OUTPUT_DIMENSIONALITYASoutput_dimensionality]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa remote model over a Vertex AItext embedding model.

  You can confirm what LLM is used by the remote model by opening the Google Cloud console and looking at theRemote endpoint field in the model details page.

• TABLE_NAME: the name of theBigQuery table that contains aSTRING column to embed.The text in the column that's namedcontent is sent to the model. Ifyour table doesn't have acontent column, use aSELECT statement forthis argument to provide an alias for an existing table column. An erroroccurs if nocontent column exists.

• QUERY_STATEMENT: a query whose result contains aSTRING column that's namedcontent. For information about thesupported SQL syntax of theQUERY_STATEMENT clause, seeGoogleSQL querysyntax.

• FLATTEN_JSON_OUTPUT: aBOOL value thatdetermines whether theJSON content returned by the function is parsedinto separate columns. The default isTRUE.

• TASK_TYPE: aSTRING literal that specifies theintended downstream application to help the model produce better qualityembeddings. TheTASK_TYPE argument accepts the following values:

  • RETRIEVAL_QUERY: specifies that the given text is a query in asearch or retrieval setting.

  • RETRIEVAL_DOCUMENT: specifies that the given text is a document in asearch or retrieval setting.

    When using this task type, it is helpful to include the document titlein the query statement in order to improve embedding quality.The document title must be in a column either namedtitle oraliased astitle, for example:

    SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.embedding_model`,(SELECTabstractascontent,headerastitle,publication_numberFROM`mydataset.publications`),STRUCT(TRUEASflatten_json_output,'RETRIEVAL_DOCUMENT'astask_type));

    Specifying the title column in the input query populates thetitle fieldof the request body sent to the model.If you specify atitle value when using any other task type, thatinput is ignored and has no effect on the embedding results.

  • SEMANTIC_SIMILARITY: specifies that the given text will be used forSemantic Textual Similarity (STS).

  • CLASSIFICATION: specifies that the embeddings will be used forclassification.

  • CLUSTERING: specifies that the embeddings will be used forclustering.

  • QUESTION_ANSWERING: specifies that the embeddings will be used for question answering.

  • FACT_VERIFICATION: specifies that the embeddings will be used for fact verification.

  • CODE_RETRIEVAL_QUERY: specifies that the embeddings will be used for code retrieval.

• OUTPUT_DIMENSIONALITY: anINT64 value in the range[1, 768]that specifies the number of dimensions to use when generatingembeddings. For example, if you specify256 AS output_dimensionality,then theml_generate_embedding_result output column contains 256embeddings for each input value.The default value is768.

Details

The model and input table must be in the same region.

Open models

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([FLATTEN_JSON_OUTPUTASflatten_json_output]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa remote model over a supported open model.

  You can confirm the type of model by opening the Google Cloud console and looking at theModel type field in the model details page.

• TABLE_NAME: the name of theBigQuery table that contains aSTRING column to embed.The text in the column that's namedcontent is sent to the model. Ifyour table doesn't have acontent column, use aSELECT statement forthis argument to provide an alias for an existing table column. An erroroccurs if nocontent column exists.

• QUERY_STATEMENT: a query whose result contains aSTRING column that's namedcontent. For information about thesupported SQL syntax of theQUERY_STATEMENT clause, seeGoogleSQL querysyntax.

• FLATTEN_JSON_OUTPUT: aBOOL value thatdetermines whether theJSON content returned by the function is parsedinto separate columns. The default isTRUE.

Details

The model and input table must be in the same region.

multimodalembedding

# Syntax for standard tablesML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([FLATTEN_JSON_OUTPUTASflatten_json_output][,OUTPUT_DIMENSIONALITYASoutput_dimensionality]))

# Syntax for object tablesML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([FLATTEN_JSON_OUTPUTASflatten_json_output][,START_SECONDASstart_second][,END_SECONDASend_second][,INTERVAL_SECONDSASinterval_seconds][,OUTPUT_DIMENSIONALITYASoutput_dimensionality]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa remote model over a Vertex AImultimodalembedding@001 model.

  You can confirm what LLM is used by the remote model by opening the Google Cloud console and looking at theRemote endpoint field in the model details page.

• TABLE_NAME: one of the following:

  • If you are creating embeddings for text ina standard table, the name of the BigQuery tablethat contains the content. The content must be in aSTRINGcolumn namedcontent. If your table does not have acontent column, use theQUERY_STATEMENT argument instead andprovide aSELECT statement that includes an alias for an existingtable column. An error occurs if nocontent column is available.

  • If you are creating embeddings for visual content using data from anan object table, the name of a BigQueryobject table thatcontains the visual content.

• QUERY_STATEMENT: the GoogleSQL querythat generates the input data for the function.

  • If you are creating embeddings from a standard table, the query mustproduce a column namedcontent, which you can generate as follows:

    • For text embeddings, you can pull the value from aSTRINGcolumn, or you can specify a string literal in the query.

    • For visual content embeddings, you can provide anObjectRefRuntimevalue for thecontent column. You can generateObjectRefRuntime values by using theOBJ.GET_ACCESS_URL function.TheOBJ.GET_ACCESS_URL function takes anObjectRefvalue as input, which you can provide by either specifyingthe name of a column that containsObjectRef values, or byconstructing anObjectRef value.

      ObjectRefRuntime values must have theaccess_url.read_url anddetails.gcs_metadata.content_type elements of the JSON valuepopulated.

  • If you are creating embeddings from an object table, the query doesn'thave to return acontent column. You can only specifyWHERE,ORDER BY, andLIMIT clauses in the query.

• FLATTEN_JSON_OUTPUT: aBOOL value thatdetermines whether theJSON content returned by the function is parsedinto separate columns. The default isTRUE.

• START_SECOND: aFLOAT64 value that specifies thesecond in the videoat which to start the embedding. The default value is0.If you specify this argument, you mustalso specify theEND_SECOND argument. This value must be positive andless than theEND_SECOND value. This argument only applies to videocontent.

• END_SECOND: aFLOAT64 value that specifies thesecond in the video at which to end the embedding. TheEND_SECOND valuecan't be higher than120. The default value is120. If you specifythis argument, you must also specify theSTART_SECOND argument. Thisvalue must be positive and greater than theSTART_SECOND value. Thisargument only applies to video content.

• INTERVAL_SECONDS: aFLOAT64 value that specifiesthe interval to use when creating embeddings. For example, if you setSTART_SECOND =0,END_SECOND =120, andINTERVAL_SECONDS =10,then the video is split into twelve 10 second segments ([0, 10), [10,20), [20, 30)...) and embeddings are generated for each segment. Thisvalue must be greater thanor equal to4 and less than120. The default value is16.This argument only applies to video content.

• OUTPUT_DIMENSIONALITY: anINT64 value thatspecifies the number of dimensions to use when generating embeddings.Valid values are128,256,512, and1408. The default value is1408. For example, if you specify256 AS output_dimensionality, thentheml_generate_embedding_result output column contains 256 embeddingsfor each input value.

  You can only use this argument when creating text or image embeddings. If you use this argument when creating video embeddings, the function returns an error.

Details

The model and input table must be in the same region.

PCA

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)})

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa PCA model.

  You can confirm the type of model by opening the Google Cloud console and looking at theModel type field in the model details page.

• TABLE_NAME: the name of theBigQuery table that contains the input data for the PCAmodel.

• QUERY_STATEMENT: a query whose result contains theinput data for the PCA model.

Details

The model and input table must be in the same region.

Autoencoder

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,{TABLE`PROJECT_ID.DATASET.TABLE_NAME`|(QUERY_STATEMENT)},STRUCT([TRIAL_IDAStrial_id]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofan autoencoder model.

  You can confirm the type of model by opening the Google Cloud console and looking at theModel type field in the model details page.

• TABLE_NAME: the name of theBigQuery table that contains the input data for theautoencoder model.

• QUERY_STATEMENT: a query whose result contains theinput data for the autoencoder model.

• TRIAL_ID: anINT64 value that identifies thehyperparameter tuning trial that you want the function to evaluate. Thefunction uses the optimal trial by default. Only specify this argument ifyou ran hyperparameter tuning when creating the model.

Details

The model and input table must be in the same region.

Matrix factorization

ML.GENERATE_EMBEDDING(MODEL`PROJECT_ID.DATASET.MODEL_NAME`,STRUCT([TRIAL_IDAStrial_id]))

Arguments

ML.GENERATE_EMBEDDING takes the following arguments:

• PROJECT_ID: the project that contains theresource.

• DATASET: the BigQuery dataset thatcontains the resource.

• MODEL_NAME: the name ofa matrix factorization model.

  You can confirm the type of model by opening the Google Cloud console and looking at theModel type field in the model details page.

• TRIAL_ID: anINT64 value that identifies thehyperparameter tuning trial that you want the function to evaluate. Thefunction uses the optimal trial by default. Only specify this argument ifyou ran hyperparameter tuning when creating the model.

gemini-embedding-001

ML.GENERATE_EMBEDDING returns the input table and the following columns:

• ml_generate_embedding_result:

  • Ifflatten_json_output isFALSE, this is theJSON responsefrom theprojects.locations.endpoints.predict call to the model. Thegenerated embeddings are in thevalues element.
  • Ifflatten_json_output isTRUE, this is anARRAY<FLOAT64>value that contains the generated embeddings.

• ml_generate_embedding_statistics: aJSON value that contains atoken_count field with the number of tokens in the content, and atruncated field that indicates whether the content was truncated. Thiscolumn is returned whenflatten_json_output isTRUE.

• ml_generate_embedding_status: aSTRING value that contains the APIresponse status for the corresponding row. This value is empty if theoperation was successful.

text-embedding

ML.GENERATE_EMBEDDING returns the input table and the following columns:

• ml_generate_embedding_result:

  • Ifflatten_json_output isFALSE, this is theJSON responsefrom theprojects.locations.endpoints.predict call to the model. Thegenerated embeddings are in thevalues element.
  • Ifflatten_json_output isTRUE, this is anARRAY<FLOAT64>value that contains the generated embeddings.

• ml_generate_embedding_statistics: aJSON value that contains atoken_count field with the number of tokens in the content, and atruncated field that indicates whether the content was truncated. Thiscolumn is returned whenflatten_json_output isTRUE.

• ml_generate_embedding_status: aSTRING value that contains the APIresponse status for the corresponding row. This value is empty if theoperation was successful.

Open models

ML.GENERATE_EMBEDDING returns the input table and the following columns:

• ml_generate_embedding_result:

  • Ifflatten_json_output isFALSE, this is theJSON responsefrom theprojects.locations.endpoints.predict call to the model. Thegenerated embeddings are in the first element of thepredictionsarray.
  • Ifflatten_json_output isTRUE, this is anARRAY<FLOAT64>value that contains the generated embeddings.

• ml_generate_embedding_status: aSTRING value that contains the APIresponse status for the corresponding row. This value is empty if theoperation was successful.

multimodalembedding

ML.GENERATE_EMBEDDING returns the input table and the following columns:

• ml_generate_embedding_result:

  • Ifflatten_json_output isFALSE, this is theJSON responsefrom theprojects.locations.endpoints.predict call to the model. Thegenerated embeddings are in thetextEmbedding,imageEmbedding, orvideoEmbeddingselement, depending on the type of input data youused.
  • Ifflatten_json_output isTRUE, this is anARRAY<FLOAT64>value that contains the generated embeddings.

• ml_generate_embedding_status: aSTRING value that contains the APIresponse status for the corresponding row. This value is empty if theoperation was successful.

• ml_generate_embedding_start_sec: for video content, anINT64 valuethat contains the starting second of the portion of the video that theembedding represents. For image content, the value isNULL.This column isn't returned for text content.

• ml_generate_embedding_end_sec: for video content, anINT64 valuethat contains the ending second of the portion of the video that theembedding represents. For image content, the value isNULL.This column isn't returned for text content.

PCA

ML.GENERATE_EMBEDDING returns the input table and the following column:

• ml_generate_embedding_result: this is anARRAY<FLOAT> value thatcontains the principal components for the input data. The number of arraydimensions is equal to the PCA model'sNUM_PRINCIPAL_COMPONENTS optionvalue if that option is used when the model is created. If thePCA_EXPLAINED_VARIANCE_RATIO option is usedinstead, the array dimensions vary depending on the input table andthe option ratio determined by BigQuery ML.

Autoencoder

ML.GENERATE_EMBEDDING returns the input table and the following column:

• trial_id: anINT64 value that identifies the hyperparameter tuningtrial used by the function. This column is only returned if you ranhyperparameter tuning when creating the model.
• ml_generate_embedding_result: this is anARRAY<FLOAT> value thatcontains the latent space dimensions for the input data.The number of array dimensions is equal to the number in the middleof the autoencoder model'sHIDDEN_UNITS optionarray value.

Matrix factorization

ML.GENERATE_EMBEDDING returns the following columns:

• trial_id: anINT64 value that identifies the hyperparameter tuningtrial used by the function. This column is only returned if you ranhyperparameter tuning when creating the model.
• ml_generate_embedding_result: this is anARRAY<FLOAT> value thatcontains the weights of the feature, and also the intercept or bias termfor the feature. The intercept value is the last value in the array.The number of array dimensions is equal to the matrix factorizationmodel'sNUM_FACTORS optionvalue.
• processed_input: aSTRING value that contains the name of the user oritem column. The value of this column matches the name of theuser or item column provided in thequery_statement clausethat was used when the matrix factorization model was trained.
• feature: aSTRING value that contains the names of the specificusers or items used during training.

text-embedding

This example shows how to generate an embedding of a single piece ofsample text by using a remote model that references atext-embedding model.

Create the remote model:

CREATEORREPLACEMODEL`mydataset.text_embedding`REMOTEWITHCONNECTIONDEFAULTOPTIONS(ENDPOINT='text-embedding-005')

Generate the embedding:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.text_embedding`,(SELECT"Example text to embed"AScontent),STRUCT(TRUEASflatten_json_output));

multimodalembedding

This example shows how to generate embeddings from visual content by using aremote model that references amultimodalembedding model.

Create the remote model:

CREATEORREPLACEMODEL`mydataset.multimodalembedding`REMOTEWITHCONNECTIONDEFAULTOPTIONS(ENDPOINT='multimodalembedding@001')

Use anObjectRefRuntime value

Generate embeddings from visual content in anObjectRef columnin a standard table:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.multimodalembedding`,(SELECTOBJ.GET_ACCESS_URL(art_image,'r')ascontentFROM`mydataset.art`));

Use an object table

Generate embeddings from visual content in an object table:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.multimodalembedding`,TABLE`mydataset.my_object_table`);

PCA

This example shows how to generate embeddings that represent the principalcomponents of a PCA model.

Create the PCA model:

CREATEORREPLACEMODEL`mydataset.pca_nyc_trees`OPTIONS(MODEL_TYPE='PCA',PCA_EXPLAINED_VARIANCE_RATIO=0.9)AS(SELECTtree_id,block_id,tree_dbh,stump_diam,curb_loc,status,health,spc_latinFROM`bigquery-public-data.new_york_trees.tree_census_2015`);

Generate embeddings that represent principal components:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.pca_nyc_trees`,(SELECTtree_id,block_id,tree_dbh,stump_diam,curb_loc,status,health,spc_latinFROM`bigquery-public-data.new_york_trees.tree_census_2015`));

Autoencoder

This example shows how to generate embeddings that represent the latent spacedimensions of an autoencoder model.

Create the autoencoder model:

CREATEORREPLACEMODEL`mydataset.my_autoencoder_model`OPTIONS(model_type='autoencoder',activation_fn='relu',batch_size=8,dropout=0.2,hidden_units=[32,16,4,16,32],learn_rate=0.001,l1_reg_activation=0.0001,max_iterations=10,optimizer='adam')ASSELECT*EXCEPT(Time,Class)FROM`bigquery-public-data.ml_datasets.ulb_fraud_detection`;

Generate embeddings that represent latent space dimensions:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.my_autoencoder_model`,TABLE`bigquery-public-data.ml_datasets.ulb_fraud_detection`);

Matrix factorization

This example shows how to generate embeddings that represent the underlyingweights that the matrix factorization model uses during prediction.

Create the matrix factorization model:

CREATEORREPLACEMODEL`mydataset.my_mf_model`OPTIONS(model_type='matrix_factorization',user_col='user_id',item_col='item_id',l2_reg=9.83,num_factors=34)ASSELECTuser_id,item_id,AVG(rating)asratingFROMmovielens.movielens_1mGROUPBYuser_id,item_id;

Generate embeddings that represent model weights and intercepts:

SELECT*FROMML.GENERATE_EMBEDDING(MODEL`mydataset.my_mf_model`)