Data Objects

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

In Vector Search 2.0, Collections store data as individual JSON objectscalled Data Objects. This page provides information on how to createData Objects or import them from Cloud Storage buckets, and how to update anddelete them.

Creating a Data Object

The following example demonstrates adding a Data Object to a Collection namedmovies.

curl -X POST \'https://vectorsearch.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/movies/dataObjects?dataObjectId=the-shawshank-redemption' \  -H 'Bearer $(gcloud auth print-access-token)' \  -H 'Content-Type: application/json' \  -d '{ \    "data": { \      "title": "The Shawshank Redemption", \      "genre": "Drama", \      "year": 1994, \      "director": "Frank Darabont" \    }, \    "vectors": { \      "plot_embedding": { \        "dense": { \          "values": [ \            0.4752082440607731, \            0.09026746166854707, \            0.8752307753619009 \          ] \        } \      }, \      "genre_embedding": { \        "dense": { \          "values": [ \            0.38638010860523064, \            0.739343471733759, \            0.16189056837017107, \            0.5271366865924485 \          ] \        } \      }, \      "soundtrack_embedding": { \        "dense": { \          "values": [ \            0.5920451749052875, \            0.08301644173787519, \            0.1264733498775969, \            0.6196429624200321, \            0.4925828581737443 \          ] \        } \      }, \      "sparse_embedding": { \        "sparse": { \          "values": [ \            1, \            6, \            3, \            2, \            8, \            5, \            2 \          ], \          "indices": [ \            4065, \            13326, \            17377, \            25918, \            28105, \            32683, \            42998 \          ] \        } \      } \    } \  }'

Embedding fields that have an auto-embedding specified in the Collection Schemaare automatically populated. You can also bring your own embeddings (BYOE)to set vector field values that are not automatically populated.

Note: Creating Data Objects is an atomic operation. All Data Objects are created,or the entire request fails if there is an error.

Importing Data Objects

The following example demonstrates how to import a Data Object from Cloud Storageinto a Collection namedmovies.

curl -X POST \"https://vectorsearch.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/movies:importDataObjects" \  -H "Authorization: Bearer $(gcloud auth print-access-token)" \  -H "Content-Type: application/json" \  -d '{ \    "gcs_import": { \      "contents_uri": "gs://your-bucket/path/to/your-data.jsonl", \      "error_uri": "gs://your-bucket/path/to/import-errors/" \    } \  }'

For very large datasets, you can bulk import data from a Cloud Storagebucket. The file format for Vector Search 2.0 is JSONL, where each line is aJSON object having three top level properties:data_object_id,data, andvectors.

The following provides an example of the JSONL with the required properties.

{"data_object_id":"movie-789","data":{"title":"The Shawshank Redemption","plot":"...","year":1994,"avg_rating":8.5,"movie_runtime_info":{"hours":2,"minutes":5},},"vectors":{"title_embedding":[-0.23,0.88,0.11,...],"sparse_embedding":{"values":[0.01,-0.93,0.27,...],"indices":[23,83,131,...]}}}
Note: "plot_embedding" is automatically populated and shouldn't be specified inthe JSONL file.

Get a Data Object

The following example demonstrates how to get a Data Object namedthe-shawshank-redemption from themovies collection.

curl -X GET \'https://vectorsearch.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/movies/dataObjects/the-shawshank-redemption'  \    -H 'Bearer $(gcloud auth print-access-token)' \    -H 'Content-Type: application/json'

Updating a Data Object

The following example demonstrates how to update thetitle field in the DataObject with the namethe-shawshank-redemption in themovies Collection.

curl -X PATCH \'https://vectorsearch.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/movies/dataObjects/the-shawshank-redemption' \  -H 'Bearer $(gcloud auth print-access-token)' \  -H 'Content-Type: application/json' \  -d '{ \    "data": { \      "title": "The Shawshank Redemption (updated)" \    }, \    "vectors": { \      "plot_embedding": { \        "dense": { \          "values": [ \            1.0, \            1.0, \            1.0 \          ] \        } \      } \    } \  }'

Delete Data Objects

You can delete individual Data Objects by name or batch delete Data Objectsthat match a filter.

The following shows how to delete the Data Objectthe-shawshank-redemptionfrom themovies Collection.

curl -X DELETE \'https://vectorsearch.googleapis.com/v1beta/projects/PROJECT_ID/locations/LOCATION/collections/movies/dataObjects/the-shawshank-redemption' \  -H 'Bearer $(gcloud auth print-access-token)' \  -H 'Content-Type: application/json'

What's next?

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.