This quickstart demonstrates how to create and use the three types ofVision API Product Search resources: aproduct set which contains agroup ofproducts, andreference images associated with those products.

In this quickstart you will create a product set, products, and theirreference images in a single step by batch import.

After the product set has beenindexed, you can query the productset using Vision API Product Search.

This quickstart steps you through the process of:

• Using a CSV and bulk import to create aproduct set,products, andreference images.
• Making a request to the Vision API Product Search with an image stored in aCloud Storage bucket.

Before you begin

If you haven't done so already, set up your project as explained below.

Set up your project

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
Install the Google Cloud CLI.
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.
Toinitialize the gcloud CLI, run the following command:
gcloudinit
Create or select a Google Cloud project.
Roles required to select or create a project
• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.
Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.
• Create a Google Cloud project:

  gcloud projects createPROJECT_ID

  ReplacePROJECT_ID with a name for the Google Cloud project you are creating.

• Select the Google Cloud project that you created:

  gcloud config set projectPROJECT_ID

  ReplacePROJECT_ID with your Google Cloud project name.
Verify that billing is enabled for your Google Cloud project.
Enable the Vision API:
Roles required to enable APIs
To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.
gcloudservicesenablevision.googleapis.com
Grant roles to your user account. Run the following command once for each of the following IAM roles:roles/storage.objectViewer
gcloudprojectsadd-iam-policy-bindingPROJECT_ID--member="user:USER_IDENTIFIER"--role=ROLE
Replace the following:
• PROJECT_ID: Your project ID.
• USER_IDENTIFIER: The identifier for your user account. For example,myemail@example.com.
• ROLE: The IAM role that you grant to your user account.

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

Toinitialize the gcloud CLI, run the following command:

gcloudinit

Create or select a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

• Create a Google Cloud project:

  gcloud projects createPROJECT_ID

  ReplacePROJECT_ID with a name for the Google Cloud project you are creating.

• Select the Google Cloud project that you created:

  gcloud config set projectPROJECT_ID

  ReplacePROJECT_ID with your Google Cloud project name.

Verify that billing is enabled for your Google Cloud project.

Enable the Vision API:

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

gcloudservicesenablevision.googleapis.com

Grant roles to your user account. Run the following command once for each of the following IAM roles:roles/storage.objectViewer

gcloudprojectsadd-iam-policy-bindingPROJECT_ID--member="user:USER_IDENTIFIER"--role=ROLE

Replace the following:

• PROJECT_ID: Your project ID.
• USER_IDENTIFIER: The identifier for your user account. For example,myemail@example.com.
• ROLE: The IAM role that you grant to your user account.

Using a dataset

In this quickstart you use a dataset of ~100apparel-v2 product categoryentries. This publicly available dataset is located in a publicCloud Storage bucket at:

• gs://cloud-samples-data/vision/product_search/product_catalog.csv

The CSV format is as follows:

gs://cloud-ai-vision-data/product-search-tutorial/images/filename1.jpg,image0,product_set0,product_id0,apparel-v2,,"style=women,category=shoe",gs://cloud-ai-vision-data/product-search-tutorial/images/filename2.jpg,image1,product_set0,product_id1,apparel-v2,,"style=men,category=shoe",gs://cloud-ai-vision-data/product-search-tutorial/images/filename3.jpg,image2,product_set0,product_id2,apparel-v2,,"style=women,category=dress",

Use bulk import to create a product set, products, and reference images

Use the followingcurl command to create a new product set with products andreference images. This set is namedproduct_set0, a value declared inthe import CSV.

First create a request JSON file calledimport_request.json and save it inyour current working directory:

import_request.json

{  "inputConfig": {    "gcsSource": {      "csvFileUri": "gs://cloud-samples-data/vision/product_search/product_catalog.csv"    }  }}

After creating the request JSON file, send the request:

curl -X POST \    -H "Authorization: Bearer $(gcloud auth print-access-token)" \    -H "x-goog-user-project:PROJECT_ID" \    -H "Content-Type: application/json; charset=utf-8" \    -d @import_request.json \    https://vision.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/productSets:import

Replace the following:

• LOCATION_ID: The location that will run yourtutorial, for example,us-east1. Valid location identifiers are:us-west1,us-east1,europe-west1, andasia-east1.

A successful response contains a long-running operation object:

{  "name": "locations/LOCATION_ID/operations/0a0aec86192599fa"}

The response also contains a relative operation ID (for example,0a0aec86192599fa) that can be used to get the status of the operation.

Get import operation status

You can use theOPERATION_ID returned from the import operationto check the status of the bulk import operation:

curl -X GET \    -H "Authorization: Bearer $(gcloud auth print-access-token)" \    -H "x-goog-user-project:PROJECT_ID" \    -H "Content-Type: application/json" \    https://vision.googleapis.com/v1/locations/LOCATION_ID/operations/OPERATION_ID

A successful response looks like:

{"name":"locations/LOCATION_ID/operations/0a0aec86192599fb","metadata":{"@type":"type.googleapis.com/google.cloud.vision.v1.BatchOperationMetadata","state":"SUCCESSFUL","submitTime":"2018-11-30T03:11:04.808114024Z","endTime":"2018-11-30T03:11:38.624444324Z"},"done":true,"response":{"@type":"type.googleapis.com/google.cloud.vision.v1.ImportProductSetsResponse","referenceImages":[{"name":"projects/PROJECT_ID/locations/LOCATION_ID/products/product_id0/referenceImages/image0","uri":"gs://cloud-ai-vision-data/product-search-tutorial/images/46a0cbcf70ba11e89399d20059124800.jpg"},{"name":"projects/PROJECT_ID/locations/LOCATION_ID/products/product_id1/referenceImages/image1","uri":"gs://cloud-ai-vision-data/product-search-tutorial/images/46a1aea370ba11e888d4d20059124800.jpg"},...{"name":"projects/PROJECT_ID/locations/LOCATION_ID/products/product_id93/referenceImages/image93","uri":"gs://cloud-ai-vision-data/product-search-tutorial/images/4697319970ba11e8a7bfd20059124800.jpg"},{"name":"projects/PROJECT_ID/locations/LOCATION_ID/products/product_id94/referenceImages/image94","uri":"gs://cloud-ai-vision-data/product-search-tutorial/images/4698596370ba11e8bf6ad20059124800.jpg"}],"statuses":[{},{},[...]{},{}]}}

Indexing

The Product Search index of products is updatedapproximately every 30 minutes. When images are added or deleted, the change won't bereflected in your Product Search responses until the index is next updated.

To make sure that indexing has completed successfully, check theindexTime fieldof a product set.

List product sets and check indexing

You can list all your product sets and use theindexTime field to verify thatindexing has completed successfully:

curl -X GET \    -H "Authorization: Bearer $(gcloud auth print-access-token)" \    -H "x-goog-user-project:PROJECT_ID" \    -H "Content-Type: application/json" \    https://vision.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/productSets

A successful response lists all your product sets, including a product set ID(for example,product_set0) as well as theindexTime field indicatingwhen indexing completed:

{  "productSets": [    {      "name": "projects/PROJECT_ID/locations/LOCATION_ID/productSets/product_set0",      "displayName": " ",      "indexTime": "2019-11-30T18:33:40.093508652Z",      "indexError": {}    }  ]}

List products

You can use thePRODUCT_SET_ID returned from the list of product setsto list all products in your product set:

curl -X GET \    -H "Authorization: Bearer $(gcloud auth print-access-token)" \    -H "x-goog-user-project:PROJECT_ID" \    -H "Content-Type: application/json" \    https://vision.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/productSets/PRODUCT_SET_ID/products?pageSize=15

A successful response lists product details.

In this request you use the optional query parameterpageSize to set the result list to 15 products. ThenextPageToken in the response also indicates there are moreproducts to list. You can use the token listed to retrieve further results. Formore information on using apageToken, seeGetting and listing resources.

{  "products": [    {      "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id0",      "displayName": " ",      "productCategory": "apparel",      "productLabels": [        {          "key": "style",          "value": "women"        },        {          "key": "category",          "value": "shoe"        }      ]    },    {      "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id1",      "displayName": " ",      "productCategory": "apparel",      "productLabels": [        {          "key": "style",          "value": "men"        },        {          "key": "category",          "value": "shoe"        }      ]    },    ...    {      "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id21",      "displayName": " ",      "productCategory": "apparel",      "productLabels": [        {          "key": "style",          "value": "women"        },        {          "key": "category",          "value": "dress"        }      ]    }  ],  "nextPageToken": "1LqhSgZfM_uWKOxvog"}

Search for matching products with Vision API Product Search

After indexing is complete, you can search for products that match a sampleimage. In this quickstart you use an image stored in a GoogleCloud Storage bucket such as the following image.

Search using a remote image

Use the following request to search using the image stored in a publicCloud Storage bucket.

First create a request JSON file calledsearch_request.json and save it inyour current working directory. Change the following values in the requestJSON to match your project's information:

• PROJECT_ID
• LOCATION_ID
• PRODUCT_SET_ID

search_request.json

{  "requests": [    {      "image": {        "source": {          "gcsImageUri": "gs://cloud-ai-vision-data/product-search-tutorial/images/468f782e70ba11e8941fd20059124800.jpg"        }      },      "features": [        {          "type": "PRODUCT_SEARCH"        }      ],      "imageContext": {        "productSearchParams": {          "productSet": "projects/PROJECT_ID/locations/LOCATION_ID/productSets/PRODUCT_SET_ID",          "productCategories": [            "apparel-v2"          ],          "filter": "style=womens OR style=women"        }      }    }  ]}

After creating the request JSON file, send the request:

curl -X POST \    -H "Authorization: Bearer $(gcloud auth print-access-token)" \    -H "x-goog-user-project:PROJECT_ID" \    -H "Content-Type: application/json; charset=utf-8" \    -d @search_request.json \    https://vision.googleapis.com/v1/images:annotate

A successful request returns a list of matching products, indicated by theirproduct ID. These results are further broken down by individual productsidentified by bounding boxes if there are multiple products in a single image.

For an example of single-product detection and multi-detection of products in animage, seeUnderstanding search responses & multi-detection.

A fieldscore is returned as well. This field indicates theconfidence at which the service feels the product matches the supplied image,on a scale of 0 (no confidence) to 1 (full confidence).

TheindexTime field shows which version of the index is being searched. Imagechanges made after this time are not reflected in the results.

{  "responses": [    {      "productSearchResults": {        "indexTime": "2020-01-07T18:33:45.300851144Z",        "results": [          {            "product": {              "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id16",              "displayName": " ",              "productCategory": "apparel-v2",              "productLabels": [                {                  "key": "style",                  "value": "women"                },                {                  "key": "category",                  "value": "dress"                }              ]            },            "score": 1,            "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id16/referenceImages/image16"          },          {            "product": {              "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id29",              "displayName": " ",              "productCategory": "apparel-v2",              "productLabels": [                {                  "key": "style",                  "value": "women"                },                {                  "key": "category",                  "value": "dress"                }              ]            },            "score": 0.27578148,            "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id29/referenceImages/image29"          },          ...          {            "product": {              "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id22",              "displayName": " ",              "productCategory": "apparel-v2",              "productLabels": [                {                  "key": "style",                  "value": "women"                },                {                  "key": "category",                  "value": "dress"                },                {                  "key": "kids",                  "value": "true"                }              ]            },            "score": 0.20356013,            "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id22/referenceImages/image22"          }        ],        "productGroupedResults": [          {            "boundingPoly": {              "normalizedVertices": [                {                  "x": 0.25542393,                  "y": 0.15517132                },                {                  "x": 0.8196448,                  "y": 0.15517132                },                {                  "x": 0.8196448,                  "y": 0.92001766                },                {                  "x": 0.25542393,                  "y": 0.92001766                }              ]            },            "results": [              {                "product": {                  "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id16",                  "displayName": " ",                  "productCategory": "apparel-v2",                  "productLabels": [                    {                      "key": "style",                      "value": "women"                    },                    {                      "key": "category",                      "value": "dress"                    }                  ]                },                "score": 1,                "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id16/referenceImages/image16"              },              {                "product": {                  "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id57",                  "displayName": " ",                  "productCategory": "apparel-v2",                  "productLabels": [                    {                      "key": "style",                      "value": "women"                    },                    {                      "key": "category",                      "value": "dress"                    }                  ]                },                "score": 0.27369013,                "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id57/referenceImages/image57"              },              ...              {                "product": {                  "name": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id18",                  "displayName": " ",                  "productCategory": "apparel-v2",                  "productLabels": [                    {                      "key": "style",                      "value": "women"                    },                    {                      "key": "category",                      "value": "dress"                    }                  ]                },                "score": 0.21221256,                "image": "projects/PROJECT_ID/locations/LOCATION_ID/products/product_id18/referenceImages/image18"              }            ],            "objectAnnotations": [              {                "mid": "/m/01d40f",                "name": "Dress",                "score": 0.968603              }            ]          }        ]      }    }  ]}

Congratulations! You've made your firstimages.annotate request to theVision API Product Search service.

Clean up

1. Optional: Revoke credentials from the gcloud CLI.

   gcloudauthrevoke

2. Caution: Deleting a project has the following effects:
   • Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
   • Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as anappspot.com URL, delete selected resources inside the project instead of deleting the whole project.

   If you plan to explore multiple architectures, tutorials, or quickstarts, reusing projects can help you avoid exceeding project quota limits.

   Delete a Google Cloud project:

   gcloud projects deletePROJECT_ID

What's next

• Get started with the Vision API Product Search in your language of choice by usingaVision API Product Search Client Library.
• Work through theHow-To Guides.
• Work through theTutorial.