Vertex AI Search can provide search snippets, extractive answers, andextractive segments with each search response to enhance your results.

• Snippets: A snippet is a brief extract of text from thesearch result document that provides a preview of a search result's content.It includes hit highlighting that you can render in your UI. Snippets aretypically displayed under each search result to help end users assess therelevance and usefulness of that result. Snippets are available for datastores with unstructured data and website data (both basic website search and advanced website indexing).

• Extractive answers: An extractive answeris verbatim text that is returned with each search result. It is extracteddirectly from the original document. Extractive answers are typicallydisplayed near the top of web pages to provide an end user with a brief answerthat is contextually relevant to their query. Extractive answers are availablefor data stores with unstructured data and with advanced website indexing.

• Extractive segments: An extractivesegment is verbatim text that is returned with each search result. Anextractive segment is usually more verbose than an extractive answer.Extractive segments can be displayed as an answer to a query, and can be usedto perform post-processing tasks and as input for large language models togenerate answers or new text. Extractive segments are availablefor data stores with unstructured data and with advanced website indexing.

Examples

The following examples help illustrate the differences between snippets,extractive answers, and extractive segments.

Query: "what is vertex ai search?"

• Snippet:

  To enable this, we are announcing our newVertex AI Search,the fastest way for developers to jumpstart the creation ofgen apps such as bots, ...

• Extractive answer:

  Vertex AI Search allows developers to quickly ship newexperiences including bots, chat interfaces, custom search engines, digitalassistants, and more. Developers have API access to Google's foundationmodels and can use out-of-the-box templates to jumpstart the creation of genapps in minutes or hours.

• Extractive segment:

  Businesses and governments also want to make customer, partner, and employeeinteractions more effective and helpful with this new AI technology. Toenable this, we are announcing our new Vertex AI Search.

  Vertex AI Search allows developers to quickly ship newexperiences including bots, chat interfaces, custom search engines, digitalassistants, and more. Developers have API access to Google's foundationmodels and can use out-of-the-box templates to jumpstart the creation of genapps in minutes or hours. With Vertex AI Search, developerswill also:

  • Combine organizational data and information retrieval techniques to provide relevant answers.
  • Search and respond with more than just text.
  • Combine natural conversations with structured flows.
  • Don't just inform — transact.

Before you begin

Depending on the type of app you have, complete the following prerequisites:

• Unstructured app:

  • Forsnippets there is no requirement.

  • Forextractive answers andextractive segments, turn onEnterprise edition features.

• Website app:

  • For snippets, turn on:

    • Enterprise edition features.

  • For extractive answers, turn on:

    • Enterprise edition features.

    • Advanced website indexing.

Snippets

Snippets are short pieces of extracted verbatim from each search resultdocument. They include hit highlighting in bold HTML tags for rendering previewsof search results in a UI. Typically, snippets are rendered as preview textunderneath a search result to help end users decide if clicking that searchresult will be useful.

Snippets are available for website and unstructured search.

1. Send a search request that includesContentSearchSpec.SnippetSpecand setsreturnSnippet to true.

   The following example ofSnippetSpec specifies that asnippet can be returned for each search result.

   "contentSearchSpec":{"snippetSpec":{"returnSnippet":true}}

   • returnSnippet: If set totrue, return a snippet.

2. Get snippets from the search response. Snippets are returned with each searchresult inderivedStructData.snippets.

   In this example of a document that was returned as one of the results in asearch response, a snippet with bold hit highlighting is included with theresult:

   {"id":"54321","document":{"name":"projects/123/locations/global/collections/default_collection/dataStores/example-datastore/branches/0/documents/54321","id":"54321","derivedStructData":{"link":"gs://cloud-samples-data/gen-app-builder/search/alphabet-investor-pdfs/2008_google_annual_report.pdf","snippets":[{"snippet":"Google Chrome. Google Chrome is an open-sourcebrowser that combines a minimal design with technologies to make the web faster, safer, and easier to navigate.","snippet_status":"SUCCESS"}]}}}

   • snippet: Contains a snippet generated for the document search result.Hit highlighting is included in bold HTML tags.
   • snippet_status: If a snippet is generated, this field is returnedasSUCCESS. If no snippet is generated, this field is returned asNO_SNIPPET_AVAILABLE.

Extractive answers

An extractive answer is a section of text derived verbatim from a document. Whena document is returned as a search result in a search response, arelevant extractive answer can be returned with that result.

An extractive answer can be text such as a paragraph, table, or bulleted listthat is extracted from the search result document. Extractive answers areshorter thanextractive segments.

Extractive answers can be used as an alternative to summarized responses incases where precise, verbatim answers are preferable to rephrased summaries.

Extractive answers are available for data stores with unstructured data and withadvanced website indexing.

Get extractive answers

To get extractive answers:

1. Send a search request that usesContentSearchSpec.extractiveContentSpecto specifymaxExtractiveAnswerCount.

   The following example ofextractiveContentSpec specifies that ananswer can be returned for each search result.

   "contentSearchSpec":{"extractiveContentSpec":{"maxExtractiveAnswerCount":1}}

   • maxExtractiveAnswerCount: The number of extractive answers to returnfor each search result. The default value is 0 and the maximum is 5.

2. Get extractive answers from the search response. Extractive answers arereturned with each search result inextractive_answers.

   In this example of a document that was returned as one of the results in asearch response, an extractive answer was included with the result:

   {"id":"54321","document":{"name":"projects/123/locations/global/collections/default_collection/dataStores/example-datastore/branches/0/documents/54321","id":"54321","derivedStructData":{"extractive_answers":[{"pageNumber":"2","content":"Google saw growth throughout the year both in our domestic business and internationally, both on Google owned sites and on the Google Network. Specifically, revenues from Google owned sites increased 101% on a year over year basis, from $792 million to $1.6 billion."}],"link":"gs://cloud-samples-data/gen-app-builder/search/alphabet-investor-pdfs/2004Q4_earnings_google.pdf"}}}

   • pageNumber: If page numbers can be extracted from the document, thisfield indicates where the answer was extracted from.
   • content: The content of the extractive answer.

Extractive segments

An extractive segment is a section of text that is extracted verbatim from asearch result document. Extractive segments are similar toextractiveanswers, but extractive segments usually more complete andverbose. Typically, extractive segments are used as input for your own LLMs togenerate answers or new text.

Extractive segments can be multiple paragraphs, including formatted text suchas tables and bulleted lists.

Extractive segments are available for data stores with unstructured dataand with advanced website indexing.

Extractive segment options

The following options are available for extractive segments:

• Number of segments: You can specify up to 10 extractive segments toreturn for each search result.

• Relevance scores: Relevance scores are based on the similarity of thequery to the extracted segment. You can specify that extractive segments bereturned with relevance scores. Scores range from -1.0 (less relevant) to 1.0(more relevant). Turning on relevance scores can increase latency.

  Note: The order in which results are returned is based on many factors.Although relevance scores are considered during ranking, they don't solelydetermine the ranking of results.

• Adjacent segments: You can setnumPreviousSegments andnumNextSegments to get up to 3 segments from immediately before andafter the relevant segment. Adjacent segments can add context and accuracy tothe relevant segment.

  Turning on adjacent segments can increase latency.

Get extractive segments

The following steps show how to get extractive segments for unstructured data.You can follow similar steps to get extractive segments for website data.

1. Send a search request that usesContentSearchSpec.extractiveContentSpecto specifymaxExtractiveSegmentCount.

   The following example ofextractiveContentSpec specifies that onesegment can be returned for each search result.

   "contentSearchSpec":{"extractiveContentSpec":{"maxExtractiveSegmentCount":1}}

   • maxExtractiveSegmentCount: The number of extractive segments to returnfor each search result. The default value is 0 and the maximum is 10.

   Additional options:

   • returnExtractiveSegmentScore: Set totrue to returna relevance score with each segment returned.
   • numPreviousSegments: The number of adjacent segmentsto return before the relevant segment. The default value is 0 and themaximum is 3. Using adjacent segments can increase latency.
   • numNextSegments: The number of adjacent segments toreturn after the relevant segment. The default value is 0 and themaximum is 3. Using adjacent segments can increase latency.

   For more information about these options, seeExtractive segmentoptions.

2. Get segments from the search response. Segments are returned with each searchresult inextractive_segments.

   In this example of a document that was returned as one of the results in asearch response, a segment was included with the result:

   {"id":"54321","document":{"name":"projects/123/locations/global/collections/default_collection/dataStores/example-datastore/branches/0/documents/54321","id":"54321","derivedStructData":{"extractive_segments":[{"pageNumber":"2","content":"Client\nGoogle Toolbar. Google Toolbar is a free application that adds a Google search box to web browsers (Internet\nExplorer and Firefox) and improves user web experience through features such as a pop-up blocker that blocks\npop-up advertising, an autofill feature that completes web forms with information saved on a user's computer, and\ncustomizable buttons that let users search their favorite web sites and stay updated on their favorite feeds.\n\nGoogle Chrome. Google Chrome is an open-source browser that combines a minimal design with\ntechnologies to make the web faster, safer, and easier to navigate.\nGoogle Pack. Google Pack is a free collection of safe, useful software programs from Google and other\ncompanies that improve the user experience online and on the desktop. It includes programs that help users\nbrowse the web faster, remove spyware and viruses.\n\nPicasa. Picasa is a free service that allows users to view, manage and share their photos. Picasa enables users\nto import, organize and edit their photos, and upload them to Picasa Web Albums where the photos can be shared\nwith others on the internet.\n\nGoogle Desktop. Google Desktop lets people perform a full-text search on the contents of their own\ncomputer, including email, files, instant messenger chats and web browser history. Users can view web pages they\nhave visited even when they are not online. Google Desktop also includes a customizable Sidebar that includes\nmodules for weather, stock tickers and news.\n\n5"}],"link":"gs://cloud-samples-data/gen-app-builder/search/alphabet-investor-pdfs/2004Q4_earnings_google.pdf"}}}

   • pageNumber: If page numbers can be extracted from the document, thisfield indicates where the answer was extracted from.
   • content: The content of the extractive segment.

What's next

• Preview search results