This page introduces and lists the capabilities of Vertex AI Searchfor custom apps. The page also provides links to the availablefeatures, tutorials, and checklists, to get you started withVertex AI Search for custom apps.

What is Vertex AI Search for custom apps?

Vertex AI Search for custom apps is a powerful,Google-quality search and content discovery engine that you can integrate intoyour applications that contain website data and other structured or unstructureddata. The search capability is beyond basic keyword matching and uses AI todeliver highly relevant results, provide personalized browse and searchexperiences, and generate AI answers grounded in your data.

You can use custom search app for vertical-agnostic datathat's on public websites or is in structured or unstructured format.Additionally, Vertex AI Search offers other vertical-specific search andrecommendations apps.

• For information about media data, seeIntroduction to media search and recommendations.
• For information about commerce and retail data, seeVertex AI Search for commerce.
• For information about healthcare data, seeHealthcare search checklist.

Key capabilities

The key capabilities of Vertex AI Search are as follows:

• High-quality search: Leverages Google's search expertise to understanduser intent, even with complex queries and natural language queries. Itcombines keyword and semantic search to serve the best results.
• Personalized browse: Provides personalized results without aspecific search query and personalized feed based on a user's context andnavigation patterns. It is ideal for discovery experiences to viewpersonalized category pages and home feeds.
• Data sources: Works with the following variety of data sources:
  • Website: Index your public websites and use advanced features, such asindex enrichment with the structured data in your websites.
  • Structured Data: Search over data organized in a defined format, such asdatabases, JSON files in Cloud Storage, or BigQuerytables—for example, hotel catalogs, real estate listings, andrestaurant directories.
  • Unstructured Data: Search over documents like PDFs, HTML files, and TXTfiles or image files like JPEG and PNG files that are stored inCloud Storage or BigQuery.
  • Blended Search: Search over multiple data stores thatblend data fromthe data sources mentioned above. For example, you can create a search appand connect it to a website data store and a document data store. Thislets your users search over all of your content at once.
• Grounded AI answer generation: Generates AI answers grounded in your data,with citations to the source documents. You can also ask follow up questionsand related queries.
• Personalization: Improves results and ranking over time by learning fromuser interactions captured in user events, such as clicks and conversions.
• Customization: Offers several ways to tune and configure the search andbrowse experience fit for your business needs.

Overview

The following diagram shows the key components of custom search andhow they work together:

The components of Vertex AI Search for custom search can beexplained as follows:

• Data store: Your content from different data sources is stored in aVertex AI Search data store. The source data can be public websitedata or structured and unstructured data.
• Data processing and indexing: Vertex AI Search understands andindexes your data, creating a searchable and retrievable representation. Thisincludes the following:
  • Keyword extraction: Identifies and generates important terms necessaryto retrieve the correct information.
  • Semantic understanding using embeddings: Creates vector embeddings tocapture the meaning of the content.
  • Metadata processing: Processes your documents using the document'sstructured data or metadata. For example, location in a hotel catalog,modification or creation dates in a web page's metadata.
  • Advanced document parsing: Understands document structure and annotatesadvanced information, such as tables, images, and graphs, using OCR orlayout parsing.
• Search app: At the heart of the custom search is a search app,which connects to one or more data stores that bring data from differentsources. For blended search, the data is ingested throughconnectors. Youconfigure the search and browse behavior at the app level.
• User query: The input from a user intended to retrieve information fromyour app, which can be of two types:
  • Search query: The user enters a targeted search query using text or images.Textual search is powered by autocomplete.
  • Navigational query or browse: An exploratory search to deliver personalizedrelevant content with no specific query. It is powered by the user's pastactivity and other signals, such as current category page and location.
• Retrieval and ranking: There are several sub-components to retrieval andranking of results:
  • Query understanding for search: Vertex AI Search analyzes asearch query using the following:
    • Natural language processing: To understand the intent.
    • Filters with natural language understanding: Translates locationsfrom natural languagequeries into geo-coordinates and the conditions in natural languagequeries into filters.
    • Knowledge graph: To disambiguate terms and expand the search.
    • Optional features: Includes spelling correction, synonyms, and queryrephrasing.
  • Retrieval: Vertex AI Search finds the most relevant documentsor chunks based on the following methods:
    • Keyword matching for search: Conventional search based on terms.
    • Semantic search: Using embeddings to find conceptually similarcontent.
    • Filtering: Applying any filters you've configured—for example, date,category, or relevance score.
  • Ranking: Vertex AI Search ranks the results based on thefollowing factors:
    • Relevance: A combination of keyword and semantic matching duringsearch.
    • Web signals for website search: Factors like page quality andpopularity.
    • Boosting and burying: Your custom rules to promote or demote certainresults.
    • Personalization: Learning from user interactions. This is optionalbut highly recommended.
    • Ordering: Applying ordering instructions, for example, by date.
• Results and answer generation:
  • Search results: A ranked list of relevant documents or chunks isreturned with optional features, such as snippets, extractive answers, andextractive segments. The results that are served can be configured with thehelp of serving controls. You can also tune the search results.
  • Answer generation: A concise, synthesized answer is generated based onthe top and relevant results, with citations. This uses advanced LLMfeatures.
  • Personalized browse: A personalized set of documents with the highestpredicted likelihood of engagement or conversion is returned. Thisprediction uses an advanced model that learns from user interactions.
• User events: A tracker for user interactions, such as clicks and views,that helps Vertex AI Search learn and improve search andpersonalization. User events aid in optimizing your business KPIs includingengagement, conversion, and revenue.

Key features and configurations

The following features and configurations are available for yourcustom search apps. At each stage you can customize these settings toserve the best results to your users.

To elaborate, here are the available configurations:

• Data preparation:
  • Schema for structured data: Define the structure of your data with fieldnames and types. You can provide your own schema or letVertex AI Search auto-detect it. For more information, seeProvide or auto-detect a schema.
  • Metadata: Add metadata to your structured or unstructured documents andweb pages to improve search and enable filtering. For more information, seethe following:
    • Use structured data for advanced website indexing.
    • Unstructured data with metadata.
    • Structured data.
  • Chunking: Break documents into smaller pieces for better relevance andLLM processing. For more information, seeParse and chunk documents.
  • Parsing: Use one of the following options to extract text, images, and other annotations:
    • Digital parser toextract machine readable text.
    • OCR parser for PDFs toextract text from scanned PDFs or images.
    • Layout Parser todetect document structure and annotate images andgraphs for better chunking and document understanding, which is requiredfor complex documents and RAG applications.
  • Custom embeddings: Upload your own prepared vector embeddings.For more information, seeUse custom embeddings.
• Data ingestion: Vertex AI Searchoffers different types of ingestion for data from different sources, such as:
  • Crawling for website data
  • Ingestingstructured andunstructured data fromCloud Storage and BigQuery orthrough the REST API
• Search and browse configuration:
  • Field settings: Control how fields are configured for search and answergeneration, such as searchable, retrievable, or indexable. For moreinformation, seeConfigure field settings.
  • Autocomplete: Provide query suggestions as users type. For moreinformation, seeConfigure autocomplete.
  • Serving Controls: Create rules to modify search behavior. For moreinformation, seeConfigure serving controls for search.
    • Boost/bury: Promote or demote specific results based on filters.
    • Filter: Remove results based on filters.
    • Synonyms: Treat certain terms as equivalent.
    • Redirect: Send users to a specific URL for certain queries.
    • Promote controls: Promote results within and beyond yourVertex AI Search data store.
  • Search tuning (Preview): Further train the model based on query and textextract pairs. For more information, seeImprove search results with search tuning.
  • Website specific configurations:
    • Advanced indexing: For more details, seeTurn on advanced website indexing.
    • Refresh web pages: You can refresh documents in your websiteregularly usingautomatic or manual refresh or optforsitemap-based refresh.
• Retrieval: Configure the following retrieval methods:
  • Filter with natural-language understanding.
  • Filter searches by document-level relevance.
  • Filter using metadata.
• Ranking:Boost or bury the retrievedresults,order web search results, ororder results from structured data stores.
• Search results: Use Vertex AI Search to do the following:
  • Get answers and follow-ups.
  • Get search summaries.
  • Get snippets and extractive content.
  • Generate personalized browse experience that requires user events
• User events: Record user interactions, such as clicks and views to improvesearch and personalization. For more information, seeAbout user events for custom apps.

What's next

• Get started with custom search.
• Follow thecustom search checklistand thewebsite search checklist toset up your own custom search app with all configurations.