Introduction to AI in BigQuery

Note: This feature may not be available when using reservations that are created with certain BigQuery editions. For more information about which features are enabled in each edition, seeIntroduction to BigQuery editions.

BigQuery offers various AI capabilities that let you do the following:

• Do predictive machine learning (ML).
• Run inference against large language models (LLMs) such as Gemini.
• Build applications using embeddings and vector search.
• Use built-in agents to assist with coding.
• Create data pipelines.
• Access BigQuery functionality with agent tools.

Machine learning

With BigQuery ML, you can train, evaluate, and run inference onmodels for tasks such as time series forecasting, anomaly detection,classification, regression, clustering, dimensionality reduction, andrecommendations.

You can work with BigQuery ML capabilities through theGoogle Cloud console, the bq command-line tool, the REST API, or inColab Enterprise notebooks.Because BigQuery ML lets SQL practitioners useexisting SQL tools and skills to build and evaluate models, it democratizes MLand speeds up model development by bringing ML to the data instead of requiringdata movement. You can use BigQuery ML to help you with thefollowing types of ML tasks:

• Create and run ML modelsby using GoogleSQL queries.
• Create Colab Enterprise notebooksto perform ML workflows. Notebookslet you use SQL and Python interchangeably, and use any AI or ML Pythonlibraries for your development.
• Understand the results of your predictive ML models withexplainable AI.
• Use theTimesFM,ARIMA_PLUS, andARIMA_PLUS_XREG models to performforecasting andanomaly detectionon time series data.
• Generate insights about changes to key metrics in your multi-dimensional datawithcontribution analysis.

To learn more, see theIntroduction to ML in BigQuery.

AI functions

BigQuery offers various SQL functions that you can use forAI tasks such as text generation, text or unstructured data analysis, andtranslation. These functions access Gemini andpartner LLM models available from Vertex AI, Cloud AI APIs, orbuilt-in BigQuery models to perform these tasks.

There are several categories of AI functions:

• Generative AI functions. These functions help you perform tasks such ascontent generation, analysis, summarization, structured data extraction,classification, embedding generation, and data enrichment. There are twotypes of generative AI functions:

  • General-purpose AI functionsgive you full control and transparency on the choice of model, prompt,and parameters to use.
  • Managed AI functionsoffer a streamlined syntax for routine tasks such as filtering, rating,and classification. BigQuery can choose a model for youto optimize for cost and quality.

• Task-specific functions. These functions help you use Cloud AI APIs fortasks such as the following:

  • Natural language processing
  • Machine translation
  • Document processing
  • Audio transcription
  • Computer vision

For more information, seeTask-specific solutions overview.

Search

BigQuery offers a variety of search functions and features tohelp you efficiently find specific data or discover similarities between dataincluding multimodal data.

• Text search. You can use theSEARCH functionto perform tokenized search on unstructured text or semi-structuredJSONdata. You can improve search performance by creating asearch index, which letsBigQuery optimize queries that use theSEARCH function, aswell as other functions and operators.For more information, seeSearch indexed data.

• Embedding generation. Embeddings are high-dimensional numerical vectorsthat represent entities such as text or images, and are often generated by MLmodels. You cangenerate multimodal embeddings by usingmodels provided by or hosted on Vertex AI, or by using modelsimported and run in BigQuery.

  You can also have BigQuery automatically maintain a column ofembeddings by enablingautonomous embedding generation(Preview).

• Vector search. You can use theVECTOR_SEARCH functionto search embeddings to find semantically similar items. You can use theAI.SEARCH function(Preview) to searchon tables that have autonomous embedding generation enabled. You can improvevector search performance by creating avector index, which uses Approximate NearestNeighbor search techniques to provide faster, more approximate results.

  Common use cases for vector search include semantic search, recommendation,and retrieval-augmented generation (RAG). For more information, seeIntroduction to vector search.

Assistive AI features

AI-powered assistance features in BigQuery, collectivelyreferred to asGemini in BigQuery,help you discover, prepare, query, and visualize your data.

• Data insights. Generatenatural language questions about your data, along with the SQLqueries to answer those questions.
• Data preparation. Generatecontext aware recommendations to clean, transform, and enrich your data.
• SQL code assist.Generate, complete, and explain SQL queries.
• Python code assist.Generate, complete, and explain Python code, including PySpark andBigQuery DataFrames.
• Data canvas. Query your data using naturallanguage, visualize results with charts, and ask follow-up questions.
• SQL translator. CreateGemini-enhanced SQL translation rules to help you migratequeries written in a different dialect to GoogleSQL.

Agents

Agents are software tools that can use AI to complete tasks on your behalf.You can use built-in agents or create your own agents to help you process,manage, analyze, and visualize your data:

• Use theData Science Agent toautomate exploratory data analysis, data processing, ML tasks,and visualization insights within a Colab Enterprise notebook.

• Use theData Engineering Agentto build, modify, and manage data pipelines to load and process data inBigQuery. You can use natural language prompts to generate datapipelines from various data sources or adapt existing data pipelines to suityour data engineering needs.

• Use theConversational Analytics Agentto chat with your data using conversational language. This agent consists ofone or more data sources and a set of use case-specific instructions forprocessing that data. Conversation analytics supports the use ofsomeBigQuery ML functions.

• Use theGemini CLI tointeract with BigQuery data in your terminal by usingnatural language prompts.

• Build using theOpen source MCP toolboxorADK tools forquick, iterative agent development.

What's next

• For more information about ML, seeIntroduction to ML in BigQuery.
• For more information about generative AI functions in SQL, seeGenerative AI overview.
• For more information about searching your data, seeSearch indexed data andIntroduction to vector search.
• For more information about assistive AI features, seeGemini in BigQuery.
• For more information about using agents with BigQuery, see:
  • Use the BigQuery MCP Server.
  • Connect LLMs to BigQuery with MCPusing IDEs such as Antigravity.
  • Analyze data with the Gemini CLI.