Run a query

This document shows you how to run a query in BigQuery and understandhow much data the query will process before execution by performing adry run.

Types of queries

You canquery BigQuery databy using one of the following query job types:

  • Interactive query jobs. Bydefault, BigQuery runs queries as interactive query jobs, whichare intended to start executing as quickly as possible.

  • Batch query jobs. Batch querieshave lower priority than interactive queries. When a project or reservationis using all of its available compute resources, batch queries are morelikely to be queued and remain in the queue. After a batch query startsrunning, the batch query runs the same as an interactive query. For moreinformation, seequery queues.

  • Continuous query jobs(Preview).With these jobs, the query runs continuously, letting you analyzeincoming data in BigQuery in real time and then write theresults to a BigQuery table, or export the results toBigtable or Pub/Sub. You can use this capability toperform time sensitive tasks, such as creating and immediately acting oninsights, applying real time machine learning (ML) inference, andbuilding event-driven data pipelines.

You can run query jobs by using the following methods:

BigQuery saves query results to either atemporary table (default) or permanent table.When you specify a permanent table as the destination table for the results, youcan choose whether to append or overwrite an existing table, or create a newtable with a unique name.

Note: If you query data from a project to data stored in a different project,the querying project is billed for the query job while the project storing thedata is billed for the amount of data stored in BigQuery.

Required roles

To get the permissions that you need to run a query job, ask your administrator to grant you the following IAM roles:

For more information about granting roles, seeManage access to projects, folders, and organizations.

These predefined roles contain the permissions required to run a query job. To see the exact permissions that are required, expand theRequired permissions section:

Required permissions

The following permissions are required to run a query job:

  • bigquery.jobs.create on the project from which the query is being run, regardless of where the data is stored.
  • bigquery.tables.getData on all tables and views that your query references. To query views, you also need this permission on all underlying tables and views. If you're usingauthorized views orauthorized datasets, you don't need access to the underlying source data.

You might also be able to get these permissions withcustom roles or otherpredefined roles.

Troubleshooting

Access Denied: Project [project_id]: User does not have bigquery.jobs.createpermission in project [project_id].

This error occurs when a principal lacks permission to create a queryjobs in the project.

Resolution: An administrator must grant you thebigquery.jobs.createpermission on the project you are querying. This permission is required inaddition to any permission required to access the queried data.

For more information about BigQuery permissions, seeAccess control with IAM.

Run an interactive query

To run an interactive query, select one of the following options:

Console

  1. Go to theBigQuery page.

    Go to BigQuery

  2. ClickSQL query.

  3. In the query editor, enter a valid GoogleSQL query.

    For example, query theBigQuery public datasetusa_namesto determine the most common names in the United States between theyears 1910 and 2013:

    SELECTname,gender,SUM(number)AStotalFROM`bigquery-public-data.usa_names.usa_1910_2013`GROUPBYname,genderORDERBYtotalDESCLIMIT10;
  4. Optional: To select additionalquery settings, clickMore, and then clickQuery settings.

  5. ClickRun.

    If you don't specify a destination table, the query job writes theoutput to a temporary (cache) table.

    You can now explore the query results in theResults tab of theQuery results pane.

  6. Optional: To sort the query results by column, clickOpen sort menunext to the column name and select a sort order. If the estimated bytesprocessed for the sort is more than zero, then the number of bytes isdisplayed at the top of the menu.

  7. Optional: To see visualization of your query results, go to theChart tab. You can zoom in or zoom out of the chart, download thechart as a PNG file, or toggle the legend visibility.

    In theChart configuration pane, you can change the chart type(line, bar, or scatter) and configure the measures and dimensions of thechart. Fields in this pane are prefilled with the initial configurationinferred from the destination table schema of the query. Theconfiguration is preserved between following query runs in the samequery editor. Dimensions supportINT64,FLOAT64,NUMERIC,BIGNUMERIC,TIMESTAMP,DATE,DATETIME,TIME, andSTRING data types, while measures supportINT64,FLOAT64,NUMERIC, andBIGNUMERIC data types.

  8. Optional: In theJSON tab, you can explore the query results in theJSON format, where the key is the column name and the value is theresult for that column.

bq

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, aCloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Use thebq query command.In the following example, the--use_legacy_sql=false flag lets you useGoogleSQL syntax.

    bqquery\    --use_legacy_sql=false\    'QUERY'

    ReplaceQUERY with a valid GoogleSQL query. Forexample, query theBigQuery public datasetusa_namesto determine the most common names in the United States between the years1910 and 2013:

    bqquery\--use_legacy_sql=false\'SELECT      name, gender,      SUM(number) AS total    FROM      `bigquery-public-data.usa_names.usa_1910_2013`    GROUP BY      name, gender    ORDER BY      total DESC    LIMIT      10;'

    The query job writes the output to a temporary (cache) table.

    Optionally, you can specify the destination table andlocationfor the query results. To write the results to an existing table, includethe appropriate flag to append (--append_table=true) or overwrite(--replace=true) the table.

    bqquery\    --location=LOCATION\    --destination_table=TABLE\    --use_legacy_sql=false\    'QUERY'

    Replace the following:

    • LOCATION: the region or multi-region for the destinationtable—for example,US

      In this example, theusa_names dataset is stored in the USmulti-region location. If you specify a destination table for thisquery, the dataset that contains the destination table must also be inthe US multi-region. You cannot query a dataset in one location andwrite the results to a table in another location.

      You can set a default value for the location using the.bigqueryrc file.

    • TABLE: a name for the destination table—for example,myDataset.myTable

      If the destination table is a new table, then BigQuerycreates the table when you run your query. However, you must specifyan existing dataset.

      If the table isn't in your current project, then add theGoogle Cloud project ID using the formatPROJECT_ID:DATASET.TABLE—for example,myProject:myDataset.myTable. If--destination_table is unspecified,a query job is generated that writes the output to a temporary table.

API

To run a query using the API,insert a new joband populate thequery job configuration property. Optionally specify yourlocation in thelocation property in thejobReference section of thejob resource.

Poll for results by callinggetQueryResults.Poll untiljobComplete equalstrue. Check for errors and warnings in theerrors list.

C#

Before trying this sample, follow theC# setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryC# API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

usingGoogle.Cloud.BigQuery.V2;usingSystem;publicclassBigQueryQuery{publicvoidQuery(stringprojectId="your-project-id"){BigQueryClientclient=BigQueryClient.Create(projectId);stringquery=@"            SELECT name FROM `bigquery-public-data.usa_names.usa_1910_2013`            WHERE state = 'TX'            LIMIT 100";BigQueryJobjob=client.CreateQueryJob(sql:query,parameters:null,options:newQueryOptions{UseQueryCache=false});// Wait for the job to complete.job=job.PollUntilCompleted().ThrowOnAnyError();// Display the resultsforeach(BigQueryRowrowinclient.GetQueryResults(job.Reference)){Console.WriteLine($"{row["name"]}");}}}

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""cloud.google.com/go/bigquery""google.golang.org/api/iterator")// queryBasic demonstrates issuing a query and reading results.funcqueryBasic(wio.Writer,projectIDstring)error{// projectID := "my-project-id"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %v",err)}deferclient.Close()q:=client.Query("SELECT name FROM `bigquery-public-data.usa_names.usa_1910_2013` "+"WHERE state = \"TX\" "+"LIMIT 100")// Location must match that of the dataset(s) referenced in the query.q.Location="US"// Run the query and print results when the query job is completed.job,err:=q.Run(ctx)iferr!=nil{returnerr}status,err:=job.Wait(ctx)iferr!=nil{returnerr}iferr:=status.Err();err!=nil{returnerr}it,err:=job.Read(ctx)for{varrow[]bigquery.Valueerr:=it.Next(&row)iferr==iterator.Done{break}iferr!=nil{returnerr}fmt.Fprintln(w,row)}returnnil}

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.QueryJobConfiguration;importcom.google.cloud.bigquery.TableResult;publicclassSimpleQuery{publicstaticvoidrunSimpleQuery(){// TODO(developer): Replace this query before running the sample.Stringquery="SELECT corpus FROM `bigquery-public-data.samples.shakespeare` GROUP BY corpus;";simpleQuery(query);}publicstaticvoidsimpleQuery(Stringquery){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();// Create the query job.QueryJobConfigurationqueryConfig=QueryJobConfiguration.newBuilder(query).build();// Execute the query.TableResultresult=bigquery.query(queryConfig);// Print the results.result.iterateAll().forEach(rows->rows.forEach(row->System.out.println(row.getValue())));System.out.println("Query ran successfully");}catch(BigQueryException|InterruptedExceptione){System.out.println("Query did not run \n"+e.toString());}}}

To run a query with a proxy, seeConfiguring a proxy.

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client library using default credentialsconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctionquery(){// Queries the U.S. given names dataset for the state of Texas.constquery=`SELECT name    FROM \`bigquery-public-data.usa_names.usa_1910_2013\`    WHERE state = 'TX'    LIMIT 100`;// For all options, see https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/queryconstoptions={query:query,// Location must match that of the dataset(s) referenced in the query.location:'US',};// Run the query as a jobconst[job]=awaitbigquery.createQueryJob(options);console.log(`Job${job.id} started.`);// Wait for the query to finishconst[rows]=awaitjob.getQueryResults();// Print the resultsconsole.log('Rows:');rows.forEach(row=>console.log(row));}

PHP

Before trying this sample, follow thePHP setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPHP API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

use Google\Cloud\BigQuery\BigQueryClient;use Google\Cloud\Core\ExponentialBackoff;/** Uncomment and populate these variables in your code */// $projectId = 'The Google project ID';// $query = 'SELECT id, view_count FROM `bigquery-public-data.stackoverflow.posts_questions`';$bigQuery = new BigQueryClient([    'projectId' => $projectId,]);$jobConfig = $bigQuery->query($query);$job = $bigQuery->startQuery($jobConfig);$backoff = new ExponentialBackoff(10);$backoff->execute(function () use ($job) {    print('Waiting for job to complete' . PHP_EOL);    $job->reload();    if (!$job->isComplete()) {        throw new Exception('Job has not yet completed', 500);    }});$queryResults = $job->queryResults();$i = 0;foreach ($queryResults as $row) {    printf('--- Row %s ---' . PHP_EOL, ++$i);    foreach ($row as $column => $value) {        printf('%s: %s' . PHP_EOL, $column, json_encode($value));    }}printf('Found %s row(s)' . PHP_EOL, $i);

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()query="""    SELECT name, SUM(number) as total_people    FROM `bigquery-public-data.usa_names.usa_1910_2013`    WHERE state = 'TX'    GROUP BY name, state    ORDER BY total_people DESC    LIMIT 20"""rows=client.query_and_wait(query)# Make an API request.print("The query data:")forrowinrows:# Row values can be accessed by field name or index.print("name={}, count={}".format(row[0],row["total_people"]))

Ruby

Before trying this sample, follow theRuby setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryRuby API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

require"google/cloud/bigquery"defquerybigquery=Google::Cloud::Bigquery.newsql="SELECT name FROM `bigquery-public-data.usa_names.usa_1910_2013` "\"WHERE state = 'TX' "\"LIMIT 100"# Location must match that of the dataset(s) referenced in the query.results=bigquery.querysqldo|config|config.location="US"endresults.eachdo|row|putsrow.inspectendend

Run a batch query

To run a batch query, select one of the following options:

Console

  1. Go to theBigQuery page.

    Go to BigQuery

  2. ClickSQL query.

  3. In the query editor, enter a valid GoogleSQL query.

    For example, query theBigQuery public datasetusa_namesto determine the most common names in the United States between theyears 1910 and 2013:

    SELECTname,gender,SUM(number)AStotalFROM`bigquery-public-data.usa_names.usa_1910_2013`GROUPBYname,genderORDERBYtotalDESCLIMIT10;
  4. ClickMore, and thenclickQuery settings.

  5. In theResource management section, selectBatch.

  6. Optional: Adjust yourquery settings.

  7. ClickSave.

  8. ClickRun.

    If you don't specify a destination table, the query job writes theoutput to a temporary (cache) table.

bq

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, aCloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Use thebq query commandand specify the--batch flag. In the following example, the--use_legacy_sql=false flag lets you use GoogleSQL syntax.

    bqquery\    --batch\    --use_legacy_sql=false\    'QUERY'

    ReplaceQUERY with a valid GoogleSQL query. Forexample, query theBigQuery public datasetusa_namesto determine the most common names in the United States between the years1910 and 2013:

    bqquery\--batch\--use_legacy_sql=false\'SELECT      name, gender,      SUM(number) AS total    FROM      `bigquery-public-data.usa_names.usa_1910_2013`    GROUP BY      name, gender    ORDER BY      total DESC    LIMIT      10;'

    The query job writes the output to a temporary (cache) table.

    Optionally, you can specify the destination table andlocationfor the query results. To write the results to an existing table, includethe appropriate flag to append (--append_table=true) or overwrite(--replace=true) the table.

    bqquery\    --batch\    --location=LOCATION\    --destination_table=TABLE\    --use_legacy_sql=false\    'QUERY'

    Replace the following:

    • LOCATION: the region or multi-region for the destinationtable—for example,US

      In this example, theusa_names dataset is stored in the USmulti-region location. If you specify a destination table for thisquery, the dataset that contains the destination table must also be inthe US multi-region. You cannot query a dataset in one location andwrite the results to a table in another location.

      You can set a default value for the location using the.bigqueryrc file.

    • TABLE: a name for the destination table—for example,myDataset.myTable

      If the destination table is a new table, then BigQuerycreates the table when you run your query. However, you must specifyan existing dataset.

      If the table isn't in your current project, then add theGoogle Cloud project ID using the formatPROJECT_ID:DATASET.TABLE—for example,myProject:myDataset.myTable. If--destination_table is unspecified,a query job is generated that writes the output to a temporary table.

API

To run a query using the API,insert a new joband populate thequery job configuration property. Optionally specify yourlocation in thelocation property in thejobReference section of thejob resource.

When you populate the query job properties, include theconfiguration.query.priority property and set the value toBATCH.

Poll for results by callinggetQueryResults.Poll untiljobComplete equalstrue. Check for errors and warnings in theerrors list.

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""time""cloud.google.com/go/bigquery")// queryBatch demonstrates issuing a query job using batch priority.funcqueryBatch(wio.Writer,projectID,dstDatasetID,dstTableIDstring)error{// projectID := "my-project-id"// dstDatasetID := "mydataset"// dstTableID := "mytable"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %v",err)}deferclient.Close()// Build an aggregate table.q:=client.Query(`SELECT  corpus,  SUM(word_count) as total_words,  COUNT(1) as unique_wordsFROM `+"`bigquery-public-data.samples.shakespeare`"+`GROUP BY corpus;`)q.Priority=bigquery.BatchPriorityq.QueryConfig.Dst=client.Dataset(dstDatasetID).Table(dstTableID)// Start the job.job,err:=q.Run(ctx)iferr!=nil{returnerr}// Job is started and will progress without interaction.// To simulate other work being done, sleep a few seconds.time.Sleep(5*time.Second)status,err:=job.Status(ctx)iferr!=nil{returnerr}state:="Unknown"switchstatus.State{casebigquery.Pending:state="Pending"casebigquery.Running:state="Running"casebigquery.Done:state="Done"}// You can continue to monitor job progress until it reaches// the Done state by polling periodically.  In this example,// we print the latest status.fmt.Fprintf(w,"Job %s in Location %s currently in state: %s\n",job.ID(),job.Location(),state)returnnil}

Java

To run a batch query,set the queryprioritytoQueryJobConfiguration.Priority.BATCHwhen creating aQueryJobConfiguration.

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.QueryJobConfiguration;importcom.google.cloud.bigquery.TableResult;// Sample to query batch in a tablepublicclassQueryBatch{publicstaticvoidrunQueryBatch(){// TODO(developer): Replace these variables before running the sample.StringprojectId="MY_PROJECT_ID";StringdatasetName="MY_DATASET_NAME";StringtableName="MY_TABLE_NAME";Stringquery="SELECT corpus"+" FROM `"+projectId+"."+datasetName+"."+tableName+" GROUP BY corpus;";queryBatch(query);}publicstaticvoidqueryBatch(Stringquery){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();QueryJobConfigurationqueryConfig=QueryJobConfiguration.newBuilder(query)// Run at batch priority, which won't count toward concurrent rate limit..setPriority(QueryJobConfiguration.Priority.BATCH).build();TableResultresults=bigquery.query(queryConfig);results.iterateAll().forEach(row->row.forEach(val->System.out.printf("%s,",val.toString())));System.out.println("Query batch performed successfully.");}catch(BigQueryException|InterruptedExceptione){System.out.println("Query batch not performed \n"+e.toString());}}}

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client library and create a clientconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctionqueryBatch(){// Runs a query at batch priority.// Create query job configuration. For all options, see// https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#jobconfigurationqueryconstqueryJobConfig={query:`SELECT corpus            FROM \`bigquery-public-data.samples.shakespeare\`            LIMIT 10`,useLegacySql:false,priority:'BATCH',};// Create job configuration. For all options, see// https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#jobconfigurationconstjobConfig={// Specify a job configuration to set optional job resource properties.configuration:{query:queryJobConfig,},};// Make API request.const[job]=awaitbigquery.createJob(jobConfig);constjobId=job.metadata.id;conststate=job.metadata.status.state;console.log(`Job${jobId} is currently in state${state}`);}

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()job_config=bigquery.QueryJobConfig(# Run at batch priority, which won't count toward concurrent rate limit.priority=bigquery.QueryPriority.BATCH)sql="""    SELECT corpus    FROM `bigquery-public-data.samples.shakespeare`    GROUP BY corpus;"""# Start the query, passing in the extra configuration.query_job=client.query(sql,job_config=job_config)# Make an API request.# Check on the progress by getting the job's updated state. Once the state# is `DONE`, the results are ready.query_job=client.get_job(query_job.job_id,location=query_job.location)# Make an API request.print("Job{} is currently in state{}".format(query_job.job_id,query_job.state))

Run a continuous query

Running a continuous query job requires additional configuration. For moreinformation, seeCreate continuous queries.

Query settings

When you run a query, you can specify the following settings:

Optional job creation mode

Optional job creation mode (formerly Short query optimized mode) can improve theoverall latency of queries that run for a short duration, such as those fromdashboards or data exploration workloads. This mode executes the query and returns the results inline forSELECT statements without requiring the use ofjobs.getQueryResultsto fetch the results. Queries using optional job creation mode don't create ajob when executed unless BigQuery determines that a job creationis necessary to complete the query.

To enable optional job creation mode, set thejobCreationMode field of theQueryRequestinstance toJOB_CREATION_OPTIONAL in thejobs.query request body.

When the value of this field is set toJOB_CREATION_OPTIONAL,BigQuery determines if the query can use the optional jobcreation mode. If so, BigQuery executes the query and returnsall results in therows field of the response. Since a job isn't created forthis query, BigQuery doesn't return ajobReference in theresponse body. Instead, it returns aqueryId field, which you can use to getinsights about the query using theINFORMATION_SCHEMA.JOBSview. Since nojob is created, there is nojobReference that can be passed tojobs.get andjobs.getQueryResultsAPIs to lookup these queries.

If BigQuery determines that a job is required to complete thequery, ajobReference is returned. You can inspect thejob_creation_reasonfield inINFORMATION_SCHEMA.JOBSview to determinethe reason that a job was created for the query. In this case, you should usejobs.getQueryResultsto fetch the results when the query is complete.

When you use theJOB_CREATION_OPTIONAL value, thejobReference field mightnot be present in the response. Check if the field exists before accessing it.

WhenJOB_CREATION_OPTIONAL is specified for multi-statement queries (scripts),BigQuery might optimize the execution process. As part of thisoptimization, BigQuery might determine that it can complete thescript by creating fewer job resources than the number of individual statements,potentially even executing the entire script without creating any job at all.This optimization depends on BigQuery's assessment of the script, and theoptimization might not be applied in every case. The optimization is fullyautomated by the system. No user controls or actions are required.

To run a query using optional job creation mode, select one of the followingoptions:

Console

  1. Go to theBigQuery page.

    Go to BigQuery

  2. ClickSQL query.

  3. In the query editor, enter a valid GoogleSQL query.

    For example, query theBigQuery public datasetusa_namesto determine the most common names in the United States between theyears 1910 and 2013:

    SELECTname,gender,SUM(number)AStotalFROM`bigquery-public-data.usa_names.usa_1910_2013`GROUPBYname,genderORDERBYtotalDESCLIMIT10;
  4. ClickMore, and thenchoose theOptional job creation query mode. To confirm this choice,clickConfirm.

  5. ClickRun.

bq

  1. In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, aCloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

  2. Use thebq query commandand specify the--job_creation_mode=JOB_CREATION_OPTIONAL flag. In the following example, the--use_legacy_sql=false flag lets you use GoogleSQL syntax.

    bqquery\    --rpc=true\    --use_legacy_sql=false\    --job_creation_mode=JOB_CREATION_OPTIONAL\    --location=LOCATION\    'QUERY'

    ReplaceQUERY with a valid GoogleSQL query, and replaceLOCATION with a valid region where the dataset is located. Forexample, query theBigQuery public datasetusa_namesto determine the most common names in the United States between the years1910 and 2013:

    bqquery\--rpc=true\--use_legacy_sql=false\--job_creation_mode=JOB_CREATION_OPTIONAL\--location=us\'SELECT      name, gender,      SUM(number) AS total    FROM      `bigquery-public-data.usa_names.usa_1910_2013`    GROUP BY      name, gender    ORDER BY      total DESC    LIMIT      10;'

    The query job returns the output inline in the response.

    Note: you may use--apilog=stdout to log API requests and responses to extract thequeryId if needed.

API

To run a query in optional job creation mode using the API,run a query synchronouslyand populate theQueryRequest property. Include thejobCreationMode property and set its value toJOB_CREATION_OPTIONAL.

Check the response. IfjobComplete equalstrue andjobReference is empty, read the results from therows field. You can also get thequeryId from the response.

IfjobReference is present, you can checkjobCreationReason for why a job was created by BigQuery. Poll for results by callinggetQueryResults.Poll untiljobComplete equalstrue. Check for errors and warnings in theerrors list.

Java

Available version: 2.51.0 and up

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.JobId;importcom.google.cloud.bigquery.QueryJobConfiguration;importcom.google.cloud.bigquery.QueryJobConfiguration.JobCreationMode;importcom.google.cloud.bigquery.TableResult;// Sample demonstrating short mode query execution.//// This feature is controlled by setting the defaultJobCreationMode// field in the BigQueryOptions used for the client. JOB_CREATION_OPTIONAL// allows for the execution of queries without creating a job.publicclassQueryJobOptional{publicstaticvoidmain(String[]args){Stringquery="SELECT name, gender, SUM(number) AS total FROM "+"bigquery-public-data.usa_names.usa_1910_2013 GROUP BY "+"name, gender ORDER BY total DESC LIMIT 10";queryJobOptional(query);}publicstaticvoidqueryJobOptional(Stringquery){try{// Initialize client that will be used to send requests. This client only needs// to be created once, and can be reused for multiple requests.BigQueryOptionsoptions=BigQueryOptions.getDefaultInstance();options.setDefaultJobCreationMode(JobCreationMode.JOB_CREATION_OPTIONAL);BigQuerybigquery=options.getService();// Execute the query. The returned TableResult provides access information// about the query execution as well as query results.TableResultresults=bigquery.query(QueryJobConfiguration.of(query));JobIdjobId=results.getJobId();if(jobId!=null){System.out.println("Query was run with job state.  Job ID: "+jobId.toString());}else{System.out.println("Query was run in short mode.  Query ID: "+results.getQueryId());}// Print the results.results.iterateAll().forEach(row->{System.out.print("name:"+row.get("name").getStringValue());System.out.print(", gender: "+row.get("gender").getStringValue());System.out.print(", total: "+row.get("total").getLongValue());System.out.println();});}catch(BigQueryException|InterruptedExceptione){System.out.println("Query not performed \n"+e.toString());}}}

To run a query with a proxy, seeConfiguring a proxy.

Python

Available version: 3.34.0 and up

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

# This example demonstrates executing a query without requiring an associated# job.fromgoogle.cloudimportbigqueryfromgoogle.cloud.bigquery.enumsimportJobCreationMode# Construct a BigQuery client object, specifying that the library should# avoid creating jobs when possible.client=bigquery.Client(default_job_creation_mode=JobCreationMode.JOB_CREATION_OPTIONAL)query="""    SELECT        name,        gender,        SUM(number) AS total    FROM        bigquery-public-data.usa_names.usa_1910_2013    GROUP BY        name, gender    ORDER BY        total DESC    LIMIT 10"""# Run the query.  The returned `rows` iterator can return information about# how the query was executed as well as the result data.rows=client.query_and_wait(query)ifrows.job_idisnotNone:print("Query was run with job state.  Job ID:{}".format(rows.job_id))else:print("Query was run without creating a job.  Query ID:{}".format(rows.query_id))print("The query data:")forrowinrows:# Row values can be accessed by field name or index.print("name={}, gender={}, total={}".format(row[0],row[1],row["total"]))

Node

Available version: 8.1.0 and up

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Demonstrates issuing a query that may be run in short query mode.// Import the Google Cloud client libraryconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery({// default behavior is to create jobs when using the jobs.query APIdefaultJobCreationMode:'JOB_CREATION_REQUIRED',});asyncfunctionqueryJobOptional(){// SQL query to run.constsqlQuery=`    SELECT name, gender, SUM(number) AS total    FROM bigquery-public-data.usa_names.usa_1910_2013    GROUP BY name, gender    ORDER BY total DESC    LIMIT 10`;// Run the queryconst[rows,,res]=awaitbigquery.query({query:sqlQuery,// Skip job creation to enable short mode.jobCreationMode:'JOB_CREATION_OPTIONAL',});if(!res.jobReference){console.log(`Query was run in short mode. Query ID:${res.queryId}`);}else{constjobRef=res.jobReference;constqualifiedId=`${jobRef.projectId}.${jobRef.location}.${jobRef.jobId}`;console.log(`Query was run with job state. Job ID:${qualifiedId}, Query ID:${res.queryId}`,);}// Print the resultsconsole.log('Rows:');rows.forEach(row=>console.log(row));}

Go

Available version: 1.69.0 and up

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""cloud.google.com/go/bigquery""google.golang.org/api/iterator")// queryJobOptional demonstrates issuing a query that doesn't require a// corresponding job.funcqueryJobOptional(wio.Writer,projectIDstring)error{// projectID := "my-project-id"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID,bigquery.WithDefaultJobCreationMode(bigquery.JobCreationModeOptional),)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %w",err)}deferclient.Close()q:=client.Query(`SELECT  name, gender,  SUM(number) AS totalFROMbigquery-public-data.usa_names.usa_1910_2013GROUP BYname, genderORDER BYtotal DESCLIMIT 10`)// Run the query and process the returned row iterator.it,err:=q.Read(ctx)iferr!=nil{returnfmt.Errorf("query.Read(): %w",err)}// The iterator provides information about the query execution.// Queries that were run in short query mode will not have the source job// populated.ifit.SourceJob()==nil{fmt.Fprintf(w,"Query was run in optional job mode.  Query ID: %q\n",it.QueryID())}else{j:=it.SourceJob()qualifiedJobID:=fmt.Sprintf("%s:%s.%s",j.ProjectID(),j.Location(),j.ID())fmt.Fprintf(w,"Query was run with job state.  Job ID: %q, Query ID: %q\n",qualifiedJobID,it.QueryID())}// Print row data.for{varrow[]bigquery.Valueerr:=it.Next(&row)iferr==iterator.Done{break}iferr!=nil{returnerr}fmt.Fprintln(w,row)}returnnil}

JDBC Driver

Available version: JDBC v1.6.1 and up

Requires settingJobCreationMode=2 in the connection string.

jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;JobCreationMode=2;Location=US;
Note: you may appendLogLevel=6;LogPath=log.txt to the connection string to enableTRACE level logging and extract troubleshooting information, includingqueryId, if needed.

ODBC Driver

Available version: ODBC v3.0.7.1016 and up

Requires settingJobCreationMode=2 in the.ini file.

[ODBCDataSources]SampleDSN=SimbaGoogleBigQueryODBCConnector64-bit[SampleDSN]JobCreationMode=2
Note: you may appendLogLevel=6 andLogPath=log.txt to the.ini file to enable detailed level logging and extract troubleshooting information, includingqueryId, if needed.

Quotas

For information about quotas regarding interactive and batch queries, seeQuery jobs.

Monitor queries

You can get information about queries as they are executing by using thejobs explorer or by querying theINFORMATION_SCHEMA.JOBS_BY_PROJECT view.

Dry run

A dry run in BigQuery provides the following information:

Dry runs don't use query slots, and you are not charged for performing a dry run.You can use the estimate returned by a dry run to calculate query costs inthepricing calculator.

Note: A dry run of a federated query that uses an external data source mightreport a lower bound of 0 bytes of data, even if rows are returned. This isbecause the amount of data processed from the external table can't be determineduntil the actual query completes. Running the federated query still incursa cost for processing this data.

Perform a dry run

To perform a dry run, do the following:

Console

  1. Go to the BigQuery page.

    Go to BigQuery

  2. Enter your query in the query editor.

    If the query is valid, then a check mark automatically appears along with the amount of data that the query will process. If the query is invalid, then an exclamation point appears along with an error message.

bq

Enter a query like the following using the--dry_run flag.

bqquery\--use_legacy_sql=false\--dry_run\'SELECT   COUNTRY,   AIRPORT,   IATA FROM   `project_id`.dataset.airports LIMIT   1000'

For a valid query, the command produces the following response:

Query successfully validated. Assuming the tables are not modified,running this query will process 10918 bytes of data.
Note: If your query processes a small amount of data, you might need toconvert the bytes that are processed from KB to MB. MB is the smallestmeasure used by the pricing calculator.

API

To perform a dry run by using the API, submit a query job withdryRun set totrue in theJobConfigurationtype.

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""cloud.google.com/go/bigquery")// queryDryRun demonstrates issuing a dry run query to validate query structure and// provide an estimate of the bytes scanned.funcqueryDryRun(wio.Writer,projectIDstring)error{// projectID := "my-project-id"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %v",err)}deferclient.Close()q:=client.Query(`SELECTname,COUNT(*) as name_countFROM `+"`bigquery-public-data.usa_names.usa_1910_2013`"+`WHERE state = 'WA'GROUP BY name`)q.DryRun=true// Location must match that of the dataset(s) referenced in the query.q.Location="US"job,err:=q.Run(ctx)iferr!=nil{returnerr}// Dry run is not asynchronous, so get the latest status and statistics.status:=job.LastStatus()iferr:=status.Err();err!=nil{returnerr}fmt.Fprintf(w,"This query will process %d bytes\n",status.Statistics.TotalBytesProcessed)returnnil}

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.Job;importcom.google.cloud.bigquery.JobInfo;importcom.google.cloud.bigquery.JobStatistics;importcom.google.cloud.bigquery.QueryJobConfiguration;// Sample to run dry query on the tablepublicclassQueryDryRun{publicstaticvoidrunQueryDryRun(){Stringquery="SELECT name, COUNT(*) as name_count "+"FROM `bigquery-public-data.usa_names.usa_1910_2013` "+"WHERE state = 'WA' "+"GROUP BY name";queryDryRun(query);}publicstaticvoidqueryDryRun(Stringquery){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();QueryJobConfigurationqueryConfig=QueryJobConfiguration.newBuilder(query).setDryRun(true).setUseQueryCache(false).build();Jobjob=bigquery.create(JobInfo.of(queryConfig));JobStatistics.QueryStatisticsstatistics=job.getStatistics();System.out.println("Query dry run performed successfully."+statistics.getTotalBytesProcessed());}catch(BigQueryExceptione){System.out.println("Query not performed \n"+e.toString());}}}

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client libraryconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctionqueryDryRun(){// Runs a dry query of the U.S. given names dataset for the state of Texas.constquery=`SELECT name    FROM \`bigquery-public-data.usa_names.usa_1910_2013\`    WHERE state = 'TX'    LIMIT 100`;// For all options, see https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/queryconstoptions={query:query,// Location must match that of the dataset(s) referenced in the query.location:'US',dryRun:true,};// Run the query as a jobconst[job]=awaitbigquery.createQueryJob(options);// Print the status and statisticsconsole.log('Status:');console.log(job.metadata.status);console.log('\nJob Statistics:');console.log(job.metadata.statistics);}

PHP

Before trying this sample, follow thePHP setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPHP API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

use Google\Cloud\BigQuery\BigQueryClient;/** Uncomment and populate these variables in your code */// $projectId = 'The Google project ID';// $query = 'SELECT id, view_count FROM `bigquery-public-data.stackoverflow.posts_questions`';// Construct a BigQuery client object.$bigQuery = new BigQueryClient([    'projectId' => $projectId,]);// Set job configs$jobConfig = $bigQuery->query($query);$jobConfig->useQueryCache(false);$jobConfig->dryRun(true);// Extract query results$queryJob = $bigQuery->startJob($jobConfig);$info = $queryJob->info();printf('This query will process %s bytes' . PHP_EOL, $info['statistics']['totalBytesProcessed']);

Python

Set theQueryJobConfig.dry_runproperty toTrue.Client.query()always returns a completedQueryJobwhen provided a dry run query configuration.

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()job_config=bigquery.QueryJobConfig(dry_run=True,use_query_cache=False)# Start the query, passing in the extra configuration.query_job=client.query(("SELECT name, COUNT(*) as name_count ""FROM `bigquery-public-data.usa_names.usa_1910_2013` ""WHERE state = 'WA' ""GROUP BY name"),job_config=job_config,)# Make an API request.# A dry run query completes immediately.print("This query will process{} bytes.".format(query_job.total_bytes_processed))

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-07-02 UTC.