Stay organized with collections Save and categorize content based on your preferences.

Listing datasets

This document describes how to list and get information about datasets in BigQuery.

Before you begin

Grant Identity and Access Management (IAM) roles that give users the necessary permissions to perform each task in this document.

Required role

To get the permission that you need to list datasets or get information on datasets, ask your administrator to grant you theBigQuery Metadata Viewer (roles/bigquery.metadataViewer) IAM role on your project. For more information about granting roles, seeManage access to projects, folders, and organizations.

This predefined role contains the bigquery.datasets.get permission, which is required to list datasets or get information on datasets.

You might also be able to get this permission withcustom roles or otherpredefined roles.

When you apply theroles/bigquery.metadataViewer role at the project ororganization level, you can list all the datasets in the project. When youapply theroles/bigquery.metadataViewer role at the dataset level, you canlist all the datasets for which you have been granted that role.

List datasets

Select one of the following options:

Console

  1. In the navigation menu, clickStudio.

  2. In theExplorer panel, expand a project name to see the datasets inthat project, or use the search box to search by dataset name.

SQL

Query theINFORMATION_SCHEMA.SCHEMATA view:

  1. In the Google Cloud console, go to theBigQuery Studio page.

    Go to BigQuery Studio

  2. In the query editor, enter the following statement:

    SELECTschema_nameFROMPROJECT_ID.`region-REGION`.INFORMATION_SCHEMA.SCHEMATA;

    Replace the following:

    • PROJECT_ID: the ID of yourGoogle Cloud project. If not specified, the default project is used.
    • REGION: anydataset region name. For example,us.

  3. ClickRun.

For more information about how to run queries, seeRun an interactive query.

bq

Issue thebq ls command to list datasets by dataset ID. The--formatflag can be used to control the output. If you are listing dataset in aproject other than your default project, add the--project_id flag to thecommand.

To list all datasets in a project, includinghidden datasets,use the--all flag or the-a shortcut.

To list all datasets in a project, excluding hidden datasets, use the--datasets flag or the-d shortcut. This flag is optional. By default,hidden datasets are not listed.

Additional flags include:

  • --filter: List datasets that match the filter expression. Use aspace-separated list of label keys and values in the formlabels.key:value. For more information onfiltering datasets using labels, seeAdding and using labels.
  • --max_results or-n: An integer indicating the maximum number ofresults. The default value is50.
bqls--filterlabels.key:value\--max_resultsinteger\--format=prettyjson\--project_idproject_id

Replace the following:

  • key:value: a label key and value
  • integer: an integer representing the number of datasets tolist
  • project_id: the name of your project

Examples:

Enter the following command to list datasets in your default project.--format is set to pretty to return a basic formatted table.

bq ls --format=pretty

Enter the following command to list datasets inmyotherproject.--formatis set toprettyjson to return detailed results in JSON format.

bq ls --format=prettyjson --project_id myotherproject

Enter the following command to list all datasets including hiddendatasets in your default project. In the output, hidden datasets beginwith an underscore.

bq ls -a

Enter the following command to return more than the default output of 50datasets from your default project.

bq ls --max_results 60

Enter the following command to list datasets in your default project withthe labelorg:dev.

bq ls --filter labels.org:dev

API

To list datasets using the API, call thedatasets.listAPI method.

C#

Before trying this sample, follow theC# setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryC# API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

usingGoogle.Cloud.BigQuery.V2;usingSystem;usingSystem.Collections.Generic;usingSystem.Linq;publicclassBigQueryListDatasets{publicvoidListDatasets(stringprojectId="your-project-id"){BigQueryClientclient=BigQueryClient.Create(projectId);// Retrieve list of datasets in projectList<BigQueryDataset>datasets=client.ListDatasets().ToList();// Display the resultsif(datasets.Count >0){Console.WriteLine($"Datasets in project {projectId}:");foreach(vardatasetindatasets){Console.WriteLine($"\t{dataset.Reference.DatasetId}");}}else{Console.WriteLine($"{projectId} does not contain any datasets.");}}}

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""cloud.google.com/go/bigquery""google.golang.org/api/iterator")// listDatasets demonstrates iterating through the collection of datasets in a project.funclistDatasets(projectIDstring,wio.Writer)error{// projectID := "my-project-id"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %v",err)}deferclient.Close()it:=client.Datasets(ctx)for{dataset,err:=it.Next()iferr==iterator.Done{break}iferr!=nil{returnerr}fmt.Fprintln(w,dataset.DatasetID)}returnnil}

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.api.gax.paging.Page;importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQuery.DatasetListOption;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.Dataset;publicclassListDatasets{publicstaticvoidrunListDatasets(){// TODO(developer): Replace these variables before running the sample.StringprojectId="MY_PROJECT_ID";listDatasets(projectId);}publicstaticvoidlistDatasets(StringprojectId){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();Page<Dataset>datasets=bigquery.listDatasets(projectId,DatasetListOption.pageSize(100));if(datasets==null){System.out.println("Dataset does not contain any models");return;}datasets.iterateAll().forEach(dataset->System.out.printf("Success! Dataset ID: %s ",dataset.getDatasetId()));}catch(BigQueryExceptione){System.out.println("Project does not contain any datasets \n"+e.toString());}}}

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client libraryconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctionlistDatasets(){/**   * TODO(developer): Uncomment the following lines before running the sample.   */// const projectId = "my_project_id";// Lists all datasets in the specified project.// If projectId is not specified, this method will take// the projectId from the authenticated BigQuery Client.const[datasets]=awaitbigquery.getDatasets({projectId});console.log('Datasets:');datasets.forEach(dataset=>console.log(dataset.id));}

PHP

Before trying this sample, follow thePHP setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPHP API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

use Google\Cloud\BigQuery\BigQueryClient;/** Uncomment and populate these variables in your code */// $projectId  = 'The Google project ID';$bigQuery = new BigQueryClient([    'projectId' => $projectId,]);$datasets = $bigQuery->datasets();foreach ($datasets as $dataset) {    print($dataset->id() . PHP_EOL);}

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()datasets=list(client.list_datasets())# Make an API request.project=client.projectifdatasets:print("Datasets in project{}:".format(project))fordatasetindatasets:print("\t{}".format(dataset.dataset_id))else:print("{} project does not contain any datasets.".format(project))

Ruby

Before trying this sample, follow theRuby setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryRuby API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

require"google/cloud/bigquery"deflist_datasetsproject_id="your-project-id"bigquery=Google::Cloud::Bigquery.newproject:project_idputs"Datasets in project#{project_id}:"bigquery.datasets.eachdo|dataset|puts"\t#{dataset.dataset_id}"endend

Get information about datasets

Select one of the following options:

Console

  1. In theExplorer panel, expand your project and select a dataset.

  2. Expand theActions option and clickOpen. The description and detailsappear in the details panel. The tables for a dataset are listed with thedataset name in theExplorer panel.

By default,hidden datasetsare hidden from the Google Cloud console. To show information abouthidden datasets, use the bq command-line tool or the API.

SQL

Query theINFORMATION_SCHEMA.SCHEMATA view:

  1. In the Google Cloud console, go to theBigQuery Studio page.

    Go to BigQuery Studio

  2. In the query editor, enter the following statement:

    SELECT*EXCEPT(schema_owner)FROMPROJECT_ID.`region-REGION`.INFORMATION_SCHEMA.SCHEMATA;

    Replace the following:

    • PROJECT_ID: the ID of your Google Cloud project. If not specified, the default project is used.
    • REGION: anydataset region name. For example,us.

  3. ClickRun.

For more information about how to run queries, seeRun an interactive query.

You can also query theINFORMATION_SCHEMA.SCHEMATA_OPTIONS view.

SELECT*FROMPROJECT_ID.`region-REGION`.INFORMATION_SCHEMA.SCHEMATA_OPTIONS;

bq

Issue thebq show command. The--format flag can be used to control theoutput. If you are getting information about a dataset in a project otherthan your default project, add the project ID to the dataset name in thefollowing format:project_id:dataset.The output displays the dataset's information such as accesscontrol, labels, and location. This command doesn't display a dataset'sinherited permissions, but you can see them in the Google Cloud console.

To show information about ahidden dataset,use thebq ls --allcommand to list all datasets and then use the name of the hidden datasetin thebq show command.

bqshow--format=prettyjsonproject_id:dataset

Replace the following:

  • project_id is the name of your project.
  • dataset is the name of the dataset.

Examples:

Enter the following command to display information aboutmydataset in yourdefault project.

bq show --format=prettyjson mydataset

Enter the following command to display information aboutmydataset inmyotherproject.

bq show --format=prettyjson myotherproject:mydataset

Enter the following command to display information about the hidden dataset_1234abcd56efgh78ijkl1234 in your default project.

bq show --format=prettyjson _1234abcd56efgh78ijkl1234

API

Call thedatasets.getAPI method and provide any relevant parameters.

Go

Before trying this sample, follow theGo setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryGo API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

import("context""fmt""io""cloud.google.com/go/bigquery""google.golang.org/api/iterator")// printDatasetInfo demonstrates fetching dataset metadata and printing some of it to an io.Writer.funcprintDatasetInfo(wio.Writer,projectID,datasetIDstring)error{// projectID := "my-project-id"// datasetID := "mydataset"ctx:=context.Background()client,err:=bigquery.NewClient(ctx,projectID)iferr!=nil{returnfmt.Errorf("bigquery.NewClient: %v",err)}deferclient.Close()meta,err:=client.Dataset(datasetID).Metadata(ctx)iferr!=nil{returnerr}fmt.Fprintf(w,"Dataset ID: %s\n",datasetID)fmt.Fprintf(w,"Description: %s\n",meta.Description)fmt.Fprintln(w,"Labels:")fork,v:=rangemeta.Labels{fmt.Fprintf(w,"\t%s: %s",k,v)}fmt.Fprintln(w,"Tables:")it:=client.Dataset(datasetID).Tables(ctx)cnt:=0for{t,err:=it.Next()iferr==iterator.Done{break}cnt++fmt.Fprintf(w,"\t%s\n",t.TableID)}ifcnt==0{fmt.Fprintln(w,"\tThis dataset does not contain any tables.")}returnnil}

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.api.gax.paging.Page;importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQuery.TableListOption;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.Dataset;importcom.google.cloud.bigquery.DatasetId;importcom.google.cloud.bigquery.Table;publicclassGetDatasetInfo{publicstaticvoidrunGetDatasetInfo(){// TODO(developer): Replace these variables before running the sample.StringprojectId="MY_PROJECT_ID";StringdatasetName="MY_DATASET_NAME";getDatasetInfo(projectId,datasetName);}publicstaticvoidgetDatasetInfo(StringprojectId,StringdatasetName){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();DatasetIddatasetId=DatasetId.of(projectId,datasetName);Datasetdataset=bigquery.getDataset(datasetId);// View dataset propertiesStringdescription=dataset.getDescription();System.out.println(description);// View tables in the dataset// For more information on listing tables see:// https://javadoc.io/static/com.google.cloud/google-cloud-bigquery/0.22.0-beta/com/google/cloud/bigquery/BigQuery.htmlPage<Table>tables=bigquery.listTables(datasetName,TableListOption.pageSize(100));tables.iterateAll().forEach(table->System.out.print(table.getTableId().getTable()+"\n"));System.out.println("Dataset info retrieved successfully.");}catch(BigQueryExceptione){System.out.println("Dataset info not retrieved. \n"+e.toString());}}}

Node.js

Before trying this sample, follow theNode.js setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryNode.js API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

// Import the Google Cloud client libraryconst{BigQuery}=require('@google-cloud/bigquery');constbigquery=newBigQuery();asyncfunctiongetDataset(){// Retrieves dataset named "my_dataset"./**   * TODO(developer): Uncomment the following lines before running the sample   */// const datasetId = "my_dataset";// Retrieve dataset referenceconst[dataset]=awaitbigquery.dataset(datasetId).get();console.log('Dataset:');console.log(dataset.metadata.datasetReference);}getDataset();

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigquery# Construct a BigQuery client object.client=bigquery.Client()# TODO(developer): Set dataset_id to the ID of the dataset to fetch.# dataset_id = 'your-project.your_dataset'dataset=client.get_dataset(dataset_id)# Make an API request.full_dataset_id="{}.{}".format(dataset.project,dataset.dataset_id)friendly_name=dataset.friendly_nameprint("Got dataset '{}' with friendly_name '{}'.".format(full_dataset_id,friendly_name))# View dataset properties.print("Description:{}".format(dataset.description))print("Labels:")labels=dataset.labelsiflabels:forlabel,valueinlabels.items():print("\t{}:{}".format(label,value))else:print("\tDataset has no labels defined.")# View tables in dataset.print("Tables:")tables=list(client.list_tables(dataset))# Make an API request(s).iftables:fortableintables:print("\t{}".format(table.table_id))else:print("\tThis dataset does not contain any tables.")

Verify the dataset name

The following samples show how to check if a dataset exists:

Java

Before trying this sample, follow theJava setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryJava API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.Dataset;importcom.google.cloud.bigquery.DatasetId;// Sample to check dataset existpublicclassDatasetExists{publicstaticvoidmain(String[]args){// TODO(developer): Replace these variables before running the sample.StringdatasetName="MY_DATASET_NAME";datasetExists(datasetName);}publicstaticvoiddatasetExists(StringdatasetName){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();Datasetdataset=bigquery.getDataset(DatasetId.of(datasetName));if(dataset!=null){System.out.println("Dataset already exists.");}else{System.out.println("Dataset not found.");}}catch(BigQueryExceptione){System.out.println("Something went wrong. \n"+e.toString());}}}

Python

Before trying this sample, follow thePython setup instructions in theBigQuery quickstart using client libraries. For more information, see theBigQueryPython API reference documentation.

To authenticate to BigQuery, set up Application Default Credentials. For more information, seeSet up authentication for client libraries.

fromgoogle.cloudimportbigqueryfromgoogle.cloud.exceptionsimportNotFoundclient=bigquery.Client()# TODO(developer): Set dataset_id to the ID of the dataset to determine existence.# dataset_id = "your-project.your_dataset"try:client.get_dataset(dataset_id)# Make an API request.print("Dataset{} already exists".format(dataset_id))exceptNotFound:print("Dataset{} is not found".format(dataset_id))

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-03-05 UTC.