Introduction to datasets

This page provides an overview of datasets in BigQuery.

Datasets

A dataset is contained within a specificproject. Datasets are top-level containers that are used to organize and control access to yourtables andviews. A table or view must belong to a dataset, so you need to create at least one dataset beforeloading data into BigQuery. Use the formatprojectname.datasetname to fully qualify a dataset name when using GoogleSQL, or the formatprojectname:datasetname to fully qualify a dataset name when using the bq command-line tool.

Location

You specify a location for storing your BigQuery data when youcreate a dataset. For a list of BigQuery dataset locations, seeBigQuery locations. After you createthe dataset, the location cannot be changed, but you cancopy datasets to different locations,or manuallymove (recreate) the dataset in a differentlocation.

BigQuery processes queries in the same location as the dataset thatcontains the tables you're querying. BigQuery stores your data in theselected locationin accordance with theService Specific Terms.

Data retention

Datasets usetime travel inconjunction with thefail-safe periodto retain deleted and modified data for a short time, in case you need torecover it. For more information, seeData retention with time travel and fail-safe.

Storage billing models

You can be billed for BigQuery data storage in either logical orphysical (compressed) bytes, or a combination of both.The storage billing model you choose determines yourstorage pricing. The storage billing model youchoose doesn't impact BigQuery performance. Whichever billingmodel you choose, your data is stored as physical bytes.

You set the storage billing model at the dataset level.If you don't specify a storage billing model when you create a dataset, itdefaults to using logical storage billing. However, you canchange a dataset's storage billing modelafter you create it. If you change a dataset's storagebilling model, you must wait 14 days before you can change the storage billingmodel again.

When you change a dataset's billing model, it takes 24 hours for thechange to take effect. Any tables or table partitions in long-term storageare not reset to active storage when you change a dataset's billing model.Query performance and query latency are not affected by changing a dataset'sbilling model.

Datasets usetime travel andfail-safe storage for data retention.Time travel and fail-safe storage are charged separately at active storage rateswhen you use physical storage billing, but are included in the base rate you arecharged when you use logical storage billing. You can modify the time travelwindow you use for a dataset in order to balance physical storage costs withdata retention. You can't modify the fail-safe window. For more informationabout dataset data retention, seeData retention with time travel and fail-safe.For more information on forecasting your storage costs, seeForecast storage billing.

You can't enroll a dataset in physical storage billing if your organization hasany existing legacyflat-rate slot commitmentslocated in the same region as the dataset. This doesn't apply to commitmentspurchased with aBigQuery edition.

External datasets

In addition to BigQuery datasets, you can create external datasets, which are links to external data sources:

• Spanner external dataset
• AWS Glue federated dataset

External datasets are also known asfederated datasets; both terms are used interchangeably.

Once created, external datasets contain tables from a referenced external data source. Data from these tables aren't copied into BigQuery, but queried every time they are used. For more information, seeSpanner federated queries.

Limitations

BigQuery datasets are subject to the following limitations:

• Thedataset location can only be set at creationtime. After a dataset is created, its location cannot be changed.
• All tables that are referenced in a query must be stored in datasets in thesame location.

• External datasets don't support table expiration, replicas, time travel, default collation, default rounding mode, or the option to enable or disable case-insensitive table names.

• Whenyou copy a table, thedatasets that contain the source table and destination table must reside inthe same location.

• Dataset names must be unique for each project.

• If you change a dataset'sstorage billing model, you must wait 14days before you can change the storage billing model again.

• You can't enroll a dataset in physical storage billing if you have anyexisting legacyflat-rate slot commitmentslocated in the same region as the dataset.

Quotas

For more information on dataset quotas and limits, seeQuotas and limits.

Pricing

You are not charged for creating, updating, or deleting a dataset.

For more information on BigQuery pricing, seePricing.

Security

To control access to datasets in BigQuery, seeControlling access to datasets.For information about data encryption, seeEncryption at rest.

What's next

• For more information on creating datasets, seeCreating datasets.
• For more information on assigning access controls to datasets, seeControlling access to datasets.