Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit809e4a2

Browse files
authored
docs(samples): add more clustering code snippets (#330)
Add sample code for creating a clustered table from a query result.File: samples/client_query_destination_table_clustered.pySection: [https://cloud.google.com/bigquery/docs/creating-clustered-tables#creating_a_clustered_table_from_a_query_result](https://cloud.google.com/bigquery/docs/creating-clustered-tables#creating_a_clustered_table_from_a_query_result)Add sample code for creating a clustered table when you load data.File: samples/load_table_clustered.pySection: [https://cloud.google.com/bigquery/docs/creating-clustered-tables#creating_a_clustered_table_when_you_load_data](https://cloud.google.com/bigquery/docs/creating-clustered-tables#creating_a_clustered_table_when_you_load_data)Fixes#329 🦕
1 parent168f0ec commit809e4a2

File tree

6 files changed

+172
-1
lines changed

6 files changed

+172
-1
lines changed

‎docs/usage/tables.rst‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,23 @@ Load table data from a file with the
8585
:start-after: [START bigquery_load_from_file]
8686
:end-before: [END bigquery_load_from_file]
8787

88+
Creating a clustered table from a query result:
89+
90+
..literalinclude::../samples/client_query_destination_table_clustered.py
91+
:language: python
92+
:dedent: 4
93+
:start-after: [START bigquery_query_clustered_table]
94+
:end-before: [END bigquery_query_clustered_table]
95+
96+
Creating a clustered table when you load data with the
97+
:func:`~google.cloud.bigquery.client.Client.load_table_from_uri` method:
98+
99+
..literalinclude::../samples/load_table_clustered.py
100+
:language: python
101+
:dedent: 4
102+
:start-after: [START bigquery_load_table_clustered]
103+
:end-before: [END bigquery_load_table_clustered]
104+
88105
Load a CSV file from Cloud Storage with the
89106
:func:`~google.cloud.bigquery.client.Client.load_table_from_uri` method:
90107

‎google/cloud/bigquery/__init__.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
fromgoogle.cloud.bigquery.datasetimportDataset
3838
fromgoogle.cloud.bigquery.datasetimportDatasetReference
3939
fromgoogle.cloud.bigqueryimportenums
40+
fromgoogle.cloud.bigquery.enumsimportSqlTypeNames
4041
fromgoogle.cloud.bigquery.enumsimportStandardSqlDataTypes
4142
fromgoogle.cloud.bigquery.external_configimportExternalConfig
4243
fromgoogle.cloud.bigquery.external_configimportBigtableOptions
@@ -137,8 +138,9 @@
137138
"Encoding",
138139
"QueryPriority",
139140
"SchemaUpdateOption",
140-
"StandardSqlDataTypes",
141141
"SourceFormat",
142+
"SqlTypeNames",
143+
"StandardSqlDataTypes",
142144
"WriteDisposition",
143145
# EncryptionConfiguration
144146
"EncryptionConfiguration",
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
16+
defclient_query_destination_table_clustered(table_id):
17+
18+
# [START bigquery_query_clustered_table]
19+
fromgoogle.cloudimportbigquery
20+
21+
# Construct a BigQuery client object.
22+
client=bigquery.Client()
23+
24+
# TODO(developer): Set table_id to the ID of the destination table.
25+
# table_id = "your-project.your_dataset.your_table_name"
26+
27+
sql="SELECT * FROM `bigquery-public-data.samples.shakespeare`"
28+
cluster_fields= ["corpus"]
29+
30+
job_config=bigquery.QueryJobConfig(
31+
clustering_fields=cluster_fields,destination=table_id
32+
)
33+
34+
# Start the query, passing in the extra configuration.
35+
query_job=client.query(sql,job_config=job_config)# Make an API request.
36+
query_job.result()# Wait for the job to complete.
37+
38+
table=client.get_table(table_id)# Make an API request.
39+
iftable.clustering_fields==cluster_fields:
40+
print(
41+
"The destination table is written using the cluster_fields configuration."
42+
)
43+
# [END bigquery_query_clustered_table]

‎samples/load_table_clustered.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
16+
defload_table_clustered(table_id):
17+
18+
# [START bigquery_load_table_clustered]
19+
fromgoogle.cloudimportbigquery
20+
21+
# Construct a BigQuery client object.
22+
client=bigquery.Client()
23+
24+
# TODO(developer): Set table_id to the ID of the table to create.
25+
# table_id = "your-project.your_dataset.your_table_name"
26+
27+
job_config=bigquery.LoadJobConfig(
28+
skip_leading_rows=1,
29+
source_format=bigquery.SourceFormat.CSV,
30+
schema=[
31+
bigquery.SchemaField("timestamp",bigquery.SqlTypeNames.TIMESTAMP),
32+
bigquery.SchemaField("origin",bigquery.SqlTypeNames.STRING),
33+
bigquery.SchemaField("destination",bigquery.SqlTypeNames.STRING),
34+
bigquery.SchemaField("amount",bigquery.SqlTypeNames.NUMERIC),
35+
],
36+
time_partitioning=bigquery.TimePartitioning(field="timestamp"),
37+
clustering_fields=["origin","destination"],
38+
)
39+
40+
job=client.load_table_from_uri(
41+
["gs://cloud-samples-data/bigquery/sample-transactions/transactions.csv"],
42+
table_id,
43+
job_config=job_config,
44+
)
45+
46+
job.result()# Waits for the job to complete.
47+
48+
table=client.get_table(table_id)# Make an API request.
49+
print(
50+
"Loaded {} rows and {} columns to {}".format(
51+
table.num_rows,len(table.schema),table_id
52+
)
53+
)
54+
# [END bigquery_load_table_clustered]
55+
returntable
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from ..importclient_query_destination_table_clustered
16+
17+
18+
deftest_client_query_destination_table_clustered(capsys,random_table_id):
19+
20+
client_query_destination_table_clustered.client_query_destination_table_clustered(
21+
random_table_id
22+
)
23+
out,err=capsys.readouterr()
24+
assert (
25+
"The destination table is written using the cluster_fields configuration."
26+
inout
27+
)
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Copyright 2020 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
from ..importload_table_clustered
16+
17+
18+
deftest_load_table_clustered(capsys,random_table_id,client):
19+
20+
table=load_table_clustered.load_table_clustered(random_table_id)
21+
22+
out,_=capsys.readouterr()
23+
assert"rows and 4 columns"inout
24+
25+
rows=list(client.list_rows(table))# Make an API request.
26+
assertlen(rows)>0
27+
asserttable.clustering_fields== ["origin","destination"]

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp