- Notifications
You must be signed in to change notification settings - Fork2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Adding configurable BigQuery ProjectID on contrib jobs#3019
base:master
Are you sure you want to change the base?
Adding configurable BigQuery ProjectID on contrib jobs#3019
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for the PR. Could you please add test cases if possible?
| @@ -527,6 +527,12 @@ def allow_quoted_new_lines(self): | |||
| """Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.""" | |||
| return False | |||
| @property | |||
| def project_id(self): | |||
| """Project ID on which to run the BigQuery Job | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Could you please add short description for the default value as well?
| @@ -610,6 +616,12 @@ def use_legacy_sql(self): | |||
| """ | |||
| return True | |||
| @property | |||
| def project_id(self): | |||
| """Project ID on which to run the BigQuery Job | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same here.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions. |
Description
Adding a way to configure BigQuery Project ID on both
BigQueryLoadTaskandBigQueryRunQueryTask.They are both added as properties of the class, keeping the same default as in previous versions.
Motivation and Context
BigQuery jobs can be run on different Project IDs than the ones configured by the output Dataset. This is especially useful for shared BigQuery Datasets, or when you usually need to defer the slot allocation between projects.
Have you tested this? If so, how?
I ran a local test using this feature.