- Notifications
You must be signed in to change notification settings - Fork1.6k
Add --params option to %%bigquery magic#6277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Add --params option to %%bigquery magic#6277
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
guillermo-carrasco commentedOct 19, 2018
@tseaver I am not familiar with the review process in these repositories. Should I push a new commit with the suggested changes, and rebase at the end? or should I rebase now? Thanks for reviewing! |
tseaver commentedOct 19, 2018
Push a new commit on the same branch. I will review those changes. We iterate until I hit the "approve" button for the review, and then I hit the merge button. Occasionally I might ask for a rebase against the current
Thank you for the patch! |
guillermo-carrasco commentedOct 21, 2018
@tseaver thanks for the clarification. I just amended the suggested changes. |
tseaver commentedOct 22, 2018
@shollyman,@tswast I'm fine with the patch as it stands now. Please comment on suitability from the DPE side. |
Uh oh!
There was an error while loading.Please reload this page.
guillermo-carrasco commentedOct 26, 2018
This new commit makes it easier to use dictionaries with the |
6c32111 to0baa021Compareguillermo-carrasco commentedOct 26, 2018
@tswast please disregard my last comment. I have folowed your suggestion and used |
alixhami left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thank you for your contribution. I have some comments on how to document this addition, because we want to be really clear to users how to use this.
Also something to note for any future contributions is that this magic is intentionally simple and is not intended to duplicate all functionality of the library. We want to avoid the maintenance burden of duplicating all the library's functionality and also keep the interface simple because magics are generally used as short hand for simple operations.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
tswast left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for updating the helper. LGTM once you make the changes@alixhami requested.
guillermo-carrasco commentedOct 26, 2018
Sorry for closing, it was a mistake. Thanks for the review@alixhami. I'll implement the suggested changes. |
guillermo-carrasco commentedOct 29, 2018
@alixhami I have implemented the following changes:
I am not sure that changing the docstring to reflect the dict approach is what we want though. It is still a JSON string what is sent in the cell magic. It's just that it is also possible to expand a dictionaryinstead of writing the JSON string. But at the end, it is a JSON string what is treated in Regarding contributions: I totally understand, and I appreciate you're taking the time to review this. Query parametrization is something we use a lot, and we have a heavy notebook-drived development, so this would be a great addition for us. Thanks again! |
6a1ebe3 tobfd94afComparebfd94af to2c17f19Comparealixhami commentedOct 29, 2018
Thanks@guillermo-carrasco! @tswast - what is your preference on how this parameter should be documented? |
tswast left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'm okay with the current documentation, given the fact that all magic arguments are passed in as strings.
guillermo-carrasco commentedOct 30, 2018
I've slightly changed the docstring now. I had to remove the dictionary reference example from the documentation. The reason is that since |
alixhami commentedOct 30, 2018
@guillermo-carrasco I'm looking into the docs build issue now. I think that example is very helpful, so I'm going to see if there's a way to include it without throwing errors. |
tseaver commentedOct 30, 2018
@alixhami PTAL one final pass. |
alixhami commentedOct 30, 2018
Ok I pushed some commits, which make the following updates:
|
guillermo-carrasco commentedOct 30, 2018
amazing, thanks a lot for all the help and reviews :) |
Often it is useful to be able to parametrize query strings. While there is the option of parametrizing queriesusing the raw sdk, there is no option to do so using the
%%bigquerymagic in Jupyter notebooks.This is useful for several reasons, specially when the same parameter appears several times in the query. This could be for example a cutoff date appearing in several
JOINconditions.This PR adds the option
--paramsto the%%bigquerymagic for Jupyter notebooks.--paramsaccepts a JSON string that will be used to format the string contained in the query.An example of how this would work: