Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

BigQuery: Add support for BigQuery Storage API Arrow format into_dataframe andto_arrow.#8551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
tswast merged 20 commits intogoogleapis:masterfromTheNeuralBit:bq-storage-arrow
Jul 12, 2019

Conversation

@TheNeuralBit
Copy link
Contributor

  • Makes_StreamParser abstract, and breaks it into two implementations: one for arrow and one for avro. The implementation is selected is based on the schema set in theReadSession.
  • Modifiies BigQuery client library to use the Arrow format for calls toclient.list_rows(table).to_dataframe(bq_storage_client)

@TheNeuralBitTheNeuralBit requested review froma team andtswastJuly 1, 2019 19:34
@googlebotgooglebot added the cla: yesThis human has signed the Contributor License Agreement. labelJul 1, 2019
@tswasttswast added api: bigqueryIssues related to the BigQuery API. api: bigquerystorageIssues related to the BigQuery Storage API. do not mergeIndicates a pull request not ready for merge, due to either quality or timing. labelsJul 1, 2019
@tswast
Copy link
Contributor

Added thedo not merge label. We'll need to wait for the Arrow changes to hit prod before merging.

@tswast
Copy link
Contributor

Once#8609 goes in, I'll aim to rebase this on top of that change, providing a fast version ofto_arrow, too.

@tswast
Copy link
Contributor

to_arrow is added in#8609 usingtabledata.list. Once that's in, we can update this PR to use the BQ Storage API for that method as well.

@tswasttswast changed the titleBigQuery Storage: Add support for arrow format in BQ Read APIBigQuery: Add support for BigQuery Storage API Arrow format into_dataframe andto_arrow.Jul 11, 2019
@tswast
Copy link
Contributor

Let's wait for#8644 to be merged and released to PyPI before merging this one.

@tswasttswast removed api: bigquerystorageIssues related to the BigQuery Storage API. do not mergeIndicates a pull request not ready for merge, due to either quality or timing. labelsJul 11, 2019
@tswasttswast requested a review fromshollymanJuly 11, 2019 21:22
@tswasttswast merged commit8852687 intogoogleapis:masterJul 12, 2019
parthea pushed a commit that referenced this pull requestAug 21, 2025
…n using BigQuery Storage API. (#8551)* Use Arrow format in client.list_rows(..).to_dataframe(..) with BQ Storage client* Add system test for arrow wire format.* Add system test for to_arrow.* Exclude bad pyarrow release.
parthea pushed a commit that referenced this pull requestSep 16, 2025
…n using BigQuery Storage API. (#8551)* Use Arrow format in client.list_rows(..).to_dataframe(..) with BQ Storage client* Add system test for arrow wire format.* Add system test for to_arrow.* Exclude bad pyarrow release.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@tswasttswasttswast approved these changes

@shollymanshollymanAwaiting requested review from shollyman

Assignees

No one assigned

Labels

api: bigqueryIssues related to the BigQuery API.cla: yesThis human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@TheNeuralBit@tswast@googlebot
[8]ページ先頭
©2009-2025 Movatter.jp