- Notifications
You must be signed in to change notification settings - Fork1.6k
BigQuery Storage: Add support for arrow format in BQ Read API#8644
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| _AVRO_BYTES_OPERATION="parse ReadRowResponse messages with Avro bytes" | ||
| _ARROW_BYTES_OPERATION="parse ReadRowResponse messages with Arrow bytes" | ||
| _FASTAVRO_REQUIRED="fastavro is required to {operation}." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I assume these parameterized errors are for when you do things like to_arrow with avro bytes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yeah, this is vestigial from when I was planning to implementto_arrow for Avro streams. Removed for now.
| used in the stream messages. | ||
| """ | ||
| iffastavroisNone: | ||
| raiseImportError(_FASTAVRO_REQUIRED) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
does this need to be parameterized as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Removed the{operation} from error message since I didn't actually need it.
_StreamParserabstract, and breaks it into two implementations: one for arrow and one for avro. The implementation is selected is based on the schema set in the ReadSession.to_arrowto reader classes.Split out from#8551 so that the changes to
google-cloud-bigquery-storagecan be submitted and released separately from the changes togoogle-cloud-bigquery.