NotificationsYou must be signed in to change notification settings
Fork396
Star55

update retl bigquery setup with storage-api#7945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

gilsegment merged 3 commits intodevelopfromupdateBQStorage-API

Sep 29, 2025

Merged

update retl bigquery setup with storage-api#7945

gilsegment merged 3 commits intodevelopfromupdateBQStorage-API

Sep 29, 2025

Conversation

Copy link

Contributor

gilsegment commentedSep 16, 2025•
edited
Loading

ticket

https://twilio-engineering.atlassian.net/browse/CLAS-1069

Proposed changes

Updating Reverse-ETL Setup with BigQuery Storage-API

Merge timing

ASAP

update retl bigquery setup with storage-api

40d65b0

gilsegment requested a review froma team as acode owner

September 16, 2025 12:09

gilsegment requested a review fromstayseesong

September 16, 2025 12:09

stayseesong reviewed

Sep 18, 2025

View reviewed changes

src/connections/reverse-etl/reverse-etl-source-setup-guides/bigquery-setup.md Outdated

Comment on lines 84 to 88

		When setting up a BigQuery Reverse ETL source, you can choose which API Segment uses to read from BigQuery. You can make this selection during the initial setup, or later from Sources > Reverse ETL > BigQuery Source > Settings > Connection Settings.
		- REST API
		Recommended for most tables. REST is generally more cost-efficient, but syncs may be slower for very large datasets.
		- Storage API
		Recommended for large tables. The Storage API can significantly improve throughput and reduce sync times, but may incur higher costs. See [BigQuery Storage API client libraries](https://cloud.google.com/bigquery/docs/reference/storage/libraries) and [BigQuery pricing for data extraction](https://cloud.google.com/bigquery/pricing?hl=en#data_extraction_pricing) for more information

Copy link

Contributor

stayseesongSep 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@gilsegment is there a reason why you want this info added after step 10? the placement of this info is very weird as step 10 is about the data location, unless you're saying that choosing which API Segment uses is related to the data location?

Copy link

ContributorAuthor

gilsegmentSep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I changed the order.@stayseesong

change order [netlify-build]

7c222eb

gilsegment requested a review fromstayseesong

September 26, 2025 13:52

Copy link

netlifybot commentedSep 26, 2025

✅ Deploy Preview forsegment-docs ready!

Name	Link
🔨 Latest commit	`7c222eb`
🔍 Latest deploy log	https://app.netlify.com/projects/segment-docs/deploys/68d69a3dd7bf57000824b733
😎 Deploy Preview	https://deploy-preview-7945--segment-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.