Collectives™ on Stack Overflow
Find centralized, trusted content and collaborate around the technologies you use most.
Learn more about Collectives
Tooling
0votes
1replies
25views
I would like to change values that were input as zero since they were missing into null values to avoid getting errors
However, you’re working with the newest data and it hasn’t been cleaned yet. Missing values were incorrectly entered as zeroes, and you need to change them to null values before you look for trends. ...
1vote
0answers
42views
Why is clustering on _id not reducing bytes scanned in BigQuery for a table synced via Datastream?
I have a BigQuery table (approx. 140 GB in size) that is synchronized from MongoDB via a Google Cloud Datastream. I have set the _id column as the clustering column. However, when I run a query ...
0votes
1answer
55views
How to bulk update descriptions for datasets shared via Analytics Hub without modifying each source dataset manually?
I have a GCP project (let's call it "Tier") that acts as a central data-sharing hub across multiple teams. Using Analytics Hub, I've created an exchange with several listings that reference ...
- 147
Advice
0votes
4replies
32views
Passing table content as parameter for BigQuery UDF
I would like to have the UDF getting table content as parameter, likeCREATE OR REPLACE PROCEDURE dataset.function1(tab TABLE)so that I am easier to pass the whole dataset for unit testing, is that ...
- 2,091
Advice
0votes
4replies
67views
BigQuery Standard SQL: UPDATE to convert incorrect 0 values to NULL returns an error
I’m using Google BigQuery (Standard SQL) to clean 2023 weather data. Missing values were mistakenly stored as 0, but 0 is a valid value for some rows, so I only want to convert the incorrect zeroes to ...
0votes
1answer
60views
AWS Glue Connection for BigQuery gives "SparkProperties is missing but it is required" and "secretId is not defined in the schema" when using Go SDK
I'm trying to programmatically create a native Google BigQuery connection in AWS Glue using the AWS SDK for Go v2 (github.com/aws/aws-sdk-go-v2/service/glue).According to the AWS docs (Glue 4.0+ ...
- 9,268
3votes
1answer
86views
Filtering dataset by geolocation data using SQL in BigQuery
I trying to determine if NCAA basketball games have an affect on liquor sales in the surrounding areas around the stadiums where the teams play their games. I am writing a query that filters and sorts ...
0votes
1answer
63views
Airflow BigQueryInsertJobOperator and job re-attachment
The inciting problemOne of our DAGs suffers from an issue wherein a BigQueryInsertJobOperator task fails due to it becoming a "zombie task" (from Airflow's perspective); in the "Event ...
- 300
Best practices
0votes
8replies
101views
How can I best optimize a Bigquery SQL by appending completely redundant filters to the query?
I have two different SQL calls to BigQuery. One completes in 22 seconds, the other is identical except that I add completely redundant filter at the end. This second completes in 8 seconds. The ...
- 635
1vote
0answers
90views
GA4 BigQuery Daily Export Missing Data After Specific Date
I have a GA4 property linked to BigQuery for daily export (both iOS and Web data streams selected, “export all events” enabled, no filters).Everything was working fine until 2025-11-23, when the ...
-1votes
1answer
139views
Facing issue in Big Query while extracting JSON Value
We are leveraging BigQuery to create reports, with some column values represented in JSON. Below is a sample payload. I can successfully retrieve the Template value, but the objectType value remains ...
Tooling
1vote
2replies
65views
Windows Shortcut
What shortcuts can I use to split strings in my database (BigQuery). I have been trying a lot of functions and I make use of the google bigquery platform. I'm getting really confused now. I tried ...
0votes
1answer
118views
Optional nanoseconds when using `timestamp_format` in BigQuery Load Jobs?
I'm loading data into BigQuery and using LoadJobConfig. There's a timestamp_format field where you can specify the format of timestamps.Checking the data I have, timestamps come in different format:...
0votes
1answer
84views
"Was expecting: <EOF>" when using BigQuery
I am trying to use BigQuery and want to search from the Google Cloud SDK Shell.I logged in and set up with gcloud config init successfully.I used the BigQuery console to put data in my dataset table....
- 3,826
0votes
0answers
88views
How to achieve dynamic partition pruning in BigQuery TVF using a subquery lookup?
I have a Table Valued Function (TVF) that takes an array of IDs as input.source_table: A very large table (TBs) partitioned by day on capturedTimestamp.selection_table: A small lookup table that ...
This tag is in a collective: a subcommunity defined by tags with relevant content and experts.
- The Overflow Blog
- Featured on Meta
Hot Network Questions
- When are Closure nodes necessary in Geometry Nodes compared to Node Groups?
- What distinguishes engineering validation from simple functional demonstration in prototype design projects?
- Would this be an appropriate soffit mounting and top plate partition attachment?
- How should I patch drywall around a new subpanel?
- Seven things from the USA
- How to wash produce to remove pathogens (including fecal-contamination)
- Do virtue ethics systems usually expect every successful agent to train for all the virtues to roughly the same extent, OR...?
- Bias of asymptotically linear estimators
- Did Jesus literally come out from God in John 8:42?
- Lilypond: How to put a short “smile” slur under a single note?
- Create and verify a large single file in parallel-mode
- Please tell me a construction of an example in which the above fact is not true if one uses the box topology instead of the product topology. Munkres.
- Line of Sight function for a hex-based wargame
- How do I encourage more interaction with Fate's mechanics?
- Why does 74x138 have 3 EN and why does it have to be 1 EN active high and 2 active low?
- Leibniz on what is reasoning
- What does the height of the curve represent in normal distribution?
- How do you write helpless screaming in dialogue?
- Is computation and information part of epistemology?
- Create a directed graph
- Why can we add up the internal energy of a moving gas and its kinetic energy due to flow velocity?
- Book about 3 main characters who gain powers from injection from crazy scientist
- Is there a better way to define this piecewise function?
- Should we use have or has in this case?