- Notifications
You must be signed in to change notification settings - Fork2.7k
No sampling over 281TB#19978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:v1.4-andium
Are you sure you want to change the base?
No sampling over 281TB#19978
Changes from1 commit
File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
- Loading branch information
Uh oh!
There was an error while loading.Please reload this page.
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -44,11 +44,8 @@ unique_ptr<SampleOptions> Transformer::TransformSampleOptions(optional_ptr<duckd | ||
| } else { | ||
| // sample size is given in rows: use reservoir sampling | ||
| auto rows = sample_value.GetValue<int64_t>(); | ||
| if (rows < 0 || sample_value.GetValue<uint64_t>() >= Allocator::MAXIMUM_ALLOC_SIZE) { | ||
Collaborator There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others.Learn more. The sample size is in rows but the allocator max size is in bytes - I suspect that sampling for Alternatively, we might want to not allocate all the space for the sample up-front if the sample size is very large - but that's obviously a larger change. | ||
| throw ParserException("Sample rows %lld out of range, must be between 0 and %lld", rows, | ||
| Allocator::MAXIMUM_ALLOC_SIZE); | ||
| } | ||
| result->sample_size = Value::BIGINT(rows); | ||
Uh oh!
There was an error while loading.Please reload this page.