- Notifications
You must be signed in to change notification settings - Fork0
Handle Bit-Flags to record Quality for Modeled Data Products
License
bitfloat/bitfield
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Thebitfield package enables efficient storage and transmission ofmetadata and intermediate results across scientific workflows byencoding computational decisions into sequences of bits that transformto integer values. This approach allows storing rich contextualinformation - including quality assessments, uncertainty metrics, modelparameters, and processing thresholds - in a single column of a table orraster layer.
Think of a bit as a switch representing off and on states. A sequence ofn bits can accommodate 2^n states, enabling the encoding of booleanresponses, categorical cases, integers, or even floating-point valueswith limited precision. The resulting bitfield creates a ‘computationalfootprint’ that preserves the context needed for cross-workflowintegration and downstream reuse.
Install the official version from CRAN:
install.packages("bitfield")Install the latest development version from GitHub:
devtools::install_github("bitfloat/bitfield")
library(bitfield)library(dplyr,warn.conflicts=FALSE)# Example data with quality issuesbf_tbl#> # A tibble: 9 × 5#> x y commodity yield year#> <dbl> <dbl> <fct> <dbl> <chr>#> 1 25.3 59.5 soybean 11.2 2021#> 2 27.9 58.1 maize 12.0 <NA>#> 3 27.8 57.8 soybean 13.2 2021r#> 4 27 59.2 <NA> 4.43 2021#> 5 259 Inf honey 13.0 2021#> 6 27.3 59.1 maize 8.55 2021#> 7 26.1 58.4 soybean 11.3 2021#> 8 26.5 NaN maize 10.6 2021#> 9 0 0 soybean 9.01 2021
Create a registry to capture metadata about your workflow:
reg<- bf_registry(name="data_quality",description="Quality assessment for agricultural data")# Test for missing valuesreg<- bf_map(protocol="na",data=bf_tbl,x=commodity,registry=reg)# Encode yield values with limited precisionreg<- bf_map(protocol="numeric",data=bf_tbl,x=yield,format="half",registry=reg)# View the registry structurereg#> width 17#> flags 2 -|----------------#>#> pos encoding type col#> 1 0.0.1/0 na commodity#> 2 1.5.10/15 numeric yield
Encode the flags into integer representation:
(field<- bf_encode(registry=reg))#> # A tibble: 9 × 1#> bf_int1#> <int>#> 1 18840#> 2 18942#> 3 19101#> 4 83054#> 5 19071#> 6 18502#> 7 18851#> 8 18770#> 9 18561
Decode the bitfield in a downstream application:
(flags<- bf_decode(x=field,registry=reg,verbose=FALSE))#> # A tibble: 9 × 2#> na_commodity numeric_yield#> <chr> <chr>#> 1 0 0100100110011000#> 2 0 0100100111111110#> 3 0 0100101010011101#> 4 1 0100010001101110#> 5 0 0100101001111111#> 6 0 0100100001000110#> 7 0 0100100110100011#> 8 0 0100100101010010#> 9 0 0100100010000001# -> legend is available in bf_legendbf_tbl|> bind_cols(flags)|>knitr::kable()
| x | y | commodity | yield | year | na_commodity | numeric_yield |
|---|---|---|---|---|---|---|
| 25.3 | 59.5 | soybean | 11.192915 | 2021 | 0 | 0100100110011000 |
| 27.9 | 58.1 | maize | 11.986793 | NA | 0 | 0100100111111110 |
| 27.8 | 57.8 | soybean | 13.229386 | 2021r | 0 | 0100101010011101 |
| 27.0 | 59.2 | NA | 4.431376 | 2021 | 1 | 0100010001101110 |
| 259.0 | Inf | honey | 12.997422 | 2021 | 0 | 0100101001111111 |
| 27.3 | 59.1 | maize | 8.548882 | 2021 | 0 | 0100100001000110 |
| 26.1 | 58.4 | soybean | 11.276921 | 2021 | 0 | 0100100110100011 |
| 26.5 | NaN | maize | 10.640715 | 2021 | 0 | 0100100101010010 |
| 0.0 | 0.0 | soybean | 9.010452 | 2021 | 0 | 0100100010000001 |
The decoded information is also available in the package environment forprogrammatic access:
# access values manuallyls(.GlobalEnv)#> [1] "bf_legend" "field" "flags" "na_commodity"#> [5] "numeric_yield" "reg".GlobalEnv[["na_commodity"]]#> [1] 0 0 0 1 0 0 0 0 0.GlobalEnv[["numeric_yield"]]#> [1] 11.187500 11.984375 13.226562 4.429688 12.992188 8.546875 11.273438#> [8] 10.640625 9.007812# beware that numeric values are stored with the precision you have specifiedbf_tbl$yield#> [1] 11.192915 11.986793 13.229386 4.431376 12.997422 8.548882 11.276921#> [8] 10.640715 9.010452
- Best Practices: Guidelines foreffective bitfield design, protocol selection, and common pitfalls
- Community Contributions:How to contribute protocols to the community standards repository
- Applications: Examples and use casesusing the bitfield package
- Browse thefunction reference for detaileddocumentation
- Report bugs atgithub.com/bitfloat/bitfield/issues
- Join discussions about community standards atgithub.com/bitfloat/standards
About
Handle Bit-Flags to record Quality for Modeled Data Products
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.