Repository files navigation

The goal ofdataspice is to make it easier for researchers to createbasic, lightweight, and concise metadata files for their datasets byediting the kind of files they’re probably most familiar with: CSVs. Tospice up their data with a dash of metadata. These metadata files canthen be used to:

• Make useful information available during analysis.
• Create a helpful dataset README webpage for your data similar to howpkgdown creates websites for Rpackages.
• Produce more complex metadata formats for richer description of yourdatasets and to aid dataset discovery.

Metadata fields are based onSchema.org/Dataset and othermetadatastandards and represent a lowest common denominator whichmeans converting between formats should be relatively straightforward.

An basic example repository for demonstrating what usingdataspicemight look like can be found athttps://github.com/amoeba/dataspice-example.From there, you can also check out a preview of the HTMLdataspicegenerates athttps://amoeba.github.io/dataspice-exampleand how Google sees it athttps://search.google.com/test/rich-results?url=https%3A%2F%2Famoeba.github.io%2Fdataspice-example%2F.

A much more detailed example has been created byAnnaKrystalli athttps://annakrystalli.me/dataspice-tutorial/ (GitHubrepo).

You can install the latest version fromCRAN:

install.packages("dataspice")

create_spice()# Then fill in template CSV files, more on this belowwrite_spice()build_site()# Optional

create_spice() creates template metadata spreadsheets in a folder (bydefault created in thedata folder in the current working directory).

The template files are:

• biblio.csv - for title, abstract, spatial and temporal coverage,etc.
• creators.csv - for data authors
• attributes.csv - explains each of the variables in the dataset
• access.csv - for files, file types, and download URLs (ifappropriate)

The user needs to fill in the details of the four template files. Thesecsv files can be directly modified, or they can be edited using eitherthe associated helper function and/orShiny app.

• prep_attributes() populates thefileName andvariableName columns of theattributes.csv file using theheader row of the data files.

• prep_access() populates thefileName,name andencodingFormat columns of theaccess.csv file from the filesin the folder containing the data.

To see an example of howprep_attributes() works, load the data filesthat ship with the package:

data_files<- list.files(system.file("example-dataset/",package="dataspice"),pattern=".csv",full.names=TRUE)

This function assumes that the metadata templates are in a folder calledmetadata within adata folder.

attributes_path<- file.path("data","metadata","attributes.csv")

Usingpurrr::map(), this function can be applied over multiple filesto populate the header names

data_files %>%purrr::map(~ prep_attributes(.x,attributes_path),attributes_path=attributes_path  )

The output ofprep_attributes() has the first two columns filled out:

Each of the metadata templates can be edited interactively using aShiny app:

• edit_attributes() opens a Shiny app that can be used to editattributes.csv. The Shiny app displays the currentattributestable and lets the user fill in an informative description and units(e.g. meters, hectares, etc.) for each variable.
• edit_access() opens an editable version ofaccess.csv
• edit_creators() opens an editable version ofcreators.csv
• edit_biblio() opens an editable version ofbiblio.csv

Remember to click onSave when finished editing.

The first few rows of the completed metadata tables in this example willlook like this:

access.csv has one row for each file

attributes.csv has one row for each variable in each file

biblio.csv is one row containing descriptors including spatial andtemporal coverage

creators.csv has one row for each of the dataset authors

write_spice() generates a json-ld file (“linked data”) to aid indatasetdiscovery,creation of more extensive metadata(e.g. EML), and creating a website.

Here’s a view of thedataspice.json file of the example data:

• build_site() creates a bare-bonesindex.html file in therepositorydocs folder with a simple view of the dataset with themetadata and an interactive map. For example, thisrepository resultsin thiswebsite

The metadata fieldsdataspice uses are based largely on theircompatibility with terms fromSchema.org. However,dataspice metadata can be converted to Ecological Metadata Language(EML), a much richer schema. The conversion isn’t perfect butdataspice will do its best to convert yourdataspice metadata toEML:

library(dataspice)# Load an example dataspice JSON that comes installed with the packagespice<- system.file("examples","annual-escapement.json",package="dataspice")# Convert it to EMLeml_doc<- spice_to_eml(spice)#> Warning: variableMeasured not crosswalked to EML because we don't have enough#> information. Use `crosswalk_variables` to create the start of an EML attributes#> table. See ?crosswalk_variables for help.#> You might want to run EML::eml_validate on the result at this point and fix what validations errors are produced. You will commonly need to set `packageId`, `system`, and provide `attributeList` elements for each `dataTable`.

You may receive warnings depending on whichdataspice fields youfilled in and this process will very likely produce an invalid EMLrecord which is totally fine:

library(EML)#>#> Attaching package: 'EML'#> The following object is masked from 'package:magrittr':#>#>     set_attributeseml_validate(eml_doc)#> [1] FALSE#> attr(,"errors")#> [1] "Element '{https://eml.ecoinformatics.org/eml-2.2.0}eml': The attribute 'packageId' is required but missing."#> [2] "Element '{https://eml.ecoinformatics.org/eml-2.2.0}eml': The attribute 'system' is required but missing."#> [3] "Element 'dataTable': Missing child element(s). Expected is one of ( physical, coverage, methods, additionalInfo, annotation, attributeList )."#> [4] "Element 'dataTable': Missing child element(s). Expected is one of ( physical, coverage, methods, additionalInfo, annotation, attributeList )."#> [5] "Element 'dataTable': Missing child element(s). Expected is one of ( physical, coverage, methods, additionalInfo, annotation, attributeList )."

This is because some fields indataspice store information indifferent structures and because EML requires many fields thatdataspice doesn’t have fields for. At this point, you should look overthe validation errors produced byEML::eml_validate and fix those.Note that this will likely require familiarity with theEMLSchema and theEMLpackage.

Once you’re done, you can write out an EML XML file:

out_path<- tempfile()write_eml(eml_doc,out_path)#> NULL

Like convertingdataspice to EML, we can convert an existing EMLrecord to a set ofdataspice metadata tables which we can then workfrom withindataspice:

library(EML)eml_path<- system.file("example-dataset/broodTable_metadata.xml",package="dataspice")eml<- read_eml(eml_path)

# Creates four CSVs files in the `data/metadata` directorymy_spice<- eml_to_spice(eml,"data/metadata")

A few existing tools & data standards to help users in specific domains:

• Darwin Core
• Ecological MetadataLanguage(EML) (&EML)
• ISO 19115 - GeographicInformation Metadata
• ISO 19139 - GeographicInfo Metadata XML schema
• Minimum Information for Biological and BiomedicalInvestigations (MIBBI)

…And others indexed inFairsharing.org & theRDA metadatadirectory.

Please note that this package is released with aContributor Code ofConduct. By contributing to thisproject, you agree to abide by its terms.

This package was developed at rOpenSci’s 2018 unconf by (in alphabeticalorder):

• Carl Boettiger
• Scott Chamberlain
• Auriel Fournier
• Kelly Hondula
• Anna Krystalli
• Bryce Mecum
• Maëlle Salmon
• Kate Webbink
• Kara Woo
• Irene Steves