Repository files navigation

Tools to convertESA CCI SM image files into a time series format.

This python package and all required dependencies can be installed frompypiviapip

pip install esa_cci_sm

On macOS if you getImportError: Pykdtree failed to import its C extension,then it might be necessary to install the pykdtree package from conda-forge

conda install -c conda-forge pykdtree

Download ESA CCI SM data files either from the publicCEDA data store via httpsor the CEDA FTP server using, e.g.FileZillaorwget

Host (FTP):anon-ftp.ceda.ac.uk no password or user required

Directory (FTP):/neodc/esacci/soil_moisture)

E.g the following command will download v9.1 COMBINED data for the year 2023 from the CEDA FTP via wget

wget -np -nH --cut-dirs 7 -r ftp://anon-ftp.ceda.ac.uk/neodc/esacci/soil_moisture/data/daily_files/COMBINED/v09.1/2023/

After installing this package via pip, you have access to the command line tool to convert ESA CCI SM image filesto CF conform time series. We use an Orthogonal multidimensional array representationas implemented in thepynetCF python library.

Note that we assume that the downloaded images are stored in yearly subfolderslike

/tmp/img/├── 1978/│   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781101000000-fv09.1.nc│   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781102000000-fv09.1.nc│   ├── ......├── 2023/│   ├── ...│   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-20231231000000-fv09.1.nc

The following command would then take the daily images from 1991to 2023 in the path/tmp/img and convert data for grid cells over land into times series.Time series are then stored in/tmp/ts.

ccism_reshuffle /tmp/img /tmp/ts 1991-01-01 2023-12-31 --land_points True

Afterwards, in python, the data can be read as pandas DataFrames.

>>fromesa_cci_sm.interfaceimportCCITs>>ds=CCITs("/tmp/ts",ioclass_kws={'read_bulk':True})>>ds.read(15,45)# lon, latsmsm_uncertaintyflag  ...modesensort01991-01-010.4248800.0945070  ...127670.1750001991-01-02NaNNaN24  ...22NaN1991-01-03NaNNaN8  ...02NaN...              ...             ...   ...  ...   ...     ...           ...2023-12-290.4954480.0399830  ...32153619720.0515752023-12-300.4261070.0550600  ...31641619721.1470662023-12-310.3901030.0302940  ...32160019722.117129

At the moment this package supports ESA CCI soil moisture data versions3 to 9 in netCDF format (reading and time series creation)with a spatial sampling of 0.25 degrees.

We are happy if you want to contribute. Please raise an issue explaining whatis missing or if you find a bug. We will also gladly accept pull requestsagainst our master branch for new features or bug fixes.

Setup of a complete development environment withconda can be performed using the followingcommands:

git clone git@github.com:TUW-GEO/esa_cci_sm.git --recursive esa_cci_smcd ./esa_cci_smconda create -n esa_cci_sm python=3.12conda activate esa_cci_smpip install -e .[testing]

To checkout our testdata files, you need to haveGit LFS installed on your machine.

If you want to contribute please follow these steps:

• Fork the esa_cci_sm repository to your account
• Clone the repository, make sure you usegit clone ... --recursive to also getthe test data repository.
• make a new feature branch from the esa_cci_sm master branch
• Add your feature
• Please include tests for your contributions in one of the test directories.We use pytest so a simple function called test_my_feature is enough
• submit a pull request to our master branch