Movatterモバイル変換

davemlz/eemontPublic

NotificationsYou must be signed in to change notification settings
Fork68
Star425

A python package that extends Google Earth Engine.

eemont.readthedocs.io/

License

MIT license

425 stars 68 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 1,035 Commits
.github		.github
docs		docs
eemont		eemont
paper		paper
tests		tests
.gitignore		.gitignore
CITATION.cff		CITATION.cff
CONTRIBUTING.rst		CONTRIBUTING.rst
LICENSE		LICENSE
README.md		README.md
readthedocs.yml		readthedocs.yml
requirements.txt		requirements.txt
setup.py		setup.py
tox.ini		tox.ini

Repository files navigation

A python package that extends Google Earth Engine

GitHub:https://github.com/davemlz/eemont

Documentation:https://eemont.readthedocs.io/

PyPI:https://pypi.org/project/eemont/

Conda-forge:https://anaconda.org/conda-forge/eemont

Tutorials:https://github.com/davemlz/eemont/tree/master/docs/tutorials

Paper:https://joss.theoj.org/papers/10.21105/joss.03168

Overview

Google Earth Engine is a cloud-based service forgeospatial processing of vector and raster data. The Earth Engine platform has aJavaScript and a Python API withdifferent methods to process geospatial objects. Google Earth Engine also provides aHUGE PETABYTE-SCALE CATALOG ofraster and vector data that users can process online (e.g. Landsat Missions ImageCollections, Sentinel Missions Image Collections, MODIS Products Image Collections, WorldDatabase of Protected Areas, etc.). The eemont package extends theGoogle Earth Engine Python APIwith pre-processing and processing tools for the most used satellite platforms by addingutility methods for differentEarth Engine Objectsthat are friendly with the Python method chaining.

Google Earth Engine Community: Developer Resources

The eemont Python package can be found in theEarth Engine Community: Developer Resourcestogether with other awesome resources such asgeemap andrgee.

How does it work?

The eemont python package extends the following Earth Engine classes:

New utility methods and constructors are added to above-mentioned classes in orderto create a more fluid code by being friendly with the Python method chaining. Thesemethods are mandatory for some pre-processing and processing tasks (e.g. clouds masking,shadows masking, image scaling, spectral indices computation, etc.), and they arepresented as simple functions that give researchers, students and analysts the chance toanalyze data with far fewer lines of code.

Look at this simple example where aSentinel-2 Surface Reflectance Image Collectionis pre-processed and processed in just one step:

importee,eemontee.Authenticate()ee.Initialize()point=ee.Geometry.PointFromQuery('Cali, Colombia',user_agent='eemont-example')# Extended constructorS2= (ee.ImageCollection('COPERNICUS/S2_SR')    .filterBounds(point)    .closest('2020-10-15')# Extended (pre-processing)    .maskClouds(prob=70)# Extended (pre-processing)    .scaleAndOffset()# Extended (pre-processing)    .spectralIndices(['NDVI','NDWI','BAIS2']))# Extended (processing)

And just like that, the collection was pre-processed, processed and ready to be analyzed!

Installation

Install the latest version from PyPI:

pip install eemont

Upgradeeemont by running:

pip install -U eemont

Install the latest version from conda-forge:

conda install -c conda-forge eemont

Install the latest dev version from GitHub by running:

pip install git+https://github.com/davemlz/eemont

Features

Let's see some of the main features of eemont and how simple they are compared to the GEEPython API original methods:

Overloaded Operators

The following operators are overloaded: +, -, *, /, //, %, **\ , <<, >>, &, |, <, <=,==, !=, >, >=, -, ~. (and you can avoid theee.Image.expression() method!)

GEE Python API	eemont-style
ds='COPERNICUS/S2_SR'S2= (ee.ImageCollection(ds).first())defscaleImage(img):scaling=img.select('B.')x=scaling.multiply(0.0001)scaling=img.select(['AOT','WVP'])scaling=scaling.multiply(0.001)x=x.addBands(scaling)notScaling=img.select(['SCL','TCI.','MSK.','QA.' ]))returnx.addBands(notScaling)S2=scaleImage(S2)exp='2.5(N-R)/(N+(6R)-(7.5*B)+1)'imgDict= {'N':S2.select('B8'),'R':S2.select('B4'),'B':S2.select('B2')}EVI=S2.expression(exp,imgDict)	ds='COPERNICUS/S2_SR'S2= (ee.ImageCollection(ds).first().scale())N=S2.select('B8')R=S2.select('B4')B=S2.select('B2')EVI=2.5(N-R)/(N+(6R)-(7.5*B)+1)

GEE Python API

eemont-style

ds='COPERNICUS/S2_SR'S2= (ee.ImageCollection(ds).first())defscaleImage(img):scaling=img.select('B.*')x=scaling.multiply(0.0001)scaling=img.select(['AOT','WVP'])scaling=scaling.multiply(0.001)x=x.addBands(scaling)notScaling=img.select(['SCL','TCI.*','MSK.*','QA.*'    ]))returnx.addBands(notScaling)S2=scaleImage(S2)exp='2.5*(N-R)/(N+(6*R)-(7.5*B)+1)'imgDict= {'N':S2.select('B8'),'R':S2.select('B4'),'B':S2.select('B2')}EVI=S2.expression(exp,imgDict)

ds='COPERNICUS/S2_SR'S2= (ee.ImageCollection(ds).first().scale())N=S2.select('B8')R=S2.select('B4')B=S2.select('B2')EVI=2.5*(N-R)/(N+(6*R)-(7.5*B)+1)

Clouds and Shadows Masking

Masking clouds and shadows can be done using eemont with just one method:maskClouds()!

GEE Python API	eemont-style
ds='LANDSAT/LC08/C01/T1_SR'defmaskCloudsShadows(img):c= (1<<3)s= (1<<5)qa='pixel_qa'qa=img.select(qa)cm=qa.bitwiseAnd(c).eq(0)sm=qa.bitwiseAnd(s).eq(0)mask=cm.And(sm)returnimg.updateMask(mask) (ee.ImageCollection(ds) .map(maskCloudsShadows))	ds='LANDSAT/LC08/C01/T1_SR' (ee.ImageCollection(ds) .maskClouds())

GEE Python API

eemont-style

ds='LANDSAT/LC08/C01/T1_SR'defmaskCloudsShadows(img):c= (1<<3)s= (1<<5)qa='pixel_qa'qa=img.select(qa)cm=qa.bitwiseAnd(c).eq(0)sm=qa.bitwiseAnd(s).eq(0)mask=cm.And(sm)returnimg.updateMask(mask)    (ee.ImageCollection(ds)    .map(maskCloudsShadows))

ds='LANDSAT/LC08/C01/T1_SR'          (ee.ImageCollection(ds)    .maskClouds())

Image Scaling and Offsetting

Scaling and offsetting can also be done using eemont with just one method:scale()!

GEE Python API	eemont-style
defscaleBands(img):scaling=img.select(['NDVI','EVI','sur.' ])x=scaling.multiply(0.0001)scaling=img.select('.th')scaling=scaling.multiply(0.01)x=x.addBands(scaling)notScaling=img.select(['DetailedQA','DayOfYear','SummaryQA' ])returnx.addBands(notScaling)ds='MODIS/006/MOD13Q1'(ee.ImageCollection(ds) .map(scaleBands))	ds='MODIS/006/MOD13Q1' (ee.ImageCollection(ds) .scaleAndOffset())

GEE Python API

eemont-style

defscaleBands(img):scaling=img.select(['NDVI','EVI','sur.*'    ])x=scaling.multiply(0.0001)scaling=img.select('.*th')scaling=scaling.multiply(0.01)x=x.addBands(scaling)notScaling=img.select(['DetailedQA','DayOfYear','SummaryQA'    ])returnx.addBands(notScaling)ds='MODIS/006/MOD13Q1'(ee.ImageCollection(ds)    .map(scaleBands))

ds='MODIS/006/MOD13Q1'          (ee.ImageCollection(ds)    .scaleAndOffset())

Complete Preprocessing

The complete preprocessing workflow (Masking clouds and shadows, and image scaling andoffsetting) can be done using eemont with just one method:preprocess()!

GEE Python API	eemont-style
ds='LANDSAT/LC08/C01/T1_SR'defmaskCloudsShadows(img):c= (1<<3)s= (1<<5)qa='pixel_qa'qa=img.select(qa)cm=qa.bitwiseAnd(c).eq(0)sm=qa.bitwiseAnd(s).eq(0)mask=cm.And(sm)returnimg.updateMask(mask)defscaleBands(img):scaling=img.select('B[1-7]')x=scaling.multiply(0.0001)scaling=img.select(['B10','B11' ])scaling=scaling.multiply(0.1)x=x.addBands(scaling)notScaling=img.select(['sr_aerosol','pixel_qa','radsat_qa' ])returnx.addBands(notScaling) (ee.ImageCollection(ds) .map(maskCloudsShadows) .map(scaleBands))	ds='LANDSAT/LC08/C01/T1_SR' (ee.ImageCollection(ds) .preprocess())

GEE Python API

eemont-style

ds='LANDSAT/LC08/C01/T1_SR'defmaskCloudsShadows(img):c= (1<<3)s= (1<<5)qa='pixel_qa'qa=img.select(qa)cm=qa.bitwiseAnd(c).eq(0)sm=qa.bitwiseAnd(s).eq(0)mask=cm.And(sm)returnimg.updateMask(mask)defscaleBands(img):scaling=img.select('B[1-7]')x=scaling.multiply(0.0001)scaling=img.select(['B10','B11'    ])scaling=scaling.multiply(0.1)x=x.addBands(scaling)notScaling=img.select(['sr_aerosol','pixel_qa','radsat_qa'    ])returnx.addBands(notScaling)    (ee.ImageCollection(ds)    .map(maskCloudsShadows)    .map(scaleBands))

ds='LANDSAT/LC08/C01/T1_SR'          (ee.ImageCollection(ds)    .preprocess())

Spectral Indices

Do you need to compute several spectral indices? Use thespectralIndices() method! Theindices are taken fromAwesome Spectral Indices.

GEE Python API	eemont-style
ds='LANDSAT/LC08/C01/T1_SR'defscaleImage(img):scaling=img.select('B[1-7]')x=scaling.multiply(0.0001)scaling=img.select(['B10','B11'])scaling=scaling.multiply(0.1)x=x.addBands(scaling)notScaling=img.select(['sr_aerosol','pixel_qa','radsat_qa' ]))returnx.addBands(notScaling)defaddIndices(img):x= ['B5','B4']a=img.normalizedDifference(x)a=a.rename('NDVI')x= ['B5','B3']b=img.normalizedDifference(x)b=b.rename('GNDVI')x= ['B3','B6']c=img.normalizedDifference(x)c=b.rename('NDSI')returnimg.addBands([a,b,c]) (ee.ImageCollection(ds) .map(scaleImage) .map(addIndices))	ds='LANDSAT/LC08/C01/T1_SR' (ee.ImageCollection(ds) .scaleAndOffset() .spectralIndices(['NDVI','GNDVI','NDSI']))

GEE Python API

eemont-style

ds='LANDSAT/LC08/C01/T1_SR'defscaleImage(img):scaling=img.select('B[1-7]')x=scaling.multiply(0.0001)scaling=img.select(['B10','B11'])scaling=scaling.multiply(0.1)x=x.addBands(scaling)notScaling=img.select(['sr_aerosol','pixel_qa','radsat_qa'    ]))returnx.addBands(notScaling)defaddIndices(img):x= ['B5','B4']a=img.normalizedDifference(x)a=a.rename('NDVI')x= ['B5','B3']b=img.normalizedDifference(x)b=b.rename('GNDVI')x= ['B3','B6']c=img.normalizedDifference(x)c=b.rename('NDSI')returnimg.addBands([a,b,c])                    (ee.ImageCollection(ds)    .map(scaleImage)    .map(addIndices))

ds='LANDSAT/LC08/C01/T1_SR'          (ee.ImageCollection(ds)    .scaleAndOffset()    .spectralIndices(['NDVI','GNDVI','NDSI']))

The list of available indices can be retrieved by running:

eemont.listIndices()

Information about the indices can also be checked:

indices=eemont.indices()indices.BAIS2.formulaindices.BAIS2.reference

Closest Image to a Specific Date

Struggling to get the closest image to a specific date? Here is the solution: theclosest() method!

GEE Python API	eemont-style
ds='COPERNICUS/S5P/OFFL/L3_NO2'xy= [-76.21,3.45]poi=ee.Geometry.Point(xy)date=ee.Date('2020-10-15')date=date.millis()defsetTimeDelta(img):prop='system:time_start'prop=img.get(prop)prop=ee.Number(prop)delta=prop.subtract(date)delta=delta.abs()returnimg.set('dateDist',delta) (ee.ImageCollection(ds) .filterBounds(poi) .map(setTimeDelta) .sort('dateDist') .first())	ds='COPERNICUS/S5P/OFFL/L3_NO2'xy= [-76.21,3.45]poi=ee.Geometry.Point(xy)(ee.ImageCollection(ds) .filterBounds(poi) .closest('2020-10-15'))

GEE Python API

eemont-style

ds='COPERNICUS/S5P/OFFL/L3_NO2'xy= [-76.21,3.45]poi=ee.Geometry.Point(xy)date=ee.Date('2020-10-15')date=date.millis()defsetTimeDelta(img):prop='system:time_start'prop=img.get(prop)prop=ee.Number(prop)delta=prop.subtract(date)delta=delta.abs()returnimg.set('dateDist',delta)                     (ee.ImageCollection(ds)    .filterBounds(poi)    .map(setTimeDelta)    .sort('dateDist')    .first())

ds='COPERNICUS/S5P/OFFL/L3_NO2'xy= [-76.21,3.45]poi=ee.Geometry.Point(xy)(ee.ImageCollection(ds)    .filterBounds(poi)    .closest('2020-10-15'))

Time Series By Regions

The JavaScript API has a method for time series extraction (included in theui.Chartmodule), but this method is missing in the Python API... so, here it is!

PD: Actually, there are two methods that you can use:getTimeSeriesByRegion() andgetTimeSeriesByRegions()!

f1=ee.Feature(ee.Geometry.Point([3.984770,48.767221]).buffer(50),{'ID':'A'})f2=ee.Feature(ee.Geometry.Point([4.101367,48.748076]).buffer(50),{'ID':'B'})fc=ee.FeatureCollection([f1,f2])S2= (ee.ImageCollection('COPERNICUS/S2_SR')    .filterBounds(fc)    .filterDate('2020-01-01','2021-01-01')    .maskClouds()    .scaleAndOffset()    .spectralIndices(['EVI','NDVI']))# By Regionts=S2.getTimeSeriesByRegion(reducer= [ee.Reducer.mean(),ee.Reducer.median()],geometry=fc,bands= ['EVI','NDVI'],scale=10)# By Regionsts=S2.getTimeSeriesByRegions(reducer= [ee.Reducer.mean(),ee.Reducer.median()],collection=fc,bands= ['EVI','NDVI'],scale=10)

Constructors by Queries

Don't you have the coordinates of a place? You can construct them by using queries!

usr='my-eemont-query-example'seattle_bbox=ee.Geometry.BBoxFromQuery('Seattle',user_agent=usr)cali_coords=ee.Feature.PointFromQuery('Cali, Colombia',user_agent=usr)amazonas_river=ee.FeatureCollection.MultiPointFromQuery('Río Amazonas',user_agent=usr)

JavaScript Modules

This is perhaps the most important feature ineeExtra! What if you could use aJavaScript module (originally just useful for the Code Editor) in python or R? Well,wait no more for it!

JS (Code Editor)	Python (eemont)	R (rgee+)
varusr='users/sofiaermida/'varrep='landsat_smw_lst:'varfld='modules/'varfle='Landsat_LST.js'varpth=usr+rep+fld+flevarmod=require(pth)varLST=mod.collection(ee.Geometry.Rectangle([-8.91,40.0,-8.3,40.4]),'L8','2018-05-15','2018-05-31',true)	importee,eemontee.Initialize()usr='users/sofiaermida/'rep='landsat_smw_lst:'fld='modules/'fle='Landsat_LST.js'pth=usr+rep+fld+fleee.install(pth)mod=ee.require(pth)LST=mod.collection(ee.Geometry.Rectangle([-8.91,40.0,-8.3,40.4 ]),'L8','2018-05-15','2018-05-31',True)	library(rgee)library(rgeeExtra)ee_Initialize()usr<-'users/sofiaermida/'rep<-'landsat_smw_lst:'fld<-'modules/'fle<-'Landsat_LST.js'pth<- paste0(usr,rep,fld,fle)mod<-ee$require(pth)LST=mod$collection(ee$Geometry$Rectangle(c(-8.91,40.0,-8.3,40.4 )),'L8','2018-05-15','2018-05-31',TRUE)

JS (Code Editor)

Python (eemont)

R (rgee+)

varusr='users/sofiaermida/'varrep='landsat_smw_lst:'varfld='modules/'varfle='Landsat_LST.js'varpth=usr+rep+fld+flevarmod=require(pth)varLST=mod.collection(ee.Geometry.Rectangle([-8.91,40.0,-8.3,40.4]),'L8','2018-05-15','2018-05-31',true)

importee,eemontee.Initialize()usr='users/sofiaermida/'rep='landsat_smw_lst:'fld='modules/'fle='Landsat_LST.js'pth=usr+rep+fld+fleee.install(pth)mod=ee.require(pth)LST=mod.collection(ee.Geometry.Rectangle([-8.91,40.0,-8.3,40.4    ]),'L8','2018-05-15','2018-05-31',True)

library(rgee)library(rgeeExtra)ee_Initialize()usr<-'users/sofiaermida/'rep<-'landsat_smw_lst:'fld<-'modules/'fle<-'Landsat_LST.js'pth<- paste0(usr,rep,fld,fle)mod<-ee$require(pth)LST=mod$collection(ee$Geometry$Rectangle(c(-8.91,40.0,-8.3,40.4    )),'L8','2018-05-15','2018-05-31',TRUE)

License

The project is licensed under the MIT license.

How to cite

Do you like using eemont and think it is useful? Share the love by citing it!:

Montero, D., (2021). eemont: A Python package that extends Google Earth Engine. Journal of Open Source Software, 6(62), 3168, https://doi.org/10.21105/joss.03168

If required, here is the BibTex!:

@article{Montero2021,    doi = {10.21105/joss.03168},    url = {https://doi.org/10.21105/joss.03168},    year = {2021},    publisher = {The Open Journal},    volume = {6},    number = {62},    pages = {3168},    author = {David Montero},    title = {eemont: A Python package that extends Google Earth Engine},    journal = {Journal of Open Source Software}}

Artists

David Montero Loaiza: Lead Developer of eemont and eeExtra.
César Aybar: Lead Developer of rgee and eeExtra.
Aaron Zuspan: Plus Codes Constructors and Methods, Panchromatic Sharpening and Histogram Matching Developer.

Credits

Special thanks toJustin Braaten for featuring eemont intutorials and the GEE Community: Developer Resources Page, toCésar Aybar for the formidable help with Awesome SpectralIndices and to the JOSS Review Team (Katy Barnhart,Jayaram Hariharan,Qiusheng WuandPatrick Gray) for the comments, suggestions and contributions!