This vignette provides a brief overview of the most importantfunctions ineia. Other vignettes go into greater depth onspecific topics and API endpoints.
Obtaining an API key is easy, and free!
Pulling data from the US Energy Information Administration (EIA) APIrequires a registered API key. A key can be obtained at no costhere. A valid emailand agreement to the API Terms of Service is required to obtain akey.
It is important to store the API key somewhere secure. Do not commitit to a repository or otherwise share it. For example, one option is tostore it in the.Renviron file.
While thekey argument can be provided to every APIfunction call, it is not necessary. There are “get” and “set” helpersavailable to make usingeia functions a more seamlessexperience.
eia_set_key() provides the option of storing the key forthe duration of the R session.
If the key already exists in the system environment and the plan isto passkey to functions explicitly, then start asfollows.
In general, however, if the key is set globally - such as in.Renviron - then no further action regarding the key isrequired when using the package. See the vignette on API details formore information about all the available options for key storage.
Once the fully registered EIA API key is obtained and properly storedin the active R session, by whichever method preferred, the ability toexplore the API directory (folder structure) and obtain the data withinis set.
Note, the EIA’s APIv2 has been redesigned to be more human-readablefriendly; and, rather than rely on numeric ID values, it is now builtaround a self-searchable folder structure where the ID values are nowjust natural language, i.e. words.
Here is the top-level directory information:
eia_dir()#> # A tibble: 14 × 3#> id name description#> <chr> <chr> <chr>#> 1 coal Coal EIA coal energy data#> 2 crude-oil-imports Crude Oil Imports Crude oil imports by country to destination, includes type, grade, quantity. Sour…#> 3 electricity Electricity EIA electricity survey data#> 4 international International Country level production, consumption, imports, exports by energy source (petroleu…#> 5 natural-gas Natural Gas EIA natural gas survey data#> 6 nuclear-outages Nuclear Outages EIA nuclear outages survey data#> 7 petroleum Petroleum EIA petroleum gas survey data#> 8 seds State Energy Data System (SEDS) Estimated production, consumption, price, and expenditure data for all energy sour…#> 9 steo Short Term Energy Outlook Monthly short term (18 month) projections using STEO model. Report and interactiv…#> 10 densified-biomass Densified Biomass EIA densified biomass data#> 11 total-energy Total Energy These data represent the most recent comprehensive energy statistics integrated ac…#> 12 aeo Annual Energy Outlook Annual U.S. projections using National Energy Modelling System (NEMS) for release …#> 13 ieo International Energy Outlook Annual international projections using the World Energy Projection System (WEPS) m…#> 14 co2-emissions State CO2 Emissions EIA CO2 Emissions dataNavigate deeper into this directory by supplying a folder id (e.g. id= “electricity”).
eia_dir("electricity")#> # A tibble: 6 × 3#> id name description#> <chr> <chr> <chr>#> 1 retail-sales Electricity Sales to Ultimate Customers "Electricity sales to ulti…#> 2 electric-power-operational-data Electric Power Operations (Annual and Monthly) "Monthly and annual electr…#> 3 rto Electric Power Operations (Daily and Hourly) "Hourly and daily electric…#> 4 state-electricity-profiles State Specific Data "State Specific Data"#> 5 operating-generator-capacity Inventory of Operable Generators "Inventory of operable gen…#> 6 facility-fuel Electric Power Operations for Individual Power Plants (Annual and Monthly) "Annual and monthly electr…And, because APIv2 is a directory listing, to go any deeper simplyrequires appending the next id (folder name) separated by"/", where the first folder ID in from the top-leveldirectory is “electricity” and the next deeper folder ID from there is“retail-sales”. As a result, supply “electricity/retail-sales” to thefirst argument ineia_dir(), as shown below, to see thenext next layer in.
Finally, the end of the directory path has been reached, as shown bythe message provided in the console output above; i.e. there are no moresub-folders to explore. This message is prompting to explore theavailable data at the end of this directory path witheia_metadata().
The default is to return tidy data in a tibble data frame. Foreia_dir(), settidy = FALSE to return a listas returned byjsonlite::fromJSON. Additionally, settidy = NA to return a raw character string of the JSONpayload.
eia_dir(tidy =FALSE)#> $response#> $response$id#> [1] ""#>#> $response$name#> [1] ""#>#> $response$description#> [1] ""#>#> $response$routes#> id name#> 1 coal Coal#> 2 crude-oil-imports Crude Oil Imports#> 3 electricity Electricity#> 4 international International#> 5 natural-gas Natural Gas#> 6 nuclear-outages Nuclear Outages#> 7 petroleum Petroleum#> 8 seds State Energy Data System (SEDS)#> 9 steo Short Term Energy Outlook#> 10 densified-biomass Densified Biomass#> 11 total-energy Total Energy#> 12 aeo Annual Energy Outlook#> 13 ieo International Energy Outlook#> 14 co2-emissions State CO2 Emissions#> description#> 1 EIA coal energy data#> 2 Crude oil imports by country to destination, \r\n includes type, grade, quantity. Source: EIA-814 Interactive data \r\n product: www.eia.gov/petroleum/imports/companylevel/#> 3 EIA electricity survey data#> 4 Country level production, consumption, imports, exports by energy source (petroleum, natural gas, electricity, renewable, etc.) \r\n Interactive product: https://www.eia.gov/international/data/world#> 5 EIA natural gas survey data#> 6 EIA nuclear outages survey data#> 7 EIA petroleum gas survey data#> 8 Estimated production, consumption, price, and expenditure data for all energy sources by state and sector. \r\n Source: https://www.eia.gov/state/seds/seds-technical-notes-complete.php \r\n Product: SEDS (https://www.eia.gov/state/seds/)#> 9 Monthly short term (18 month) projections using STEO model. \r\n Report and interactive projection data browser: STEO (www.eia.gov/steo/)#> 10 EIA densified biomass data#> 11 These data represent the most recent comprehensive energy statistics integrated across all energy sources. The data includes total energy production, consumption, stocks, and trade; energy prices; overviews of petroleum, natural gas, coal, electricity, nuclear energy, renewable energy, and carbon dioxide emissions; and data unit conversions values. Source: https://www.eia.gov/totalenergy/data/monthly/pdf/mer_a_doc.pdf Report: MER (https://www.eia.gov/totalenergy/data/monthly/)#> 12 Annual U.S. projections using National Energy Modelling System (NEMS) for release year. Report, documentation, and interactive projection data browser: AEO (www.eia.gov/aeo/)#> 13 Annual international projections using the World Energy Projection System (WEPS) model for release year. Report and interactive projection data browser: IEO (www.eia.gov/ieo/)#> 14 EIA CO2 Emissions data#>#>#> $request#> $request$command#> [1] "/v2/"#>#> $request$params#> $request$params$api_key#> [1] "dUlpeaO4fNAmo9nJcgIhtCfBN5VtiZnoZDzuYGuG"#>#>#>#> $apiVersion#> [1] "2.1.4"eia_dir(tidy =NA)#> [1] "{\"response\":{\"id\":\"\",\"name\":\"\",\"description\":\"\",\"routes\":[{\"id\":\"coal\",\"name\":\"Coal\",\"description\":\"EIA coal energy data\"},{\"id\":\"crude-oil-imports\",\"name\":\"Crude Oil Imports\",\"description\":\"Crude oil imports by country to destination, \\r\\n includes type, grade, quantity. Source: EIA-814 Interactive data \\r\\n product: www.eia.gov\\/petroleum\\/imports\\/companylevel\\/\"},{\"id\":\"electricity\",\"name\":\"Electricity\",\"description\":\"EIA electricity survey data\"},{\"id\":\"international\",\"name\":\"International\",\"description\":\"Country level production, consumption, imports, exports by energy source (petroleum, natural gas, electricity, renewable, etc.) \\r\\n Interactive product: https:\\/\\/www.eia.gov\\/international\\/data\\/world\"},{\"id\":\"natural-gas\",\"name\":\"Natural Gas\",\"description\":\"EIA natural gas survey data\"},{\"id\":\"nuclear-outages\",\"name\":\"Nuclear Outages\",\"description\":\"EIA nuclear outages survey data\"},{\"id\":\"petroleum\",\"name\":\"Petroleum\",\"description\":\"EIA petroleum gas survey data\"},{\"id\":\"seds\",\"name\":\"State Energy Data System (SEDS)\",\"description\":\"Estimated production, consumption, price, and expenditure data for all energy sources by state and sector. \\r\\n Source: https:\\/\\/www.eia.gov\\/state\\/seds\\/seds-technical-notes-complete.php \\r\\n Product: SEDS (https:\\/\\/www.eia.gov\\/state\\/seds\\/)\"},{\"id\":\"steo\",\"name\":\"Short Term Energy Outlook\",\"description\":\"Monthly short term (18 month) projections using STEO model. \\r\\n Report and interactive projection data browser: STEO (www.eia.gov\\/steo\\/)\"},{\"id\":\"densified-biomass\",\"name\":\"Densified Biomass\",\"description\":\"EIA densified biomass data\"},{\"id\":\"total-energy\",\"name\":\"Total Energy\",\"description\":\"These data represent the most recent comprehensive energy statistics integrated across all energy sources. The data includes total energy production, consumption, stocks, and trade; energy prices; overviews of petroleum, natural gas, coal, electricity, nuclear energy, renewable energy, and carbon dioxide emissions; and data unit conversions values. Source: https:\\/\\/www.eia.gov\\/totalenergy\\/data\\/monthly\\/pdf\\/mer_a_doc.pdf Report: MER (https:\\/\\/www.eia.gov\\/totalenergy\\/data\\/monthly\\/)\"},{\"id\":\"aeo\",\"name\":\"Annual Energy Outlook\",\"description\":\"Annual U.S. projections using National Energy Modelling System (NEMS) for release year. Report, documentation, and interactive projection data browser: AEO (www.eia.gov\\/aeo\\/)\"},{\"id\":\"ieo\",\"name\":\"International Energy Outlook\",\"description\":\"Annual international projections using the World Energy Projection System (WEPS) model for release year. Report and interactive projection data browser: IEO (www.eia.gov\\/ieo\\/)\"},{\"id\":\"co2-emissions\",\"name\":\"State CO2 Emissions\",\"description\":\"EIA CO2 Emissions data\"}]},\"request\":{\"command\":\"\\/v2\\/\",\"params\":{\"api_key\":\"dUlpeaO4fNAmo9nJcgIhtCfBN5VtiZnoZDzuYGuG\"}},\"apiVersion\":\"2.1.4\"}"Now back to the example regarding the Retail Sales ofElectricity…
eia_metadata() provides as the name suggests: metadataon a given set of data provided by a terminal (i.e. final) directorypath. This includes the following:
eia_metadata("electricity/retail-sales")#> $Name#> [1] "Electricity Sales to Ultimate Customers"#>#> $Description#> [1] "Electricity sales to ultimate customer by state and sector (number of customers, average price, revenue, and megawatthours of sales). \n Sources: Forms EIA-826, EIA-861, EIA-861M"#>#> $Data#> # A tibble: 4 × 3#> id alias units#> <chr> <chr> <chr>#> 1 revenue Revenue from Sales to Ultimate Customers million dollars#> 2 sales Megawatthours Sold to Utlimate Customers million kilowatthours#> 3 price Average Price of Electricity to Utlimate Customers cents per kilowatthour#> 4 customers Number of Ultimate Customers number of customers#>#> $Facets#> # A tibble: 2 × 2#> id description#> <chr> <chr>#> 1 stateid State / Census Region#> 2 sectorid Sector#>#> $Frequency#> # A tibble: 3 × 4#> id description query format#> <chr> <chr> <chr> <chr>#> 1 monthly One data point for each month. M YYYY-MM#> 2 quarterly One data point every 3 months. Q YYYY-QQ#> 3 annual One data point for each calendar year. A YYYY#>#> $Defaults#> # A tibble: 1 × 2#> format frequency#> <chr> <chr>#> 1 YYYY-MM monthly#>#> $Period#> # A tibble: 1 × 2#> start end#> <chr> <chr>#> 1 2001-01 2023-08The metadata from above can be used to have a better understanding ofwhat data is available and how that data can be pulled from the APIusingeia_data(). However, simply supplying the abovedirectory path from above (“electricity/retail-sales”) will only providethe data structure, i.e. no meaningful data values will be present (moreon this below).
eia_data("electricity/retail-sales")|>head()#> # A tibble: 6 × 5#> period stateid stateDescription sectorid sectorName#> <chr> <chr> <chr> <chr> <chr>#> 1 2008-07 WSC West South Central IND industrial#> 2 2008-07 WSC West South Central OTH other#> 3 2008-07 WSC West South Central RES residential#> 4 2008-07 WSC West South Central TRA transportation#> 5 2008-07 MTN Mountain ALL all sectors#> 6 2008-07 MTN Mountain COM commercialNote the console warning that appears. The API can only provide amaximum of 5000 records. If there are more data available than returned,a warning message like above will be provided informing the user oftheir “incomplete return”.
“Data Values” are just the names of the columns that hold thespecific, respective data value. For this example, the overall number ofpower units sold to customers is of main interest, and the associatedcolumn name for this information is"sales".
To get data values, e.g. sales, supply the proper column name id, asprovided in the Data Values section of theeia_metadata()console output, again, in this case"sales".
eia_data("electricity/retail-sales",data ="sales",length =6)# length = 6 instead of `|> head()`#> # A tibble: 6 × 7#> period stateid stateDescription sectorid sectorName sales `sales-units`#> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>#> 1 2008-07 WSC West South Central IND industrial 14800. million kilowatthours#> 2 2008-07 WSC West South Central OTH other NA million kilowatthours#> 3 2008-07 WSC West South Central RES residential 22406. million kilowatthours#> 4 2008-07 WSC West South Central TRA transportation 7.41 million kilowatthours#> 5 2008-07 MTN Mountain ALL all sectors 27756. million kilowatthours#> 6 2008-07 MTN Mountain COM commercial 9123. million kilowatthoursSee now that sales data values have been right-column-bound to thedata structure as originally provided byeia_data("electricity/retail-sales").
But what if data is wanted for just one sector, say “residential”?This is where facets arrive, and for any given terminal directory, thefacet values can be found witheia_facets() by supplyingthe terminal directory path and facet id, the latter of which can,again, be found from the console output ofeia_metadata().
eia_facets("electricity/retail-sales","sectorid")#> # A tibble: 6 × 3#> id name alias#> <chr> <chr> <chr>#> 1 RES residential (RES) residential#> 2 COM commercial (COM) commercial#> 3 ALL all sectors (ALL) all sectors#> 4 OTH other (OTH) other#> 5 TRA transportation (TRA) transportation#> 6 IND industrial (IND) industrial# oreia_facets("electricity/retail-sales","stateid")#> # A tibble: 62 × 3#> id name alias#> <chr> <chr> <chr>#> 1 MA Massachusetts (MA) Massachusetts#> 2 VT Vermont (VT) Vermont#> 3 WA Washington (WA) Washington#> 4 HI Hawaii (HI) Hawaii#> 5 WNC West North Central Region: (WNC) West North Central#> 6 NC North Carolina (NC) North Carolina#> 7 ENC East North Central Region: (ENC) East North Central#> 8 MTN Mountain Region: (MTN) Mountain#> 9 MD Maryland (MD) Maryland#> 10 VA Virginia (VA) Virginia#> # … with 52 more rowsThe above output provides the ids for both the State/Region andSector that will be required for obtaining Retail Sales for Residentialcustomers in Vermont.
"stateid" for Vermont is “VT”"sectorid" for residential is “RES”Now, the ID values from the above output can be used to further limitthe returned data with thefacets argument ineia_data().
eia_data(dir ="electricity/retail-sales",data ="sales",facets =list(sectorid ="RES",stateid ="VT"),length =6)#> # A tibble: 6 × 7#> period stateid stateDescription sectorid sectorName sales `sales-units`#> <chr> <chr> <chr> <chr> <chr> <dbl> <chr>#> 1 2008-07 VT Vermont RES residential 188. million kilowatthours#> 2 2010-07 VT Vermont RES residential 198. million kilowatthours#> 3 2023-08 VT Vermont RES residential 180. million kilowatthours#> 4 2016-02 VT Vermont RES residential 185. million kilowatthours#> 5 2015-05 VT Vermont RES residential 143. million kilowatthours#> 6 2010-06 VT Vermont RES residential 158. million kilowatthoursIf multiple data values (salesand price) or facets(residentialand commercial) are of interest, simplyconcatenate these entries withc(), as shown below:
eia_data(dir ="electricity/retail-sales",data =c("sales","price"),facets =list(sectorid =c("RES","COM"),stateid ="VT"),length =6)#> # A tibble: 6 × 9#> period stateid stateDescription sectorid sectorName sales price `sales-units` `price-units`#> <chr> <chr> <chr> <chr> <chr> <dbl> <dbl> <chr> <chr>#> 1 2008-07 VT Vermont COM commercial 187. 12.5 million kilowatthours cents per kilowatthour#> 2 2008-07 VT Vermont RES residential 188. 14.5 million kilowatthours cents per kilowatthour#> 3 2010-07 VT Vermont COM commercial 189. 13.3 million kilowatthours cents per kilowatthour#> 4 2010-07 VT Vermont RES residential 198. 15.4 million kilowatthours cents per kilowatthour#> 5 2023-08 VT Vermont COM commercial 174. 17.3 million kilowatthours cents per kilowatthour#> 6 2023-08 VT Vermont RES residential 180. 20.4 million kilowatthours cents per kilowatthourThis electric retail sales data is available in multiple frequenciesand for a defined date range, as shown below:
#> Frequency:#> id description query format#> 1 monthly One data point for each month. M YYYY-MM#> 2 quarterly One data point every 3 months. Q YYYY-QQ#> 3 annual One data point for each calendar year. A YYYY#> #> Date Range:#> 2001-01 to 2023-08The default frequency for this data is monthly, but maybe annualizeddata for a truncated time frame - say the last five years - is all thatis needed. The ability to alter the granularity or truncate time frameof the returned data is provided by thefreq,start, andend arguments ofeia_data().
eia_data(dir ="electricity/retail-sales",data =c("sales","price"),facets =list(sectorid =c("RES","COM"),stateid ="VT"),freq ="annual",start ="2013",end ="2023")#> # A tibble: 20 × 9#> period stateid stateDescription sectorid sectorName sales price `sales-units` `price-units`#> <int> <chr> <chr> <chr> <chr> <dbl> <dbl> <chr> <chr>#> 1 2018 VT Vermont COM commercial 2004. 15.2 million kilowatthours cents per kilowatthour#> 2 2018 VT Vermont RES residential 2116. 18.0 million kilowatthours cents per kilowatthour#> 3 2015 VT Vermont COM commercial 2011. 14.5 million kilowatthours cents per kilowatthour#> 4 2015 VT Vermont RES residential 2089. 17.1 million kilowatthours cents per kilowatthour#> 5 2014 VT Vermont COM commercial 2031. 14.6 million kilowatthours cents per kilowatthour#> 6 2017 VT Vermont RES residential 2023. 17.7 million kilowatthours cents per kilowatthour#> 7 2017 VT Vermont COM commercial 1977. 14.6 million kilowatthours cents per kilowatthour#> 8 2021 VT Vermont COM commercial 1867. 16.6 million kilowatthours cents per kilowatthour#> 9 2021 VT Vermont RES residential 2174. 19.3 million kilowatthours cents per kilowatthour#> 10 2022 VT Vermont COM commercial 1916. 17.3 million kilowatthours cents per kilowatthour#> 11 2022 VT Vermont RES residential 2187. 19.9 million kilowatthours cents per kilowatthour#> 12 2014 VT Vermont RES residential 2121. 17.5 million kilowatthours cents per kilowatthour#> 13 2020 VT Vermont COM commercial 1806. 16.4 million kilowatthours cents per kilowatthour#> 14 2020 VT Vermont RES residential 2157. 19.5 million kilowatthours cents per kilowatthour#> 15 2013 VT Vermont RES residential 2125. 17.1 million kilowatthours cents per kilowatthour#> 16 2013 VT Vermont COM commercial 2017. 14.7 million kilowatthours cents per kilowatthour#> 17 2019 VT Vermont COM commercial 1934. 16.0 million kilowatthours cents per kilowatthour#> 18 2019 VT Vermont RES residential 2082. 17.7 million kilowatthours cents per kilowatthour#> 19 2016 VT Vermont COM commercial 2014. 14.5 million kilowatthours cents per kilowatthour#> 20 2016 VT Vermont RES residential 2056. 17.4 million kilowatthours cents per kilowatthourAs shown above, the returned data frame has 20 records - 10 annualobservations for each sector (residential and commercial). However,notice the hectic ordering of the returned data. Thankfully, the APIservice can sort this in either ascending or descending order using thesort argument.
Similar tofacets,sort requires a namedlist object with two components:
"cols" - a list of column names on which to sort."order" - a single character value, either “asc” or“desc” for ascending and descending, respectively.eia_data(dir ="electricity/retail-sales",data =c("sales","price"),facets =list(sectorid =c("RES","COM"),stateid ="VT"),freq ="annual",start ="2013",end ="2023",sort =list(cols =c("period","sectorid"),order ="asc"))#> # A tibble: 20 × 9#> period stateid stateDescription sectorid sectorName sales price `sales-units` `price-units`#> <int> <chr> <chr> <chr> <chr> <dbl> <dbl> <chr> <chr>#> 1 2013 VT Vermont COM commercial 2017. 14.7 million kilowatthours cents per kilowatthour#> 2 2013 VT Vermont RES residential 2125. 17.1 million kilowatthours cents per kilowatthour#> 3 2014 VT Vermont COM commercial 2031. 14.6 million kilowatthours cents per kilowatthour#> 4 2014 VT Vermont RES residential 2121. 17.5 million kilowatthours cents per kilowatthour#> 5 2015 VT Vermont COM commercial 2011. 14.5 million kilowatthours cents per kilowatthour#> 6 2015 VT Vermont RES residential 2089. 17.1 million kilowatthours cents per kilowatthour#> 7 2016 VT Vermont COM commercial 2014. 14.5 million kilowatthours cents per kilowatthour#> 8 2016 VT Vermont RES residential 2056. 17.4 million kilowatthours cents per kilowatthour#> 9 2017 VT Vermont COM commercial 1977. 14.6 million kilowatthours cents per kilowatthour#> 10 2017 VT Vermont RES residential 2023. 17.7 million kilowatthours cents per kilowatthour#> 11 2018 VT Vermont COM commercial 2004. 15.2 million kilowatthours cents per kilowatthour#> 12 2018 VT Vermont RES residential 2116. 18.0 million kilowatthours cents per kilowatthour#> 13 2019 VT Vermont COM commercial 1934. 16.0 million kilowatthours cents per kilowatthour#> 14 2019 VT Vermont RES residential 2082. 17.7 million kilowatthours cents per kilowatthour#> 15 2020 VT Vermont COM commercial 1806. 16.4 million kilowatthours cents per kilowatthour#> 16 2020 VT Vermont RES residential 2157. 19.5 million kilowatthours cents per kilowatthour#> 17 2021 VT Vermont COM commercial 1867. 16.6 million kilowatthours cents per kilowatthour#> 18 2021 VT Vermont RES residential 2174. 19.3 million kilowatthours cents per kilowatthour#> 19 2022 VT Vermont COM commercial 1916. 17.3 million kilowatthours cents per kilowatthour#> 20 2022 VT Vermont RES residential 2187. 19.9 million kilowatthours cents per kilowatthour