Open energy system database projects employopen data methods to collect,clean, and republish energy-relateddatasets for open use. The resulting information is then available, given a suitableopen license, for statistical analysis and for building numericalenergy systemmodels, includingopen energy system models. Permissive licenses likeCreative CommonsCC0 andCC BY are preferred, but some projects will house data made public under market transparency regulations and carrying unqualifiedcopyright.

The databases themselves may furnish information on national power plant fleets, renewable generation assets, transmission networks,time series for electricity loads, dispatch,spot prices, and cross-border trades, weather information, and similar. They may also offer other energy statistics including fossil fuel imports and exports, gas, oil, and coal prices, emissions certificate prices, and information onenergy efficiency costs and benefits.

Much of the data is sourced from official or semi-official agencies, includingnational statistics offices,transmission system operators, andelectricity market operators. Data is alsocrowdsourced using publicwikis and public upload facilities.^[1] Projects usually also maintain a strict record of theprovenance and version histories of the datasets they hold. Some projects, as part of their mandate, also try to persuade primary data providers to release their data under more liberal licensing conditions.^[a]

Two drivers favor the establishment of such databases. The first is a wish to reduce the duplication of effort that accompanies each new analytical project as it assembles and processes the data that it needs from primary sources. And the second is an increasing desire to makepublic policy energy models more transparent to improve their acceptance by policymakers and the public.^[3] Better transparency dictates the use of open information, able to be accessed and scrutinized by third-parties, in addition to releasing thesource code for the models in question.^[4]

In the mid-1990s, energy models used structured text files for data interchange but efforts were being made to migrate torelational database management systems for data processing.^[5] These early efforts however remained local to a project and did not involve online publishing or open data principles.

The first energy information portal to go live wasOpenEI in late 2009, followed byreegle in 2011.

A 2012 paper marks the first scientific publication to advocate thecrowdsourcing of energy data.^[6] The 2012 PhD thesis by Chris Davis also discusses the crowdsourcing of energy data in some depth.^[7] A 2016 thesis surveyed the spatial (GIS) information requirements for energy planning and finds that most types of data, with the exception of energy expenditure data, are available but nonetheless remain scattered and poorly coordinated.^[8]

In terms of open data, a 2017 paper concludes that energy research has lagged behind other fields, most notably physics, biotechnology, and medicine.^{[9]: 213–214} The paper also lists the benefits of open data and open models and discusses the reasons that many projects nonetheless remain closed.^{[9]: 211–213} A one-page opinion piece from 2017 advances the case for using open energy data and modeling to build public trust in policy analysis. The article also argues thatscientific journals have a responsibility to require that data and code be submitted alongside text forpeer review.^[10]

Data models are central to the design and organization of databases. Open energy database projects generally try to develop and adhere to well resolved data models, usingde facto andpublished standards where applicable. Some projects attempt to coordinate their data models in order to harmonize their data and improve its utility. Defining and maintaining suitablemetadata is also a key issue. Thelife-cycle management of data includes, but is not limited to, the use ofversion control to track the provenance of incoming andcleansed data. Some sites allow users to comment on and rate individual datasets.

Issues surroundingcopyright remain at the forefront with regard to open energy data. As noted, most energy datasets are collated and published by official or semi-official sources. But many of the publicly available energy datasets carry no license, limiting their reuse in numerical and statistical models, open or otherwise. Copyright protected material cannot lawfully be circulated, nor can it be modified and republished.

Measures to enforcemarket transparency have not helped much because the associated information is again not licensed to enable modification and republication. Transparency measures include the 2013 European energy market transparency regulation 543/2013.^[11] Indeed, 543/2013 "is only an obligation to publish, not an obligation to license".^{[12]: slide 14} Notwithstanding, 543/2013 does enable downloaded data to be computer processed with legal certainty.^[13]: 5

Energy databases with hardware located with the European Union are protected under a generaldatabase law, irrespective of the legal status of the information they hold.^[12]Database rights not waived by public sector providers significantly restrict the amount of data a user can lawfully access.

A December 2017 submission by energy researchers in Germany and elsewhere highlighted a number of concerns over the re-use of public sector information within the Europe Union.^[14]The submission drew heavily on a recent legal opinion covering electricity data.^[13]

National and international energy statistics are published regularly by governments and international agencies, such as theIEA.^[15] In 2016 the United Nations issued guidelines for energy statistics.^[16] While the definitions and sectoral breakdowns are useful when defining models, the information provided is rarely in sufficient detail to enable its use in high-resolution energy system models.^[9]: 213

There are few published standards covering the collection and structuring of high-resolution energy system data. TheIECCommon Information Model (CIM) defines data exchange protocols for low and high voltage electricity networks.

Although this page is about genuinely open data, some important databases remain closed.

Data collected by theInternational Energy Agency (IEA) is widely quoted in policy studies but remains nonethelesspaywalled.Researchers atOxford University have called for this situation to change.^[17]

Energy system models are data intensive and normally require detailed information from a number of sources. Dedicated projects to collect, collate, document, and republish energy system datasets have arisen to service this need. Most database projects prefer open data, issued underfree licenses, but some will accept datasets with proprietary licenses in the absence of other options.

TheOpenStreetMap project, which uses theOpen Database License (ODbL), contains geographic information about energy system components, including transmission lines.^[18]Wikimedia projects such asWikidata and Wikipedia have a growing set of information related to national energy systems, such as descriptions of individual power stations.^{[7]: 156–159}

The followingtable summarizes projects that specifically publish open energy system data. Some are general repositories while others (for instance,oedb) are designed to interact with open energy system models in real-time.

Three of the projects listed work withlinked open data (LOD), a method of publishing structured data on theweb so that it can be networked and subject tosemantic queries. The overarching concept is termed thesemantic web. Technically, such projects supportRESTful APIs,RDF, and theSPARQL query language. A 2012 paper reviews the use of LOD in the renewable energy domain.^[19]

The Climate Compatible Growth (CCG) programme provides starter kits for the following 69 countries: Algeria, Angola, Argentina, Benin, Botswana, Bolivia, Brazil, Burkina Faso, Burundi, Cambodia, Cameroon, Central African Republic, Chad, Chile, Colombia, Côte d'Ivoire, Democratic Republic of Congo, Djibouti, Ecuador, Egypt, Equatorial Guinea, Eritrea, Eswatini, Ethiopia, Gabon, Gambia, Ghana, Guinea, Guinea-Bissau, Indonesia, Kenya, Laos, Lesotho, Liberia, Libya, Malawi, Malaysia, Mali, Mauritania, Morocco, Mozambique, Myanmar, Namibia, Niger, Nigeria, Papua New Guinea, Paraguay, Peru, Philippines, Republic of Congo, Republic of Korea, Rwanda, Senegal, Sierra Leone, Somalia, South Africa, South Sudan, Sudan, Taiwan, Tanzania, Thailand, Togo, Tunisia, Uganda, Uruguay, Venezuela, Viet Nam, Zambia, and Zimbabwe.

The datasets are hosted on theZenodo science archive site, visit that site and search for "ccg starter kit".

The Energy Research Data Portal for South Africa is being developed by theEnergy Research Centre,University of Cape Town,Cape Town, South Africa. Coverage includes South Africa and certain other African countries where the Centre undertakes projects.^[b] The website uses theCKAN open source data portal software. A number of data formats are supported, includingCSV andXLSX. The site also offers anAPI for automated downloads. As of March 2017^[update], the portal contained 65 datasets.

The energydata.info project from theWorld Bank Group,Washington, DC, USA is an energy database portal designed to support national development by improving public access to energy information.^[20] As well as sharing data, the platform also offers tools to visualize and analyze energy data. Although the World Bank Group has made available a number of dataset and apps, external users and organizations are encouraged to contribute. The concepts of open data and open source development are central to the project. energydata.info uses its ownfork of theCKAN open source data portal as its web-based platform. The Creative CommonsCC BY 4.0 license is preferred for data but other open licenses can be deployed. Users are also bound by the terms of use for the site.^[21]

As of January 2017^[update], the database held 131 datasets, the great majority related todeveloping countries. The datasets are tagged and can be easily filtered. A number of download formats, includingGIS files, are supported:CSV,XLS,XLSX,ArcGIS,Esri,GeoJSON,KML, andSHP. Some datasets are also offered asHTML. Again, as of January 2017^[update], four apps are available. Some are web-based and run from a browser.

The semanticwiki-site anddatabase Enipedia lists energy systems data worldwide.^[7][22] Enipedia is maintained by theEnergy and Industry Group, Faculty of Technology, Policy and Management,Delft University of Technology,Delft, the Netherlands. A key tenet of Enipedia is that data displayed on the wiki is not trapped within the wiki, but can be extracted viaSPARQL queries and used to populate new tools. Any programming environment that can download content from aURL can be used to obtain data.^[23] Enipedia went live in March 2011, judging by traffic figures quoted by Davis.^{[7]: 185  : fig 9.17}

A 2010 study describes how community driven data collection, processing, curation, and sharing is revolutionizing the data needs ofindustrial ecology and energy system analysis.^[24] A 2012 chapter introduces asystem of systems engineering (SoSE) perspective and outlines howagent-based models and crowdsourced data can contribute to the solving of global issues.^{[citation needed]}

As of April 2019^[update], the site has gone offline pending a move to theenipedia.org domain.

The Open Energy Platform (OEP) is a collaborative versioneddataset repository for storing open energy system model datasets. A dataset is presumed to be in the form of adatabase table, together withmetadata. Registered users can upload and download datasets manually using a web-interface or programmatically via anAPI usingHTTPPOST calls. Uploaded datasets are screened for integrity using deterministic rules and then subject to confirmation by a moderator. The use of versioning means that any prior state of the database can be accessed (as recommended in this 2012 paper).^[4] Hence, the repository is specifically designed to interoperate with energy system models. Thebackend is aPostgreSQLobject-relational database undersubversionversion control. Open-data licenses are specific to each dataset. Unlike other database projects, users can download the current version (the public tables) of the entirePostgreSQL database or any previous version. The development is being led by a cross-project community.^[25]

The Open Data Energy Networks (Open Data Réseaux Énergies orODRÉ) portal is run by eight partners, led by the French nationaltransmission system operator (TSO)Réseau de Transport d'Électricité (RTE). The portal was previously known as Open Data RTE. The site offers electricity system datasets under a Creative CommonsCC BY 2.0 compatible license, withmetadata, anRSS feed for notifying updates, and an interface for submitting questions.Re-users of information obtained from the site can also register third-party URLs (be they publications or webpages) against specific datasets.^[27][28]

The portal uses the French GovernmentLicence Ouverte license and this is explicitly compatible with the United KingdomOpen Government Licence (OGL), the Creative CommonsCC BY 2.0 license (and thereby later versions), and the Open Data CommonsODC-BY license.^[26]: 2

The site hosts electricity, gas, and weather information related to France.

The Open Data Portal is run by UK Power Networks, a GBDistribution Network Operator (DNO), hosted on theOpenDataSoft platform. The Portal offers electricity network datasets under a Creative CommonsCC BY 4.0 compatible license, withmetadata, a newsfeed, and a data request form.Re-users of information obtained from the site can also register third-party URLs (be they publications or webpages) against specific datasets. A number of download formats, includingGIS files, are supported:CSV,XLS,GeoJSON,KML, andSHP. The site also offers an API for automated downloads.

The portal uses theCreative Commons License and also hosts datasets from other sources which are licensed under theOpen Government Licence (OGL).

The site hosts electricity datasets related to UK Power Networks' three license areas in London, the East and South East of England.

The Open Power System Data (OPSD) project seeks to characterize theGerman andwestern Europeanpower plant fleets, their associatedtransmission network, and related information and to make that data available to energy modelers and analysts.^[31] The platform was originally implemented by theUniversity of Flensburg,DIW Berlin,Technische Universität Berlin, and the energy economics consultancy Neon Neue Energieökonomik, all from Germany. The first phase of the project, from August 2015 to July 2017, was funded by theFederal Ministry for Economic Affairs and Energy (BMWi) for€490000.^[32][33] The project later received funding for a second phase, from January 2018 to December 2020, withETH Zurich replacing Flensburg University as a partner.^[34]

Developers collate and harmonize data from a range of government, regulatory, and industry sources throughout Europe. The website and the metadata utilize English, whereas the original material can be in any one of24 languages. Datasets follow the emergingfrictionless data package standard being developed byOpen Knowledge Foundation (OKF). The website was launched on 28 October 2016. As of June 2018^[update], the project offers the following primary packages, for Germany and other European countries:

• details, includinggeolocation, of conventional power plants and renewable energy power plants
• aggregated generation capacity by technology and country
• hourlytime series covering electrical load, day-ahead electricity spot prices, and wind and solar resources
• a script to filter and downloadNASA MERRA-2 satellite weather data^[c][35]

In addition, the project hosts selected contributed packages:

• electricity demand and self-generation time series for representative south German households
• simulatedPV and wind generationcapacity factor time series for Europe, generated by theRenewables.ninja project

To facilitate analysis, the data is aggregated into large structured files (inCSV format) and loaded into data packages with standardized machine-readable metadata (inJSON format).^[36][37] The same data is usually also provided asXLSX (Excel) andSQLite files. The datasets can be accessed in real-time using stableURLs. ThePython scripts deployed for data processing are available onGitHub and carry anMIT license. The licensing conditions for the data itself depends on the source and varies in terms of openness. Previous versions of the datasets and scripts can be recovered in order to track changes or replicate earlier studies. The project also engages with energy data providers, such astransmission system operators (TSO) andENTSO-E, to encourage them to make their data available underopen licenses (for instance,Creative Commons andODbL licenses).^[2]

In a 2019 publication, OPSD developers describe their design choices, implementation, and provisioning.^[38] Information integrity remains key, with each data package having traceable provenance, curation, and packing. From October 2018, each new or revised data package is assigned a uniqueDOI to ensure that external references to current and prior versions remain stable.

A number of published electricity market modeling analyses are based on OPSD data.^{[39][40][41][42]}

In 2017, the Open Power System Data project won the Schleswig-Holstein Open Science Award ^[43] and the Germany Land of Ideas award.^[44]

Open Energy Information (OpenEI) is acollaborativewebsite, run by the US government, providingopen energy data to software developers, analysts, users, consumers, and policymakers.^[45][46] The platform is sponsored by theUnited States Department of Energy (DOE) and is being developed by theNational Renewable Energy Laboratory (NREL).^[46] OpenEI launched on 9 December 2009.^[47] While much of its data is from US government sources, the platform is intended to be open and global in scope.

OpenEI provides two mechanisms for contributing structured information: asemantic wiki (usingMediaWiki and theSemantic MediaWiki extension) for collaboratively-managed resources and adataset upload facility for contributor-controlled resources. US government data is distributed under aCC0public domain dedication, whereas other contributors are free to select an open data license of their choice. Users can rate data using a five-star system, based on accessibility, adaptability, usefulness, and general quality.^[46] Individual datasets can be manually downloaded in an appropriate format, often asCSV files.^[46]Scripts for processing data can also be shared through the site. In order to build a community around the platform, a number offorums are offered covering energy system data and related topics.^[45]

Most of the data on OpenEI is exposed aslinked open data (LOD) (describedelsewhere on this page). OpenEI also uses LOD methods to populate its definitions throughout the wiki with real-time connections toDBPedia,reegle, andWikipedia.^{[46][48]: 46–49}

OpenEI has been used to classify geothermal resources in the United States.^[49] And to publicize municipal utility rates, again within the US.^[50]

OpenGridMap employscrowdsourcing techniques to gather detailed data on electricity network components and then infer a realistic network structure using methods fromstatistics andgraph theory. The scope of the project is worldwide and both distribution and transmission networks can bereverse engineered. The project is managed by the Chair of Business Information Systems,TUM Department of Informatics,Technical University of Munich,Munich, Germany. The project maintains a website and aFacebook page and provides anAndroidmobile app to help the public document electrical devices, such astransformers andsubstations. The bulk of the data is being made available under aCreative CommonsCC BY 3.0 IGO license.^[51][d] The processing software is written primarily inPython andMATLAB and is hosted onGitHub.^[52][53]

OpenGridMap provides a tailoredGIS web application, layered onOpenStreetMap, which contributors can use to upload and edit information directly. The same database automatically stores field recordings submitted by the mobile app. Subsequent classification by experts allows normal citizens to document and photograph electrical components and have them correctly identified. The project is experimenting with the use of hobby drones to obtain better information on associated facilities, such asphotovoltaic installations. Transmission line data is also sourced from and shared with OpenStreetMap. Each componentrecord is verified by a moderator.

Once sufficient data is available, the transnet software is run to produce a likely network, usingstatistical correlation,Voronoi partitioning, andminimum spanning tree (MST) algorithms. The resulting network can be exported inCSV (separate files for nodes and lines),XML, andCIM formats. CIM models are well suited for translation into software-specific data formats for further analysis, includingpower grid simulation. Transnet also displaysdescriptive statistics about the resulting network for visual confirmation.^{[53]: 3–5}

The project is motivated by the need to provide datasets for high-resolution energy system models, so that energy system transitions (like the GermanEnergiewende) can be better managed, both technically and policy-wise.^[54] The rapid expansion ofrenewable generation and the anticipated uptake ofelectric vehicles means that electricity system models must increasingly represent distribution and transmission networks in some detail.

As of 2017^[update], OpenGridMap techniques have been used to estimate the low voltage network in the German city ofGarching and to estimate the high voltage grids in several other countries.

The Power Explorer portal is a part of the larger Resource Watch platform, hosted by theWorld Resources Institute. The initial Global Power Plant Database, an open source database of the power plants globally, was released in April 2018.^[55] As of May 2021^[update], the portal itself is still under development.

Power Explorer is also supported byGoogle with various research partners, includingKTH, Global Energy Observatory,Enipedia, andOPSD.

The PowerGenome project aims to provide a coherent dataset covering the United States electricity system. PowerGenome was initially designed to service theGenX model,^[56] but support for other modeling frameworks is in planning.^[57] The PowerGenome utility also pulls from upstream datasets hosted by the Public Utility Data Liberation project (PUDL) and theEIA, so those dependencies need to be met by users. Datasets are occasionally archived onZenodo.^[58] A video describing the project is available.^[59]

reegle is aclean energy informationportal coveringrenewable energy,energy efficiency, andclimate compatible development topics.^{[48]: 41 [60][61]} reegle was launched in 2006 byREEEP andREN21 with funding from the Dutch (VROM), German (BMU), and UK (Defra) environment ministries.^[62] Originally released as a specializedinternet search engine, reegle was relaunched in 2011 as an information portal.

reegle offers and utilizeslinked open data (LOD) (describedelsewhere on this page).^{[48]: 43–46} Sources of data includeUN andWorld Bank databases, as well as dedicated partners around the world. reegle maintains a comprehensive structuredglossary (driven by an LOD-compliantthesaurus) of energy and climate compatible development terms to assist with the tagging of datasets. The glossary also facilitates intelligent web searches.^{[e][61]: 191, 193 [63][64]}

reegle offers country profiles which collate and display energy data on a per-country basis for most of the world.^[65] These profiles are kept current automatically using LOD techniques.^{[61]: 193–194} As of 2021, the portal is no longer active.

Renewables.ninja is a website that can calculate the hourly power output fromsolar photovoltaic installations andwind farms located anywhere in the world. The website is a joint project between theDepartment of Environmental Systems Science,ETH Zurich,Zürich, Switzerland and theCentre for Environmental Policy,Imperial College London,London, United Kingdom. The website went live during September 2016. The resulting time series are provided under a Creative CommonsCC BY-NC 4.0 license (which is unfortunately not open data conformant) and the underlying power plant models are published using aBSD-new license. As of February 2017^[update], only the solar model, written inPython, has been released.^[66]

The project relies on weather data derived frommeteorological reanalysis models and weathersatellite images. More specifically, it uses the 2016 MERRA-2 reanalysis dataset fromNASA ^[c] and satellite images from CM-SAF SARAH.^[67] For locations in Europe, this weather data is further "corrected" by country so that it better fits with the output from known PV installations and windfarms. Two 2016 papers describe the methods used in detail in relation to Europe. The first covers the calculation of PV power.^[68] And the second covers the calculation of wind power.^[69]

The website displays an interactive world map to aid the selection of a site. Users can then choose a plant type and enter some technical characteristics. As of February 2017^[update], only year 2014 data can be served, due to technical restrictions. The results are automatically plotted and are available for download in hourlyCSV format with or without the associated weather information. The site offers anAPI for programmatic dataset recovery using token-based authorization. Examples deployingcURL andPython are provided.

A number of studies have been undertaking using the power production datasets underpinning the website (these studies predate the launch of the website), with the bulk focusing on energy options forGreat Britain.^{[70][71][72][73][74][75][76]}

The SMARD site (pronounced "smart") serves electricity market data fromGermany,Austria, andLuxembourg and also provides visual information. The electricity market plots and their underlyingtime series are released under a permissiveCC BY 4.0 license.^[77] The site itself was launched on 3 July 2017 in German and an English translation followed shortly. The data portal is mandated under the German Energy Industry Act (Energiewirtschaftsgesetz orEnWG) section §111d, introduced as an amendment on 13 October 2016. Four table formats are offered:CSV,XLS,XML, andPDF. The maximum sampling resolution is15 min. Market data visuals orplots can be downloaded in PDF,SVG,PNG, andJPG formats. Representative output is shown in the thumbnail (on the left), in this case mid-winter dispatch over two days for the whole of Germany. The horizontal ordering by generation type is first split intorenewable and conventional generation and then based onmerit. A user guide is updated as required.^[78]

• Comprehensive Knowledge Archive Network (CKAN) – a web-based open data management system
• Climate change mitigation scenarios
• Crowdsourcing
• Energy modeling – the process of building computer models of energy systems
• Energy system – the interpretation of the energy sector in system terms
• Open Energy Modelling Initiative – a European-based energy modeling community
• Open energy system models – a review ofenergy systemmodels that are alsoopen source
• Open Knowledge Foundation – a global non-profit network that promotes and shares information

• Open energy data wiki maintained by theOpen Energy Modelling Initiative
• De Felice, Matteo (2020)."Freely available datasets of energy variables".openmod forum.Open Energy Modelling Initiative. Retrieved1 December 2020. The list is under a Creative Commons CC‑BY‑4.0 license and many of the datasets cited are similarly licensed.

• De-risking Energy Efficiency Platform (DEEP) – an open energy efficiency data platform for Europe
• European Climatic Energy Mixes project (ECEM) — the role that climate change may play on future energy systems
• OpenEnergy Database (oedb) – an open energy system database being developed in Germany
• OpenEnergyMonitor – an open source energy use monitoring project
• Domain‑wide data projects – a list of data related projects designed to support open energy system modeling

• Energy
• Energy models
• Energy policy
• Open data
• Open science
• Systems theory

• CS1: unfit URL
• CS1: long volume value
• CS1 maint: location missing publisher
• CS1 French-language sources (fr)
• CS1 German-language sources (de)
• All articles with dead external links
• Articles with dead external links from April 2020
• Articles with permanently dead external links
• Articles with short description
• Short description matches Wikidata
• Use dmy dates from December 2016
• Use American English from December 2016
• All Wikipedia articles written in American English
• Articles containing potentially dated statements from March 2017
• All articles containing potentially dated statements
• Articles containing potentially dated statements from January 2017
• All articles with unsourced statements
• Articles with unsourced statements from December 2019
• Articles containing potentially dated statements from April 2019
• Articles containing French-language text
• Articles containing potentially dated statements from June 2018
• Articles containing potentially dated statements from 2017
• Articles containing potentially dated statements from May 2021
• Articles containing potentially dated statements from February 2017