External data links

SENSE is part of the wider family of data services and sources in the UK.  Below are links to some of the major data sources that can be linked with our energy focused data to unlock a rich picture of UK energy use combined with financial, demographic and climate data.

Smart Data Research UK

We have 5 sister data services in the Smart Data Research UK family, all the data from these services is catalogues alongside our own at https://data.sdruk.ukri.org/. These data can be linked to our own using common identifiers including Census areas (LSOA, MSOA), timestamps, latitude and longitude

Electricity Market Data

Elexon offer electricity wholesale market data openly, with user friendly APIs available to extract detailed information about market settlement and balancing market pricing. See more at https://www.elexon.co.uk/bsc/data/kinnect-insights-solution/

Energy system data for models and forecasting

One of the outputs of the AI for Decarbonisation’s Virtual Centre of Excellence project is a list of data sources. While this was focussed on providing data for AI models, many of the sources can be used for research with or without AI. See more at the ADViCE data resources page. For low voltage network forecasting resources, a useful review of models and data sources with multiple links to useful datasets was compiled by one of our team in 2021 – see https://low-voltage-loadforecasting.github.io/

Weather and climate data

  • Provides weather forecast data and archive datasets including climate reanalysis. Requires registration and login, but many open datasets.

  • provides weather forecast data and archive datasets of modelled weather for arbitrary locations worldwide, offered under permissive licensing (see https://open-meteo.com/en/licence)

  • provides API access to climate and weather data for the UK including forecast and observation data.

  • A long running climate reanalysis data project run by NASA. Blends observations and models to provide a continuous dataset of climate variables from 1980 to present, worldwide. On a 0.5° × 0.66° grid. Full documentation at https://gmao.gsfc.nasa.gov/gmao-products/merra-2/documentation_merra-2/

  • Note that models of weather an renewables output based on MERRA-2 or CM-SAF SARAH are available at https://www.renewables.ninja/.  Free signup and logon is required to access the full data range for models, licensing is permissive – see https://www.renewables.ninja/downloads

Energy system data for models and forecasting

One of the outputs of the AI for Decarbonisation’s Virtual Centre of Excellence project is a list of data sources. While this was focussed on providing data for AI models, many of the sources can be used for research with or without AI. See more at the ADViCE data resources page. For low voltage network forecasting resources, a useful review of models and data sources with multiple links to useful datasets was compiled by one of our team in 2021 – see https://low-voltage-loadforecasting.github.io/

Geographical linking, translation and analysis

One of the challenges in linking contextual data to energy use can be the different geographical labels applied to data. Network data is often based on the network topologies, with some GIS information available and some point locations, for example substations, tagged with latitude and longitude. In contrast, contextual demographic information is often presented with census geographies as their label (LSOA, MSO etc), while other data such as mobility is commonly presented using local authority geography. Postcodes may also be used as geographical tags. Useful lookup datasets for translation are maintained by the Office for National Statistics at https://geoportal.statistics.gov.uk/

https://kepler.gl/ provide a useful toolbox to conduct and present your geospatial analysis

Extra data links!