Earthserver - Opportunities and challenges of serving ECMWF's peta-sized archive through OGC web-services

Publication date: 
Julia Wagemann
Stephan Siemen
Sylvie Lamy-Thepaut
Medium / Event: 
Geophysical Research Abstracts, Vol. 18, EGU2016-16899

ECMWF is partner of the EU-funded (Horizon2020) EarthServer-2 project and is setting up a web service that facilitates climate data access, exploration, analysis and visualisation based on Open Geospatial Consortium (OGC) standards.  By doing this, ECMWF data shall become easier accessible to researchers and decision-makers of the MetOcean and GIS community.

MARS is ECMWF’s Meteorological Archive and Retrieval System, the world’s largest archive of meteorological data. In November 2015, the MARS archive held ~87 PB of data and grew by additional ~3 PB every month. In order for users to fully benefit from the potential of a data volume beyond the PB, it is in the interest of ECMWF as a data provider to minimize the necessary data transport and yet, to provide access to the full range of data and information.

The aim of the three-year project is to establish a connection between the rasdaman server technology and ECMWF’s MARS archive and thus, provide access to more than 1 PB of global reanalysis served by the OGC-based standard data access protocols Web Coverage Service (WCS) and Web Coverage Processing Service (WCPS).

By presenting first results of serving meteorological data, the presentation will show opportunities for data users using OGC web services. A further focus will be set on current challenges of serving climate data from ECMWF’s archive and specific requirements of the MetOcean community, e.g. related to the support of GRIB and netCDF data, in order to collectively work on mature Big Data standards across all Earth Science disciplines.