Big Earth Data at your fingertips - this is the vision of EarthServer, an intercontinental initiative for unleashing the potential of Big Data through a disruptive paradigm shift in technology:
- from isolated silos of data with disparate functionality towards a single, uniform information space;
- from a difficult, artificial differentiation between data and metadata access to unified retrieval;
- from zillions of files towards few whatever-size datacubes;
- from limited functionality to the freedom of asking anything, anytime, any server in a peer network of data centers worldwide.
In phase 1, EarthServer has established open ad-hoc analytics on massive Earth Science data, based on and extending leading-edge Array Database technology, rasdaman. According to EU Commission and phase 1 reviewers, rasdaman will "significantly transform the way that scientists in different areas of Earth Science will be able to access and use data in a way that hitherto was not possible". Six Lighthouse Applications with over 200 TB have demonstrated this. Additionally, EarthServer-1 "with no doubt has been shaping the Big Earth Data landscape through the standardization activities within OGC, ISO and beyond".
Phase 2 of EarthServer has started with an even more ambitious goal: each data center will provide at least 1 Petabyte of 3-D and 4-D datacubes. Technology advance will allow real-time scaling of such Petabyte cubes, and intercontinental fusion. This power of data handling will be wrapped into direct visual interaction based on multi-dimensional visualization techniques, in particular: NASA WorldWind. Following the motto "a cube says more than a million images" EarthServer-2 has set out to redefine the Big Data service landscape even more.
EarthServer-2 makes Agile Analytics on Big Earth Data Cubes of sensor, image, simulation, and statistics data a commodity for non-experts and experts alike through
- navigation, extraction, aggregation, and recombining of any-size space/time data cubes;
- easy to install & maintain value-adding services extending the existing portfolio of data and compute centers;
- based on open standards, in particular: the OGC Big Data standards and the forthcoming ISO SQL/MDA (“Multi-Dimensional Arrays”) standard.
In the Joint Research Activity, the project will advance the existing, world-leading rasdaman Array Database technology wrt. query functionality, inter-federation data processing with automatic data and query distribution, tape archive integration, and 3D/4D visualization based on NASA’s virtual globe technology.
In the Services Activity, large data centers (ECMWF, PML, MEEO/ESA, NCI, JacobsUni) will set up water, air, weather, and planetary services on 3D & 4D data cubes up to Petabyte-size with user-tailored clients for both visual and textual ad-hoc mix&match.
In the Networking Activity, the project will advance open Big Data standards in OGC, RDA, and ISO (in particular: write ISO SQL/MDA). Further, all adequate channels will be used for strong dissemination & exploitation, specifically: writing a monograph explaining OGC Big Geo Data standards; scientific publications & active conference organization; Earth science data user workshops for each domain addressed; actively contributing technology & experience to GEO / GEOSS and further bodies; establish standardized Big Geo Data benchmark and run it against EarthServer-2 and further relevant systems.
Altogether, EarthServer-2 will maintain and extend the lead in Big Earth Data services established in the highly successful EarthServer-1 project. Being already supported by ESA, rasdaman will form an enabling building block for COPERNICUS / Sentinel.
Earthserver-2 is funded under Horizon's 2020 call H2020-EINFRA-2014-2:
Project reference No: 654367
Project start: 1st May 2015
Project end: 1st May 2018 (36 months)
Call: H2020-EINFRA-2014-2 [H2020-EU.126.96.36.199]
Total EU contribution: EUR 2 839 743
Funding Scheme: RIA