Brian Killough CEOS Systems Engineering Office SEO WGISS40 Harwell Oxfordshire UK September 30 2015 remote presentation General CEOS Data Cube Architecture A flexible architecture that supports infinite user applications increasing and diverse datasets local or cloudbased deploy ID: 398281
Download Presentation The PPT/PDF document "CEOS Data Cube Open Source Software Stat..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CEOS Data Cube Open Source Software Status
Brian KilloughCEOS Systems Engineering Office (SEO)WGISS-40Harwell, Oxfordshire, UKSeptember 30, 2015(remote presentation)Slide2
General CEOS
Data Cube Architecture
A flexible architecture that supports infinite user applications, increasing and diverse datasets, local or cloud-based deployment, and automated ingestion of new datasets.
Open Source software to allow free and open access, Advanced Programming Interface (API) access, future data and capability growth, and commercial opportunities.
Applications
Web-based
Mobile
Data and
Application
Platform
API Toolset
Access Control
Data
ManagementJob Management
Data Cube AccessAnalyticsVisualization
API
API
Data Cube
Analysis-Ready Data Products
Satellite
Space AgencySlide3
High-Level Data Cube Requirements
Free and open access to software and APIsDocumented processes for data cube generation, new data ingestion, and application interfacesCloud-based or local deployment“Analysis Ready” data products for satellite data layers(fully processed – calibration, geolocation, radiometric)Preservation of native data product grid formatsComputational flexibility to allow reprojection into “nested grid” formats for multiple dataset interoperability and spatial consistency
Reference user interface that supports data cube statistics/analysis and optical image preparation (e.g., mosaics).
Enable user development of applications through flexible APIs
Architecture flexibility and standards to support Data Cubes for local, regional or country scalesSlide4
Kenya Project
The Kenya Data Cube Project is led by NASA-SEO and the Australian Government. Supported by USGS and the Clinton Foundation (CCI and SLEEK).Two operating versions of the Kenya Data Cube ... One is deployed on the Amazon Cloud and one is deployed on a local computer. 11.5 TB of uncompressed Landsat data (7500+ Landsat 7+8 scenes). Pixel information was extracted, reformatted into a UTM grid, and compressed into a 3.6 TB Data Cube. Application algorithms can be run directly on the compressed Data Cube. Demonstrated an operating sample user interface for mosaic creation at the recent SDCG meeting. Planning to add more data layers in the coming months ... SRTM, ALOS PALSAR Mosaics, MODIS.Slide5
Colombia Project
The Colombia Data Cube Project is led by NASA-SEO and the Colombian Government. Supported by CSIRO (Australia, CEOS Chair Team) and IDEAM (Institute of Hydrology, Meteorology and Environmental Studies).A mini-cube (4 Landsat path-row regions) will be delivered on October 14-15. Includes all historic Landsat data back to 2000. This meeting will define the objectives and detailed project development plan for 2016.Colombian researchers are eager to expand the Data Cube to the full country and to develop enhanced user interfaces to support forest and agriculture applications. Additional datasets will also be added.Slide6
Kenya Data Cube
User Interface DemoMap PanelTool Panel
Selectable data products (L7, L8)
Selectable time periods (years, months)
Selectable region (user defined, KML, Shapefile)
Selectable product (mosaic)
Selectable output image (true color, false color, NDVI)Slide7
Open Source Software
The NASA-SEO, CSIRO Australia and Geoscience Australia are managing the CEOS Data Cube Open Source Software.Website: http://www.datacube.org.auIncludes background on Australia Geoscience Data Cube, Wiki and GitHub (open source repository)Software is released using the Apache 2.0 license. This is a very permissive license that provides an express grant of patent rights from contributors to users and allows open distribution and commercial use.NASA internal approval for open source release took 2 months. This includes approvals from softwware engineering, legal and export control.NASA will release its first version of code within a week: Landsat “ingestor”, Kenya Data Cube and a reference user interface (mosaic creation).Slide8
Future Plans
Work with CEOS Space Agencies to develop plans for sustained provision of Analysis Ready Data (ARD)Sentinel-1/2 and SPOT-5 are priorities.
Reference User Interface
Data Cubes
Analysis-Ready Data Products
Develop prototype reference user interfaces for the Kenya and Colombia projects.
Consider features such as: mosaic creation, data cube analytics and visualization, forest cover classification and change detection, and agriculture NDVI and change detection
Develop an API for users to create their own user interfaces
Develop prototype Data Cubes for Kenya and Colombia projects. Test local and cloud-based deployment options.
Expand data layers to include: Landsat, SRTM, ALOS, MODIS, SPOT-5, Sentinel-1/2.
Improve data cube ingestor software (creation) and data cube application software (analysis)