Vicki Ferrini Suzanne OHara LDEO Paul Johnson Kevin Jerram UNH MBES Data Workflow Increasing Emphasis on Open Data Access Acquisition Costs Spatial amp Temporal Change Scientific Reproducibility ID: 551694
Download Presentation The PPT/PDF document "A Fleet-wide Approach to Optimizing Data..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
A Fleet-wide Approach to Optimizing Data Quality
Vicki Ferrini, Suzanne O’Hara (LDEO)
Paul Johnson, Kevin
Jerram
(UNH)Slide2
MBES Data WorkflowSlide3
Increasing Emphasis on
Open Data Access
Acquisition Costs
Spatial & Temporal Change
Scientific Reproducibility
Federal Data Policy Compliance
Data Syntheses & Big Data
Enable New AnalysesSlide4
*NSF-funded cruises can have 2-year proprietary restrictionSlide5
A lot of this high-value data is acquired opportunistically!Slide6
How can we cost-effectively
optimize data quality?
The Economist, 2010Slide7
GMRT
1992
R2R
2009
MAC
2011
Multibeam Sonar
Data
Continuum
GOAL
: Well-documented high-quality publicly available dataSlide8
Multibeam Advisory Committee
Optimize data quality
at acquisition
Encourage opportunistic data
acquisition Consolidate Tools, Resources & ExpertiseUS Academic Research FleetTechnical Teams / Ship VisitsData ResourcesBIST Database
Reference SurfacesPatch Test LocationsHelp DeskMore details tomorrow…Slide9MAC-
Supported Ships
+2 new ships coming online 2016Slide10
GMRT
1992
R2R
2009
MAC
2011
Multibeam Sonar
Data
Continuum
GOAL
: Well-documented high-quality publicly available dataSlide11
Data Stewardship of Underway Data
Unprocessed data from permanent sensors
Cruise Catalog
Cruise and data set metadata
Optimize delivery to National Data Centers
Programmatic Quality Assessment
Rolling Deck to Repository (R2R)Slide12
Identify
potential
problems in data
No judgment on scientific utility
Provide Feedback
Vessel Operators – address problems
Down-stream data users (scientists/engineers) – facilitate data use/re-use
Enable evaluation of fleet-wide system performance over time
R2R Quality Assessment: GoalsSlide13
R2R: MB Quality Assessment
Lead: S. O’Hara (LDEO)
Leverage open
s
ource (MB System)
Programmatically
introspect data files
Fully
document tests, results, and
ranking criteria
/thresholds in I/O XML
Customizable
test thresholds
Output includes QA Test Results, Ranking (R,Y,G) and other relevant infoSlide14
R2R: MB Quality Assessment
Lead: S. O’Hara (LDEO)Slide15
GMRT
1992
R2R
2009
MAC
2011
Multibeam Sonar
Data
Continuum
GOAL
: Well-documented high-quality publicly available dataSlide16
GMRT Synthesis: Overview
Open-access bathymetry product
Support specialists & non-specialists
Multi-resolutional synthesis
GEBCO + MBES + land + gridsFull-native resolution of MBES (100m+)Tiled Global Compilation
Images, grids, maskMercator, South Polar, North Polar2 scheduled releases / year (~80 cruises)Attribution to data contributorsAccess to source dataSlide17
GMRT
– Open Data Access
Java Applications (GeoMapApp,
Virtual Ocean)
Web Application (GMRT
MapTool
)
iPhone App (Earth Observer)
Web Services
Grid Server, Image Server, Attribution Service
WMS (
M
ercator, SP, + NP 2016)Point Service + Profile Service (Dec 2015)Broad distribution through collaborations
GEBCO, Google, ESRI, NOAA NCEISlide18
GMRT: MB QA/QC
Bad navigation
Noisy outer beams
Attitude problems
Bad soundings
Instrument problems
Bad weatherSound velocity
Slow speed in turnsQuality assessment Grid
weighting Grid resolutionSlide19
Raw MB
Files*
MB QA/QC
Processed MB
files
rDB
*source data in
public domain
Tiled images, grids, mask
GMRT Services
GMRT
– MBES Workflow
MB SystemSlide20
GMRT
–
MBES Content
GMRT v3.1 released Nov 2015
>175,000 data files + metadata
~4.4 million ship-track km of data
875 cruises
26 Ships
21 Swath File Formats
15 Sonar Systems
Most modern data acquired with Kongsberg systemsSlide21GMRT Metadata
Per Data Set
Processing notes
Make/Model/Ship
Quality, resolution
Contributor
Per Data File
Metrics (from mbinfo
) Track-line geometryUnder Development
Polygon geometry Area mappedProcessing metricsSlide22
GMRT
1992
R2R
2009
MAC
2011
Multibeam Sonar
Data
Continuum
GOAL
: Well-documented high-quality publicly available dataSlide23
Pulling the pieces together…Slide24
http://
gmrt.marine-geo.org
Consolidated access to resourcesSlide25
Fleet-wide Review of R2R
MBQA Tests & Results
Which tests correspond with issues corrected in GMRT processing?
Are test thresholds correct?
Other tests needed?Slide26Fleet-wide Review of GMRT
Processed Data Statistics
Are there metrics can we programmatically assemble from processed data that can help improve data quality at acquisition?Slide27Compare/Combine Results
PreliminarySlide28Compare/Combine Results
PreliminarySlide29Next Steps…
Quantitatively compare R2R MBQA results with GMRT Processed Data Statistics
Identify which tests are working
Refine MBQA tests/parameters
Code-sharing between MAC-GMRT-R2R
Use fleet-wide data review (MBQA tests + GMRT results) to help improve MAC guidelines/best practices