Underexplored Research Topics - PowerPoint Presentation

Slide1

Underexplored Research Topics

An Industry PerspectiveErik Hoelesri

e

sri

Super Secret

Research Laboratory

,

August

2011Slide2

AgendaHistoryBig DataImagery and VideoLidar, Point Clouds, and 3D

Mobile DevicesGeostreamingDynamic GIS2Slide3

Early HistoryProblems – Forrest Management, Land Management, Transportation Planning, Military, …People – Garrison, Tobler, Tomlinson, Horwood, Fisher, etc.

Places – Univ. of Washington, US Forest Service, Biological Records Centre, Canada Land Inventory, Harvard Graphics Lab3Slide4

1958Univ. of Washington Department of Geography – center of intense advanced research on scientific quantitative geography (William Garrison

and his students) – the ‘new geography’Developed now classical techniques in spatial analysis, statistical methods, measure of spatial distributions, techniques of spatial comparison, 3D and n-dimensional analysis, network analysis, and geographic modeling techniques4Slide5

The New GeographyThe change involved the greater use and emphasis placed on quantitative techniquesExperimental and statistical methodologiesResulted in a new generation of geographers trained in contemporary methods of theory construction and data analysis

William Garrison’s Geography 426 class (Statistical Methods) in 1955 was a shock; students trained in statistical methods as well as an introduction to computers and their application (on an IBM 601)5Slide6

The New GeographyStudents of William Garrison

:Brian Berry – urban and regional research sparked geography’s social-scientific revolution - the most-cited geographer for more than 25 years; member NASWilliam Bunge – theoretical geography (dependence of geographical theory on geometry and topological mathematics)John Nystuen – fundamental spatial concepts (distance, orientation, connectivity)Waldo Tobler – mathematics of projection, cartographyOther students: Duane Marble, Richard Morrill, Michael Dacey, John Kolars, Art Getis, Bob Mayfield, Ron Boyce

6Slide7

The New GeographyOther active UW faculty included Edward Ullman and Donald HudsonAs PhDs moved, many other faculty at other universities became involved in the movement; e.g.,Richard Chorley, Peter Haggett

, John Borchert, Leslie Curry, Leslie King, and Maurice Yeates7Slide8

1958TERCOM (Terrain Contour Matching) development starts at the USAF’s

Wright-Patterson AFBMissile guidance systemBasic premise is any geographic location on Earth is uniquely identified by the vertical contours of the surrounding terrainReference contour data stored in guidance system computerFirst digital terrain modelFirst used with SLAM (Project Pluto) – nuclear ramjet powered supersonic low altitude cruise missile

SLAM

8Slide9

1960

Led by Robert Miller, the US Forest Service creates new forest inventory system using punch cards on the IBM 650 electronic tabulatorsConsidered major breakthrough in compiling data summaries – a true paradigm shift in processing field dataIBM 1620 moved USFS beyond tabulators; programmable in FORTRANBegan hiring people to program (compile, edit, and analyze the data)Slide10

1962

Max Waters and Franklyn Perring (Biological Records Centre - BRC) author the Atlas of British FloraBRC held the atlas data on record cards and punched cards~1700 speciesUsed mechanical equipment for data-processing, using 40-column punched cardsOne of the earliest machine readable geographical databases

10Slide11

1963

Development of CGIS (Canada Geographic Information System) starts, led by Roger TomlinsonSystem was needed to analyze Canada's national land inventory and pioneered many aspects of GIS

A very significant milestone

First widespread use of “geographic

information system” terminology (1966)

Over 40 people actively involved in

developing CGIS between 1960-1969

Built by IBM under contract to

the Canada Land Inventory

Tomlinson

Roger won the

1995

Anderson Medal of Honor

Association of American Geographers

Roger won the

Alexander Graham Bell Medal

Extraordinary Achievement in Geographic Research

Roger became a charter

member in 2005 of the

URISA Hall of Fame

“[Tomlinson is] generally recognized as the ‘father of GIS’.”

- URISA Hall of Fame

11Slide12

CGISAdvances pioneered by CGISFirst cartographic scanner (48”)Raster to topological vector conversion (

Don Lever)Integration of scanning, digitizing, and keypunch data encodingMorton coding (indexing) and compression (Guy Morton)Topological coding of boundaries (first known use of the link/node concept); attaching polygon attributes to points (spaghetti and meatballs)Automated edge matching across tiles/sheetsSpatial coordinate systemsCommand language for data overlay

Paul Henderson

12Slide13

1963

Edgar Horwood (Washington) conducts training workshop at Northwestern on his Card Mapping Program and Tape Mapping ProgramPrograms displayed thematic data associated with statistical administrative zonesInspired Howard Fisher to create SYMAPHorwood led the creation of URISA and served as first presidentPrior to 1960, offered first academic course utilizing computer processing of geographic information (according to Nick Chrisman)

Don became a charter

member in 2005 of the

URISA Hall of FameSlide14

Horwood’s Short Laws

Good data is the data you already haveBad data drives out goodThe data you have for the present crisis was collected to relate to the previous oneThe respectability of existing data grows with elapsed time and distance from the data source to the investigatorData can be moved from one office to another but it cannot be created or destroyed14Slide15

Horwood’s Short Laws

If you have the right data you have the wrong problem and vice versaThe important thing is not what you do but how you measure itIn complex systems there is no relationship between information gathered and the decision madeAcquisition from knowledge is an exceptionKnowledge flows at half the rate at which academic courses proliferate15Slide16

1964

The Harvard Lab for Computer Graphics and Spatial Analysis was established by Howard FisherLarge grant from the Ford FoundationVery significant research center, created

pioneering software for spatial data handling

Many key individuals in industry participated:

Jack Dangermond, Scott Morehouse, Hugh Keegan,

Duane Niemeyer, and Lawrie Jordon (Esri)

David Sinton (Intergraph), Lawrie Jordon and Bruce Rado (ERDAS)

Fisher

The Lab became a charter

member in 2005 of the

URISA Hall of Fame

“[The Lab] was an important early moment in the development of what has evolved in GIS over the past four decades. The contributions of the lab included the training of many creative students and researchers who left the lab to make greater advances elsewhere.”

- URISA Hall of Fame

16Slide17

Harvard LabMany key academics also participated:Nick ChrismanGeoff Dutton

Randolf FranklinTom PoikerCarl SteinitzWilliam Warntz17Slide18

Harvard PackagesSYMAP – general purpose mapping, output on line printer, simple to use, enormous interest

CALFORM – SYMAP on a pen plotter, table of point locationsSYMVU – 3D perspective views of SYMAP output, first new form of spatial displayGRID – raster cells, multiple layersPOLYVRT – topology, format conversionODYSSEY – comprehensive vector analysis, first robust and efficient polygon overlay (including sliver removal)

18Slide19

1970First Law of Geography by Waldo Tobler

Everything is related to everything else, but near things are more related than distant thingsFirst GIS conference sponsored by the International Geographical Union (IGU)Representatives of all known GIS systems invited40 participants

ToblerSlide20

Big DataSlide21

Big DataBasically, extremely scalable analyticsThe three VsVolume – petabytes and more

Velocity – real time acquisition and analysisVariety – structured, unstructured, and semi-structured21Slide22

Big DataExample dataFull motion videoMulti/hyper-spectral imageryCell phone calls

Register transactionsLidar/point cloudsEmail/tweetsSpace/time critical22Slide23

Big DataQuery OptimizationTraditional data types solved long agoBig problems with extended data typesRevert to full table scans

One solution: massively parallel systems, data partitioning, etc.IBM’s Netezza, Oracle’s Exadata, Microsoft’s SQL Azure, Apache’s Hadoop, Teradata, among othersCan a finely tuned query win?23

IBM’s

Blue GeneSlide24

Big DataHow to stream for real time event processingStore to disk/post processAnalyst with manual inspectionSlow

How to persist/partition and rapidly search24Slide25

Big DataSearch criterion controlling storageBased upon predicate filteringTemporal, then spatial, or vice versaPlacenames

Type of attribute/tagsSensor platform attributesColumn-oriented stores25Slide26

Big DataPeeking at data as it flows inIdentify interesting bits, ignore mostWhen is something near, when does something cross …Query optimization problem

Existing frameworksMicrosoft, Oracle, IBM, etc.26Slide27

Move is to dynamic data, applying analytics to large volumes

, reporting facts as available 27Data Streams

Creates the analytics to apply to the data stream based on

Pattern

Analysis

CLOUD BASE

Analytics run in

GRID Computing,

Hadoop

, and

Map Reduce

environment

Analyst

Question focused datasets

Analytics, filters, correlation results

Reports, analysis, other web pages

Alerts, facts, etc.

feed into other Analysts

Discovery

Filter/Store/Analyze

DisseminateSlide28

Big DataDetecting patterns, connecting thing togetherSocial media type stuff with spatial/temporalCash register transactions, cell phone callsPattern of life

“Connecting the dots spatially”Knowns and unknownsHow to assign unknowns to knowsHow to assign confidences28Slide29

Temporal AnomaliesA half-million Enron e-mails from ~150 accounts were sent from 1999 to 2001, a period when Enron executives were manipulating financial data, making false public statements, engaging in insider trading, and the company was coming under scrutiny by regulatorsThe

graph reveals a map of a week's e-mail patterns in May 2001, when a new name suddenly appearedThis week's pattern differed greatly from others, suggesting different conversations were taking place that might interest investigatorsSlide30

Temporal

AnomaliesSlide31

Big DataSpatio-temporal web crawlersTrends and spatial activitySocial mediaMeaningful persistence

Fast, geolocate, query31Slide32

Non-traditional DataLots of non-spatial dataCSV/TXT files, Excel spreadsheets, news feeds, social mediaCoarse grained spatial dataCity level, not down to 10 meters …

E.g., Fukushima radiationGeoprocessing and trend analysis/detection32Slide33

Imagery and VideoSlide34

ImageryData management problems solvedReliable feature extraction Very high value/demand

areaData fusionFull motion video34Slide35

Reliable Feature ExtractionNeed ability to combine image sources with algorithmsE.g., GeoEye-1 + easy to use tools -> rooftopsUsers willing to tie capabilities to sources

With EROS B, here are 4 things you can doWith SPOT5, here are 5 things you can do …Currently, image processing systems need PhD level analysts to run semi-automated systemsSolutions attempt to be too general35Slide36

36

Not so good user experienceSlide37

Reliable Feature ExtractionFeature extraction is a huge deal even if tied to a specific commercial data sourceKey is not to be too generic3D feature extractionTrees from point clouds

Signage from car imagery37Slide38

Object RecognitionIdentification of objects at a coarse levelSimpler than feature extraction (a car vs. Fred’s car)Analyze shape with simple transformationExamples:

Where are the planes at the airport (not which planes are at the airport)?Where are the parking spaces in a city, how many are there?Image understandingNot edge detection, …<image>38Slide39

Object RecognitionIdentification of objects at a coarse levelSimpler than feature extraction (a car vs. Fred’s car)Analyze shape with simple transformationExamples:

Where are the planes at the airport (not which planes are at the airport)?Where are the parking spaces in a city, how many are there?Image understandingNot edge detection, …<image>39Slide40

Object RecognitionIdentification of objects at a coarse levelSimpler than feature extraction (a car vs. Fred’s car)Analyze shape with simple transformationExamples:

Where are the planes at the airport (not which planes are at the airport)?Where are the parking spaces in a city, how many are there?Image understandingNot edge detection, …<image>Family eating a weekend breakfast in the late 1950s/early 1960sSlide41

Full Motion VideoWay too much data to store/archiveNeed to isolate interesting time slices in video streamsE.g., detect the moving objects, store themWant to identify moving objects and tag them

Over time, if a previously identified object reappears in the video stream, highlight it, store it, etc.UASs, security cameras, environmental management (gorillas in the jungle), etc.41Slide42

Full Motion VideoIntegrating full motion video into 3D mapsGeopositioned (and oriented) video insetsE.g., video of truck moving down a dirt road being projected appropriately onto interactive map display

42Slide43

CompressionLots of good general purpose compressionWireless carriers doing a lot of work (3G vs. 4G)Need data specific compression techniques

Compression tied to data or type of requestOptimize the transport; huge issue with wirelessE.g., User A wants 10m accuracy data, User B wants 50m accuracy dataNeed table compression, not row by row compression43Slide44

Data FusionCombining different sensors on different platforms and fuse into derived product that is usefulExamples: Lidar

+ multispectral imagery = RGB Lidar (simplistic classification – green implies tree)AIS + satellite SAR = ships causing surface pollution on the ocean44Slide45

Data FusionFuture example: imagery + range detection = mensurationHow tall is a building?Platform – combination of hardware and software that is placed on satellite, UAS, or aircraft

Platform to find IEDs – hyperspectral imagery and SARPlatform to find rooftops – which sensors and software?45Slide46

ImageryImagery is more than a pretty background in GISHard to convince people it is useful

46Slide47

Topology Editing

Lidar, Point Clouds and 3DSlide48

LidarThere is lots of Lidar out there, what can be done in specific problem domains?E.g, vegetation growth near

powerlinesLidar needs to move beyond basic classification, visualization, surface generation, and change detection; e.g., Immersive point clouds Inside buildings<Petrovic video>48Slide49

Point CloudsUse as reference; interesting opportunities for automated (or

semi-automated) feature extractionComplex examples: oil refineries, manufacturing plants49Slide50

3DStill large disconnect between 2D and 3DCollected and stored differentlyE.g., store a tree as a point with attributes and rules, applied at runtime for visualization or analysis

AutoCADAnalytical 3D still in infancyMining industry is where big interest isVisual analyticsSeeing trends/relationships50Slide51

3DThe ability to manage 3D features is problematicEditing workflows and input devices are still awkward to useLook to Hollywood?

51Slide52

1973First call on a mobile cell phone made by its inventor Martin Cooper

at MotorolaCall placed to his rival Joel Engell, Bell Labs' head of researchResulted in a fundamental technology and communications market shift toward the person and away from the placeCooper stated that his research was inspired by watching Capt. James T. Kirk using his communicator on Star Trek

Cooper

Kirk

James will receive the

2267 Starfleet Medal of HonorSlide53

Modeling HierarchiesModeling building interiors and cities very difficultHierarchies (table, room, apartment, floor, building, block, city, …) without redundancy3D, networks, infrastructure, underground

How to merge/abstract<Sydney video>53Slide54

Versioning

Mobile DevicesSlide55

Mobile DevicesDeveloping for mobile “feels like Windows 95”Primitive by today’s standardsData storage remains the biggest challengeRDBMS-based storage problematic

Still using traditional techniques – e.g., R-tree in conjunction with SQLite>>95% of time is spent drawingRDBMS not optimized for this55Slide56

Mobile DevicesNeed good spatial clustering and column-based structuringNeed to move beyond RDBMS and SQLExpressing search still hard – time, space, attributes, network connectivity, etc.

56Slide57

Mobile DevicesHow do these fit into the remote sensing community?Lots of info with all the photosConsider micro-platforms on mobile devices

Everyone is a walking sensorImagery, accelerometers, inclinometersDoes it make sense to build platforms on mobile devices?57Slide58

Mobile Devices

With mobile, users are standing in the mapDiffers from desktopAre there more effective ways of communicating situational awareness in the field?E.g., driving directions and tilting the displayAugmented realityAudio58Slide59

Mobile DevicesCurrent examplesEarthquake detectionPothole detectionFuture examples

GM’s OnStar-like. system – your phone detects travel on road and sudden violent deceleration; auto-calls for helpYou have a skin rash – take a multispectral image of it and upload to a doctor service. What kind of rash is it? Do I need to be seen by the doctor? Is over the counter OK?Don Cooke’s cat59Slide60

Mobile DevicesCurrent examplesEarthquake detectionPothole detectionFuture examples

GM’s OnStar system – your phone detects travel on road and sudden violent deceleration; auto-calls for helpYou have a skin rash – take a multispectral image of it and upload to a doctor service. What kind of rash is it? Do I need to be seen by the doctor? What do I do?Don Cooke’s cat60

Don Cooke

Mary the GPS CatSlide61

Mobile DevicesTablets are a disruptorScreen real estate, power, usability, portabilityReplacing traditional devices

E.g., Field engineers replacing high-end GPS units and ruggedized PCs (~$5000) with an iPad (~$800) that has built in GPS and wireless; or United Airlines replacing 34 lbs. of pilot charts (11,000 units)Cheaper, lighter, more usefulAre disks needed in the future?Slide62

Data Model Design

Geostreaming Slide63

GeostreamingLots of work during past decade on stream processingBig, sophisticated systems in place, particularly in financial services/Wall Street and defense/intel

Available commercial frameworks:Microsoft, Oracle, IBM, StreamBase, Tibco, etc.Need to extend into spatial domain63Slide64

GeostreamingGeoprocessing on geostreamsCollections of geoprocessing functions that can be assembled on geostreams, much like conventional streams and operators/functions

E.g., real-time heat map generation, geofencing, Detecting abnormal behavior is big topic; e.g.,Ships at sea (AIS data) – smuggling patternsCriminals – flash mobs, flocking, evacuation64Slide65

AIS Data

70,000+ ships being tracked (>300T displacement)65Slide66

Patterns

t

1

t

2

t

4

t

3

t

5

t

15

t

16

t

17

t

18

t

19

?

66Slide67

Patterns

t1t2

t

4

t

3

t

5

t

5

t

4

t

3

t

2

t

1

t

6

t

6

t

7

t

7

t

8

t

8

t

9

t

9

t

10

t

10

t

11

67Slide68

Patterns

t1t2

t

4

t

3

t

5

t

5

t

4

t

3

t

2

t

1

t

6

t

6

t

7

t

7

t

8

t

8

t

9

t

9

t

10

t

10

t

11

?

68Slide69

PatternsMany interesting patterns:Moving too slowly (engine problems)Moving too quickly in bad weather (safety)

Stoppages (accidents)Stoppages, then moving quickly (repairs)Orbiting, scan line patterns (fishing)69Slide70

Patterns70Slide71

Patterns71Slide72

Sensor NetworksMaintain real-time databasesSpecialized forms existE.g., weather, air traffic control, traffic loops, SOSUS, electric power distribution (SCADA, Smart Grid, AMI)

Very difficult and expensive to implement; very custom functionality72Slide73

Visualization

Non-spatial data visualization challengingSocial networks, telco circuitsDerived geometry based upon containmentSchematic representationsManuel Lima’s Visual Complexity is today’s Tufte for non-spatial visualization73Slide74

ProceduralProcedural definitionsPolynomial with coefficients describing 3D solidsParallelGeometry (Quebec)

Procedural descriptions of houses and buildings<CityEngine video>74Slide75

Dynamic GIS Slide76

Dynamic GISManaging dynamic knowledgeMost observations are about a point or a line or a polygon, not their interaction with other points, lines, and

polygonsPairs of places and the dynamics that happen between themRelated to social networking problems 76

Dangermond

GoodchildSlide77

Dynamic GIS“Pairs of places" involves:Proximity and flows

Interactions in time and space – e.g., migrationsDisplay of complex interactions like communications that occur in a social network“Pairs of classes" and their interactions and the ability to link facts at a point to other factsWhat's happening at one location is related to what's happening at other points, both in the spatial and temporal dimensions77Slide78

Dynamic GISNeed to look at spatio-temporal patterns and infer something about the processes that are going on

Can be done best visually with maps, where things like footprints lead to inferences about a patternE.g., flash mobs, flocking, cotravelor, etc.Over time, we've attempted to build these kinds of relationships like point and polygon, polygon overlay, and other geometric static relationships and space78Slide79

Dynamic GISWe have not invented ways to look at spatial interaction, using pairs of things in time like how many phone calls are being made between multiple cars moving in time and spaceThis is basically what people are after today when they're wanting to analyze social networks and community interactions …

A new kind of GIS79Slide80

Dynamic GISDynamic GIS is a paradigm for GIS that enables continuous processing of spatial information to deliver temporally relevant map and analysis outputs

80Slide81

Final ObservationsSlide82

Final ObservationsFocus on disruptive technologyToo much delta (and epsilon) researchWorkflow based researchSolve useful problems that you could explain to your mom

Don’t try to develop the grand unification theoryE.g., procedural / rule-based construction of only buildings and roads, not everything82Slide83

Final ObservationsHow can we take science and make it really easy?Universe is changing; rise of the

Neo-GeosVery difficult for everyoneHow can you make bad data good?83Slide84

Final ObservationsInject some pragmatic engineeringUse real databases (open source/web-based/whatever)

Simple scales, complex failsPrecompute as if your life depends upon itSignificant performance advantages may be gained if reasonable constraints may be madeDisk space cheap, as is hardware running in background84Slide85
Slide86

86