Introducing Azure Stream Analytics - PowerPoint Presentation

Slide1
Slide2

Introducing Azure Stream Analytics

Judy MeyerPrincipal Group Program Manager

DBI-B316Slide3

Breakout Sessions

CDP-B307 Azure Event Hub

(Fri 2:45)

Related content

Lab DBI-IL204 Speed Lab Azure Stream Analytics (Fri 8:30)

Find Me Later At…

Microsoft Solutions Experience Location (MSE):

Data Platform & Business Intelligence

Ask the ExpertsSlide4

Introducing…Azure Stream AnalyticsSlide5

Data at Rest

Data in Motion

whySlide6

What are customers wanting to do?

Smart grid

CRM alerting sales with customer scenario

Data and identity protection services

Real-time fraud detection

Click-stream analysis

Real-time financial portfolio alerts

Connected car scenario

Real-time financial sales trackingSlide7

How do customers create a real-time streaming solution?

Time

Development and operations resources

Infrastructure – Procure and setup

Develop solution (code) for ingress, processing and egress

Develop solutions to integrate with other components like ML, BI etcDevelop solutions to manage resiliency, such as infrastructure failuresDevelop solutions and infrastructure for increasing scale with business growthMonitoring and Troubleshooting of solutionSlide8

Customers using Azure Stream Analytics

Infrastructure – Procure and setup

Develop solution (code) for ingress, processing and egress

Develop solutions to integrate with other components like ML, BI

etc

Develop solutions to manage resiliency, such as infrastructure failuresDevelop solutions and infrastructure for increasing scale with business growthMonitoring and Troubleshooting of solution

From Event or Data Streams to Real Time Insights

in less time with less people resourcesSlide9

High Throughput

Low Latency

what

Dashboard

Monitoring

Internet of Things

Command & Control

Real Time

Blob ArchivingSlide10

Canonical Event-driven Scenario

Ingestor

(broker)

Collection

Presentation and action

Event producers

Transformation

Long-term storage

Event hubs

Storage

adapters

Stream

processing

Cloud gateways

(web APIs)

Field

gateways

Applications

Legacy IOT

(custom protocols)

Devices

IP-capable devices

(Windows/Linux)

Low-power devices (RTOS)

Search and query

Data analytics (Excel)

Web/thick client

dashboards

Service bus

Azure DBs

Azure storage

HDInsight

Stream Analytics

Devices to take actionSlide11

Aerocrine Experience

Join me in Welcoming…Anders Murman,

CTO of Aerocrine Slide12

Improving Asthma Diagnosis and Treatment

NIOX

®

MINO

®

NIOX

®

VERO

®

Slide13

Better Asthma Outcomes

FeNO testing improves patient outcomes while decreasing exacerbations.

Cost-Effectiveness

FeNO testing saves healthcare costs by decreasing ER visits and hospitalizations.

Physician and Patient Behavior

FeNO testing improves appropriate medication use, predicts relapse, and provides compliance monitoring.

Value Added by FeNO Testing

Unmet Need

Aerocrine is building support through promoting the value of FeNO to KOLs, payers and providers

1

Establish FeNO as Standard of Care Slide14

2

Drive Penetration in Defined U.S. Professional Segment

Currently,

Aerocrine

has 26 sales territories staffed, 4 regional managers and 3

cslsSlide15

Microsoft Connectivity project

NAV

CRM

Azure

MS/AER Streaming Analytics

MS/AER PowerBI reporting and app publishing

Customer Support

Local sales reps

MgmtSlide16

More AboutAzure Stream AnalyticsSlide17

Introducing stream analytics

Mission critical reliability and scale

Enables rapid development

Fully managed

real-time analyticsSlide18

Intake millions of events per second

Process data from connected devices/apps

Integrated with highly-scalable

publish-subscriber

ingestor

Easy processing on continuous streams of data Transform, augment, correlate, temporal operationsDetect patterns and anomalies in streaming dataCorrelate streaming with reference data

Real-time analyticsSlide19

No challenges with deployment

No hardware acquisition and maintenance

Bypasses deployment expertise

Up and running in a few clicks (and within minutes)

No software provisioning and maintaining

Easily expand your business globallySlide20

Introducing stream analytics

Mission critical reliability and scale

Enables rapid development

Fully managed

real-time analyticsSlide21

Guaranteed events delivery

Guaranteed not to lose events or incorrect

 output

Preserves event order on per-device basis

Guaranteed

business continuityGuaranteed uptime (three nines of availability)Auto-recovery from failures Built in state management for fast recovery

Mission critical reliabilitySlide22

Elasticity of the cloud for scale up or scale down

Spin up any number of resources on demand

Scale from small to large when required

Distributed, scale-out architecture

Scale using slider in Azure Portal and not writing code

Low startup costsProvision and run Streaming solution for as low as $25/month Pay only for the resources you useAbility to incrementally add resourcesReduce costs when business needs changes

No challenges with scale Slide23

Introducing stream analytics

Mission critical reliability and scale

Enables rapid development

Fully managed

real-time analyticsSlide24

Decrease bar to create Stream Processing Solutions

via SQL-like Language

Easily filter, project, aggregate, join streams, add static data with streaming data, detect patterns or lack of patterns with a few lines of SQL

Built-in temporal

semantics

Development and debugging experience through Azure PortalManage out-of-order events & actions on late arriving events via configurationsRapid DevelopmentRapid DevelopmentSlide25

Scheduling and

monitoring built in

Built-in monitoring

View your system’s performance at a glance

Help you find the cost-optimal way of deploymentSlide26

End-to-End Architecture Overview

Data Source

Collect

Process

Consume

DeliverEvent InputsEvent HubAzure BlobTransformTemporal joinsFilter

Aggregates

Projections

Windows

Etc.

Enrich

Correlate



Outputs

SQL Azure

Azure Blobs

Event Hub

☁

BI

Dashboards

Predictive Analytics



Azure

Storage

Temporal Semantics

Guaranteed delivery

Guaranteed up time

Azure Stream Analytics

Reference Data

Azure BlobSlide27

Azure Stream Analytics

In ActionDipanjan Banik

Program ManagerSlide28

SELECT

count(*), Topic

FROM

Tweets

GROUP

BY Topic, TumblingWindow(second, 5)Let’s count tweets by topic…That’s all. Just 2 (very short) lines of code.Slide29

Contoso is about to launch a new product to the market. To do an effective product launch they want to get real-time insights into what customers are talking about their products by tapping into social feeds.

Real-time analytics demoSlide30

Pain Points with other Streaming Solutions

Not an end to end solution

Hard to develop

Need expertise and special skills

Costs lot of money on Development

@ApplicationAnnotation(name="WordCountDemo")public class Application implements StreamingApplication{  protected String fileName = "com/datatorrent/demos/wordcount/samplefile.txt";  private Locality locality = null;   @Override  public void populateDAG(DAG dag, Configuration

conf

)  

{   

locality =

Locality.

CONTAINER_LOCAL

;     

WordCountInputOperator

input =

dag.

addOperator

("

wordinput

", new

WordCountInputOperator

());    

input.

setFileName

(

fileName

);    

UniqueCounter

<String> wordCount = dag.

addOperator("count", new UniqueCounter<String>());     dag.

addStream("wordinput-count

",

input.

outputPort

,

wordCount.

data

).

setLocality

(locality);     

ConsoleOutputOperator

consoleOperator

=

dag.

addOperator

("

console

", new

ConsoleOutputOperator

());    dag.addStream("count-console",

wordCount.count, consoleOperator.input);   }  

}

 Slide31
Slide32

Azure Stream Analytics Query LanguageSlide33

Query Language

You write declarative queries in SQL

No code compilation, easy to author and deploy

Unified programming model

Brings together event streams, reference data and machine learning extensions

Temporal Semantics All operators respect, and some use, the temporal properties of eventsBuilt-in operators and functionsThese should (mostly) look familiar if you know relational databasesFilters, projections, joins, windowed (temporal) aggregates, text and date manipulationSlide34

Scenario – Toll Station

Our toll station has

multiple toll booths

, where a sensor placed on top of the booth scans an RFID card affixed to the windshield of the

vehicles

as they pass the toll booth. The passage of vehicles through these toll stations can be modelled as event streams over which interesting operations can be performed.Entry data streamExit data streamToll IdEntryTimeLicensePlateStateMakeModel

Vehicle Type

Vehicle Weight

Toll

Tag

1

2014-09-10 12:01:00.000

JNB 7001

NY

Honda

CRV

1

1535

7

2

2014-09-10 12:02:00.000

YXZ 1001

NY

Toyota

Camry

1

1399

4

123456789

…

Toll Id

ExitTime

LicensePlate

1

2014-09-10T12:03:00.0000000Z

JNB 7001

2

2014-09-10T12:03:00.0000000Z

YXZ 1001

…

 Slide35

Projections

1, 1450, “VW”,

“Golf”, (…)

2, 1230, “Toyota”,

“Camry”, (…)

1, 2400, “VW”,“Passat”, (…)1, 980, “Ford”,“Fiesta”, (…)SELECT

TollId

,

VehicleWeight

/ 1000

AS

Tons

FROM

EntryStream

1, 1.45

2, 1.23

1, 2.40

1, 0.980

Show me the Toll Id and Vehicle Weight in Tons for all vehicles passing through

the Toll Booth

TimeSlide36

Filters

SELECT

Model

FROM

EntryStream WHERE Make = "VW"1, 1450, “VW”, “Golf”, (…)2, 1230, “Toyota”,“Camry”, (…)1, 2400, “VW”,“Passat”, (…)1, 980, “Ford”,“Fiesta”, (…)

“Golf”

“Passat”

Show me the Model of vehicles manufactured by Volkswagen

TimeSlide37

Tumbling Windows

SELECT

TollId

,

COUNT(*) FROM EntryStream GROUP BY TollId, TumblingWindow(minute,5)How many vehicles entered each toll both every 5 minutes?Slide38

Aggregate functions

Count, Min, Max,

Avg

, Sum

Scalar functions

CastDate and time: Datename, Datepart, Day, Month, Year, Datediff, DateaddString: Len, Concat, Charindex, Substring, PatindexTypesBuilt-in functions and supported typesType

Description

bigint

Integers in the range -2^63 (-9,223,372,036,854,775,808) to 2^63-1 (9,223,372,036,854,775,807).

float

Floating point numbers in the range - 1.79E+308 to -2.23E-308, 0, and 2.23E-308 to 1.79E+308.

nvarchar

(max)

Text values, comprised of Unicode characters. Note: A value other than max is not supported.

datetime

Defines a date that is combined with a time of day with fractional seconds that is based on a 24-hour clock and relative to UTC (time zone offset 0).Slide39

PricingSlide40

Stream Analytics is priced on two variables:

Volume of data processed Streaming units required to process the data stream

Pricing

Meter

Price (USD)

Volume of Data ProcessedVolume of data processed by the streaming job (in GB)$.001 per GBStreaming UnitBlended measure of CPU, memory, throughput. $0.031 per hour* Streaming unit is a unit of compute capacity with a maximum throughput of 1MB/sSlide41

Daily Azure Stream Analytics cost for 1 MB/sec of average

processingVolume of Data Processed Cost -$0.0005 /GB * 84.375 GB = $0.04

per day, streaming max 1 MB/s

non-stop

Streaming Unit Cost -$.016 /hr

* 24 hrs = $0.38 per day, for 1 MB/sec max. throughputTotal cost -$0.38 + $0.04 = $0.42 per day -or- ~$12.60 per monthExample Pricing for Public Preview @ 50%Slide42

27 Hands on Labs + 8 Instructor Led Labs in Hall 7

DBI Track resources

Free SQL Server 2014 Technical Overview e-book

microsoft.com/sqlserver

and

Amazon Kindle Store

Free online training at Microsoft Virtual Academy

microsoftvirtualacademy.com

Try new Azure data services previews!

Azure Machine Learning

,

DocumentDB

, and

Stream Analytics Slide43

Resources

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Developer Network

http

://developer.microsoft.com Slide44

Please Complete An Evaluation FormYour input is important

!

TechEd Schedule Builder

CommNet

station

or PC

TechEd Mobile

app

Phone or Tablet

QR codeSlide45

Evaluate this sessionSlide46

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.