Markus Horstmann Microsoft Shawn C Cutter Fielding Systems DEVB379 Why IoT DataDriven Insight Telemetry More efficient use of resources reduce cost environmental impact More targeted products and services increase revenue social impact ID: 137064
Download Presentation The PPT/PDF document "Using Service Bus to Connect to the Inte..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1Slide2
Using Service Bus to Connect to the Internet of Things (IoT)
Markus Horstmann, MicrosoftShawn C. Cutter, Fielding Systems
DEV-B379Slide3
Why IoT?
Data-Driven Insight (“Telemetry”)More efficient use of resources (reduce cost, environmental impact)
More targeted products and services (increase revenue, social impact)
=> “Things” = a rapidly expanding source of raw material for the Insight pipeline
Enable Action at a Distance (“Command & Control”)
Engage with customers beyond the point of sale
Preventive maintenance, Best practices guidance, Proactive sales, Remote servicing etc.
=>
From CRM to PRM – “Product Relationship Management”Slide4
That’s what this talk is all about…
Why Service Bus for IoT?Slide5
Service Bus: The Azure Cloud Device Gateway
IoT in the real world: Fielding SystemsTechnology drill down:In-vehicle diagnostics recorder: .Net
Micro Framework and AMQP
Fielding Systems:
Beagleboard
– Embedded Linux and AMQP
“Reykjavik” accelerator: scalable device gateway, HDInsight analytics and moreNew in and around Service BusExpress Queues and Topics: trade off latency vs. durability guaranteesAMQP.Net Lite: client library for .NET Micro Framework, Windows Runtime etc.http://amqpnetlite.codeplex.com/
What we’ll cover:Slide6
Enter details of an upcoming trip into your calendar application
Calendar is configured to share data with the D2D navigation service
Before your trip, you get departure time notifications on your phone
Service calculates when you should leave based on traffic and weather
It also considers when you need to allow time to refuel your car
Service sends notifications to your smart phone as departure time draws near
When you start your car, map and directions are already on display
Service recommends a different path due to construction or congestionGuides you to open parking spots near your destinationAutomatically pays parking with pre-configured accountWhen you exit your car, the navigation experience transfers to your phoneService delivers walking directions to complete your journey
Helps you find your car again when finished
Example: Door-to-Door NavigationSlide7
100,000s
of
Vehicles
100,000s of
Users/Drivers
100,000s of
Users/Drivers
Cloud Platform Enablers for Smart Mobility
Identity Services (WAAD)
Command Router
Telemetry Router
Service Bus + BizTalk
Services/Server
+ Virtual Networks
Notification
Hubs
Mobile
Services
Web Sites
Service Bus
SignalR
Service Bus
Notification Hubs
Custom Protocol Gateway
Mobile Platform
Push Messaging
Mobile Solution Backend
Web Portal
Mobile
Platform
Push Messaging
Vehicle
Information System
Web Sites
Navigation System
Calendar
Management System
Mobile Experience
Portal Experience
3
rd
Party Data
& Services
(Maps, Weather, Traffic,
Events)
Corporate
and Divisional Line of Business and Information
Systems
Service Bus & Web API
Telemetry Adapters
Azure Database
, Blob and
Table Storage
Real-Time Analytics,
HDInsight
Virtual Machines (
IaaS
)
Cloud Services and Web Sites (
PaaS
)
Media Services
Service Bus & Web API
Service Bus & Web APISlide8
IoT Cloud Platform “Stack”
Non-IP
Capable (or willing) Devices
IP
Capable Devices
Cloud
Device Gateway
Custom Code
Cloud Platform Services
Enterprise
Systems
Third-Party
Data
and Services
Field GatewaySlide9
IoT Cloud Platform “Stack”:
Cloud Device Gateway
Non-IP
Capable (or willing) Devices
IP
Capable Devices
Cloud
Device Gateway
Custom Code
Cloud Platform Services
Enterprise
Systems
Third-Party
Data
and Services
Field Gateway
Custom
Cloud Gateway
Pattern 1: Device Direct
Pattern 2: Custom Cloud Gateway
Field Gateway
Field Gateway
Service Bus
Service BusSlide10
Connectivity
Addressability (non-IP devices, firewalls/NATs, online/offline, roaming)Heterogeneity (OS/firmware, power/network constraints, protocols)Security (identity, authorization, privacy, data integrity)Efficient Scale (millions of devices per tenant, at a reasonable cost)
Messaging
Telemetry (collection, filtering, routing, throughput, per-message QoS)
Notifications (targeting devices/device groups within large populations)
Command/Query and Inquiries (correlation, sessions/batching)
Cloud Device Gateway ChallengesSlide11
IPv6 + Virtual Private Networks
Give every device a routable IP addressDevices expose services
for control/query
operations
Remote access enabled within the VPN’s routing domain
Issues
Addressability:
network-layer intervention required, loosely connected devicesSecurity: Devices not protected from each other, control via IP addresses/ports not app-levelScale: Many devices not VPN-capable, infrastructure costly and hard to maintainThe Default IoT Connectivity ModelSlide12
Service-Assisted Communication
Connections are device-initiated and outbound
NAT/Firewall Device (Router)
IP
NAT
Cloud Gateway
Command Source
Port mapping is automatic, outbound
Device does not listen for unsolicited traffic
No inbound ports open, attack surface is minimized
Access-controlled command API
Secure, managed hosting platform
DNS
myapp.cloudapp.netSlide13
Devices connect via open
standard protocolsAMQP 1.0 and HTTP: natively in Service BusMQTT,
CoAP
and others: implement via custom gateway/adapter model
Sockets secured via TLS (or a lightweight variant
)
Each
device gets a dedicated Inbox/Outbox “queue” on the GatewayOutbox: Telemetry, alerts and service invocationsInbox: commands and queriesRequest/reply: correlated messageon top of inbox/outbox messaging channelsSecurity: device can only access its own specific inbox/outbox endpoints (URI’s)
Service Bus as a Cloud Gateway
Backend Components
Cloud Gateway
Outbox
(Telemetry)
Inbox
(Command & Control)
Protocol Head
Command API
DeviceSlide14
Telemetry Routing with the Azure Service Bus
Split the stream
Enable parallel processing
Implement different
QoS
levels
Level and balance the load
Topic
Subs
Filters
Service Bus
Device 2
Receiver 2b
Device 1
Device 3
Receiver 2a
Alerts
Data
Receiver 1
Alert Processor
Storage
Pre-processorSlide15
Routing Commands with the Azure Service Bus
Topic
Subs
Filters
Service Bus
Device 2
Device 1
Device 3
Sender 2
Model A Device 1
Model T
Device 3
Sender 1
Model T
Device 2
Model T
Model A
Target individuals or groups
Set delivery timeouts (TTL)
Deal with spotty connectivity
Traverse NATs/firewalls securelySlide16
Microsoft Service with Cloud Device Gateways:
Azure Intelligent System Service (in Preview)Offers a rich SaaS solution and higher-level PaaS
components for IoT
Service Bus is the Cloud Device Gateway component inside ISS
Visual Studio 2013 Profile Sync
Service Bus terminates all client connection in the Cloud
Halo 4 Game statistics
Service Bus fans-out to backend systems behind a Custom GatewayMore exist and many more to come…The Azure Device Gateway: Service BusSlide17
Fielding Systems: The Digital Oilfield on Azure
Data from peopleData from devicesOil and Gas: Legacy Devices and NetworksSlave devices and Serial networks
Azure Service Bus
Controlled distribution of workload
Redundancy and scalability
Oil and
Gas: Smart Devices and
IoTMake the devices in the field smarterFielding Systems Scenario & DemoSlide18
Azure Service Bus: Pull ArchitectureSlide19
Azure Service Bus: Pull ArchitectureSlide20
Demo
Fielding Systems: ScadaVisorSlide21
Scalability
ResiliencyLocal data processingLoggingCompression
Enhanced device security
Reduction in latency for alarm notifications
Operate on web standards
Problems
addressed
by the Push approachSlide22
Azure Service Bus: Push ArchitectureSlide23
Demo
Fielding Systems: Smart-device Push
Beaglebone
with
Qpid Proton PythonSlide24
Digital Oilfield Big PictureSlide25
Technology Drill down: Vehicle Diagnostics
Blob
Storage Adapter
HIVE DDL
HDInsight
Azure
Table Storage
Office 365 BI:
Excel, Power Maps
HIVE ODBC Adapter
In-Vehicle Device:
.Net
Gadgeteer
FEZ Spider: Arm7, 16MB Ram, 4MB Flash
-
Onboard Diagnostics Module (OBD2)
- Temperature and Humidity Sensor
-
Ethernet module, tethered via WP (“GSM module”)
Service Bus
Outbox Topic(s)
(Telemetry)
Inbox Topic(s)
(Command & Control)
Table Storage Adapter
Orleans Adapter
Companion App:
How is my car?
Where’s my car?
Unlock my car
…
Mobile Service
Custom Real-time alerting
Commands
Alerts
Web Site
Device / User Pairing
Notifications
Device Status Checks
Real-time updates / Alerts
Telemetry
Alerts
Commands
Blob StorageSlide26
Demo / Code walkthrough:
Device-to-Cloud (and back)
- NetMF with AMQP.Net Lite
-Slide27
Use ISS if you want a rich, end-to-end IoT platform
Turn-key client agents, open framework and protocols to roll your ownBackend with rich device
repository and device
management
portal/APIs
Event storage, alerting and analytics, per-device billing model
More information:
http://www.microsoft.com/intelligentsystems/services Use Service Bus directly if you need to Front ISS with a Custom Gateway for custom protocols or device data modelsBuild your own IoT platform or solutionService Bus is the Gateway inside ISSNo matter where you start, your backend and device investments
accrue
Intelligent System Service and Service BusSlide28
Service Bus Messaging
Custom Protocol Gateway
Telemetry Pump and Adapters
Command Gateway
Provisioning Service and Metadata Store
Custom Protocol Gateway Host
MQTT
CoAP
…
Telemetry/Request
Router
Notification/Command
Router
Adapters
Command API Host
Provisioning Service
Device Metadata and Key Store
HDInsight
BizTalk
Orleans
Azure Storage
Azure
Dbs
Service Bus
HTTP
HTTP
Devices
AMQP
1
2
3
4
Configuration
HTTP
Cloud Device Gateway – Reference ArchitectureSlide29
Device Gateway – Partition Topology
The “Partition” is a set of resources dedicated to a specific device population
(or
subset
thereof).
The “Master
”
role manages partition deployment and device provisioning into the partitions.Partition
Master
Partition Repo
Command Topics
Service Bus Standard Protocol
Custom Protocol
Device Repo
AMQP
HTTP
MQTT
Custom Protocol Host
Protocol
Adapters
Telemetry
Pump/Router
N Instances
Telemetry
Adapter Analytics (
HDInsight
)
Telemetry
Adapter
LastValue
(Table)
Telemetry
Adapter
RealTime
(Orleans)
Deployment
Runtime
out0000
outFFFF
…
out0001
out0002
s0001
s0002
s03E7
s0001
s0002
s03E7
s0001
s0002
s03E7
s0001
s0002
s03E7
g0000
/
rte0000
g0000
/rte0001
out0
out1
out2n Groups of m Routers
out0
out1
out2
g0001/rte0000
g0001/
rte0001out0out1
out2
out0
out1
out2Provisioning Runtime
Ingestion Topics
Command
API
Host
Ingestion Topic
Analytics
Subscription
LastValue
Subscription
RealTime
Subscription
Command Response
in0000
inFFFF
…
in0001
in0002
C
A
L
R
C
A
L
R
C
A
L
R
C
A
L
R
…Slide30
Device Gateway – Customer Topology
Global coverage achieved by spreading partitions across multiple Azure regions
Reference architecture supports up to 1000 distinct partitions
Number and distribution of partitions driven by data volumes, business continuity, legal and proximity considerationsSlide31
Device Gateway Demo
Device Gateway deployment & provisioning
HDInsight adapter
Table Storage
adapterSlide32
Express Queues & Topics
Trade off lower-latency and higher-throughput against possibility of message lossPer-message force-persistence flag lets you get durability backSend us your feedback!
Scenarios:
Second screen experiences for gaming
Interactive UX (
SignalR
backplane)AMQP.Net Lite libraryhttps://amqpnetlite.codeplex.com/ AMQP.Net Lite is a lightweight AMQP client library for use on a broad range of .Net Framework platforms, including .Net Micro Framework, .Net Compact.Vehicle Diagnostics Demo will be available as a sample soonWhat’s new in Service BusSlide33
Roadmap
First and third-party IoT services are being built today on Service BusIoT related enhancements to Service Bus are coming soon
We will simplify and enrich support for Device Gateway patterns
Learn more
Clemens Vasters’
video blog:
http://channel9.msdn.com/Blogs/Subscribe
Explore new SB features: Express Queues and Topics, AMQP.Net Lite We are looking for customers and partners!Connecting devices at scale to backing services on AzureWilling to work closely with the product team and share learnings/resultsContact:
Markus Horstmann –
markush@microsoft.com
Shawn C. Cutter -
shawn.cutter@fieldingsystems.com
Next StepsSlide34
Visit the Developer Platform & Tools Booth
Having a
friend
buy
your coffee?
Yea, it’s kind of like that.
MSDN Subscribers get up to $150/
mo
in Azure credits.
Stop by the Developer Platform and Tools
booth and
visit
the MSDN
Subscriptions station
to activate
your benefits and
receive
a gift
!
http
://
aka.ms/msdn_teched
3 Steps to New
Gear! With
Application Insights
Create a Visual Studio Online account
http://visualstudio.com
Install Application Insights Tools for Visual
Studio Online
http://aka.ms/aivsix
Come to our booth for a
t-shirt and a chance to win!
VSIP QR Tag
Contests
Visit
our booth to
join the hunt for cool prizes!Slide35
Resources
Microsoft Engineering Stories
How
Microsoft Builds Software
http://
aka.ms/EngineeringStories
Visual Studio
Industry
Partner
Program
Meet Our New Visual Studio Online Partners
or
Join Now.
http://vsipprogram.com
Visual Studio | Integrate
Create Your Own Dev Environment
http://integrate.visualstudio.com
Development tools & services
for
teams
of all sizes
http
://www.visualstudio.com Slide36
Complete an evaluation
and
enter to win!Slide37
Evaluate this session
Scan this
QR
code
to evaluate
this
session.Slide38
©
2014
Microsoft Corporation. All rights reserved. Microsoft, Windows,
and
other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.