/
Azure, Caas and Storage Practical Data Driven Configuration as a Azure, Caas and Storage Practical Data Driven Configuration as a

Azure, Caas and Storage Practical Data Driven Configuration as a - PowerPoint Presentation

luanne-stotts
luanne-stotts . @luanne-stotts
Follow
353 views
Uploaded On 2019-01-26

Azure, Caas and Storage Practical Data Driven Configuration as a - PPT Presentation

Service with Azure Storage Copyright 2016 Monza Clound LLC Agenda Who we are Monza Cloud LLC Note Were not here to read MSDN articles code snippets to you Were here to demonstrate production code techniques and share lessons ID: 748320

table storage service configuration storage table configuration service queue azure cloud queues

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Azure, Caas and Storage Practical Data D..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Azure, Caas and Storage

Practical Data Driven Configuration as a Service with Azure Storage

Copyright © 2016 Monza Clound, LLCSlide2

Agenda

Who we are, Monza Cloud, LLC.

Note: We’re not here to read MSDN articles / code snippets to you. We’re here to demonstrate production code techniques and share lessons

learned

How we use CaaS (Configuration as a Service) with database driven configuration repositories

Quick introduction to AzStudio, our cloud APIs and Tools

Demonstrate the basic steps to work with storage using MSDE code

Practical Demonstrations :

CaaS at work – Shortcuts and environments explained

Blob storage creation, routing, and testing

Queue creation, routing and testing. Demo BigItem

Table Storage creation, routing and testing. Demo BigItem

Closing statement

Q&A, reference to resourcesSlide3

Monza Cloud, LLC

Founded in 2015Team worked for several years on various Azure projects before forming company

We specialize in rapid development of SaaS offerings that take advantage of our Azure framework

We’re not a big “move this VM” or “Modify this existing app” team, we’re here to write your apps in the cloud for the cloud

Extensive experience deploying “internet of things” solutionsSlide4

What is CaaS

It is a web service that examines the client’s DNS information and optional attributes to determine it’s config informationIt delivers either a JSON master configuration file or a database connection string to the configuration database

CaaS can be considered the “bootstrap” for our environments

Configuration manager then brokers the config information to the rest of the codeSlide5

CaaS Advantages

No more config files. No need for specialized tools to manage these filesDatabase driven config means referential integrity checks and data enforcement, easy to back up and restore

Configuration data can be segmented and management of specific portions of configuration can be distributed to different personnel

Configuration changes can be picked up on-the-fly (runtime) by your services without recycling or redeploying the service

Can be made exceptionally redundant, can gracefully handle multiple layers of failure without redeploymentSlide6

CaaS - Bootstrap

Service Starts

List of

Master endpoints

Pop next endpoint

Call endpoint, asking for configuration

data

Configuration service uses reverse DNS / keys to lookup config

Missing lookup or service request fails

Return ENVIRONMENT settings and DATABASE connection

Service Runs

Service sets environment values and config database info (singleton)

No more endpoints!

Service Fails

Note: JSON file can replace database connectionSlide7

Service – Post Bootstrap

CaaS response has now set the global property to an environment id. This drives ALL configuration lookup operationsApplication now consumes services / functionality as necessary

and a connection manager dictates routing

Programmers ignore all of this, they use

shortcuts

Control and management are abstracted across multiple projects, teams and responsibilities are segmented, making cloud configuration a breezeSlide8

AzStudio : Quick Overview

For today’s demos we’re using our the Monza Cloud product suite for Azure, AzStudioAzStudio C# API’s == JQuery or Angular in the JavaScript world

AzStudio == Visual toolset used by programmers and DevOps

If you’re developing in Azure you WILL develop something like this in your environment. Either using a collection of tools / scripts or something consolidated like our product

You will use these patterns and codify them in your organization in order to survive the rapid decomposition of resources in a SaaS /

Paas

EnvironmentSlide9

Common Pattern in Azure Storage

Create your storage accountUse connection string to connect to account, preserving this connection reference in memory (singleton pattern)

Get a reference to the storage type (blob, queue, table storage). Keep that reference in memory as well

Get a reference to the specific resource (container, queue client , table client). Yes, preserve the reference for future operations

Perform actions. Using singleton patterns will speed up subsequent steps dramatically. Using “infrequent refresh” techniques will allow you to update the above information without recycling rolesSlide10

BigItem

Before we go to practical demos, let’s talk about overcoming limitations in Queues and Table StorageBoth systems have a maximum item size (64K and 1MB)

The work around to this is to synchronize additional information in a blob storage container

The trick to this work around is abstracting the mechanics of this extension out so developers are not constantly having to be aware of this arrangement

Our solution is called “BigItem” and we’ll demonstrate it during our practical demosSlide11

Topic #1: Blobs

Blobs are any binary (file) data you want to store in the cloudBlobs are stored in a

container

in a

storage account

Containers can be set public (with or without browsing) or private.

Most errors occur with containers that are private and you want public access

Blob performance depends on many things, size and how you access will effect performance the most

SAS (Shared Access Service) allows you to grant limited-time access to a file on a private container

CORS is now available at the storage account level, allows you to publish web services / scripts to storage containersSlide12

Practical Demonstration #1: Blobs

We’re going to set up a storage container type (Crowd picks name)Set up routing table elements for DEV and QA (but not production)

We’ll create a blob “shortcut” to consolidate routing

Test 1 – Read, write and delete contents in DEV

Test 2 – Read, write and delete contents in QA

Test 3 – Demonstrate error raised when attempting to hit productionSlide13

Blob Code Examples

Here’s how our code looks when using Blob ServicesYou’ll want to create a basic set of library calls for your product(s)

Makes inheritance of things like

StorageAccount

connections reusable and cacheable

Upload

:

Az.BlobStorageManager.UploadFileToStorage

(

path,shortcutName

);

Download:

Az.BlobStorageManager.

DownloadFileFromStorage(

sourcePath, destPath

, shortcutName);Delete: Az.BlobStorageManager.DeleteFile

(fileName,shortcutName);List:

Az.BlobStorageManager.GetContainerFiles(shortcutName);Slide14

Blobs – MSDN Links and AzStudio Example

How to set up a storage account: https

://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account

/

Blobs

and Containers:

https://

msdn.microsoft.com/library/azure/dd179376.aspx

These links are raw MSDN documentation.

AzStudio’s

APIs wrap these in much easier code. For example, to write a file to a blob is as simple as:

Az.BlobStorageManager.UploadFileToStorage

(

pathToBlob

,”

myshortcutname”);Slide15

Topic 2: Queues

Queues are used to pipe information between systemsQueues reside in a

container

in a

storage account

“Queue Driven Workflow” is a basic tenant of cloud architecture, so you’ll need to use queues quite a bit

Queue throughput: (very roughly) around 5000 1K transactions a minute

Size of transaction has a high impact on queue throughput

Have plans in place to federate your queues and a robust library around this to reduce complexity in

your systemsSlide16

Practical Demonstration #2: Queues

Set up a queue container type and routing informationCreate queue shortcut for DEV and QA, but not production

Demonstrate writing the reading with the queue.

Explain our wrapper and encryption in queues “at rest”

Demonstrate encryption of queue item contents

Talk about federation and BigItem. Demonstrate BigItemSlide17

Queues – MSDN Links and AzStudio Example

Setting up Queue and using it in MSDN:

https

://

msdn.microsoft.com/library/azure/dd179353.aspx

Using queues in AzStudio uses the same concepts as blobs. Just use a shortcut (encryption and BigItem are transparent). Here’s an example.

Az.QueueManager.GetQueue

(“

myqueueshortcut

”).

Enqueue

(

myObject

);Slide18

Topic 3: Table Storage

Table storage is high-volume schema-less entity storageTables store any class that implements

ITableEntity

, which has several restrictions

Two properties make up the primary key for a row

PartitionKey

and

RowKey

You can only really effectively query against these two keys. While you can query against any of you’re classes properties, not using these two properties will invoke a full table scan

Careful planning around keys is the core principle of using table storage

See

https://azure.microsoft.com/en-us/documentation/articles/storage-table-design-guide/#

table-design-patterns

for detailed pattern / anti-pattern informationSlide19

Practical Demonstration #3: Table Storage

Create the table as a container type

Map the table in DEV and QA

Write and read from DEV

Write and Read from QA

Show that PROD is not responding (errors due to missing table routing)

Explain how LINQ and FILTERS work

Explain BigItem with table storageSlide20

Conclusion

You’re cloud development and deployment can get overly complicated with configuration filesEven with specialized scripts and tools the files just don’t lend themselves to clean segmentation and security practices

The tools and techniques demonstrated tonight should give you ideas about your own efforts

Often overlooked is having these tools enforces practices, which help you overcome business culture issues

Now we’ll open up to Q&A