Designing and Delivering Scalable and Resilient Web Services - PowerPoint Presentation

Slide1

Designing and Delivering Scalable and Resilient Web Services

Ron Jacobs

Sr. Technical Evangelist, Microsoft

http://blogs.msdn.com/rjacobs

Slide2

Agenda

Slide3

Simple

Do the simplest thing that will possibly work

Slide4

Scalability

Able

to support the required quality of service as the system load

increases-Wiktionary

Slide5

Typical Web Architecture

Slide6

Web Explosion

Database

IIS/ASP.NET

Application

Web Site’s too slow!!

Database is hot!!

Where did my shopping cart go?

IIS/ASP.NET

Application

IIS/ASP.NET

Application

Servers are crashing

Services are slow

Slide7

Agenda

Slide8

Data Near Processing

ASP.NET

Web Service

Database

Smart Client

Browser

Cache

Cache

Cache

Cache

Cache

Slide9

Good but…

Cache is scoped to machine / process

Machines die

Processes recycleCache memory is limited

Slide10

What if?

You

could have as much cache as you wanted?

You could share a giant cache across servers, services and even clients?

What if this was something you could simply add to the platform for

1

free?

1

Some features may require certain editions of Windows Server

Slide11

Windows Server AppFabric

AppFabric

CACHING

MONITORING

WORKFLOW HOSTING

SERVICE HOSTING

SCALE OUT

HIGH AVAILABILITY

MANAGEMENT

AppFabric

Cache – Formerly known as Code Name “Velocity”

Slide12

Unified Cache View

What is

AppFabric

Caching?

An explicit, distributed, in-memory application cache for all kinds of data

Caching clients can be across machines or processes

Clients Access the Cache as if it was a large single cache

Cache Layer distributes data across the various cache nodes

Slide13

AppFabric Cache

Slide14

Data Distribution - Partitioned Cache

…

Web Tier

Cache Tier

Cache Service

Cache Service

Cache Service

A

C

E

D

B

G

I

H

F

Scale on Data Size -

More machines => More memory to cache

Scale on Cache Throughput -

More machines => keys distributed across more machines => better throughput

ASP.Net

App

Caching Client

ASP.Net

App

Caching Client

ASP.Net

App

Caching Client

A

C

E

D

B

G

I

H

F

Slide15

Scale Test Output

Load

Throughput

Latency

1 Cache Server

As load increases,

throughput fails

to scale

latency increases

Caching Tier

Slide16

Add Second Cache Server

Throughput

Load Max

Throughput increases

Latency decreases

Caching Tier

Load

Latency

Slide17

Add Third Cache Server

Load

Throughput

Latency

Caching Tier

Slide18

Associated Press

Caches metadata and news

Serves

16 million hits per day

Increased the amount of cached data

6 times

.

Slide19

System.Web.Cache

Slide20

AppFabric

DataCache

Slide21

Deployment

Caching Service

Caching Service

Caching Service

Configuration Store

Install

AppFabric

Copy Client DLLs

…

Application

Application

Application

<hosts>

<host name="BL1CDB8083714“

cachePort

="22233"

cacheHostName

="

DistributedCacheService

"/>

…..

</hosts>

<

localCache

isEnabled

=“true" ../>

<security … />

Caching Access Layer

Caching Access Layer

Caching Access Layer

.NET 3.5 SP1 OR .NET 4

Configure

AppFabric

.NET 4

Update

Web.config

Slide22

Usage Pattern – Cache Aside

(Explicit Caching)

// Read from Cache

Toy toyObj = (Toy)

catalog.Get

("toy-101");

Application

Caching Service

Database

// If Not present in the cache

if (toyObj == null)

{

// Read from backend..

toyObj =

ReadFromDatabase

();

// Populate Cache

catalog.Put

("toy-101", toyObj);

return toyObj;

}

Caching Access Layer

Slide23

Administration

PowerShell

cmdlets

are used to administer the cache cluster Rich set of cmdlets for

Cache cluster management

Cache creation and monitoring

Slide24

Hello

AppFabric

Cache

Demo

Slide25

Using PowerShell

Remember – PowerShell can also be called from .NET Code!

Slide26

AppFabric

Cache

Codeplex

Toolhttp://mdcadmintool.codeplex.com/

Demo

Slide27

Security

Domain Based Security Option

Domain Account / Local Account based Authentication

Only authorized servers can join the cluster

Only authorized clients can connect to the cluster

Transport Level Security

Turn on/off Signing or Encryption

Can turn off Cache Security

Use Firewalls,

IPSec

, VLANs to protect cache

grant-cacheallowedclientaccount  RedDomain\Machine1$

grant-cacheallowedclientaccount  RedDomain\John

Slide28

AppFabric Caching Logical Hierarchy

Host

Physical processes hosting AppFabric Caching instance.

Named Caches

Can span across machines

Defined in the configuration file

Cache Item

Key, Payload (Object ), Tags, TTL, Timestamps, Version

Regions

Physically co-located Container of Cache Items

May be implicit or explicitly created

Regions

Region A

Key Payload Tags

121

xxxx

“Toy” “Child”

123

yyyy

“Toy” “Chair”..

Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects

AppFabric Caching Service

Named Cache : Product Catalog

Named Cache : Electronics Inventory

AppFabric Caching Service

AppFabric Caching Service

AppFabric Caching Service

Slide29

AppFabric Caching API

// Create instance of cachefactory (reads appconfig)

DataCacheFactory fac = new DataCacheFactory();

// Get a named cache from the factory

DataCache catalog = fac.GetCache("catalogcache");

// Simple Get/Put

catalog.Put("toy-101", new Toy("Puzzle", .,.));

// From the same or a different client

Toy toyObj = (Toy)

catalog.Get

("toy-101");

// Region based Get/Put

catalog.CreateRegion

("

toyRegion

");

// Both toy and toyparts

are put in the same region catalog.Put

("toy-101", new Toy( .,.), “toyRegion”);Catalog.Put("toypart-100", new ToyParts(…), “

toyRegion

”);Toy toyObj = (Toy)

catalog.Get("toy-101“,"toyRegion");

Slide30

Access APIs – Tagging Items

Tag

hotItem

= new Tag("

hotItem

");

catalog.Put

("toy-101", new Toy("Puzzle"),

new Tag[]{

hotItem

}

, “

toyRegion

”);catalog.Put("toy-102", new Toy("Bridge"), “toyRegion

”);// From the same or a different client

List<KeyValuePair

<string, object>> toys =

catalog.GetAnyMatchingTag("toyRegion", hotItem);

Slide31

Types of Data

Reference

Activity

Resource

Primary

Read Only

Read-Write Not shared

Read-Write, Shared

Catalog Data

Shopping Cart

Auction Data/Seat

Assignment

Web Tier

Distributed Cache

Shopping Cart

Grocery Catalog

Grocery Inventory

Grocery Shop

Slide32

Application

Application

AppFabric Caching Client

Reference Data – Performance

Catalog data doesn’t change often

Unnecessary network cost to access from different machines

Solution – Local Cache

Put(K2, v3)

Routing

Table

Cache2

Cache1

Primary for K2,V2

K2, V2

Primary for K1,V1

K1, V1

Cache3

Primary for K3,V3

K3, V3

AppFabric Caching Client

Local Cache

Routing Table

K2, V2

Get(K2)

Get(K2)

K2, V3

Slide33

Reference Data – Bulk Get

Bulk Fetch from region

200-300k ops per second

Fewer network calls

Catalog.BulkGet

(

new List<string>(){“toy-101”, “toy-102”} ,

“

toyRegion

”);

Slide34

Activity Data – Session Integration

…

Session State stored in AppFabric Caching

Cache Service

Caching Service

Load Balance Requests

No more sticky routing

Caching Service

Scale your Session Store

Dynamically

Highly Available

Drop in AppFabric Caching

SessionStoreProvider

Allows session state to be shared amongst multiple applications

Application

Caching Access Layer

Application

Application

Caching Access Layer

Caching Access Layer

<

sessionState

mode="Custom“

customProvider

="

SessionStoreProvider

">

<providers>

<add name="

SessionStoreProvider

"

type=“

Microsoft.Data.Caching.DataCacheSessionStoreProvider

,

ClientLibrary

“

cacheName

="<

YourNamedCache

>"/>

</providers>

</

sessionState

>

Slide35

Application

Application

(K2, V2)

Cache2

Cache1

Cache3

Primary for

Activity Data - Availability

Get(K2)

Primary for

Primary for

K3, V3

AppFabric Caching Client

Routing Table

K2, V2

PUT

Secondary for

K2, V2

K1, V1

Secondary for

K3, V3

Secondary for

K1, V1

AppFabric Caching Client

Routing Table

K2, V2

Replication Agent

K2, V2

Slide36

Resource Data - Optimistic Locking

GetCacheItem returns a version object

Every update to an object internally increments it's version

Supply the version obtained along with the Put/Remove

Put/Remove will succeed only if the passed in version matches the version in the cache

Version Based Update

Time

Client1

Client2

(Different

Thread

or process)

T0

CacheItem item =

catalog.GetCacheItem(“PlayerRegion”, ”Zune”);

CacheItem item =

catalog.GetCacheItem(“PlayerRegion”,

”Zune”);

T1((ZuneObject)item.Object).inventory --;((ZuneObject)item.Object).inventory

--;

T2

catalog.Put(“PlayerRegion”, “Zune”, item.Object, item.Version);T3 catalog.Put(“PlayerRegion”, “Zune”,

item.Object, item.Version);

// Version mismatch// Client must retry again

Two clients access the same item

Both update the item

Second Client gets in first; put succeeds because item version matches; atomically increments the version

First client tries put;

Fails because the versions don’t match

Slide37

K1

Resource Data - Pessimistic Locking

Take locks on non-existent keys

Allows you to co-ordinate calls for data

Client1:

GetAndLock ("k1")

Client2:

GetAndLock ("k1")

Client3:

Get ("k1")

Regular Get succeeds

GetAndLock gets lock handle

Other GetAndLock on same item fails

Slide38

Data Race

GET

GET

GET

Slide39

Lock Non-Existent Key

GET/LOCK

GET/LOCK

GET/LOCK

Cache Service

Caching Service

Cache Service

Slide40

Composite Race

CALL

WAIT

WAIT

Cache Service

Caching Service

Cache Service

Slide41

Composite Race

PUT

UNLOCK

GET

GET

Cache Service

Caching Service

Cache Service

Slide42

Resource/Activity Data – Tracking Changes

Cache Event notifications

Register on any client to notify changes

Batched Notifications

DataCache.Register

CacheLevel

Callback

(

int

filter,

DataCacheChangeCallback

delegate);

DataCache.RegisterRegionLevelCallback(

String region, int filter, DataCacheChangeCallback delegate); DataCache.RegisterKeyLevel

Callback( String region, String key, int

filter, DataCacheChangeCallback

delegate);

Slide43

Application

Cache2

Cache1

Primary for

K2, V2

Primary for

K1, V1

Cache3

Primary for

K3, V3

Scalable Notifications

AppFabric Caching Client

Routing Table

Register Notification for Key “K3"

Map Keys

to Partition

Poll

Required Nodes

Nodes Return List of Changes LSN Order

Partition: P2

Last LSN: 19

Call Delegate

Store Last LSN

Change Log

Partition P1

Add K2

Del K32

Change Log (Partition P2)

Del K32

Del K43

Change Log

33 Add K1

34 Del K22

Slide44

Agenda

Slide45

Data Center

Pre-Fetch

Hospital

Slide46

Data Center

Pre-Fetch

Hospital

Remote Clinic

Slow!!

WAN

Slide47

Data Center

Pre-Fetch

Hospital

Remote Clinic

Cache Service

WAN

Slide48

Agenda

Slide49

Web Platform Installer

Slide50

Select Enterprise

Slide51

Install AppFabric

Slide52

Install AppFabric

Slide53

endpoint.tv

Slide54

AppFabric on MSDN

http://msdn.microsoft.com/AppFabric

Slide55

Slide56

©

2010 Microsoft

Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,

it

should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.

MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.