Mike Freedman COS 461 Computer Networks http wwwcsprincetonedu coursesarchivespr20cos461 Continuation of Lec 15 2 HTTP xfer single object Web pages many objects nytimescom ID: 1017623
Download Presentation The PPT/PDF document "Content Distribution Networks (CDNs)" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Content Distribution Networks (CDNs)Mike FreedmanCOS 461: Computer Networkshttp://www.cs.princeton.edu/courses/archive/spr20/cos461/
2. Continuation of Lec 152
3. HTTP xfer = single objectWeb pages = many objects
4. nytimes.com
5. How to handle many requests?Maximize goodput by reusing connectionsAvoid connection (TCP) setupAvoid TCP slow-startClient-server will maintain existing TCP connection for up to K idle seconds5GET / HTTP/1.1Host: www.example.comConnection: Keep-AliveHTTP/1.1 200 OKDate: Tue, 27 Mar 2001 03:50:51 GMTConnection: Keep-Alive
6. Three approaches to multiple requestsPersistent ConnectionsConn 1:Request 1Response 1Request 2Response 2Request 3Response 3Pipelined ConnectionsConn 1:Request 1Request 2Request 3Response 1Response 2Response 3Parallel ConnectionsConn 1:Request 1Response 1Conn 2:Request 2Response 2
7. What are challenges with pipelining? Head-of-line blocking Small xfers can “block” behind large xferNo reorderingHTTP response does not “identify” which request it’s in response to; obvious in simple request/responseCan behave worse than parallel + persistentCan send expensive query 1 on conn 1, while sending many cheap queries on conn 27
8. Google’s SPDY -> HTTP/2Server “push” for contentOne client request, multiple responsesAfter all, server knows that after parsing HTML, client will immediately request embedded URLsBetter pipelining and xferMultiplexing multiple xfers w/o HOL blockingRequest prioritizationHeader compressionhttps://developers.google.com/web/fundamentals/performance/http2
9.
10.
11. Why Web Caching?11
12. Single Server, Poor PerformanceSingle serverSingle point of failureEasily overloadedFar from most clientsPopular contentPopular site“Flash crowd” (aka “Slashdot effect”)Denial of Service attack12
13. Skewed Popularity of Web Traffic“Zipf” or “power-law” distribution13Characteristics of WWW Client-based TracesCarlos R. Cunha, Azer Bestavros, Mark E. Crovella, BU-CS-95-01
14. Proxy CachesclientProxyserverclientHTTP requestHTTP requestHTTP responseHTTP responseHTTP requestHTTP responseorigin server14
15. Forward ProxyCache “close” to the clientUnder administrative control of client-side ASExplicit proxyRequires configuring browserImplicit proxyService provider deploys an “on path” proxy… that intercepts and handles Web requests15clientProxyserverclientHTTP requestHTTP requestHTTP responseHTTP response
16. Reverse ProxyCache “close” to serverEither by proxy run by server or in third-party content distribution network (CDN)Directing clients to the proxyMap the site name to the IP address of the proxy16ProxyserverHTTP requestHTTP responseorigin serverorigin serverHTTP requestHTTP response
17. RouterRouterData Centers . ..ServersServersClientReverse ProxyReverse ProxyRequestsClientClientPrivate BackboneInternetGoogle Design17
18. Proxy Caches(Y) Forward (M) Reverse (C) Both (A) NeitherReactively replicates popular contentReduces origin server costsReduces client ISP costsIntelligent load balancing between origin serversOffload form submissions (POSTs) and user authContent reassembly or transcoding on behalf of originSmaller round-trip times to clientsMaintain persistent connections to avoid TCP setup delay (handshake, slow start) 18
19. Proxy Caches(Y) Forward (M) Reverse (C) Both (A) NeitherReactively replicates popular content Reduces origin server costsReduces client ISP costsIntelligent load balancing between origin serversOffload form submissions (POSTs) and user authContent reassembly or transcoding on behalf of originSmaller round-trip times to clientsMaintain persistent connections to avoid TCP setup delay (handshake, slow start) 19CCYMACCC
20. Modern HTTP Video-on-DemandDownload “content manifest” from origin serverList of video segments belonging to videoEach segment 1-2 seconds in lengthClient can know time offset associated with eachStandard naming for different video resolutions and formats: e.g., 320dpi, 720dpi, 1040dpi, …Client downloads video segment (at certain resolution) using standard HTTP request. HTTP request can be satisfied by cache: it’s a static objectClient observes download time vs. segment duration, increases/decreases resolution if appropriate20
21. Content Distribution Networks21
22. Content Distribution NetworkProactive content replicationContent provider (e.g., CNN) contracts with a CDNCDN replicates the content On many servers spread throughout the InternetUpdating the replicasReactive by TTL or updates pushed to replicas when the content changes22origin server in North AmericaCDN distribution nodeCDN serverin S. AmericaCDN serverin EuropeCDN serverin Asia
23. Server Selection PolicyLive serverFor availabilityLowest loadTo balance load across the serversClosestNearest geographically, or in round-trip timeBest performanceThroughput, latency, …Cheapest bandwidth, electricity, …23Requires continuous monitoring of liveness, load, and performance
24. Server Selection MechanismApplicationHTTP redirectionAdvantagesFine-grain controlSelection based on client IP addressDisadvantagesExtra round-trips for TCP connection to serverOverhead on the serverGETRedirectGETOK24
25. Server Selection MechanismRoutingAnycast routingAdvantagesNo extra round tripsRoute to nearby serverDisadvantagesDoes not consider network or server loadDifferent packets may go to different serversUsed only for simple request-response apps1.2.3.0/241.2.3.0/2425
26. Server Selection MechanismNamingDNS-based server selectionAdvantagesAvoid TCP set-up delayDNS caching reduces overheadRelatively fine controlDisadvantageBased on IP address of local DNS server“Hidden load” effectDNS TTL limits adaptation261.2.3.41.2.3.5DNSquerylocal DNS server
27. How Akamai Works27
28. Akamai StatisticsDistributed serversServers: ~275,000Networks: 1,500Countries: 136Many customers50% of Fortune Global 500NetworkUp to 50 Tbps daily2019 Cricket World Cup: 25.3M concurrent viewers85% Internet is one network hop from Akamai servers28https://www.akamai.com/us/en/about/facts-figures.jsp
29. HTTPHow Akamai Uses DNScnn.com (content provider)DNS root server12Nearby Akamai clusterGET index.html29cache.cnn.com/foo.jpgHTTPAkamaiclusterAkamai global DNS serverAkamai regionalDNS serverEnd user
30. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Nearby Akamai cluster30DNS lookup cache.cnn.comAkamaicluster34ALIAS:g.akamai.netAkamai global DNS serverAkamai regionalDNS serverEnd user
31. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster31Akamaicluster3465ALIASa73.g.akamai.netDNS lookupg.akamai.netEnd user
32. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster32Akamaicluster346587DNS a73.g.akamai.netAddress1.2.3.4End user
33. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster33Akamaicluster3465879GET /foo.jpgHost: cache.cnn.comEnd user
34. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster34Akamaicluster3465879GET /foo.jpgHost: cache.cnn.com1211GET foo.jpgEnd user
35. HTTPHow Akamai Uses DNScnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster35Akamaicluster3465879121110End user
36. HTTPHow Akamai Works: Cache Hitcnn.com (content provider)DNS TLD server12Akamai global DNS serverAkamai regionalDNS serverNearby Akamai cluster36Akamaicluster4356End user
37. Mapping SystemEquivalence classes of IP addressesIP addresses experiencing similar performanceQuantify how well they connect to each otherCollect and combine measurementsPing, traceroute, BGP routes, server logsE.g., over 100 TB of logs per daysNetwork latency, loss, and connectivity37
38. Routing Client Requests within MapMap each IP class to a preferred server clusterBased on performance, cluster health, etc.Updated roughly every minute Short, 60-sec DNS TTLs in Akamai regional DNS accomplish thisMap client request to a server in the clusterLoad balancer selects a specific serverE.g., to maximize the cache hit rate38
39. Selecting server inside cluster“Consistent hashing”content_key = hash(URL) mod Nnode_key = hash(server ID) mod NContent belongs to server’s node_key is “closest” to URL’s content_key39 CK80N32N90N105CK20CK5Circular ID spaceContent 5Server 105
40. Adapting to FailuresFailing hard drive on a serverSuspends after finishing “in progress” requestsFailed serverAnother server takes over for the IP addressLow-level map updated quicklyFailed cluster or network pathHigh-level map updated quicklyFailed path to customer’s origin serverRoute packets through an intermediate node40
41. Akamai Transport OptimizationsBad Internet routesOverlay routing through an intermediate serverPacket lossSending redundant data over multiple pathsTCP connection set-up/teardownPools of persistent connectionsTCP congestion window and round-trip timeEstimates based on network latency measurements 41
42. Akamai Application OptimizationsSlow download of embedded objectsPrefetch when HTML page is requestedLarge objectsContent compressionSlow applicationsMoving applications to edge serversE.g., content aggregation and transformationE.g., static databases (e.g., product catalogs)42
43. ConclusionContent distribution is hardMany, diverse, changing objectsClients distributed all over the worldMoving content towards client is keyReduces latency, improves throughput, reliabilityContribution distribution solutions evolvedReactive caching, load balancing, toProactive content distribution networks43