Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues - PowerPoint Presentation

Slide1

Too Slow?: Use VS2010 Profiling & Load Testing to Manage Performance Issues

Benjamin DaySlide2

Benjamin DayConsultant, Coach, TrainerScrum.org ClassesProfessional Scrum Developer (PSD)

Professional Scrum Foundations (PSF) TechEd, VSLive

,

DevTeach

, O’Reilly OSCONVisual Studio Magazine, Redmond Developer NewsMicrosoft MVP for Visual Studio ALMTeam Foundation Server, TDD, Testing Best Practices,Silverlight, Windows Azurehttp://blog.benday.combenday@benday.comSlide3

Thanks.To my content reviewersJerri Chiu, MicrosoftDennis Bass, MicrosoftSlide4

The AgendaBasics of Web Performance TestsLoad Tests

Load Test RigsPerformance ExplorerExisting application Identify performance problems

Go from bad performance to good

ASP.NET & WCFSlide5

Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.Slide6

Here we go.Slide7

The Purpose of Load TestingIdentify the capabilities of your applicationPossible goalsExplore

Verify Find the limitsCrushSlide8

Best Practice:The Obligatory “Eat Your Vegetables”Load test throughout the development cycle.

Catch problems earlyEstablish performance baselineWatch for performance trends

Cheaper to fix early

Plan for performanceSlide9

Cheaper To Fix EarlySlide10

Dose Of RealityYou know you should work that way.You probably don’t.

This talk is for you.You have problems.How do you figure out what’s going wrong?Slide11

Inspiration for this talkReal customer engagementSpent years developing a web applicationReleased it

Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it4000+ simultaneous users

Reduced hardwareSlide12

The ToolsSlide13

Tour Of The ApplicationdemoSlide14

Disaster Strikes.The application is deployed to production.Users are losing their minds.Intermittent errors on the site.

Site is slow.Your boss is seriously angry.“Fix it!”Slide15

The Problem.“No Repro”“Works on my box.”You don’t know what the errors are.

The site seems to work fine.Wild guess: it dies under load.You need to create some load.Slide16

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide17

Web Performance Tests.Slide18

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide19

What is a Web Performance Test?Test type in Visual Studio 2010 UltimateSimulate a user using an ASP.NET applicationHTTP traffic

Building block of Load TestsSlide20

Tip: Web Tests & User StoriesWeb Tests should simulate a User StoryExample:As an administrator, I need to search for a person by username so that I can deactivate that person’s account.

As a customer, I want to order a pizza and pay with a credit card.Helps to organize your suiteSlide21

Create a basic Web TestdemoSlide22

More about Web TestsCollection of web requestsNot for Test-First DevelopmentRecorded or codedCan be data-driven

Make them do real stuff with Validation RulesExtraction RulesSlide23

Tip: Use Web Tests as a Quality Gate to QAWeb Tests are great for “smoke testing”Run against a build to learn if the build is worth QA-

ingSave QA a lot of time(They like this.)Slide24

ParameterizationMinimize hard coded valuesUse test context values in requests and validations

Web server addresses for requestsDouble “squiggle bracket” syntax{{variable_name}}

Essential for data-driven web testsSlide25

Data-driven Web TestsRun web test once per row in data sourceSimulate multiple users doing *similar* actions

Data source typesOLE DB, CSV, XMLParameterizationBetter tests, better coverage Slide26

Web Tests in the Underperforming Application

demoSlide27

Tip: Wait for stabilityWait until your screens are reasonably stableControl name changes

 broken web testsControl hierarchy changes  name changes 

broken web testsSlide28

Load Tests.Slide29

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide30

What are Load Tests?Mix of Web Performance and Unit TestsSimulates a mix of users doing a mix of things

Simulate Network speedsDifferent browsersVarying user activity loadsSlide31

Types of “Load Tests”Performance TestResponse timeTime lapses

DurationLoad TestBehavior under normal/peak workloadActual workload

Stress Test

Surfacing issues under extreme conditions and resource failures

Source: J.D. Meier,

http

://blogs.msdn.com/b/jmeier/archive/2007/03/21/performance-vs-load-vs-stress-testing.aspxSlide32

Why would you put a Unit Test in a Load Test?Helpful for testing WCFCumbersome to simulate WCF

via Web Test(You’d have to re-invent the wheel.)Unit tests can re-use the WCF libraries(Done and done.)

Important for Silverlight and SOA

http://wcfloadtest.codeplex.com

/http://sqlloadtest.codeplex.com/ Slide33

Hey Kids, Let’s create some Load Tests!

demo

(…and then you all say, “Hooray!”)Slide34

Running Load Tests.Slide35

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide36

Two ways to run load tests.Generate load from Visual Studio 2010Uses only 1 core/processorLimited to 250 simulated users

Generate load from a Test RigFar better testSimulate lots more usersSlide37

Load Test Agent LicensingMicrosoft Visual Studio Team System 2010

Test Load Virtual User PackSimulate up to 1000 virtual usersUsed to cost $$$Slide38

Visual Studio 2010 Load Test Feature PackUsed to cost ~$3500.00 per 1000 simulated usersFree starting March 8, 2011

Visual Studio 2010 Ultimate with MSDNSimulate unlimited amount of load(Limited only by your hardware.)

http

://www.microsoft.com/visualstudio

/en-us/products/2010-editions/load-test-virtual-user-packSlide39

Load Test Rig & The ApplicationControllerOrchestrates

LicensingPerf Mon CountersAgent (1..n)

Simulates

usersSlide40

Configure the ControllerInstall SQL ExpressOptional

Stores collected test run dataInstall Visual Studio 2010 Test Controller Run the process as an

ActiveDirectory

domain user

Example: LoadTestControllerAdd a Virtual User License PackSlide41

Configure the Agent(s)Install the AgentInstallation will ask which ControllerTip: Run the Agent service(s) using an Active Directory user

Example: LoadAgentSlide42

PermissionsPerformance Monitor Users Controller user needs to be in this groupTypical servers: Controller, Agent(s), Web Server, Database server

TeamTestControllerAdmins Run tests on the rigView resultsClean up the results database

TeamTestControllerUsers

Run tests on the rig

View ResultsSlide43

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide44

Run Load Tests on the RigdemoSlide45

Hopefully, I remembered to mention the following…The “Manage Test Controllers…” dialog*.TestRunConfig

settingsRunning the load testsCould spot the exceptions that were happeningFix the data access problems

Demonstrated improved performance

Compare two Load Test RunsSlide46

The Core Problem.Slide47

‘sp_who2’ says lots of open connections.Slide48

Performance ProfilingSlide49

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide50

Profiling ToolsLoad Tests help you find issues in a system“Surface” issues

Profiling helps you find issues in the codeUnit Tests, Web Tests, Load TestsRepeatable

Good for targeted tuning

System under “natural” load

Not repeatableLooking for rough patternsSlide51

Profiling MethodsSlide52

Load Test Performance Sessions Via WizardMust be only Web Performance TestsCan’t profile via Load Test Rig

ASP.NET must be running in x86 modeSlide53

Let’s go profile some stuff.demoSlide54

The Plan.Script user activity with Web Performance Tests

Simulate a mix of users using Load TestsSimulate lots of users with a Load Test Rig

Reproduce the errors...hopefully

Find code problems with the Performance Wizard

Fix the code Slide55

Any last questions?Slide56

For More Information: Articles

http://tinyurl.com/3xjqgvz

http://

tinyurl.com/2ulvvvr

Slide57

For More Information: Microsoft LinksVisual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/ Content Index for Web Tests & Load Tests

http://tinyurl.com/ye97e52Slide58

For More Information: Microsoft Bloggers VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/Ed Glas - http://blogs.msdn.com/edglas/

Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike

Taute

- http://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/Slide59

Thank you.

blog.benday.com | www.benday.com | benday@benday.comSlide60

Additional information that I practically never have time to cover.Slide61

Miscellaneous Tips.Slide62

Include Load Tests In Your Nightly BuildCreate a Test List (*.vsmdi)Include the Test List in the build

Captures trend data at 24 hours intervalsCompare the load test data with ExcelSlide63

Static Code Analysis / FxCopHelp you to find issues in your codeShow you

IDispose errorsRules are configurableUse your judgment about which are relevant for your teamSlide64

Avoid Static Methods & CA1822Static Code Analysis Rule CA1822 is evil.

Sacrifices maintainability for perfomance.Blog post:

“Static Methods Are A Code Smell”

http://

bit.ly/cnEsuV Slide65

Only optimize known performance problems. You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might have

Find and fix *REAL* problemsSlide66

Tip: WCF Message SizeData structure formatting can make a huge differenceExample: Banks and the States they operate inOption 1: Dictionary<string, string>

Key = Bank IdValue = State AbbreviationOption 2: Dictionary<string, string[]>

Key = State Abbreviation

Value = Array of Bank Id’s

Option 2 was approximately 90% smallerSlide67

Tip: Add Performance CountersCreate Performance Counters (perfmon) in your app

Do this early in the development cycleMore directed profiling of your app during Load TestsGood for managing your app when it’s in production

(Operations people love this.)Slide68

Using Performance Sessions on x64You can’t profile x64 IIS processes from the IDE

Option #1: Profile from the command line

Start ASP.NET profiling using VSPerfAspNetCmd.exe

Run the web test

Option #2: Run the ASP.NET process in x86 modeSlide69

How To Run ASP.NET in x86 ModeSet “Enable 32-bit Applications” to true on your application’s

AppPool