Problems with Visual Studio 2012 Ultimate Benjamin Day Benjamin Day Consulting Inc bendaycomblog benday About Me Cambridge MA Consultant Coach amp Trainer Microsoft MVP for Visual Studio ALM ID: 484545
Download Presentation The PPT/PDF document "Identify & Fix Performance" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1Slide2
Identify & Fix Performance Problems with Visual Studio 2012 Ultimate
Benjamin DayBenjamin Day Consulting, Inc.benday.com/blog | @benday Slide3
About MeCambridge, MAConsultant, Coach, & Trainer
Microsoft MVP for Visual Studio ALMTeam Foundation Server, Software Testing, Scrum, Software ArchitectureScrum.org ClassesProfessional Scrum Developer (PSD)
Professional Scrum Foundations (PSF)www.benday.com, benday@benday.com, @benday Slide4
Have you heard of Scrum.org?Slide5Slide6
http://www.pluralsight.com/training/Courses/TableOfContents/alm-fundamentals Slide7
http://pluralsight.com/training/Courses/TableOfContents/alm-for-developers Slide8
Thanks.To my content reviewersJerri Chiu, Microsoft
Dennis Bass, MicrosoftSlide9
The AgendaBasics of Web Performance Tests
Load TestsLoad Test RigsPerformance ExplorerExisting application Identify performance problemsGo from bad performance to goodASP.NET & WCFSlide10
The AgendaBasics of Web Performance Tests
Load TestsLoad Test RigsPerformance ExplorerExisting application Identify performance problemsGo from bad performance to goodSlide11Slide12
Disclaimer: I’m going to use the term “load testing” imprecisely…a lot.Slide13
Here we go.Slide14
The Purpose of Load TestingIdentify the capabilities of your applicationPossible goals
ExploreVerify Find the limitsCrushSlide15
The Obligatory “Eat Your Vegetables” Slide.Load test throughout the development cycle.Catch problems early
Establish performance baselineWatch for performance trendsCheaper to fix earlyPlan for performance
Best PracticeSlide16
Cheaper to Fix EarlySlide17
Dose Of RealityYou know you should work that way.
You probably don’t.This talk is for you.You have problems.How do you figure out what’s going wrong?Slide18
Inspiration for this talkReal customer engagementSpent years developing a web application
Released it Seriously grumpy customersHuge hardware~20 simultaneous usersFixed it4000+ simultaneous usersReduced hardwareSlide19
The ToolsSlide20
Tour of the application.Slide21
Disaster Strikes.The application is deployed to production.Users are losing their minds.
Intermittent errors on the site.Site is slow.Your boss is seriously angry.“Fix it!”Slide22
The Problem.“No Repro”“Works on my box.”
You don’t know what the errors are.The site seems to work fine.Wild guess: it dies under load.You need to create some load.Slide23
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide24
Web Performance Tests.Slide25
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide26
What is a Web Performance Test?Test type in Visual Studio 2012 UltimateSimulate a user using an ASP.NET application
HTTP trafficBuilding block of Load TestsSlide27
Tip: Web Tests & User StoriesWeb Tests should simulate a User StoryExample:
As an administrator, I need to search for a person by username so that I can deactivate that person’s account.As a customer, I want to order a pizza and pay with a credit card.Helps to organize your suiteSlide28
Create a basic Web TestSlide29
More about Web TestsCollection of web requestsNot for Test-First Development
Recorded or codedCan be data-drivenMake them do real stuff with Validation RulesExtraction RulesSlide30
Tip: Use Web Tests as a Quality Gate to QAWeb Tests are great for “smoke testing”Run against a build to learn if the build is worth QA-ing
Save QA a lot of time(They like this.)Slide31
ParameterizationMinimize hard coded valuesUse test context values in requests and validations
Web server addresses for requestsDouble “squiggle bracket” syntax{{variable_name}}Essential for data-driven web testsSlide32
Data-driven Web TestsRun web test once per row in data sourceSimulate multiple users doing *similar* actions
Data source typesOLE DB, CSV, XMLParameterizationBetter tests, better coverage Slide33
Web Tests in the
underperforming appSlide34
Tip: Wait for stabilityWait until your screens are reasonably stable
Control name changes broken web testsControl hierarchy changes name changes broken web testsSlide35
Load Tests.Slide36
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide37
What are Load Tests?Mix of Web Performance and Unit TestsSimulates a mix of users doing a mix of things
Simulate Network speedsDifferent browsersVarying user activity loadsSlide38
Types of “Load Tests”Performance TestResponse time
Time lapsesDurationLoad TestBehavior under normal/peak workloadActual workloadStress TestSurfacing issues under extreme conditions and resource failures
Source: J.D. Meier,
http
://blogs.msdn.com/b/jmeier/archive/2007/03/21/performance-vs-load-vs-stress-testing.aspxSlide39
Why would you put a Unit Test in a Load Test?Helpful for testing WCF
Cumbersome to simulate WCF via Web Test(You’d have to re-invent the wheel.)Unit tests can re-use the WCF libraries(Done and done.)Important for Silverlight and SOA
http://wcfloadtest.codeplex.com/http://sqlloadtest.codeplex.com/ Slide40
(Hooray!)
“Hey Kids,Let’s create some Load Tests!”Slide41
Running load tests.Slide42
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide43
Two ways to run load tests.Generate load from Visual Studio 2012
Uses only 1 core/processorLimited to 250 simulated usersGenerate load from a Test RigFar better testSimulate lots more usersSlide44
Load Test Rig & The ApplicationControllerOrchestrates
LicensingPerfMon CountersAgent (1..n)Simulates usersSlide45
Configure the ControllerInstall SQL Express
OptionalStores collected test run dataInstall Visual Studio 2012 Test Controller Run the process as an ActiveDirectory
domain userExample: LoadTestControllerSlide46
Configure the Agent(s)Install the AgentInstallation will ask which Controller
Tip: Run the Agent service(s) using an Active Directory userExample: LoadAgentSlide47
PermissionsPerformance Monitor Users Controller user needs to be in this group
Typical servers: Controller, Agent(s), Web Server, Database serverTeamTestControllerAdmins Run tests on the rigView resultsClean up the results databaseTeamTestControllerUsers
Run tests on the rigView ResultsSlide48
Load Test Rig: Manage Test ControllerSlide49
Load Test Rig: Test SettingsSlide50
Load Test Rig: Remote ExecutionSlide51
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide52
Run Load Tests on the RigSlide53
Before: Load Test SummarySlide54
Before: Load Test GraphsSlide55
Hopefully, I remembered to mention the following…The “Manage Test Controllers…” dialog
*.TestRunConfig settingsRunning the load testsCould spot the exceptions that were happeningFix the data access problemsDemonstrated improved performance
Compare two Load Test RunsSlide56
The Core Problem.Slide57
‘sp_who2’ says lots of open connections.Slide58
After: Load Test SummarySlide59
After: Load Test GraphsSlide60
Performance ProfilingSlide61
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide62
Profiling ToolsLoad Tests help you find issues in a system
“Surface” issuesProfiling helps you find issues in the codeUnit Tests, Web Tests, Load TestsRepeatableGood for targeted tuningSystem under “natural” load
Not repeatableLooking for rough patternsSlide63
Profiling MethodsSlide64
Load Test Performance Sessions Via WizardMust be only Web Performance TestsCan’t profile via Load Test RigSlide65
Let’s go profile some stuff.Slide66
The Plan.Script user activity with Web Performance Tests
Simulate a mix of users using Load TestsSimulate lots of users with a Load Test RigReproduce the errors...hopefully
Find code problems with the Performance WizardFix the code Slide67
Any last questions?Slide68
For More Information: ArticlesUses Visual Studio 2012
http://tinyurl.com/8sdjjakSlide69
For More Information: ArticlesUses Visual Studio 2010
http://tinyurl.com/3xjqgvz
http://
tinyurl.com/2ulvvvr
Slide70
For More Information: Microsoft LinksVisual Studio Performance Testing Quick Reference Guidehttp://vsptqrg.codeplex.com/
Content Index for Web Tests & Load Testshttp://tinyurl.com/ye97e52Slide71
For More Information: Microsoft Bloggers VS Team Test Blog Site http://blogs.msdn.com/b/vstsqualitytools/
Ed Glas - http://blogs.msdn.com/edglas/Bill Barnett - http://blogs.msdn.com/billbar/Sean Lumley - http://blogs.msdn.com/slumley/Dennis Stone - http://blogs.msdn.com/densto/Mike Taute - http
://blogs.msdn.com/mtaute/Neelesh Kamkolkar - http://blogs.msdn.com/b/nkamkolkar/Slide72
Thank you.
http://www.benday.com
| benday@benday.comSlide73
©
2012 Microsoft
Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the
part
of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT
MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.Slide74
Miscellaneous tips.Slide75
Include Load Tests In Your Nightly BuildCreate a Test List (*.vsmdi
)Include the Test List in the buildCaptures trend data at 24 hours intervalsCompare the load test data with ExcelSlide76
Static Code Analysis / FxCopHelp you to find issues in your code
Show you IDispose errorsRules are configurableUse your judgment about which are relevant for your teamSlide77
Avoid Static Methods & CA1822Static Code Analysis Rule CA1822 is evil.
Sacrifices maintainability for perfomance.Blog post: “Static Methods Are A Code Smell”http://
bit.ly/cnEsuV Slide78
Only optimize known performance problems. You’d be surprised by what *ISN’T* a performance problemDon’t spend a ton of time coding fancy solutions to performance problems you *THINK* you might have
Find and fix *REAL* problemsSlide79
Tip: WCF Message SizeData structure formatting can make a huge differenceExample: Banks and the States they operate in
Option 1: Dictionary<string, string>Key = Bank IdValue = State AbbreviationOption 2: Dictionary<string, string[]>Key = State AbbreviationValue = Array of Bank Id’s
Option 2 was approximately 90% smallerSlide80
Tip: Add Performance CountersCreate Performance Counters (perfmon
) in your appDo this early in the development cycleMore directed profiling of your app during Load TestsGood for managing your app when it’s in production(Operations people love this.)Slide81
Using Performance Sessions on x64You can’t profile x64 IIS processes from the IDE
Option #1: Profile from the command lineStart ASP.NET profiling using VSPerfAspNetCmd.exe Run the web test
Option #2: Run the ASP.NET process in x86 modeSlide82
How To Run ASP.NET in x86 ModeSet “Enable 32-bit Applications” to true on
your application’s AppPoolSlide83
Resources
Connect. Share. Discuss.
http://northamerica.msteched.com
Learning
Microsoft Certification & Training Resources
www.microsoft.com/learning
TechNet
Resources for IT Professionals
http://microsoft.com/technet
Resources for Developers
http://microsoft.com/msdn Slide84
Required Slide
Complete an evaluation on CommNet and enter to win!Slide85
MS Tag
Scan the Tag
to evaluate this
session now on
myTechEd
Mobile
Required Slide
*delete this box when your slide is finalized
Your MS Tag will be inserted here during the final scrub. Slide86
©
2012 Microsoft
Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the
part
of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT
MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.