Stephen Rose Vadim Arakelov Pieter Wigleven Matthew Reynolds Microsoft Corporation WCL305 Intro Discussion Fast boots slow boots the enterprise MSIT and Microsoft Services Enterprise Boot Performance Landscape ID: 227761
Download Presentation The PPT/PDF document "How Many Coffees Can You Drink While You..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
How Many Coffees Can You Drink While Your PC Boots?
Stephen Rose, Vadim Arakelov, Pieter Wigleven, Matthew Reynolds Microsoft Corporation
WCL305Slide2
Intro Discussion
Fast boots, slow boots, the enterprise, MSIT and Microsoft Services
Enterprise Boot Performance LandscapeSlide3
Fast vs
SlowSlide4
Demo
Clean Windows boots quickly
Anatomy of boot/logonHow to use ADK toolsYour boot through the eyes of the Windows ADKSlide5
Demos
Real world examples of what goes wrong with boot performance in the enterprise.
How many coffees can you drink while your PC boots?Slide6
Case Of…
Examples from the field“Look at me! Look at me!”Slide7
Look at me! | Analysis
Desktop
Idle
Massive IOSlide8
Look at me! | Lessons learned
Don’t attach work to boot unless absolutely necessary
Use scheduled tasks instead of startup script, etc. to accomplish background configurationHigh disk IO is the #1 boot performance killerUse ADK to see who the culprits areFrequent high-disk IO culprits:
Security / anti-malware agents
Management agents
WMI (details to follow)Slide9
Case Of…
Examples from the field“Is my disk LED burning?”Slide10
Desktop appears at 135 seconds
Idle/responsive at 185 seconds
Is my disk LED burning? | Analysis
Disk utilizationSlide11
Is my disk LED burning | Lessons learned
Consider (future) hardware requirements
More than minimal OS requirementsSome customers refer to KB314865Slide12
Case of…
Examples from the field“The swarm”Slide13
The swarm | Analysis
Desktop appears
Idle/responsive
Massive disk IOSlide14
The swarm | Process starts on reasonable systemSlide15
The swarm | Process starts on affected systemSlide16
The swarm | Another environment
CPU
DISKSlide17
The swarmSlide18
The swarm | Lessons learned
Standard User vs. Admin
Reduce use of boot-time scriptsUse Preferences (wisely)Use scheduled tasks to run scripts in background
Streamline use of overlapping configuration vectors
Measure performance
Before/after software installations
Periodically
Questions?Slide19
Case of…
Featuring:The 10 Minute Unintended Consequence
The Productivity MaximizerThe Slow Profile“13 minute triple play”Slide20
13 minute triple play | Analysis
~800s to idle desktop
Not CPU bound
Not disk boundSlide21
13 minute triple play | Services diagram
ProductivityAgent service takes 50 seconds to ready
Launch of all remaining services delayedSlide22
13 minute triple play | Winlogon diagram
Group Policy delay 50 seconds
Profile load 20 seconds
Group Policy delay 600 seconds!Slide23
13 minute triple
play | First GP delay
GP waiting for network
Network components waiting for slow serviceSlide24
13 minute triple
play | First GP delay
Synchronous mode (“wait for the network”)
So we wait… about 50 secondsSlide25
13 minute triple play | Winlogon diagram
Group Policy delay 50 seconds
Profile load 20 seconds
Group Policy delay 600 seconds!Slide26
13 minute triple
play | Profile delay
Roaming profile sync took several secondsSlide27
13 minute triple
play | Massive GP delay
600 second delay in Group Policy
Seems to correspond to GpScript.exeSlide28
13 minute triple
play | Massive GP delay
Group Policy service svchost.exe launches GpScript.exe launches… Notepad?
Why is notepad opening the
vbscript
?Slide29
13 minute triple play |
File associationSlide30
13
minute triple play | Lessons learned (1)
Avoid waiting in the first place by avoiding synchronous uses of Group
Policy /
Profiles
Tune GP
scripts timeout
Just
enough
time to run your intended scripts
Default
:
10 minutes!
Tune profile timeoutsSlide31
13 minute triple play | Lessons learned (2)
Don’t rely on file associations for system automation/scripts
Look for slow starting services, particularly if they are part of a load order groupQuestions?Slide32
Case of…
Examples from the field“Hottest CPU
in town”Slide33
Hottest CPU in town
CPU utilizationSlide34
Hottest CPU in town
Great stuff…. Slide35
Hottest CPU in town
Walking the stack
repdrvfs!PerformAllValidationsSlide36
Hottest CPU in town
Bing!Slide37
Hottest CPU in town
Hurray! A fix is available!Slide38
Hottest CPU in town | Lessons learned
Latest updates from Microsoft and 3rd parties often contain boot performance enhancements
WMIAnti-virusHigh CPU on boot is easy to troubleshoot with ADK toolsStacks!Slide39
Microsoft IT
Enterprise boot performance remediation examplePower-On to ProductivitySlide40
High Disk IO
Top disk consumersForefrontLync / Communicator
WMI (various callers)…
Responsive ~170sSlide41
Synchronous Policies
Boot Time Delay:
Synchronous policies caused a 43 seconds delay during boot process
Sync
Group
PoliciesSlide42
Slow Service Start Time
APP-V Service Start Time
App-V service takes 90 seconds to start
Issue is caused by the use of WMI query to gather required informationSlide43
Current Disk IO Utilization
High Disk IO remediated by:Optimizing Forefront AV engine
Disabling Lync client side logging
Responsive ~68sSlide44
Asynchronous Policies
ASync Group Policies
Reduction in boot time was
40+ seconds
by avoiding synchronous policies Slide45
Fast Service Start Time
APP-V Service Start
Time
App-V service takes 4.5 seconds to startSlide46
Before & After
System Boot time with Lync logging, Synch Policies (DA) and old FEP:
Scenario
Benefit
1
Lync Client
Side Logging
Boot time improvement is 50 sec. to several min.
2
Forefront
AV Engine Optimization
Boot time improvement is 25
sec.
3
Synchronous GPO Cleanup
Boot time improvement is 40 sec.
Synchronous GPO Cleanup
Lync Client Side Logging
Forefront AV Engine Optimization
System Boot time with Lync logging
disabled, Async Policies and New FEP
Boot Time of
193 seconds
Boot Time of
77 secondsSlide47
Power On To Productivity Program Update
Scenario
Benefit
Complete
Corrective
1
Excessive disk IO consumption by Lync logging
50 second to several min
boot time improvement
2
Excessive disk IO consumption by Forefront
25 second
boot time improvement
3
Direct
Access GPOs applying synchronously
40
second
boot time improvement
4
Starting up the App-V service requires all other services to wait until completed.
30 – 90 second
reduction to the
App-V
service start-up time during boot
5
WMI repository verification
issue
50 – 300 seconds boot time improvement on production systems (improvement times dependent on type of the system and live in production)
Proactive
6FIM Password Reset Tool resulting in boot performance impact
23 second boot time improvement7
System Center Greenup Power Management resulting in boot performance impact16 second boot time
improvement
8
New NetLogon script resulting in boot performance impact
30 second boot time improvement which included 7 second reduction from initial baseline.
3 minute, 54 second
boot time improvementSlide48
Boot Phases By Disk Report
48
SSDSlide49
Sync vs. Async GP Mode At
Logon Report
49
40 seconds boot delay caused by “Wait For The Network” sync modeSlide50
Windows 8 and Boot/Logon Performance
Improvements:General OS speed-up helps with bootHybrid boot
Faster network initializationUpdated xperf/WPR/WPA tools + assessment framework in Windows 8 ADKWinRT apps blocked from many system unfriendly behaviors Where you can still shoot yourself in the foot:Disk IO from installed componentsSynchronous net bound GP / Profile modesSlide51
Change Your IT Culture
: Prioritize “Power-On to Productivity”
Measure performance (ADK tools) and remediate top issuesCAB should not approve new scripts or software without certifying for boot impactPush back on vendors (including us) when they impact perfBe extremely selective in using synchronous network-bound features like home drives, roaming profiles, legacy user logon script, sync mode GP, etc.Slide52
Want help from us?
Microsoft Consulting ServicesLarge deployment
& optimization projectsMicrosoft Premier SupportPremier Field EngineeringProactive support / training / WDRAPCommercial Technical SupportReactive boot performance analysisSlide53
Related Content
TechEd
Session: S330C (see recording)
IT Showcase | http
://technet.microsoft.com/en-us/library/hh708771.aspx
ADK | http
://
www.microsoft.com/en-gb/download/details.aspx?id=29929
Find us to talk: Windows area in TLC, community/ATE event, hallways
Slide54
Track Resources
Resources
for Developers
http
://msdn.microsoft.com/en-us/windows/apps
Windows
8 is ready for
Business
http
://www.microsoft.com/en-us/windows/enterprise/products-and-technologies/windows-8/default.aspx
Microsoft
Desktop Optimization Pack
:
www.microsoft.com/MDOP
Microsoft Desktop Virtualization:
www.microsoft.com/dvSlide55
Track Resources
Springboard Series: www.microsoft.com/springboard
Explore > Plan > Deliver > Operate > Support for Windows 7 and Windows 8MDOP
Desktop Virtualization
Windows
Intune
Internet Explorer 8, 9 and 10
Track ResourcesSlide56
Download
http://windows.microsoft.com/en-US/windows-8/release-previewDownload the Windows 8 Release Preview TodaySlide57
Resources
Connect. Share. Discuss.
http
://europe.msteched.com
Learning
Microsoft Certification & Training Resources
www.microsoft.com/learning
TechNet
Resources for IT Professionals
http://microsoft.com/technet
Resources for Developers
http://microsoft.com/msdn Slide58
Evaluations
http://europe.msteched.com/sessions
Submit your evals online Slide59
©
2012 Microsoft
Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the
part
of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.
MICROSOFT
MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.Slide60
Bonus Ninja Demo | Stacks for high CPUSlide61
Bonus Ninja Demo | Stacks for high CPUSlide62
Bonus Ninja Demo: Wait analysis
What if that nice roaming profile event log hadn’t been there?
How could you track down this or any delay generically?Slide63Slide64