Mar 6 2019 ADVANCED COMPUTER ARCHITECTURE CS 43306501 Branch Prediction The content and concept of this course are adapted from CMU ECE 740 AGENDA Logistics Review from the last class Branch Prediction ID: 760296
Download Presentation The PPT/PDF document "Samira Khan University of Virginia" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Samira KhanUniversity of VirginiaMar 6, 2019
ADVANCED COMPUTER ARCHITECTURE CS 4330/6501Branch Prediction
The content and concept of this course are adapted from CMU ECE 740
Slide2AGENDA
Logistics
Review from the last class
Branch Prediction
Slide3LOGISTICS
Mar 18: Second Student Paper PresentationTwo papersPresenters do not need to submit reviews
Beyond the Memory Wall: A Case for Memory-centric HPC System for Deep Learning, MICRO 2018
EIE: Efficient Inference Engine on Compressed Deep Neural Network, ISCA 2016
Slide4Presentation Guidelines
30 mins for presentation and 5 mins for Q&AStart with authors’ slide and then modify them to make yoursFormat is similar to reviewsSpend significant time on the background and problem Key ideaMechanismResults. Pros, cons, what did you like, future workSend the slides at least one week before the presentationPractice at least 3 times before the classIf necessary, first write down and then practice
4
Slide5Presentation Guidelines
Background and problem statement (10 mins)Key idea/mechanism (8-10 mins)Results (2-3 mins)Pros/cons/discussion (5-7 mins)Q&A (5 mins)
5
Slide6BRANCH PREDICTION
Idea: Predict the next fetch address (to be used in the next cycle)Requires three things to be predicted at fetch stage:Whether the fetched instruction is a branch(Conditional) branch directionBranch target address (if taken)Observation: Target address remains the same for a conditional direct branch across dynamic instancesIdea: Store the target address from previous instance and access it with the PCCalled Branch Target Buffer (BTB) or Branch Target Address Cache
6
Slide7THE LAST TIME PREDICTOR
Problem: A last-time predictor changes its prediction from TNT or NTT too quickly even though the branch may be mostly taken or mostly not takenSolution Idea: Add hysteresis to the predictor so that prediction does not change on a single different outcomeUse two bits to track the history of predictions for a branch instead of a single bit Can have 2 states for T or NT instead of 1 state for eachSmith, “A Study of Branch Prediction Strategies,” ISCA 1981.
7
Slide8TWO-BIT COUNTER BASED PREDICTION
Each branch associated with a two-bit counterOne more bit provides hysteresisA strong prediction does not change with one single different outcomeAccuracy for a loop with N iterations = (N-1)/Nfor (i=0; i<N; i++) { … }Prediction: TTTT …. T TTTT ... T TTTT ... TActual: TTTT .... N TTTT ... N TTTT ... N TNTNTNTNTNTNTNTNTNTN 50% accuracy (assuming init to weakly taken)+ Better prediction accuracy-- More hardware cost (but counter can be part of a BTB entry)
8
Slide9STATE MACHINE FOR 2-BIT SATURATING COUNTER
Counter using saturating arithmeticThere is a symbol for maximum and minimum values
predtaken11
predtaken10
pred!taken01
pred!taken00
actually
taken
actually
taken
actually
!taken
actually
!taken
actually
!taken
actually
!taken
actually
taken
actually
taken
9
“
weakly
taken
”
“
strongly
taken”
“weakly!taken”
“
strongly
!taken
”
Slide10IS THIS ENOUGH?
~85-90% accuracy for many programs with 2-bit counter based prediction (also called bimodal prediction)Is this good enough?How big is the branch problem?
10
Slide11REVIEW: RETHINKING THE BRANCH PROBLEM
Control flow instructions (branches) are frequent15-25% of all instructionsProblem: Next fetch address after a control-flow instruction is not determined after N cycles in a pipelined processorN cycles: (minimum) branch resolution latencyStalling on a branch wastes instruction processing bandwidth (i.e. reduces IPC)N x IW instruction slots are wasted (IW: issue width)How do we keep the pipeline full after a branch?Problem: Need to determine the next fetch address when the branch is fetched (to avoid a pipeline bubble)
11
Slide12Assume a 5-wide superscalar pipeline with 20-cycle branch resolution latencyHow long does it take to fetch 500 instructions? Assume no fetch breaks and 1 out of 5 instructions is a branch100% accuracy 100 cycles (all instructions fetched on the correct path)No wasted work99% accuracy100 (correct path) + 20 (wrong path) = 120 cycles20% extra instructions fetched98% accuracy100 (correct path) + 20 * 2 (wrong path) = 140 cycles 40% extra instructions fetched 95% accuracy100 (correct path) + 20 * 5 (wrong path) = 200 cycles100% extra instructions fetched
REVIEW: IMPORTANCE OF THE BRANCH PROBLEM
12
Slide13CAN WE DO BETTER?
Last-time and 2BC predictors exploit “last-time” predictabilityRealization 1: A branch’s outcome can be correlated with other branches’ outcomesGlobal branch correlation Realization 2: A branch’s outcome can be correlated with past outcomes of the same branch (other than the outcome of the branch “last-time” it was executed)Local branch correlation
13
Slide14GLOBAL BRANCH CORRELATION (I)
Recently executed branch outcomes in the execution path is correlated with the outcome of the next branchIf first branch not taken, second also not takenIf first branch taken, second definitely not taken
14
Slide15GLOBAL BRANCH CORRELATION (II)
If Y and Z both taken, then X also takenIf Y or Z not taken, then X also not taken
15
Slide16GLOBAL BRANCH CORRELATION (III)
Eqntott, SPEC 1992 if (aa==2) ;; B1 aa=0; if (bb==2) ;; B2 bb=0; if (aa!=bb) { ;; B3 …. }If B1 is taken (i.e. aa=0@B3) and B2 is taken (i.e. bb=0@B2) then B3 is certainly NOT taken
16
Slide17CAPTURING GLOBAL BRANCH CORRELATION
Idea: Associate branch outcomes with “global T/NT history” of all branchesMake a prediction based on the outcome of the branch the last time the same global branch history was encounteredImplementation:Keep track of the “global T/NT history” of all branches in a register Global History Register (GHR)Use GHR to index into a table of that recorded the outcome that was seen for that GHR value in the recent past Pattern History Table (table of 2-bit counters)Global history/branch predictorUses two levels of history (GHR + history at that GHR)
17
Slide18TWO LEVEL GLOBAL BRANCH PREDICTION
First level: Global branch history register (N bits)The direction of last N branchesSecond level: Table of saturating counters for each history entryThe direction the branch took the last time the same history was seen
1 1 ….. 1 0
GHR
(global history register)
00 …. 00
00 …. 01
00 …. 10
11 …. 11
0
1
2
3
index
Pattern History Table (PHT)
previous one
Yeh and Patt,
“
Two-Level Adaptive Training Branch Prediction
,
”
MICRO 1991.
18
Slide19McFarling, “Combining Branch Predictors,” DEC WRL TR 1993.
19
This branch tests
i
Last 4 branches test jHistory: TTTNPredict taken for iNext history: TTNT (shift in last outcome)
HOW DOES THE GLOBAL PREDICTOR WORK?
Slide20target address
ONE-LEVEL BRANCH PREDICTOR
Direction predictor (2-bit counters)
Cache of Target Addresses (BTB: Branch Target Buffer)
Program
Counter
PC + inst size
taken?
Next Fetch
Address
hit?
Address of the
current instruction
20
Slide21target address
TWO-LEVEL GLOBAL HISTORY PREDICTOR
Direction predictor (2-bit counters)
Cache of Target Addresses (BTB: Branch Target Buffer)
Program
Counter
Global branch
history
PC + inst size
taken?
Next Fetch
Address
hit?
Which direction earlier
branches went
Address of the
current instruction
21
Slide22target address
TWO-LEVEL GSHARE PREDICTOR
Direction predictor (2-bit counters)
Cache of Target Addresses (BTB: Branch Target Buffer)
Program
Counter
Global branch
history
XOR
PC + inst size
taken?
Next Fetch
Address
hit?
Which direction earlier
branches went
Address of the
current instruction
22
Slide23CAN WE DO BETTER?
Last-time and 2BC predictors exploit “last-time” predictabilityRealization 1: A branch’s outcome can be correlated with other branches’ outcomesGlobal branch correlation Realization 2: A branch’s outcome can be correlated with past outcomes of the same branch (other than the outcome of the branch “last-time” it was executed)Local branch correlation
23
Slide24LOCAL BRANCH CORRELATION
McFarling, “Combining Branch Predictors,” DEC WRL TR 1993.
24
Slide25MORE MOTIVATION FOR LOCAL HISTORY
To predict a loop branch “perfectly”, we want to identify the last iteration of the loopBy having a separate PHT entry for each local history, we can distinguish different iterations of a loopWorks for “short” loops
25
Slide26CAPTURING LOCAL BRANCH CORRELATION
Idea: Have a per-branch history registerAssociate the predicted outcome of a branch with “T/NT history” of the same branchMake a prediction is based on the outcome of the branch the last time the same local branch history was encounteredCalled the local history/branch predictorUses two levels of history (Per-branch history register + history at that history register value)
26
Slide27TWO LEVEL LOCAL BRANCH PREDICTION
First level: A set of local history registers (N bits each)Select the history register based on the PC of the branchSecond level: Table of saturating counters for each history entryThe direction the branch took the last time the same history was seen
1 1 ….. 1 0
Local history registers
00 …. 00
00 …. 01
00 …. 10
11 …. 11
0
1
2
3
index
Pattern History Table (PHT)
Yeh and Patt,
“
Two-Level Adaptive Training Branch Prediction
,
”
MICRO 1991.
27
Slide28target address
TWO-LEVEL LOCAL HISTORY PREDICTOR
Direction predictor (2-bit counters)
Cache of Target Addresses (BTB: Branch Target Buffer)
Program
Counter
PC + inst size
taken?
Next Fetch
Address
hit?
Address of the
current instruction
Which directions earlier instances of *this branch* went
28
Slide29CAN WE DO EVEN BETTER?
Predictability of branches variesSome branches are more predictable using local historySome using globalFor others, a simple two-bit counter is enoughYet for others, a bit is enough Observation: There is heterogeneity in predictability behavior of branchesNo one-size fits all branch prediction algorithm for all branchesIdea: Exploit that heterogeneity by designing heterogeneous branch predictors
29
Slide30HYBRID BRANCH PREDICTORS
Idea: Use more than one type of predictor (i.e., multiple algorithms) and select the “best” predictionE.g., hybrid of 2-bit counters and global predictorAdvantages: + Better accuracy: different predictors are better for different branches + Reduced warmup time (faster-warmup predictor used until the slower-warmup predictor warms up)Disadvantages: -- Need “meta-predictor” or “selector” -- Longer access latencyMcFarling, “Combining Branch Predictors,” DEC WRL Tech Report, 1993.
30
Slide31ALPHA 21264 TOURNAMENT PREDICTOR
Minimum branch penalty: 7 cyclesTypical branch penalty: 11+ cycles48K bits of target addresses stored in I-cachePredictor tables are reset on a context switchKessler, “The Alpha 21264 Microprocessor,” IEEE Micro 1999.
31
Slide32BRANCH PREDICTION ACCURACY (EXAMPLE)
Bimodal: table of 2bc indexed by branch address
32
Slide33ARE WE DONE W/ BRANCH PREDICTION?
Hybrid branch predictors work wellE.g., 90-97% prediction accuracy on averageSome “difficult” workloads still suffer, though!E.g., gccMax IPC with tournament prediction: 9Max IPC with perfect prediction: 35
33
Slide3434
Chappell et al., “
Simultaneous Subordinate Microthreading (SSMT),” ISCA 1999.
ARE WE DONE W/ BRANCH PREDICTION?
Slide35SOME OTHER BRANCH PREDICTOR TYPES
Loop branch detector and predictorLoop iteration count detector/predictorWorks well for loops, where iteration count is predictableUsed in Intel Pentium MPerceptron branch predictorLearns the direction correlations between individual branchesAssigns weights to correlationsJimenez and Lin, “Dynamic Branch Prediction with Perceptrons,” HPCA 2001.Hybrid history length based predictorUses different tables with different history lengthsSeznec, “Analysis of the O-Geometric History Length branch predictor,” ISCA 2005.
35
Slide36Intel Pentium M Predictors
36
Gochman et al.,
“
The Intel Pentium M Processor: Microarchitecture and Performance
,
”
Intel Technology Journal, May 2003.
Slide37Perceptron Branch Predictor (I)
Idea:
Use a perceptron to learn the correlations between branch history register bits and branch outcomeA perceptron learns a target Boolean function of N inputsJimenez and Lin, “Dynamic Branch Prediction with Perceptrons,” HPCA 2001.Rosenblatt, “Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms,” 1962
37
Each branch associated with a perceptron
A perceptron contains a set of weights wi
Each weight corresponds to a bit in
the GHR
How much the bit is correlated with the
direction of the branch
Positive correlation: large + weight
Negative correlation: large - weight
Prediction:
Express GHR bits as 1 (T) and -1 (NT)
Take dot product of GHR and weights
If output > 0, predict taken
Slide38Samira KhanUniversity of VirginiaMar 6, 2019
ADVANCED COMPUTER ARCHITECTURE CS 4330/6501Branch Prediction
The content and concept of this course are adapted from CMU ECE 740