"Register File Prefetching ISCA 2022, New York City" is the property of its rightful owner. Permission is granted to
download and print the materials on this website for personal, non-commercial use only, and to display it
on your personal computer provided you do not modify the materials and that you retain all copyright
notices contained in the materials. By downloading content from our website, you accept the terms of this
agreement.
Presentation Transcript
01
Register File Prefetching ISCA 2022, New York City Sudhanshu Shukla, Sumeet Bandishte, Jayesh Gaur, Sreenivas Subramoney
Processor Architecture Research Lab (PAR-Lab), Intel India<br>
02
Scaling the Memory Wall L1
[ 5 Cycles ] L2
[ 14 Cycles ] LLC
[ 39-45 Cycles ] DRAM
[ 200-400 cycles ] Register File Modern OOO processors utilize multiple levels of Caches & Memory
Primary Solution
Prefetch data closer to Core Research has primarily focused on Prefetching from large latency DRAM<br>
03
Prefetching Perf. Headroom at various Cache Levels L1 Data Cache Latency is as important as Memory DRAM Latency !! Oracle Prefetching: Hits at level N are served at latency of level N-1 93% of all Loads hit in L1<br>
04
Agenda Related Work : Load Value/Address Prediction
Register-File Prefetching (RFP)
Identifying opportunities for Register File prefetching
RFP micro-architecture
Guaranteeing Correctness/Consistency/Coherency
Experimental Results
Conclusions<br>
05
Load I6 I5 I4 L1 I3 I2 Load I4 Value Predicted I5 can execute independently using the predicted value Benefits
Can hide complete Load latency
Breaks Dependencies Higher ILP Drawbacks
Incorrect Prediction Costly Flush
Re-Allocation & Re-Execution
Low Coverage to avoid Flushes Predict Load Values
(break dependencies) Prior Art #1: Value Prediction (VP)<br>
06
Predict Load Address Probe value from Data $ Value Predict Prior Art #2: VP using Address Prediction Benefits
Load address more predictable Higher Coverage
Can hide complete Load latency like VP Drawbacks
Fetch-Latency Optimizations Limited slack
Aliasing Inflight Stores Reduces Coverage
Double cache access to check for correctness<br>
07
Options for mitigating L1 latency Value Prediction has limited benefits
Primarily targeted for increasing ILP
Flushes/Inflight Stores limit coverage
Load re-execution adds pressure to L1 bandwidth Register File Prefetch (RFP)
True Prefetcher from L1 Register File
No costly flushes Aggressive coverage
No Load re-execution Efficient, Lesser L1 bandwidth<br>
08
RFP: Mitigating the L1 Latency Traditional Schedule L1 Hit Load Execution<br>
09
RFP: Mitigating the L1 Latency RFP Schedule Traditional Schedule Prefetch load value before its execution Allows dependent to hide the load latency<br>
10
Prefetch value at Predicted Address Prefetched
Value Prefetches must probe Store Buffer along with L1 Cache Obtain the Correct Value Prefetches can only Issue after load allocation RFP: Prefetching the Correct Value Guarantees prefetched value to be correct for correct address predictions Predict Load Address Prefetch Value Exploit the 3-Cycle Issue Slack of OOO pipeline to Prefetch Values just in time<br>
11
Register File Store the prefetched value in the same RF entry as the Demand load Prefetches only check Address. If wrong Demand load executes as is RFP Pipeline Use same Load Buffer Entry as Demand Load : Coherency/Consistency guaranteed<br>
12
Dependents only execute after the Prefetched Address is validated to be correct No Costly Pipeline Flush on Incorrect Prefetches RFP: Avoiding Pipeline Flushes<br>
13
Performance Evaluation Setup In-house execution driven, cycle accurate simulator
Models x86 Core like Intel Tigerlake @ 4GHz, 3MB LLC
65 diverse single-threaded applications
Benchmarks include – SPEC’17, SPEC’06, Cloud etc<br>
14
RFP Performance<br>
15
RFP Coverage<br>
16
RFP vs Prior Art Composite VP [Sheikh HPCA 2019] combines both the Value and Address Prediction schemes RFP coupled with VP provides the highest performance<br>
17
Conclusions Future cores need to mitigate L1 cache latency
L1 cache latency as important as DRAM memory latency
Proposed Register File Prefetching
A novel prefetcher that prefetches into Register File
Unlike cache prefetchers that prefetch from DRAM to Cache
Unlike value predictors that flush on incorrect prediction
Successfully prefetches 43% of Loads to Register File from L1
Provides 3.1% performance gains by mitigating L1 latency<br>
