Slack Space Recycling and Lazy Indirect Block Update Yongseok Oh ltysohuosackrgt The 3rd Annual Haifa Experimental Systems Conference May 24 2010 Haifa Israel 1 Yongseok Oh and ID: 816064
Download The PPT/PDF document "Optimization of LFS with" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Optimization of LFS with Slack Space Recycling and Lazy Indirect Block Update
Yongseok Oh <ysoh@uos.ac.kr>The 3rd Annual Haifa Experimental Systems ConferenceMay 24, 2010Haifa, Israel
1
Yongseok
Oh and
Donghee
Lee (University of Seoul)
Jongmoo
Choi
(
Dankook
University)
Eunsam
Kim and Sam H. Noh (
Hongik
University)
Slide2IntroductionSlack Space Recycling and Lazy Indirect Block UpdateImplementation of SSR-LFSPerformance Evaluation
Conclusions2Outline
Slide3LFS collects small write requests and writes them sequentially to the storage device [Rosenblum at el, ACM TOCS ‘91]
Advantages Superior write performance for random workloadsFast recoveryDrawbacksOn-demand cleaningCascading meta-data update
3
Log-structured File System
Storage
A
B
C
D
Segment buffer
Slide4Hard Disk Drives Mechanical components – disk head, spindle motor, and platterPoor random read/write performanceSolid State Drives
Consist of NAND flash memory, no mechanical partsHigh performance, low-power consumption, and shock resistanceSequential writes are faster than random writes4
Storage Devices
Slide5IntroductionSlack Space Recycling and Lazy Indirect Block UpdateImplementation of SSR-LFS
Performance EvaluationConclusions5Outline
Slide6To make free segments
LFS cleaner copies valid blocks to other free segment
On-demand cleaning
Overall performance decreases
Background cleaning
It does not affect the performance
6
LFS cleaning
A
B
C
D
A
B
C
D
Copy
Segment 1
Segment 2
Segment 3
Slide7Matthews et al. employed
Hole-plugging
in LFS
[
Matthews et al, ACM OSR ’97
]
The cleaner copies valid blocks to holes of
other segments
7
Hole-plugging
A
B
C
D
Copy
Segment 1
Segment 2
Segment 3
A
B
Slide8We proposed SSR scheme that directly recycles slack space to avoid on-demand cleaning
Slack Space is invalid area in used segment 8Slack Space Recycling (SSR) Scheme
SSR
SSR
A
B
C
D
Segment 1
Segment 2
Segment 3
Segment buffer
E
F
G
H
E
F
G
H
Slide9Update of a data block incurs cascading meta-data updateModification of a data block consumes 4 blocks
9Cascading meta-data update
Indirect
Data
A
A
B
Double
Indirect
B
C
i
-node
C
1
N-1
N
D
A
B
C
D
E
F
A’
Segment 1
Segment 2
Data’
A’
Update
C’
1
N-1
N
D’
Update
A’
B’
update
B’
C’
update
D’
B’
C’
Slide10We propose
LIBU scheme
to decrease cascading meta-data updateLIBU uses IBC (Indirect Block Cache)
to absorb the frequent update of indirect blocks
Indirect Map
is necessary to terminate cascading meta data update
10
Lazy Indirect Block Update (LIBU) scheme
Indirect
Data
A
A
B
Double
Indirect
B
C
i
-node
2
1
N-1
N
D
IBC (Indirect Block Cache)
E
Indirect Map
C
A
B
C
D
E
F
A’
D’
Segment 1
Segment 2
Data’
A’
Update
2
1
N-1
N
D’
Update
A’
B’
Insert
B’
C’
Insert
C’
Insert
E’
Slide11For crash recovery, LFS periodically stores checkpoint information
If power failure occurs,
search the last check pointscan all segments written after the last check pointrebuild
i
-node map, segment usage table, indirect map, and indirect blocks in IBC
11
Crash Recovery
Power failure
search the last checkpoint
Scan
Check point
Check point(last)
i
-node map
Segment usage table
indirect map
indirect blocks
flush
Write
Write
Rebuild
Consistent state
Disk
RAM
Slide12IntroductionSlack Space Recycling and Lazy Indirect Block UpdateImplementation of SSR-LFS
Performance EvaluationConclusions12
Outline
Slide13We implemented SSR-LFS (Slack Space Recycling LFS)Using FUSE (Filesystem in Userspace
) framework in LinuxSSR-LFS selectively chooses either SSR or cleaning When the system is idle, it performs background cleaningWhen the system is
busy, it performs SSR or on-
dmenad
cleaning
If
average slack size is too small
, it selects
on-demand cleaning
Otherwise, it selects SSR13
Implementation of SSR-LFS
VFS
FUSE
write(“/
mnt/file”)
SSR-LFS Core
Userspace
Kernel
Syncer
IBC
Cleaner
buffer cache
Recycler
i
-node cache
libfuse
Architecture of SSR-LFS
Slide14IntroductionSlack Space Recycling and Lazy Indirect Block Update
Implementation of SSR-LFSPerformance EvaluationConclusions14
Outline
Slide15For comparison, we used several file systemsExt3-FUSEOrg-LFS(cleaning)Org-LFS(plugging)SSR-LFS
BenchmarksIO TEST that generates random write workloadPostmark that simulates the workload of a mail server Storage Devices SSD – INTEL SSDSA2SH032G1GNHDD – SEAGATE ST3160815AS
15
Experimental Environment
Slide16SSR-LFS shows better than others for a wide range of utilizationOn HDDSSR-LFS and Org-LFS outperform Ext3-FUSE under utilization of 85%,
On SSDExt3-FUSE outperforms Org-LFS due to optimization of SSD for random writes16Random Update Performance
HDD
SSD
Slide17Medium file size (16KB ~ 256KB)Subdirectories 1000, # of files 100,000, # of transactions 100,000SSR-LFS outperforms other file systems on both devices
Org-LFS shows better performance than Ext3-FUSE on HDDExt3-FUSE shows comparative performance to Org-LFS on SSD17
Postmark Benchmark Result (1)
Slide18Small file size (4KB ~ 16KB)Subdirectories 1000, # of files 500,000 ,# of transactions 200,000Ext3-FUSE performs better than other file systems on SSD
due to meta-data optimization of Ext3 such as hash-based directory18Postmark Benchmark Result (2)
Slide19IntroductionSlack Space Recycling and Lazy Indirect Block Update
Implementation of SSR-LFSPerformance EvaluationConclusions19
Outline
Slide20SSR-LFS outperforms original style LFS for a wide range of utilizationFuture worksOptimization of meta-data structures Cost-based selection between cleaning and SSR
We plan to release the source code of SSR-LFS this yearhttp://embedded.uos.ac.kr 20
Conclusions
Slide21Thank you21
Q & A
Slide2222
Back up slides
Slide2323Storage devices
Slide24To identify the performance penalty of a user space implementationExt3-FUSE underperforms Kernel Ext3 for almost patternsDue to FUSE overhead
24Measurement of FUSE overhead
Slide25Medium file size (16KB ~ 256KB)Subdirectories 1000, # of files 100,000, # of transactions 100,000SSR-LFS outperforms other file systems on both devices
Org-LFS shows better performance then Ext3-FUSE on HDDExt3-FUSE shows comparative performance to Org-LFS on SSD25
Postmark Benchmark Result (1)
1302 cleanings
4142 SSRs
Slide26Small file size (4KB ~ 16KB)Subdirectories 1000, # of files 500,000 ,# of transactions 200,000Ext3-FUSE performs better than other file systems on SSD
due to meta-data optimization of Ext3 such as hash-based directory26Postmark Benchmark Result (2)
3018 cleanings
1692 cleanings
1451 SSRs
Slide27Large file size (disappeared in the paper)File size 256KB ~ 1MBSubdirectories 1000# of files 10,000
# of transactions 10,00027Postmark Benchmark Result (3)
Slide2828Statistics of IO-TEST
Slide29IO TEST benchmark 1st stage Creates 64KB files until 90% utilizationRandomly deletes files until target utilization
2nd stageRandomly updates up to 4GB of file capacityMeasures the elapsed time in this stageLFS has no free segmentsIncluding cleaning or SSR cost
29
Random Update Performance
Disk
90%(High threshold)
20%(desired threshold)
Create files
Delete files
Random update files
Slide3030
Experimental EnvironmentType
Specifics
OS
Ubuntu
9.10 (linux-2.6.31.4)
Processor
AMD 2.1GHz
Opteron
1352RAM
SAMSUNG 4GB DDR-2 800Mhz ECC
SSDINTEL SSDSA2SH032G1GNHDDSEAGATE ST3160815AS
Slide3131Comparison of cleaning and SSR scheme
T
seg
write
Foreground
Background
Time
Free Segments
Write Request
T
back
-ground
1
T1
T3
0
T
seg
write
1
T
on-demand
cleaning
0
1
2
3
delay
T2
x
T
idle
T
seg
write
Foreground
Background
Time
Free Segments
T
ssr
SSR
Write Request
T
back
-ground
0
1
2
T1
T2
T3
0
T
seg
write
1
3
T
idle
Cleaning case :
SSR case :
Slide32To measure the performance impact of Lazy Indirect Block Update1 worker - File size 1GB, Request size 4KB, Sync I/O modeSSR-LFS outperforms Org-LFS and Ext3-FUSE for sequential and random writes on both devices
32FIO Benchmark Result (1)
Slide334 workers - File size 1GB, Request size 4KB, Sync modeThe LIBU scheme has greater impact on performance when four workers are running
33FIO Benchmark Result (2)