Speicher Securing LSM-based KV Stores using

slide1. SpeicherSecuring LSM-based KV Stores using Shielded ExecutionMaurice Bailleu, Jörg Thalheim, Pramod Bhatotia Christof Fetzer
TU Dresden Michio Honda
NEC Labs Kapil Vaswani
Microsoft Research<br>
slide2. Storage security in the cloud How do we ensure security of data in the cloud?<br>
slide3. Trusted Execution Environments (TEEs):
Hardware extensions for trusted computing, e.g. Intel SGX and ARM TrustZone

Shielded execution:  Run-time framework based on TEE to provide strong security, e.g. HAVEN [OSDI’14] and SCONE [OSDI’16] Trusted computing Shielded
application Can we also use shielded execution for securing legacy storage systems?<br>
slide4. Shielded execution is mainly designed for securing (volatile) in-memory computation

However, storage systems require securing non-volatile state,  i.e., persistent state on an untrusted storage medium across reboot, crash, or migration Research GAP Unfortunately NOT!  
At least not directly ;-) Research challenge:
How to extend the trust beyond the "secure but stateless enclave" to the "untrusted storage in stateful setting"?<br>
slide5. Security properties:
Confidentiality
Unauthorized data access is prohibited
Integrity
Unauthorized change to data is detected
Freshness 
Stale state of data is detected (rollback / forking attacks) Our contribution A secure persistent LSM-based KV storefor the untrusted computing infrastructure<br>
slide6. Motivation
Challenges
Design
Evaluation Outline<br>
slide7. KV stores keep data in-memory for fast operations

Unfortunately, EPC is limited (only ~94MiB available)

To support larger memory region, SGX supports secure paging 

However, EPC paging incurs high overheads (2-2000X) Challenge #1: Enclave physical memory Redesigned in-memory LSM data structure (MemTable) to overcome the enclave physical memory limitation<br>
slide8. Persistent KV stores persist data on untrusted storage (SSDs)

Trust of the enclave does not naturally extends to untrusted storage

Further, the security properties need to valid across system reboot, crash, or migration Challenge #2: Untrusted storage Trusted enclave
(Volatile memory 
Region) Untrusted storage Redesigned on-disk LSM data structures (SSTable and log files) to extend the trust beyond the enclave memory<br>
slide9. Storage systems issue frequent I/O syscalls

Thread executing the syscall need to exit the enclave

Enclave exit operations are expensive since they  require TLB flushing, security checks, etc. Challenge #3: I/O syscall Designed a direct I/O library for shielded execution based on SPDK for fast I/O without exiting enclave Trusted enclave I/O call Exit enclave toissue the syscall<br>
slide10. To protect system against rollback or forking attacks, we need trusted monotonic counters

SGX counters are extremely slow (250ms) and wear out in a couple of days! 

Unsuitable for modern KV stores Challenge #4: Trusted counter Designed an asynchronous trusted counter interface taking advantage of persistency guarantees of modern KV stores<br>
slide11. Motivation
Challenges
Design
Background
System components
Algorithms
Evaluation Outline<br>
slide12. Persistent storage (SSD) Background: LSM-based KV store Level 0 MemTable SST SST SST SST SST SST Write Ahead Log (WAL) Level 1 Level 2 Log files Main memory MANIFEST …. …. …. ….<br>
slide13. Motivation
Challenges
Design
Background
System components
Algorithms
Evaluation Outline<br>
slide14. Operating system Host memory Trusted enclave memory System overview Storage
engine (RocksDB) MemTable I/O lib DMA Speicher controller SSD SSTable
Log files Trusted counter<br>
slide15. Enclave MemTable Host Split the MemTable into two parts:(1) Key part --> stored in the enclave
(2) Value part --> stored in the untrusted host memory (Key part) (Value part)<br>
slide16. KV SSTable KV … KV Block #1 KV KV … KV Block #n … Hn H1 … Storing the hashes of the blocks allows to make integrity checks for every KV pair, while still allowing fast lookup. Merkle tree<br>
slide17. Log files: WAL and Manifest Append #1 Record 1 Trusted Counter Value 1 Hash 1 Append #2 ... R2 C2 H2 Log files guarantee freshness of the data based on the trusted counter<br>
slide18. Asynchronous monotonic counter Time Unstable period Time to persist data in modern KV stores can overlap with the unstable period<br>
slide19. Motivation
Challenges
Design
Background
System components
Algorithms
Evaluation Outline See the paper
for more algorithms<br>
slide20. Algorithm #1: Put (K,V)<br>
slide21. Enclave Algorithm #1: Put Host Memory Storage
engine MemTable I/O lib DMA SSD SST, Log files Trusted Counter Client 1. Put request 2. Write record to WAL 3. Increment trusted counter 5. Write to MemTable 4. Get stable counter and expected time 6. Return success and expected time Speicher controller<br>
slide22. Algorithm #2: Get (K)<br>
slide23. Enclave Algorithm #2: Get Host Memory Storage
engine MemTable I/O lib DMA SSD SST, Log files Trusted Counter Client 3. Search in SST files 2. Search in MemTable 1.Get Request 4. Return Value Speicher controller<br>
slide24. Motivation
Challenges
Design
Evaluation Outline<br>
slide25. Questions:
What is the performance of the direct I/O library?
What is the performance overhead of SPEICHER?

Experimental setup:
Intel Xeon E3-1270 v5 (3.60 GHz, 4 cores, 8 hyper-threads) -- Skylake w/ SGX
Intel DC P3700 SSD (400GB, PCIe x4) -- w/ SPDK Evaluation See the paper
for more results<br>
slide26. Q1 : Throughput of the I/O lib Our I/O library performance at par to native SPDK<br>
slide27. Q2: Overheads of Speicher Reasonable overhead, which results mostly from en-/decryption of KV pairs!<br>
slide28. SummarySpeicher: A secure LSM based KV-Store Security properties: confidentiality + integrity + freshness Challenge: How to extend the trust beyond the "secure but stateless enclave" to the "untrusted storage in stateful setting"? Reasonably overheads<br>
slide29. Backup!<br>
slide30. Native MemTable 30 Memory K V K V Key Value<br>
slide31. Native SSTable 31<br>
slide32. Native log files: WAL and MANIFEST Append #1 Record 1 Append #2 ... Record 2<br>