Accelerating Read Mapping with - PowerPoint Presentation

1. Accelerating Read Mapping with FastHASHHongyi Xin† Donghyuk Lee† Farhad Hormozdiari ‡ Samihan Yedkar† Can Alkan § Onur Mutlu†† Carnegie Mellon University § University of Washington‡ University of California Los Angeles

2. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion2

3. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion3

4. Read MappingA post-processing procedure after DNA sequencingMap many short DNA fragments (reads) to a known reference genome with some minor differences allowed4Reference genomeReadsDNA, logicallyDNA, physicallyMapping short reads to reference genome is challenging (billions of 50-300 base pair reads)

5. ChallengesNeed to find many mappings of each readA short read may map to many locations, especially with Next Generation DNA SequencingHow can we find all mappings efficiently?Need to tolerate small variances/errors in each readEach individual is different: Subject’s DNA may slightly differ from the reference (Mismatches, insertions, deletions)How can we efficiently map each read with up to e errors present?Need to map each read very fast (i.e., performance is important)Human DNA is 3.2 billion base pairs long  Millions to billions of reads (State-of-the-art mappers take weeks to map a human’s DNA)How can we design a much higher performance read mapper?5

6. OutlineRead Mapping and its ChallengesHash Table-Based MappersPreprocess the reference into a Hash TableUse Hash Table to map readsProblem and GoalKey ObservationsMechanismsResults6

7. Hash Table-Based Mappers [Alkan+ NG’09]712324577940AAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAT13421412765889......CCCCCCCCCCCC......24459744988989............TTTTTTTTTTTT36535123NULLReference genomek-mer or 12-mer Location list—where the k-mer occurs in reference gnomeOnce for a reference

8. OutlineRead Mapping and its ChallengesHash Table-Based MappersPreprocess the reference into a Hash TableUse Hash Table to map readsProblem and GoalKey ObservationsMechanismsResults8

9. 12Hash Table-Based Mappers [Alkan+ NG’09]12324557940AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTCCCCCCCCCCCCTTTTTTTTTTTTReference GenomeHash Table (HT)readk-mersAAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTT2445974498898936535823…AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTT…AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTTAAAAAAAAAAAA324.. AAAAAAAAAAAAAACGCTTCCACCTTAATCTGGTTG..read***..****************************************..Invalid mapping9Valid mapping✔Verification/Local Alignment

10. Advantages of Hash Table Based Mappers+ Guaranteed to find all mappings+ Tolerate up to e errors10

11. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion11

12. Problem and GoalPoor performance of existing read mappers: Very slow Verification/alignment takes too long to executeVerification requires a memory access for reference genome + many base-pair wise comparisons between the reference and the readGoal: Speed up the mapper by reducing the cost of verification1295%

13. Reducing the Cost of VerificationWe observe that most verification calculations are unnecessary1 out of 1000 potential locations passes the verification processWe also observe that we can get rid of unnecessary verification calculations byDetecting and rejecting early invalid mappingsReducing the number of potential mappings13

14. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion14

15. Key ObservationsObservation 1Adjacent k-mers in the read should also be adjacent in the reference genomeHence, mapper can quickly reject mappings that do not satisfy this propertyObservation 2Some k-mers are cheaper to verify than others because they have shorter location lists (they occur less frequently in the reference genome) Mapper needs to examine only e+1 k-mers’ locations to tolerate e errorsHence, mapper can choose the cheapest e+1 k-mers and verify their locations15

16. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion16

17. FastHASH MechanismsAdjacency Filtering (AF): Rejects obviously invalid mapping locations at early stage to avoid unnecessary verificationsCheap K-mer Selection (CKS): Reduces the absolute number of potential mapping locations17

18. Adjacency Filtering (AF)Goal: detect invalid mappings at early stageKey Insight: For a valid mapping, adjacent k-mers in the read are also adjacent in the reference genomeKey Idea: search for adjacent locations in the k-mers’ location listsIf more than e k-mers fail—there must be more than e errors—invalid mapping18AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTreadReference genomeValid mappingInvalid mapping

19. 12Adjacency Filtering (AF)12324557940AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTCCCCCCCCCCCCTTTTTTTTTTTTReference GenomeHash Table (HT)readk-mersAAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTT2445974498898936535123…AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTT…AAAAAAAAAAAACCCCCCCCCCCCTTTTTTTTTTTTAAAAAAAAAAAA32424?36?336?***+12+24557569?940952?✗19

20. FastHASH MechanismsAdjacency Filtering (AF): Rejects obviously invalid mapping locations at early stage to avoid unnecessary verificationsCheap K-mer Selection (CKS): Reduces the absolute number of potential mapping locations20

21. Cheap K-mer Selection (CKS)Goal: Reduce the number of potential mappingsKey insight:K-mers have different cost to examine: Some k-mers are cheaper as they have fewer locations than others (occur less frequently in reference genome)Key idea: Sort the k-mers based on their number of locationsSelect the k-mers with fewest locations to verify21

22. Cheap K-mer Selectione=2 (examine 3 k-mers)22AAGCTCAATTTC CCTCCTTAATTT TCCTCTTAAGAA GGGTATGGCTAG AAGGTTGAGAGC CTTAGGCTTACCread3141231441492194 loc.338…………1K loc.376…………2K loc.32614512 loc.32614512 loc.388…………1K loc.Previous work needs to verify:3004 locationsFastHASH verifies only:8 locationsLocationsNumber of LocationsCheapest 3 k-mersExpensive 3 k-mers

23. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion23

24. MethodologyImplemented FastHASH on top of state-of-the-art mapper: mrFASTNew version mrFAST-2.5.0.0 over mrFAST-2.1.0.6Tested with real read sets generated from Illumina platform1M reads of a human (160 base pairs)500K reads of a chimpanzee (101 base pairs)500K reads of a orangutan (70 base pairs)Tested with simulated reads generated from reference genome1M simulated reads of human (180 base pairs)Evaluation systemIntel Core i7 Sandy Bridge machine16 GB of main memory24

25. FastHASH Speedup25orangutansimulated humanchimpanzee19xWith FastHASH, new mrFAST obtains up to 19x speedup over previous version, without losing valid mappings

26. AnalysisReduction of potential mappings with FastHASH2699%99%99%99%99%FastHASH filters out over 99% of the potential mappings without sacrificing any valid mappings

27. Other Key Results (In the paper)FastHASH finds all possible valid mappingsCorrectly mapped all simulated reads (with fewer than e artificially added errors)27

28. OutlineRead Mapping and its ChallengesHash Table-Based MappersProblem and GoalKey ObservationsMechanismsResultsConclusion28

29. ConclusionProblem: Existing read mappers perform poorly in mapping billions of short reads to the reference genome, in the presence of errorsObservation: Most of the verification calculations are unnecessaryKey Idea: To reduce the cost of unnecessary verificationReject invalid mappings early (Adjacency Filtering)Reduce the number of possible mappings to examine (Cheap K-mer Selection)Key Result: FastHASH obtains up to 19x speedup over the state-of-the-art mapper without losing valid mappings29

30. AcknowledgementsCarnegie Mellon University (Hongyi Xin, Donghyuk Lee, Samihan Yedkar and Onur Mutlu, co-authors)Bilkent University (Can Alkan, co-author)University of Washington (Evan Eichler and Can Alkan)UCLA (Farhad Hormozdiari, co-author)NIH (National Institutes of Health) for financial support30

31. Thank you! Questions?Download link to FastHASHYou can find the slides on SAFARI group website:http://www.ece.cmu.edu/~safari 31

32. Accelerating Read Mapping with FastHASHHongyi Xin† Donghyuk Lee† Farhad Hormozdiari ‡ Samihan Yedkar† Can Alkan § Onur Mutlu†† Carnegie Mellon University § University of Washington‡ University of California Los Angeles

33. Mapper Comparison: Number of Valid Mappings33Bowtie does not support error threshold larger than 3FastHASH is able to find many more valid mappings than Bowtie and BWA

34. Mapper Comparison: Execution Time34Bowtie does not support error threshold larger than 3FastHASH is slower for e <= 3, but is much more comprehensive (can find many more valid mappings)