Sridhara Dasu U Wisconsin Madison Outline General Introduction to Detector Trigger amp DAQ Introduction to LHC Trigger amp DAQ Challenges amp Architecture LHC Experiments Trigger ID: 1022135
Download Presentation The PPT/PDF document "Trigger & DAQ for Hadron Collider P..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Trigger & DAQ forHadron Collider PhysicsSridhara DasuU. Wisconsin – MadisonOutline:General Introduction to Detector Trigger & DAQIntroduction to LHC Trigger & DAQChallenges & ArchitectureLHC Experiments Trigger & DAQ
2. LHC Overviewwith every bunch crossing~25 Minimum Bias eventswith ~2000 particles produced
3. LHC Physics – Trigger ChallengeElectroweak Symmetry Breaking ScaleHiggs discovery and higgs sector characterizationQuark, lepton Yukawa couplings to higgsNew physics at TeV scale to stabilize higgs sectorSpectroscopy of new resonances (SUSY or otherwise)Find dark matter candidateMulti-TeV scale physics (loop effects)Indirect effects on flavor physics (mixing, FCNC, etc.)Bs mixing and rare B decaysLepton flavor violationRare Z and higgs decaysPlanck scale physicsLarge extra dimensions to bring it closer to experimentNew heavy bosonsBlackhole production Low PT g, e, mLow PT B, t jetsMultiple low PT objectsMissing ETLow PT leptonsHigh PT leptons and photonsMulti particle and jet events~ Dedicated triggers (CMS) or experiment (LHCB)Low 40 GeV
4. Trigger
5. Example: Scintillator SignalPhotomultiplier serves as the amplifierMeasure if pulse height is over a thresholdfrom H. Spieler “Analog and Digital Electronics for Detectors”
6. ReadoutAmplifierFilterShaperRange compressionclockSamplingDigital filterZero suppressionBufferFormat & ReadoutBufferFeature extractionDetector / Sensorto Data Acquisition SystemL1 Trigger LogicL2 Trigger Logic
7. Filtering & ShapingPurpose is to adjust signal for the measurement desiredBroaden a sharp pulse to reduce input bandwidth & noiseMake it too broad and pulses from different times mixAnalyze a wide pulse to extract the impulse time and integral ⇒Example: Signals from scintillator every 25 nsNeed to sum energy deposited over 150 nsNeed to put energy in correct 25 ns time binApply digital filtering & peak findingWill return to this example later
8. Sampling & DigitizationSignal can be stored in analog form or digitized at regular intervals (sampled)Analog readout: store charge in analog buffers (e.g. capacitors) and transmit stored charge off detector for digitizationDigital readout with analog buffer: store charge in analog buffers, digitize buffer contents and transmit digital results off detectorDigital readout with digital buffer: digitize the sampled signal directly, store digitally and transmit digital results off detectorZero suppression can be applied to not transmit data containing zerosCreates additional overhead to track suppressed dataSignal can be discriminated against a thresholdBinary readout: all that is stored is whether pulse height was over threshold
9. Range CompressionRather than have a linear conversion from energy to bits, vary the number of bits per energy to match your detector resolution and use bits in the most economical manner.Have differentranges withdifferent nos.of bits per pulseheightUse nonlinearfunctions tomatch resolution
10. Baseline SubtractionWish to measure the integral of an individual pulse on top of another signalFit slope in regions away from pulsesSubtract integral under fitted slope from pulse height
11. A wide variety of frontends
12. LHC Run-1 ParametersPile-Up – the number of proton interactions occurring during each bunch crossingDesign201020112012Beam Energy (TeV)73.53.54Bunches/Beam283536813801380Proton/Bunch(1011)1.151.31.51.5Peak Lumi.(1032 cm-2 s-1)10023060Integrated Lumi. (fb-1)100/yr0.036615*Pile-Up23~12030*expected value
13. Beam Xings: LEP. TeV, LHCLHC has ~3600 bunchesAnd same length as LEP (27 km)Distance between bunches: 27km/3600=7.5mDistance between bunches in time: 7.5m/c=25ns
14. LHC Physics & Event RatesAt design L = 1034cm-2s-123 pp events/25 ns xing~ 1 GHz input rate“Good” events contain ~ 20 bkg. events1 kHz W events10 Hz top events< 104 detectable Higgs decays/yearCan store ~ 300 Hz eventsSelect in stagesLevel-1 Triggers1 GHz to 100 kHzHigh Level Triggers100 kHz to 300 Hz
15. Collisions (p-p) at LHCEvent size: ~1 MByte Processing Power: ~X TFlopAll charged tracks with pt > 2 GeVReconstructed tracks with pt > 25 GeVOperating conditions:one “good” event (e.g Higgs in 4 muons ) + ~20 minimum bias events)Event rate
16. Processing LHC DataQCD
17. LHC Trigger & DAQ ChallengesComputing Services16 Million channels Charge TimePattern40 MHz COLLISION RATE100 - 50 kHz 1 MB EVENT DATA 1 Terabit/s READOUT 50,000 data channels200 GB buffers ~ 400 Readout memories3 Gigacell buffers 500 Gigabit/s 5 TeraIPS ~ 400 CPU farmsGigabit/s SERVICE LANPetabyte ARCHIVE EnergyTracks300 HzFILTERED EVENTEVENT BUILDER. A large switching network (400+400 ports) with total throughput ~ 400Gbit/s forms the interconnection between the sources (deep buffers) and the destinations (buffers before farm CPUs). EVENT FILTER. A set of high performance commercial processors organized into many farms convenient for on-line and off-line applications. SWITCH NETWORKLEVEL-1TRIGGERDETECTOR CHANNELSChallenges:1 GHz of Input InteractionsBeam-crossing every 25 ns with ~ 23 interactions produces over 1 MB of dataArchival Storage at about 300 Hz of 1 MB events
18. Challenges: Pile-up
19. Challenges: Time of Flightc = 30 cm/ns → in 25 ns, s = 7.5 m
20. LHC Trigger Levels100- 400 Hz
21. Level 1 Trigger Operation
22. Level 1 Trigger Organization
23. Trigger Timing & ControlSingle High-PowerLaser per zoneReliability, transmitter upgradesPassive optical coupler fanout1310 nm OperationNegligible chromatic dispersionInGaAs photodiodesRadiation resistance, low biasOptical System:
24. Detector Timing AdjustmentsNeed to Align:Detector pulse w/collision at IPTrigger data w/ readout dataDifferent detector trigger dataw/each otherBunch Crossing NumberLevel 1 Accept Number
25. Synchronization Techniques2835 out of 3564 p bunches are full, use this pattern:
26. ATLAS & CMS Trigger & Readout StructureFront end pipelinesReadout buffersProcessor farmsSwitching networkDetectorsLvl-1HLTLvl-1Lvl-2Lvl-3Front end pipelinesReadout buffersProcessor farmsSwitching networkDetectorsATLAS: 3 physical levelsCMS: 2 physical levels
27. CMS 2-Level Trigger & DAQLv1 decision is distributed to front-ends & readout via TTC system (red).Readout buffers designed to accommodate Poisson fluctuations from 100 kHz Level-1 trigger rate.Detector FrontendComputing ServicesReadoutSystemsFilterSystemsEvent ManagerBuilder NetworksLevel-1TriggerRunControlDataTriggerBack PressureThe Lv1-Accept decision causes the readout system to respond with backpressure information.Interaction rate: 1 GHzBunch Xing rate: 40 MHzLevel 1 Output: 100 kHzStorage Output: ~ 400 HzAvg Event Size: ~ 0.5 MBData production ~ 1 TB/day
28. Level 1 Trigger Data
29. Present ATLAS & CMS L1: Only Calorimeter & MuonSimple AlgorithmsSmall amounts of dataComplexAlgorithmsHugeamounts ofdataHigh Occupancy in high granularity tracking detectors
30. Present CMS L1 Trigger SystemLv1 trigger is based on calorimeter & muon detectors.At L1 trigger on:4 highest Et e/4 highest Et central jets4 highest Et forward jets4 highest Et tau-jets4 highest Pt muonsFor each of these objects rapidity, , and are also transmitted to Global Trigger for topological cuts & so Higher Level Triggers can seed on them.Also trigger on inclusive triggers:Et , MET, HtUXCUSCCalorimeters:Muon Systems:Generate L1A and send via TTC distribution to detector front-ends to initiate readoutMaximum round-trip latency 4 μsData stored in on-detector pipelines
31. CMS Trigger Levels ≈ 400 Hz
32. CMS Level-1 Trigger & DAQOverall Trigger & DAQ Architecture: 2 Levels:Level-1 Trigger:25 ns input3.2 s latencyInteraction rate: 1 GHzBunch Crossing rate: 40 MHzLevel 1 Output: 100 kHzOutput to Storage: 400 HzAverage Event Size: 1 MBData production 1 TB/dayUXCUSC
33. Calorimeter Trigger ProcessingTrigger Tower25 Xtals (TT)TCC(LLR)CCS(CERN)SRP(CEADAPNIA)DCC(LIP)TCSTTCTrigger primitives @800 Mbits/sODDAQ@100 kHzL1 Global TRIGGERRegionalCaloTRIGGERTrigger Tower Flags (TTF)Selective Readout Flags (SRF)SLB (LIP)Data path@100KHz (Xtal Datas)Trigger Concentrator CardSynchronisation & Link BoardClock & Control SystemSelective Readout ProcessorData Concentrator CardTiming, Trigger & ControlTrigger Control SystemLevel 1 Trigger (L1A)From : R. Alemany LIP
34. ECAL Trigger PrimitivesTest beam results (45 MeV per xtal):
35. CMS Electron/Photon Algorithm
36. CMS t / Jet Algorithm
37. Tau Trigger Upgrade - 2015
38. Pileup Subtraction (2015)
39. Tau Trigger Improvement (2015)
40. Example Hardware (2015) : CMS Calorimeter Trigger Systems
41. Example Card (2014-2015):Trigger Processor (CTP7)
42. Example Firmware Block
43. Reduced RE system|h| < 1.61.6ME4/1MB1MB2MB3MB4ME1ME2ME3*Double Layer*RPCSingle LayerCMS Muon Chambers
44. Muon Trigger Overview|| < 1.2|| < 2.40.8 < || || < 2.1|| < 1.6 in 2007Cavern: UXC55Counting Room: USC55
45. CMS Muon Trigger PrimitivesMemory to store patternsFast logic for matching FPGAs are ideal
46. CMS Muon TriggerTrack FindersMemory to store patternsFast logic for matching FPGAs are idealSort based on PT, Quality - keep loc.Combine at next level - matchSort again - Isolate?Top 4 highest PT and quality muons with location coord.Match with RPC Improve efficiency and quality
47. CMS Global Trigger
48. Global L1 Trigger Algorithms
49. High Level Trigger Strategy
50. CMS DAQ & HLTDAQ unit (1/8th full system):Lv-1 max. trigger rate 12.5 kHzRU Builder (64x64) .125 Tbit/sEvent fragment size 16 kBRU/BU systems 64Event filter power ≈ .5 TFlopData to surface:Average event size 1 MbyteNo. FED s-link64 ports > 512DAQ links (2.5 Gb/s) 512+512Event fragment size 2 kBFED builders (8x8) ≈ 64+64DAQ Scaling & StagingHLT: All processing beyond Level-1 performed in the Filter FarmPartial event reconstruction “on demand” using full detector resolution
51. Start with L1 Trigger Objects Electrons, Photons, t-jets, Jets, Missing ET, MuonsHLT refines L1 objects (no volunteers)GoalKeep L1T thresholds for electro-weak symmetry breaking physicsHowever, reduce the dominant QCD backgroundFrom 100 kHz down to 100 Hz nominallyQCD background reductionFake reduction: e±, g, tImproved resolution and isolation: mExploit event topology: JetsAssociation with other objects: Missing ETSophisticated algorithms necessaryFull reconstruction of the objectsDue to time constraints we avoid full reconstruction of the event - L1 seeded reconstruction of the objects onlyFull reconstruction only for the HLT passed events
52. Electron selection: Level-2“Level-2” electron:Search for match to Level-1 triggerUse 1-tower margin around 4x4-tower trigger regionBremsstrahlung recovery “super-clustering”Select highest ET clusterBremsstrahlung recovery:Road along f — in narrow -window around seedCollect all sub-clusters in road “super-cluster”basic clustersuper-cluster
53. CMS tracking for electron triggerPresent CMS electron HLTFactor of 10 rate reduction: only tracker handle: isolationNeed knowledge of vertexlocation to avoid loss of efficiency
54. t-jet tagging at HLT
55. B and t tagging
56. Prescale set used: 2E32 Hz/cm²Sample: MinBias L1-skim 5E32 Hz/cm² with 10 Pile-upUnpacking of L1 information,early-rejection triggers,non-intensive triggersMostly unpacking of calorimeter info.to form jets, & some muon triggersTriggers with intensive tracking algorithmsOverflow: Triggers doing particle flowreconstruction (esp. taus)CMS HLT Time Distribution(example from early 2011)
57. CMS DAQ 12.5 kHz12.5 kHz100 kHz12.5 kHz… Read-out of detector front-end drivers Event Building (in two stages) High Level Trigger on full events Storage of accepted events
58. 2-Stage Event Builder 12.5 kHz12.5 kHz100 kHz12.5 kHz… 1st stage “FED-builder”Assemble data from 8 front-ends into one super-fragment at 100 kHz 8 independent “DAQ slices”Assemble super-fragments into full events
59. Building the eventEvent builder : Physical system interconnecting data sources with data destinations. It has to move each event data fragments into a same destinationEvent fragments : Event data fragments are stored in separated physical memory systemsFull events : Full event data are stored into one physical memory system associated to a processing unitHardware:Fabric of switches for builder networksPC motherboards for data Source/Destination nodes
60. Myrinet Barrel-Shifter BS implemented in firmware Each source has message queue per destination Sources divide messages into fixed size packets (carriers) and cycle through all destinations Messages can span more than one packet and a packet can contain data of more than one message No external synchronization (relies on Myrinet back pressure by HW flow control) zero-copy, OS-bypass principle works for multi-stage switches
61. EVB – HLT installationEVB – input “RU” PC nodes640 times dual 2-core E5130 (2007) Each node has 3 links to GbE switch Switches8 times F10 E1200 routers In total ~4000 portsEVB – output + HLT node (“BU-FU”)720 times dual 4-core E5430, 16 GB (2008)288 times dual 6-core X5650, 24 GB (2011)Each node has 2 links to GbE switchHLT Total: 1008 nodes, 9216 cores, 18 TB memory@100 kHz: ~90 ms/eventCan be easily expanded by adding PC nodes and recabling EVB network
62. Trigger & DAQ SummaryLevel 1 TriggerSelect 100 kHz interactions from 1 GHzProcessing is synchronous & pipelinedDecision latency is 3 ms (x~2 at HL-LHC)Algorithms run on local, coarse dataCal & Muon at LHC (include tracking at LHC-HL)Use of ASICs & FPGAsHigher Level TriggersDepending on experiment, done in one or two stepsIf two steps, first is hardware region of interestThen run software/algorithms as close to offline as possible on dedicated farm of PCs