FPGAs Grzegorz Korcyl Jagiellonian University Kraków Grzegorz Korcyl PANDA TDAQ Workshop Giessen April 2010 O utline Motivation General structure GbE Buffer Packet Constructor ID: 1031643
Download Presentation The PPT/PDF document "Gigabit Ethernet implementation for" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Gigabit Ethernet implementation for FPGAsGrzegorz Korcyl - Jagiellonian University, KrakówGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
2. OutlineMotivationGeneral structureGbE BufferPacket ConstructorFrame ConstructorFrame TransmitterResources utilizationResultsSetup 1Setup 2SummaryFuture plansGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
3. 1. MotivationThe way to transport large amounts of data at high speedProtocol with routing capabilites neededUDP over GbE is going to be implemented on HUB2, MDC, Shower addonsGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010Addon + GbE1 Evt builder2 Evt builder3 Evt builderGbE Switch
4. 2. General StructureTrbNet – Hades trigger, slow controland data transport network systemGbE Buffer gathers subeventsfrom TrbNet and prepares packetconstructionPacket Constructor creates aHadesTransportUnitQueueFrame Constructor fragments data and creates Ethernet framesFrame Transmitter transmits each frame Grzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
5. 2. General StructureMessageSubevent 1Subevent 2Subevent 3Subevent 4Packet 1Packet 2Packet 3Frame 1Frame 2Frame 3Frame 1Frame 2Frame 3Frame 1Frame 2Grzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010GbE BufferPacket ConstructorFrame ConstructorTrbNetETHERNETFrame Transmitter
6. 2.a GbE BufferInterface between TrbNet and GbE logicBuffers incoming dataResponds back to readout requestAllows configuration via slow controlPepares the data for Packet ConstructorDecides about the start/close a packetSelects event builder address basing on the readout parameteresAdds additional internal subevent headers Grzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
7. 2.b Packet ConstructorConstructs HadesTransportUnitQueue:Several subevents in one queueConsistency check by termination bytesRight now up to 64kB in totalMinimum overhead (one subevent case) is 56B per queueGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
8. 2.c Frame ConstructorConstructs IP frames:Fragments the whole queue into 1.5kB (MTU) parts and encapsulates with network headers42B of headers overhead per frameGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
9. 2.d Frame TransmitterUses two IP Cores to transmit bytes over fiber:Tri-Speed MACTransmit state machineInsertion of pause framesAddition of Ethernet preamble and CRCSGMII/GbE PCSInterface between TS MAC and SERDESLink state machineAutonegotiationConversion to 8b/10bGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
10. 2.e Resources utilizationBuilt on Lattice ECP2M LFE2M100EMainly consumes RAM blocks inside FPGAGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010ResourceUtilizationPercentageSlices4256 out of 473768%LUTs5984 out of 950006%Blocks RAM76 out of 288 (~150kB)26%
11. 3.a Results – setup 1One data source – dummy front-endHUBv2Netgear optical GbE switchOne event builderTriggering by TRBGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
12. 3.a Results – setup 1No pipelining, single packet construction at a time limits throughput to half of nominal link bandwidthEvent rate limited by single trigger in the system at a timeGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
13. 3.b Results – setup 25x ADCM data sourcesHUBv2Longshine optical GbE switchOne event builderTriggering by generatorGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
14. 3.b Results – setup 2Packet size 38.8kBAccepted event rate 1.25kHz and 49MB/s throughputLong-lasting and stable connectionGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010Old DAQNew DAQFactorBandwith to Event builder6 MB/s150 MB/s25x fasterEvent rate / sector50 Hz1.25 kHz25x fasterThroughput / sector1 MB/s49 MB/s50x faster
15. 4. SummaryConstruction of HadesTransportUnitQueues of the size up to 64kBTransmission of packets at satisfying speedMultiple subevents combined into one queueConfiguration via slow controlAutonegotiationDynamic event builder selectionBasic, ready to use GbE component is readyGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010
16. 5. Future PlansStabilize fast packet constructionPipeline packet construction in order to reach the full GbLarge queues fragmentation into multiple UDP packetsAbility to transport queues (subevents) larger than one UDP packet (64kB)Test different MTU sizes (Jumbo packets) in order to find the optimal configurationControl of the transmission status and statisticsGenerate status information and create systems to handle unexpected situationsFull featured GbE component planed to be ready in MayGrzegorz Korcyl – PANDA TDAQ Workshop, Giessen April 2010