Paulo Moreira November 2010 CERN Outline GBT Project Status GBT project overview Radiation hard link GBT link bandwidth The GBT chipset The GBTIA The GBLD The GBT SCA The GBT Protocol on FPGAs ID: 448867
Download Presentation The PPT/PDF document "GBT Project" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
GBT Project
Paulo Moreira
November 2010
CERNSlide2
Outline
GBT Project Status:
GBT project overview
Radiation hard linkGBT link bandwidthThe GBT chipsetThe GBTIAThe GBLDThe GBT - SCAThe GBT Protocol on FPGAsThe E – Links:SLVS data transmission testsDriver/ReceiverThe GBT – SerDesThe GBT – SerDes ArchitectureSerializerDe-serializerPhase-ShifterLogicPower consumptionGBT Project ScheduleGLIB overview
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
2Slide3
Radiation Hard Optical Link Architecture
Defined in the “DG White Paper”
“Work Package 3-1”
Objective:Development of an high speed bidirectional radiation hard optical linkDeliverable:Tested and qualified radiation hard optical linkDuration:4 years (2008 – 2011)Radiation Hard Optical Link:Versatile link project:
Opto-electronics componentsRadiation hardness
Functionality testingGBT project:
ASIC designVerification
Radiation hardness
Functionality testing
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
3
On-Detector
Radiation Hard Electronics
Off-Detector
Commercial Off-The-Shelf (
COTS)
GBTX
GBTIA
GBLD
PD
LD
Custom ASICs
Timing & Trigger
DAQ
Slow Control
Timing & Trigger
DAQ
Slow Control
FPGA
GBT
GBT
Versatile LinkSlide4
GBT Link Bandwidth
Bandwidth:
User: 3.36 Gb/s
Line: 4.8 Gb/sGeneric data field:3.2 Gb/s (80-bits)Dedicated channels:Link control: 80 Mb/s (2-bits)Slow control channel: 80 Mb/s (2-bits)DC balance:ScramblerNo bandwidth penaltyLink is bidirectionalLink is symmetrical
Down-link highly flexible:Can convey unique data to each frontend device that it is serving
“Soft” architecture managed at the control room level
Frame Synchronization:
Redundant header
Forward Error Correction:
Interleaved Reed-Solomon double error correction
4-bit symbols (RS(15,11))
Interleaving: 2
Error correction capability:
2 Interleaving
× 2 RS
= 4 symbols 16-bitsCode efficiency: 88/120 = 73%Transmission protocol easily implemented in modern FPGAshttp://cern.ch/proj-gbt
Paulo.Moreira@cern.ch4Slide5
The GBT Chipset
Radiation tolerant chipset:
GBTIA: Transimpedance optical receiver
GBLD: Laser driverGBTX: Data and Timing TransceiverGBT-SCA: Slow control ASICSupports:Bidirectional data transmissionBandwidth:Line rate: 4.8 Gb/sEffective: 3.36 Gb/sThe target applications are:Data readoutTTCSlow control and monitoring links. Radiation tolerance:Total doseSingle Event Upsets
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
5
GBTIA
GBLD
GBTX
GBT-SCA
Frontend
Electronics
Data<119:0>
Clock<7:0>
Control<N:0>Slide6
The GBTIA
Main specs:
Bit rate 5 Gb/s (min)
Sensitivity: 20 μA P-P (10-12 BER)Total jitter: < 40 ps P-PInput overload: 1.6 mA (max)Dark current: 0 to 1 mASupply voltage: 2.5 VPower consumption: 250 mWDie size: 0.75 mm × 1.25 mmEngineers :
Ping Gui – SMU, USAMohsine Menouni – CPPM, France
Status:Chip fabricated and testedChip fully meets specifications!
Radiation tolerance proven!GBTIA + PIN-diode encapsulated in a TO Package (Versatile link project)
Future:
Version 2 will address productivity
Pad positions reworked to facilitate the wire bond operation between the package and ASIC
Mean optical power monitoring to facilitate pin-diode/fiber alignment
2.5 V supply
Migration from the LM to the DM technologies flavor
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
6Slide7
The GBLD
Main specs:
Bit rate 5 Gb/s (min)
Modulation:current sinkSingle-ended/differentialLaser modulation current: 2 to 12 mALaser bias: 2 to 43 mA“Equalization”Pre-emphasis/de-emphasisIndependently programmable forrising/falling edgesSupply voltage: 2.5 VDie size: 2 mm × 2 mm
I2C programming interfaceEngineers
:Gianni Mazza – INFN, Italy
Angelo Rivetti – INFN, ItalyKen Wyllie – CERNPing Gui – SMU, USA
Status:
Chip fabricated and tested
Chip fully functional
Performance according to specs (if corrected
for the large input capacitance of the input protection diode)
Future:
Reduce the area of the input protection diode
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch7Slide8
The GBT – SCA
GBT-SCA Main specs:
Dedicated to slow control functions
Interfaces with the GBTX using a dedicated E-link portCommunicates with the control room using a protocol carried (transparently) by the GBTImplements multiple protocol busses and functions:I2C, JTAG, Single-wire, parallel-port, etc…Implements environment monitoring functions:Temperature sensingMulti-channel ADCMulti-channel DACEngineers:
Alessandro Gabrielli – INFN, ItalyKostas Kloukinas – CERN, Switzerland
Sandro Bonacini – CERN, SwitzerlandAlessandro Marchioro – CERN, Switzerland
Filipe Sousa – CERN, Switzerland
Status
Specification work undergoing:
1
st
Draft already available
RTL design undergoingTape-out: 2011
10-bit ADC prototype submitted for fabrication in April 2010
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch8Slide9
The GBT Protocol on FPGAs
GBT-SERDES successfully implemented in FPGAs:
Scrambler/ Descrambler + Encoder/ Decoder + Serializer/CDR
FPGA Tested:XILINX Virtex-5FXT and 6LXTALTERA Stratix II and IV GXOptimization studies:Optimization of use of resources (2009)Low and “deterministic” latency (2010)Firmware:“Starter Kit” is available for download with various resources optimization schemes forStratixIIGx and Virtex5FXTAvailable soon for:StratixIVGx and Virtex6LXTLow latencyEngineers:Sophie Baron – CERN, Switzerland
Jean-Pierre Cachemiche – CPPM, FranceCsaba Soos – CERN, Switzerland
Steffen Muschter - Stockholm UniversityUsers:30 registered users from all over the world (most users from collaborating institutes)
LHC experiments, but also CLIC, PANDA, GBTVery active users are now part of the development team
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
9
Altera + opto TRx - 4.8 Gb/s
Xilinx - 4.8 Gb/sSlide10
SLVS Driver/Receiver
Receiver
Power Supply: 1.2V to 1.5V
Power Dissipation:150uW @ 320Mbs, 1.2V supply<1uW @ power downDriverPower Supply: 1.2V to 1.5 VPower Dissipation:3.1mW @ 320Mbs, 1.2 V supply<10uW @ power down
Engineer
Sandro Bonacini – CERN, Switzerland
Status:
Chip currently under testing
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
10
Electrical Specifications
Electrical Specifications
Programmable Output CurrentSlide11
E – Links: SLVS Data Transmission Tests
Scalable Low Voltage Standard (SLVS)
JEDEC standard: JESD8-13
Main features: 2 mA Differential max Line impedance: 100 Ohm Signal: +- 200 mV Common mode ref voltage: 0.2VTests on SLVS-RT chip
1 driver1 receiverVarious types of transmission media tested:
KaptonPCB
Ethernet cableTest equipmentBidirectional link
FPGAs perform pseudo-random data generation and checking
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
11
(*) PRELIMINARYSlide12
X-ray Irradiation Results
Pre-
rad
cycle-to-cycle jitter measured using a PRBS sequence generator (Agilent 81133A)is about 17 ps (rms)
All chips show a peak in the SLVS receiver supply current and then a decrease to a value smaller than the pre-rad.
SLVS transmitter supply current doesn’t change significantly with irradiation.
Chips show a worse jitter performance after irradiation
Sequence-dependence, most likely due to the receiver becoming slower for the decrease in supply
current
PMOS threshold increase responsible for bias current degradation.
New chip submitted July 2010 with a resized bias circuit
Input
from Xilinx
S3E
Input
Output
Pre-
radPost-
radSlide13
The GBT - SerDes
The GBT – SerDes is a demonstrator for:
The Serializer/De-serializer critical circuits:
Phase-Locked LoopsFrequency dividresLine driver/receiverConstant-latency barrel shifterPhase shifterThe circuit operates at 4.8 Gb/sThe chip was packaged in a custom flip-chip BGA packageEngineers:Ozgur Cobanoglu - CERN, SwitzerlandFederico Faccio - CERN, SwitzerlandRui Francisco – CERN, SwitzerlandPing Gui – SMU, USAAlessandro Marchioro - CERN, SwitzerlandPaulo Moreira - CERN, SwitzerlandChristian Paillard - CERN, SwitzerlandKen Wyllie - CERN, SwitzerlandStatus:
Chip is currently under testing
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
13Slide14
The GBT – SerDes Architecture
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
14
Serial
input
DES
Clock
Generator
Clock
reference
SER
Serial
out
Switch
Switch
120
FEC
Decoder
FEC
Encoder
De-scrambler
Header decoder
Scrambler
Header encoder
Parallel
Out/
BERT
Parallel
In/
PRBS
Control
Logic
Phase
Shifter
Switch
Switch
120
Switch
Switch
120
120
120
120
120
txDataValid
dIn [29:0]
Full custom
txClock40
txClock160
rxDataValid
dOut [29:0]
rxClock40
rxClock160
PROMPT
I2C
JTAG
AUX[n:0]
RX: 40 MHz & 160 MHz
TX: 40 MHz & 160 MHz
Data path
Clocks
Control bus
RST
rxRdy
txRdy
ClkOut3
ClkOut2
ClkOut1
ClkOut0
120
120
120
120
120
120
120
Frame
Aligner
120
Power On
RESET
resetSlide15
Serializer
Serializer:
4.8 Gb/s
120-bit shift register3 × 40-bit shift register (f=1.6 GHz)3-to-1 fast multiplexer (f=4.8 GHz)Data path:No SEU protectionSEUs handled by the Reed-Solomon CODECClock divider:Divide by 120f = 4.8 GHzTriple voted for SEU robustnessPLL:SEU hardened VCOEngineers:Ozgur Cobanoglu - CERN, SwitzerlandFederico Faccio - CERN, SwitzerlandPaulo Moreira - CERN, Switzerland
Status:Fully functional
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
15Slide16
Serializer Measurements 4.8 Gb/s (1/3)
Tx
Jitter:
Total jitter (1e-12): 53 psRandom jitter: 2.4 ps (rms)Deterministic jitter: 19 psData dependent: 4.8 psPeriodic:RMS: 4.6 psPP: 19.6 psDuty-cycle-distortion: 0.6 psInter-symbol interference: 4.8 pshttp://cern.ch/proj-gbtPaulo.Moreira@cern.ch16
More than 3 orders of magnitude
improvement due to the FEC
SNR is high, the system operates error free
SNR
relatively low
SNR very low
(noise is too high)
FEC can’t improve BERSlide17
Serializer Measurements 4.8 Gb/s (2/3)
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
17Slide18
Serializer Measurements: 6 Gb/s (3/3)
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
18Slide19
Serializer: Test Board Grounding Scheme
Electrical eye:
4.8 Gb/s
Separate digital and analogue groundsStrong low frequency jitter componentsMost jitter components below 3 MHzEasily tractable by the receiving PLLElectrical eye:4.8 Gb/sCommon digital and analogue groundsLow frequency jitter components virtually not present!http://cern.ch/proj-gbtPaulo.Moreira@cern.ch
19Slide20
De-serializer
De-Serializer:
Dual PLL CDR Loop:
1st Loop: Frequency centering PLL2nd Loop: CDRAllows to reduce the CDR VCO gain for lower JitterHalf-Rate:Phase-detectorFrequency-detectorConstant latency frame alignment circuitAs for the serializer:Unprotected data pathTMR clock dividerSEU hardened VCOEngineers:Ozgur Cobanoglu - CERN, SwitzerlandFederico Faccio - CERN, Switzerland
Rui Francisco – CERN, SwitzerlandPaulo Moreira - CERN, SwitzerlandStatus:
The receiver is fully functional Clock recovery operates up to 6 Gb/sHowever it only operates error free up to
3.0 Gb/sThis seems to be caused by the (so far unexplained) bad quality of the eye-
diagram at the input of the receiver (see later in this presentation)
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
20Slide21
CDR: Measurements
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
2140 MHz recovered clock clock
PRBS @ 4.8 Gb/s:
Total jitter (1e
-12
): 63 ps
Random jitter: 4.9 ps (rms)
Deterministic jitter: 24
ps
(pp)
Periodic:
RMS:
2
ps
PP: 5 psSlide22
De-serializer: Input eye-diagram
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
22
Signal generator eye-diagram (straight to the scope)
Empty board with a connector and a 100
W
termination (differential active probe)
Populated boardSlide23
PCB – Package Modelling (2½ D)
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
23
Package (one interconnect layer)
PCB: SFP to GBT transmission lines
Package S-Parameters
Package + PCB S-Parameters
PreliminarySlide24
Phase – Shifter
Phase-Shifter:
Main features:
8 – channels (3 in the GBT-SERDES prototype)1 PLL + Counter generates the three frequencies: 40 / 80 and 160 MHz1 DLL per channelMixed digital/analogue phase shifting technique:Coarse de-skewing – DigitalFine de-skewing – AnaloguePower consumption:PLL: 42 mW (measured)Channel: 16 mW/channel (measured)Differential non-linearity: <6.7% LSBIntegral non-linearity: INL<6.5% LSBEngineers :Ping Gui – SMU, USATim Fedorov – SMU, USAPaul
Hartin – SMU, USANataly Pico – SMU, USA
Bryan Yu – SMU, USAStatus:Fully functional
Fully meets the specsOne channel with timing problems but problem clearly identified with trivial solution
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
24Slide25
Phase – Shifter: Measurements
Resolution:
Dt = 48.83 ps Differential Non-Linearity: s = 4.7 ps (9.6% of Dt) pp = 21.5 ps (44% of Dt) Period Jitter:
s = 4.8 ps (pp = 29 ps) Integral Non-Linearity
: s = 4.3 ps (8.7% of
Dt) pp = 21.9 ps (48.7% of
D
t
)
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
25Slide26
Phase – Shifter: Measurements
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
26Slide27
Digital Functions
Digital Functions:
Parallel I/O interface MUX
Scrambler De-ScramblerEncoder decoderFrame aligner logicFrequency calibration logicI2C interfaceEngineers:Alessandro Marchioro - CERN, SwitzerlandPaulo Moreira - CERN, SwitzerlandChristian Paillard - CERN, SwitzerlandKen Wyllie - CERN, SwitzerlandStatus:Fully functionalhttp://cern.ch/proj-gbtPaulo.Moreira@cern.ch
27Slide28
GBT – SerDes Power Consumption
Circuit
Power [mW]
CDR456Serializer3303 ch Phase-Shifter (+ 2 diff. drivers = 10 mW)94 (≈ 16 mW/Ch + PLL: 42 mW)I/O
75Digital Core
27Total
980
http://cern.ch/proj-gbt
Paulo.Moreira@cern.ch
28Slide29
Project Schedule
Tasks remaining
:
GBT – SerDes:Understanding the receiver behaviour:3 Gb/s error free operation instead of4.8 Gb/sSEU testsGBTX:Receiver rework (if needed)Power down functions (SER/CDR)TX 8B/10B modeClock ManagerVXCO based PLL8 channel Phase-Shifter (only 3 on GBT - SerDes)E – LinksBi-directional C4 padSerializersPhase-AlignersControl Logic:
Watchdog and start-up state machinesIC channel logicI2C master
Configuration logic:Fuse bankChip assembly and verification
From industry:BGA package (flip-chip)80 MHz crystalTesting:
Test setup (should we use the IC tester?)
Early behavioral model needed for test development
Software
Firmware
GBLD:
Change the input protection diodes, change I/O to 1.5VGBTIAChange pad ring, add average power detector and add squelch circuit
Migration from the LM to the DM technologies flavor2.5V Supply
Project Schedule 2011
1st Q:SEU tests on GBT – SerDesGBLD submissionGBTIA submission3/4th Q: GBTX submissionhttp://cern.ch/proj-gbtPaulo.Moreira@cern.ch
29Slide30
The Gigabit Link interface Board (GLIB)
GLIB concept:
E
valuation platformEasy entry point for users of high speed optical linksIntended use:Optical link evaluation in the laboratoryControl, triggering and data acquisition of remote modules in beam or irradiation testsEach GLIB card:Can process data to/from four SFP+ transceiver modules
Each operating at bi-directional data rates of up to 6.5 Gbps
.Matches comfortably the specifications of the GBT/Versatile
Link:Target data rate of 4.8 Gbps.Basic configuration:
O
ne
GLIB
board interfaces
with up to four GBT channelsPhysical implementation:
Double width Advanced Mezzanine Card (AMC)Based
on the XC6VLX130T FPGA of the Virtex-6 familyLong lifetime:
Distribution
and support of a small set of variants over several yearsEngineering contacts:Sophie BaronFrancois VaseyPaschalis Vichoudishttp://cern.ch/proj-gbtPaulo.Moreira@cern.ch
30Slide31
GLIB Deliverables
The GLIB team envisages to deliver and
support:
SoftwareFirmwareHardware 3 Basic setups:Bench-top beam test setupBench-top front-end module test setupCrate system test setupThe required FMCs (TTC & E-Link) will also be delivered and supported.http://cern.ch/proj-gbtPaulo.Moreira@cern.ch
31
Bench-top front-end module test setup
Bench-top beam test setup
Crate system test setup
Status:
Specifications
V
1.9
available
.
Design
Schematics
:
Ready.
Layout: Ready. Verification
on-going
Fabrication:
Prototype Feb 2011
Testing:Commercial solutions will be used
Software/firmware:
Development will start in 2011