Exploring Future Measurement Methodologies for PCIe 6.0
David Bouse
Principal Technology Lead
1 2 3 4
PCI Express 6.0 Specification Updates
64 GT/s Base Tx
Measurements PAM4 Rx
Calibration &
JTOL
Tektronix Gen6 Solution Update
Agenda
PCI-SIG 6.0 Specification Updates
PCI Express Evolution Bit Rate / Lane Link BW Lane BW x16 bi BW PCIe 1.x 2.5 GT/s 2.0 Gb/s 250 MB/s 8 GB/s PCIe 2.x 5.0 GT/s 4.0 Gb/s 500 MB/s 16 GB/s PCIe 3.x 8.0 GT/s 8.0 Gb/s ~1 GB/s 32 GB/s PCIe 4.x 16.0 GT/s 16.0 Gb/s ~2 GB/s 64 GB/s PCIe 5.x 32.0 GT/s 32.0 Gb/s ~4 GB/s 128 GB/s PCIe 6.x 64.0 GT/s 64 Gb/s ~8 GB/s 256 GB/s
• PCIe 6.0 debuts 2021 during 5G critical mass
• Server nodes
• High BW network infrastructure
• HPC & AI applications
• PCIe 5.0 Released 2019
• 400G Ethernet
• Host Bus adaptors (50 GB/s min)
• Cloud AI & modeling (co-processors)
• NAND based storage
PCIe 6.0 Specification Snapshot (10/14/2021)
• PCIe 6.0 Base Specification – Rev 0.9 workgroup approved
• Describes chip-level behavior on all levels of the stack
• PCIe 6.0 CEM Specification – Rev 0.5 under development
• Card electro-mechanical (CEM) defines system and Add-in Card level
• PCIe 6.0 PHY Test Specification – Rev 0.5 under development
• Describes electrical compliance tests for Tx, Rx LEQ, & PLL Bandwidth
PCIe 4.0 to 6.0 Base Specification Updates
PCIe 4.0 PCIe 5.0 PCIe 6.0 (TBD)
Data Rate 16 GT/s 32 GT/s 64 GT/s (PAM4)
Add-in Card Loss 8dB @ 8Ghz 9.5dB @ 16GHz 8.5dB @ 16GHz
Rx Test (Channel Loss) - (27 to 30) dB @ 8GHz - (34 to 37) dB @ 16GHz - (30 to 33) dB @ 16GHz Reference CTLE 2 Poles; 1 Zero; DC Gain
Range (-6 to -12) dB 4 Poles; 2 Zero; DC Gain
Range (-5 to -15) dB 6 Poles; 3 Zero; DC Gain Range (-5 to -15) dB
Reference DFE 2-Taps 3-Taps 16-Taps
Eye Width (Rx Test) 18.75 ps 9.375 ps 3.125 ps (top eye)
Eye Height (Rx Test) 15 mV 15 mV 6 mV (top eye)
Lane Margining Required timing only Required timing/voltage Required timing/voltage Refclk Jitter Limits <= 500 fs <= 150 fs <=100 fs
Min Scope BW 25 GHz 50 GHz 50 GHz
64 GT/s Base Tx Measurements
Transmitter Equalization Expansion
Cursor Expansion: 2nd Precursor Added
o 2 pre-cursors o 1 post-cursor
Presets: Q0 to Q10
o Heavier pre-cursor weighting
o Numerous optimal presets for Rx Calibration
Measurement Method: AC Method
o Step response captured
• Equalized & Non-Equalized
o Cursors applied to Non-Equalized step o Minimized Mean Square Error (MSE)
Expected 6.0 Base Tx Measurements
TX Test Pattern Notes
SNDR Compliance Signal Noise Distortion Ratio
Voltage Differential Peak-to-Peak Compliance Measured with 64 level 3s & 64 level 0s
Transmit Equalization Compliance Q0-Q10 (PAM4) with AC step method
Tx Equalization Boost Compliance Q10 (full swing) & Q4 (reduced swing)
EIEOS Min Voltage Swing Compliance Include package loss impact
Ratio Level Mismatch Compliance PAM4 measurement only
Uncorrelated Tj 52UI Jitter Measurement Jitter computed on each unique transition Uncorrelated Dj 52UI Jitter Measurement Jitter computed on each unique transition
Uncorrelated Rj 52UI Jitter Measurement Informative
Pulse Width Jitter Tj High Swing Toggle 0303 level patten; noise comp included Pulse Width Jitter Dj_dd High Swing Toggle 0303 level patten; noise comp included
Pulse Width Jitter Rj High Swing Toggle Informative
PS21 Compliance Pseudo package loss
Explored in this webinar
Signal to Noise Distortion Ratio (SNDR)
pmax = peak of linear fit pulse
σe = standard deviation of error waveform σn = standard deviation of noise
Pmax ISI
Signal power
Noise Non-compensable
distortion
Sigma E (σe)
11
• Pulse response is used to reconstruct one repeat of the ideal linear-model waveform with superposition
◦ Linear fit pulse length approximates Rx EQ length
• Difference between averaged pattern waveform and ideal linear-model waveform becomes the error waveform
• RMS of error waveform becomes σe
• Quantifies distortion related to:
◦ Rx EQ design (length, type)
◦ Non-linearity
◦ Other non-compensable phenomenon
1000 2000 3000 4000 5000 6000 7000 8000
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
0.5 Averaged data patttern (blue) and linear fit error (red).
Time (UI)
Voltage
4950 5000 5050 5100 5150
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
0.5 Averaged data patttern (blue) and linear fit error (red).
Time (UI)
Voltage
Sigma N (σn)
0 1000 2000 3000 4000 5000 6000 7000 8000
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
0.5 Averaged waveform and samples for noise measurement.
Time (UI)
Voltage
5300 5310 5320 5330 5340 5350 5360 5370 5380 5390 -0.5
-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4
0.5 Averaged waveform and samples for noise measurement.
Time (UI)
Voltage
• Original waveform (real time) is taken
• NRZ – RMS noise is measured in low-slope portion of ‘long run’ of each level (0,1) across all pattern repeats in the
record
• PAM4 – same as above but measured for each of four levels
• σn becomes the mean of these either two or four RMS level measurements
PCIe 6.0 SNDR Requirements
• Pattern: Gen6 compliance pattern (SSC considered for SRIS)
• Tx Equalization: Q0 (C-2 = C-1 = C+1 = 0.0)
• RT Scope waveform capture
• BW of 50GHz (minimum)
• Record Length: min of 250 pattern repetitions
• Filtering: 4th order Bessel-Thomson (3dB roll-off from DC at 33GHz)
• Sigma E
• Pulse Length = Np = 600UI
• Pulse Delay = Dp = 4
• 32 points/UI (resampling)
• PRBS portion of compliance pattern only
• Clock Recovery Applied (DSP implementation not explicitly defined)
• Sigma N
• 61st UI of 64 UI run for each PAM4 voltage level considered
• Eight evenly spaced points within 61st UI averaged
64 GT/s SNDR Feasibility Study
Channel Loss SSC Scope BW Pattern Np Scope Noise Removal SNDR Pmax Sigma E Sigma N
1.5 dB (cable only) On 33 GHz BT Gen6 Compliance 600 3.8 40.7 369.6 3.31 0.84
" Off " " " " 40.0 367.0 3.47 1.18
7.43 dB (4" Replica) On " " " " 39.6 239.4 2.18 1.25
" Off " " " " 40.0 239.4 2.26 0.79
10.79 dB (8" Replica) On " " " " 39.4 195.1 1.88 0.91
" Off " " " " 39.8 195.1 1.95 0.41
14.33 dB (12" Replica) On " " " " 39.0 162.0 1.71 0.62
" Off " " " " 39.0 161.9 1.76 0.41
32 GBaud PAM4 Source 70GHz Scope
Variable BoardISI
PCIe 6.0 Uncorrelated Jitter Requirements
• Pattern: 52UI Jitter Measurement Pattern
• Four instances of all 12 PAM4 transitions & DC balancing bits
• Tx Equalization: Q0 (C-2 = C-1 = C+1 = 0.0)
• RT Scope waveform capture
• BW of 50GHz (minimum)
• Record Length: min of 2E6 UI (37.5 UI per unique edge)
• Filtering: 4th order Bessel-Thomson (3dB roll-off from DC at 33GHz)
• Jitter measured on all 48 edges separately and then averaged
• Select CTLE or No CTLE resulting in lowest Rj
• Scope noise removed separately for each transition type
64 GT/s Uncorrelated Jitter Study
32 GBaud PAM4 Source 70GHz Scope
Jrms
2.0E-13 2.5E-13 3.0E-13 3.5E-13 4.0E-13
01 02 03 10 12 13 20 21 23 30 31 32
Rj (seconds)
PAM4 Tranistion Type
Random Jitter Scope Noise Impact
Sigma Rj Sigma Rj (noise removed)
Sigma Rj Mean = 316 fs; Sigma Rj (noise removed) Mean = 285 fs
2M UI Waveform Record
(38,461 each edge)
2.7E-13 2.9E-13 3.1E-13 3.3E-13 3.5E-13 3.7E-13 3.9E 13
38462 192308 346154 538460 692304 846149 999994 1153839 1307685 1461530 1615374 1769219 1923064 2076909 2230754
Rj (seconds)
Edge Population
Random Jitter Population Impact
Rj (01) Rj (02) Rj (03) Rj (10) Rj (12) Rj (13) Rj (20) Rj (21) Rj (23) Rj (30) Rj (31) Rj (32)
~38K / each edge type per waveform Accumulate 58 captures for 22.3M population Highest Rj Edge of
Transition Type Shown
Scope Noise Removed
3.0E-12 3.1E-12 3.2E-12 3.3E-12 3.4E-12 3.5E-12 3.6E-12
38462 192308 346154 538460 692304 846149 999994 1153839 1307685 1461530 1615374 1769219 1923064 2076909 2230754
Tj (seconds)
Edge Population
Uncorrelated Total Jitter Population Impact
Tj (01) Tj (02) Tj (03) Tj (10) Tj (12) Tj (13) Tj (20) Tj (21) Tj (23) Tj (30) Tj (31) Tj (32)
ALL EDGES - Tj (38K edge) Mean = 2.85 ps; Tj (2M edge) Mean = 2.92 ps
~38K / each edge type per waveform Accumulate 58 captures for 22.3M population Highest Rj Edge of
Transition Type Shown
Scope Noise Removed
PAM4 Rx Calibration & JTOL
Receiver Equalization Expansion
CTLE: 64 GT/s
o 6 Poles & 3 Zeros
o DC Gain: -5dB to -15dB (1dB steps)
DFE: 64 GT/s
o 16 taps (-d1, -d2, … , -d16)
o DFE taps constrained to min burst errors o Empirical formula in spec
Receiver Challenges at 64 GT/s PAM4
• Repeatable stressed eye calibration for PAM4 signaling
• Validation & compliance across 6 data rates for two legacy NRZ & new PAM4 signaling
• Noise from measurement equipment must be accurately characterized and removed
• JTOL and BER Rx testing in the presence of SSC
• Updating post-processing tools to support 64 GT/s
• Back-channel equalization challenges with PAM4 signals with ~ 28dB insertion loss
Multi-level Gen6 Stressed Eye Diagram
MP1900A DPS70004SX
No External Couplers
PAM4 Stressed Eye Feasibility Study
Scope: Tektronix Real Time (DPO75004SX)
o 50 GHz with 200 GS/s
o Root Complex package embedded
Post Processing: Seasim 1.0.6
o CTLE/DFE
o Sj, Rj, & Crosstalk (DMI)
Source: PAM4 PPG (Anritsu MP1900A)
o 64 GT/s (32 GBaud PAM4) – Step Response o Tx EQ Pre Cursor 1 & 2 optimized
Channel: 33.1dB @ 16GHz
o Gen5 Base Rx Fixtures
Source: PAM4 PPG (Anritsu MP1900A)
o 64 GT/s (32 GBaud PAM4) – Step Response o Tx EQ Pre Cursor 1 & 2 optimized
Channel: 33.1dB @ 16GHz
o Gen5 Base Rx Fixtures
Solution Space
9 Closed Eyes with
Different Sj/DMI #1 #2 #3 #4 #5 #6 #7 #8 #9
Expected Jitter Tolerance Methodology
• Modified Compliance pattern during loopback
• Without SSC enabled on data signal
• Tx Equalization may be optimized for the Rx
• Independently swept 33 GHz tone
• Common Clock – swept to 1ns
• SRIS – swept to 15ns
• JTOL mask between 400 KHz and 100 MHz
• Additional 210 MHz tone holds Sj over 0.05UI
• FBER target of E-6
• Without accounting for burst errors
Tektronix Gen6 Solution Update
Tektronix PCI Express 6.0 Solution
Transmitter
Base CEM Ref CLK
TekExpress Automation
DPOJET Plugin for Debug
PAM4 Analysis App & DPOJET SDLA
PAM4 Analysis Application
Tektronix Innovative Gen6
DSP Tools
PCIe Gen5 CEM Fixtures Enable Early
Gen6 Signal Access (limited availability)
Thank You for Your Time Questions?
