way around. This ensures deterministic delay for every transmitted word, even in case of reinitialization. A clock skip mechanism enables the possibility to delay the next transition of the parallel clock by one bit clock. Therefore, the P_clk_delay signal is pulled high for one parallel clock period. The P_cdr_clk_stable wire signals as soon as the parallel clock has locked again. After P_cdr_clk_stable is high, the duty cycle of the P_cdr_clk has to be 50 %. Each lane of the link will deliver its own parallel word clock. The synchronization of the data to the system clock is done in the digital part.
5.5 Design Use and Tests
For all designs, simulations have been set up using the FIS verification environ- ment from 4.2.10 to proof the implementations against SEUs and validate the synchronization concepts. Further, all designs have been tested successfully in FPGA devices. Regarding the two FE ASICs, the STSXYTER is able to deliver the full speed of 2 Gbit/s, while the SPADIC ASIC link frequency had to be reduced. Thus only 700 Mbit/s are available in the current version, compared to the 1000 Mbit/s planned. To also gain experience from live application, a design implementation of the CBMnet link port and PHY has been intensively tested directly in a beam tests in the COSY accelerator at the Forschungszentrum Jülich, which is very convenient to carry out a test to measure single event effects. While scrubbing was required to mitigate the impact of SEUs on the configuration memory, the implementation of the CBMnet version 3.0 in a beam with 5 · 106 · protons · s−1 · cm−2 approved its reliability while running for several hours without hang-up. This test finally confirmed the successful application of SEU handling concepts for complex network logic in an FPGA.
Regarding the HUB ASIC, a first prototype has been designed, implemented, and submitted in the planned TSMC 65 nm LP process [83]. The manufactured chip is depicted in fig. 5.14. First laboratory tests also verified the implementation of front-end links in the 65 nm TSMC process. Unfortunately, the multi-gigabit SerDes in the first submission of the prototype could not be put into operation due to clock signal disturbance in the high frequency block. Although a close cooperation with the IITKGP during the bring-up phase tried to solve the problem, a final solution of the working group from India is still outstanding.
Chapter 5 CBM Design Implementations
Figure 5.14: The manufactured HUB ASIC chip mounted on a PCB. The full
custom physical layer block in the upper right corner can be clearly identified.
5.6 Conclusion
All CBMnet FPGA and ASIC designs have been frequently used in laboratory setups and beam times. The front-end ASICs with integrated CBMnet ease the synchronization of the detectors and the DAQ benefits from the high data bandwidth. Compared to implementations with the old CBMnet protocol, which did not support any SEU handling, the new generic cores improved the reliability of all designs and deliver all required features for the final experiment setup. Otherwise a meaningful operation of the DAQ system would not have been possible in the future. For every design, it has been shown how a initialization with synchronization and deterministic latency is assured in detail, like synchronous GTX links in the FLIB design using sophisticated initialization routines to achieve fix delay, synchronization of high-speed front-end links using Xilinx IOSelect capabilities, or special developed circuits, like in the HUB ASIC, and how DLMs are synchronized between clock domains with different speed. Finally, for the core logic of the designs, the easy integration of internal and external plug-ins has been described.
5.6 Conclusion
Since the back-end link SerDes in the HUB ASIC could not be taken into service successfully from the IITKGP, the decision was taken to build up competence for mixed-signal multi-gigabit SerDes design within the CAG group. The so called openMGT project is described in chapter 6.
Chapter 6
Multi-Gigabit Transmitter
While in the previous chapters only full-custom high-speed interconnects have been described, and multi-gigabit links only have been used in the context of FPGA designs and by use of external IP, this chapter gives a more detailed view on state-of-the-art multi-gigabit designs of serial interconnects, more precisely on a 20 Gbit/s SSTL transmitter. In the context of the Serializer/Deserializer development in a team of the Computer Architecture Group, the OpenMGT framework has been founded to ease the conception and verification of mixed-signal designs [68]. Regarding the transmitter, challenges are the very small time bases, signal integrity and impedance matching. A potent equalization implementation reduces data dependent jitter, which is a very important parameter to decrease the bit error rate. In the following the architectural decisions, design implementation and verification techniques are presented.
6.1 Serial I/O Challenge
A Serializer/Deserializer (SerDes) is a pair of two blocks, where the transmitter converts parallel data into a serial data stream and the receiver converts the serial stream back into parallel data. As serial interconnects usually use DDR transmission mode, in this context the definition Unit Interval (UI) means one bit time, which is equal to 1/datarate. In the previous chapters serial interfaces and interconnects have been used intensively for network communication, but always like digital interfaces. In case of FPGA designs, built-in hard macros or dedicated serial I/O capabilities have been used, where in fact a lot of sophisticated analog circuits do their work, but the real transmission magic happens in the
Chapter 6 Multi-Gigabit Transmitter Serializer Equalizer Clocking Equalizer Sampler Deserializer CDR Line Driver Parallel Data Parallel Data TX RX Differential Out Differential In Channel Termination and ESD Termination and ESD
Figure 6.1: Structural built-up of a SerDes with transmitter and receiver equal-
ization and Clock Data Recovery (CDR) circuit.
background. After successful configuration, the designer only has to deal with the digital parallel interface. For the ASIC devices, digital serializer and deserializer modules have been used, supplemented with a differential I/O buffer chain to gain the driver strength needed for off-chip signaling. Depending on the transmission channel and speed of the sent data, these simple set-ups are sufficient, but in case of multi-gigabit transmission many of challenges arise.
To overcome the speed limitation caused by skew between clock and data, the clock signal is embedded in the data stream, which requires for a Clock Data Recovery (CDR) circuit on receiver side and frequent data signal transitions in the stream to synchronize the CDR. Therefore a SerDes consists of the following functional blocks, which are depicted in fig. 6.1.
At higher frequencies the waveform is also more severely distorted by high-frequency losses, leading to attenuation and an increased Bit Error Rate (BER). This requires for equalization at transmitter, but also at receiver side to improve signal integrity and transmission reliability. Furthermore, one of the important parameters is the jitter performance of a SerDes, since the signal quality, and therefore BER, is directly affected by jitter. To reduce output jitter many design aspects must be considered, but special care must be given to reference clocks and power supply. With the smaller manufacturing sizes also the susceptibility to ElectroStatic