Constructing a sub-nanosecond time synchronization network using White Rabbit

Abstract

Highly accurate time sources, such as Cesium-based or GPS clocks, are the best candidates to provide a correct notion of time, but are too costly and impractical for implementation in every modern digital device. Ubiquitously, (crystal) oscillators are employed instead. To maintain a shared notion of time, the offsets and drift/jitter have to be compensated via the concept of synchronization, divided into time offset equalization and syntonization. Packet-based distribution protocols such as Network Time Protocol (NTP) and Precision Time Protocol (PTP) can provide time offset equalization, whilst synchronous Ethernet can deliver syntonization. To reach high synchronization accuracy, PTP and synchronous Ethernet can be combined into an improved protocol, referred to as White Rabbit. It enhances the one-way delay calculation through fine and coarse delay measurements. These improve PTP timestamps to reach sub-nanosecond synchronization accuracy.

The White Rabbit project has limitations though: it relies on extensive calibration parameter sets that are only available for limited devices and do not account for hardware variability. This is in contrast with the initial project objectives. Secondly, the update cycle of the hardware implementation is slow. Namely, the current White Rabbit switch is based on FPGA platforms from ten years ago. Thirdly, the switch contains costly components and requires full replacement of present network switches upon deployment.

This work opted to define a new switch design, which distributes the synchronization tasks over several smaller FPGAs to provide a low-cost implementation and improve applicability, whilst requiring minimal factory calibration and maintaining sub-nanosecond accuracy between devices.

The new switch concept is based on the smaller and widely applied White Rabbit node design. The proof of concept is based on two interconnected FPGAs forming a 1-port White Rabbit switch. Fine delay measurements, asymmetry estimations and fine delay compensations are applied to reach synchronization between the FPGAs. The Digital Dual Mixer Time Difference (DDMTD) is preferred for the fine delay measurements due to its superior resolution, area and invariance to large clock periods. A three-line asymmetry calculation is introduced and motivated to provide an improved one-way delay estimation. Lastly, the IDELAY element was selected to provide the fine delay compensation due to its process-voltage-temperature (PVT) invariance, negligible jitter addition and area demands.

Whilst both the fine delay measurement and fine delay compensation were thoroughly discussed by means of specifications and simulations, the three-line asymmetry calculation had to rely on a theoretical discussion. As such, measurements have shown that the transmission delay over an added bi-directional link is approximately equal when the clock signal moves from slave to master and vice versa. A large delay variability between other transmission lines is subsequently exposed. The variability leads to inaccurate one-way estimations with a traditional two-line approach. A three-line asymmetry calculation will be conclusively beneficial for the synchronization accuracy. The low-cost requirement of the distributed switch can be maintained through elimination of components in the slave and master node. The reduced switch lowers the hardware costs with approximately 25% by trading off cost-effectiveness with jitter filtering and consecutive performance.