Facebook Engineers Release Open Source Time Appliance

The Evolution of Time Synchronization in Modern Computing

It’s likely that most individuals are unaware of the extent to which our daily-used devices rely on precise timekeeping. This applies to everything from smartphones and laptops to large-scale network servers. Traditionally, maintaining accurate time has been a specialized task, largely handled by a limited number of hardware vendors.

Addressing Limitations with an Open-Source Solution

Recognizing the need for improvement, a pair of engineers at Facebook embarked on developing a more accurate timekeeping device. This device takes the form of a PCI Express (PCIe) card and has been contributed to the Open Compute Project as an open-source initiative.

Oleg Obleukhov, a production engineer at Facebook, explains the fundamental principle: regularly querying a time-keeping server to ensure consistent time reporting across all devices. “Nearly all electronic devices today utilize NTP – Network Time Synchronization Protocol – found in phones, watches, and laptops. These devices connect to NTP servers to obtain the current time,” he clarified.

Traditional Time Synchronization Methods

Prior to Facebook’s innovation, time verification typically involved two primary methods. Developers often used services like Facebook.com for time checks. However, organizations like Facebook, operating at immense scale, required a solution that functioned even without internet connectivity.

Data centers traditionally employ a hardware device known as Stratum One. This dedicated unit serves solely as a timekeeper. These boxes, developed over years by a select group of companies, proved reliable, but acquiring new features presented challenges.

The Drive for Control and Innovation

Furthermore, companies such as Facebook lacked control over these proprietary devices. Obleukhov, alongside research scientist Ahmad Byagowi, initiated a project to create these devices using readily available components, designed to fit into any PC with an available PCIe slot.

The initial design was sketched on an iPad, evolving into a functional prototype. A crucial time appliance relies on two key elements: a GNSS receiver and a high-stability oscillator.

Key Components and Their Functionality

Obleukhov and Byagowi detailed the roles of these components in a blog post: “The process begins with a GNSS receiver providing the time of day (ToD) and a 1 pulse per second (PPS) signal. Coupled with a high-stability oscillator – such as an atomic clock or an oven-controlled crystal oscillator – this system can achieve nanosecond-level accuracy. The time data is then transmitted across the network via a standard network card.”

While seemingly straightforward, the implementation is complex, which explains why it hadn't been previously addressed. However, these Facebook engineers were dissatisfied with the limitations of existing methods and resolved to create a superior alternative.

Frustration Fuels Innovation

“Much of this stemmed from frustration with the available market offerings. We required specific features, including security measures and monitoring capabilities. Requests to vendors often took six to twelve months to fulfill, and the resulting solutions rarely met our exact needs, necessitating constant adjustments,” Obleukhov stated.

The miniaturization of atomic clocks and oscillators made it feasible to integrate a timekeeping device onto a PCIe card. Combining this technological advancement with their dissatisfaction with current solutions, the engineers realized they could independently tackle the challenge.

Flexibility and Community Collaboration

As the design progressed, the engineers prioritized flexibility, allowing engineers to customize the design with components suited to their specific requirements. Some applications might demand high-end, expensive parts, while others could utilize more affordable alternatives.

From the outset, they committed to open-sourcing the design and engaging the Open Compute Project to foster community contributions. “Open sourcing was intentional from the beginning. We needed community support to prevent the project from becoming solely reliant on internal resources and potentially fading away if interest waned,” Obleukhov explained.

Current Status and Future Development

Currently, a dozen vendors are involved in the project, with several cards available, including the one designed by the original engineers, and a commercial offering from Orolia. The ongoing goal is to refine the design, and the open-source approach ensures continuous improvement through collaborative efforts.