Internet Traffic Reengineering: Outages, Pandemics & More

The Challenge of Introducing New Infrastructure Solutions

Successfully driving sales in the realm of enterprise infrastructure fundamentally revolves around addressing established client needs. However, a significant hurdle arises when potential clients are unaware of the benefits a startup’s offering could provide.

NS1: A Deep Dive into Customer Acquisition

This article, the fourth and concluding segment of our EC-1 series, focuses on NS1’s customer base – the crucial element for financial viability, expansion, and overall success. Without a robust clientele, sustained growth remains unattainable.

As previously discussed, NS1 faces the challenge of a generally satisfied customer base with their current DNS services. The market is saturated with options from established companies and public cloud providers. Consequently, many clients lack a compelling reason to fundamentally alter their existing network infrastructure.

Capitalizing on Opportunity and Building Awareness

NS1 benefited from opportune events, notably a significant disruption experienced by DynDNS in October 2016. This outage immediately drew the attention of network architects, highlighting the critical need for robust and reliable DNS infrastructure.

Furthermore, NS1 has proactively worked to demonstrate the value of a more adaptable DNS foundation for optimizing performance. In-depth interviews with Roblox revealed how the platform strategically engineered its systems to accommodate rapid expansion during the COVID-19 pandemic.

Expanding Product Lines and Future Outlook

We will also examine NS1’s ongoing product development and its future trajectory. Currently, NS1 provides services to over 760 clients, including high-traffic applications like LinkedIn, Dropbox, JetBlue, Fox, and The Guardian.

The company has solidified its position as a leading infrastructure provider, suggesting a promising and potentially exceptional future.

Key Takeaways

Customer awareness is a primary challenge for innovative infrastructure companies.
Major outages can create opportunities for alternative providers.
Education on the benefits of advanced DNS solutions is crucial.
NS1’s growth demonstrates the demand for reliable and scalable infrastructure.

A Single Outage with Far-Reaching Consequences

Numerous factors can motivate an enterprise or organization to select NS1 for its services.

Access to DNS is universally available to every business and internet user globally. However, NS1 doesn’t simply offer DNS services; the company consistently emphasizes its provision of network resilience and enhanced performance. While DNS often functions adequately, “good enough” is insufficient when milliseconds matter and users anticipate consistent, optimal connectivity.

Roblox's Transition

In 2017, Roblox utilized DynDNS for its managed DNS requirements. Upon joining the gaming company, Principal Traffic Engineer Adam Mills was encouraged by Dan Williams, VP of Corporate and Production Engineering at Roblox, to switch to NS1, based on Williams’ positive experience with it at Dropbox.

Although the October 2016 DynDNS outage influenced the decision, it wasn’t the primary driver. The core concern, according to Mills, centered on the technology itself.

outages, pandemics and the reengineering of traffic on the internet

Mills described DynDNS as somewhat outdated, lacking a robust API framework and dynamic configuration capabilities.

He viewed DynDNS in 2017 as a fundamental DNS service, merely connecting traffic to locations without additional features. Roblox aimed to leverage DNS as a crucial element of its network operations, directing users to the most suitable servers based on their geographical location, with geotargeting being a key objective. This would enhance game performance and, consequently, the user experience.

Roblox already maintained data centers in regions with significant user activity, enabling connections as close as possible to players. What was required was a more sophisticated layer to determine optimal user-to-platform connectivity. The company had developed its own infrastructure for the game platform, but constructing a dedicated DNS service wasn’t a priority.

“As a video game company, we weren’t inclined to develop DNS in-house,” Mills stated. “At the time, the traffic engineering team consisted of only two individuals, making a custom DNS implementation impractical.” NS1 delivered the necessary APIs and performance, allowing the team to optimize the platform’s performance.

Pinterest's Response

The DynDNS outage also significantly impacted NS1’s growth, serving as a catalyst for change at Pinterest. Scott Beardsley, an engineering manager at Pinterest, explained that the outage highlighted the necessity of a secondary authoritative DNS provider to guarantee the company’s uptime.

Pinterest initially engaged NS1 for “dumb DNS” – basic managed DNS for traffic routing. However, over the past five years, Pinterest has expanded its usage to incorporate NS1’s Pulsar product, enabling algorithmic and real-time traffic demand management.

“Pinterest hosts a vast amount of content, and content delivery can be costly,” Beardsley noted. “Therefore, we needed a method to optimize costs by routing traffic to multiple CDN vendors.” With Pulsar, Pinterest benefits from user feedback regarding their experience, allowing for optimization of traffic delivery across CDN providers for availability, performance, and cost-effectiveness.

Pulsar provides a feedback loop, enabling Pinterest to continuously refine its traffic management strategies.

How Roblox Leveraged Traffic Engineering to Manage Growth During the 2020 Surge

The onset of the COVID-19 pandemic in early 2020 led to a significant increase in gaming activity globally, as children and adults alike found themselves with more leisure time. For Roblox, this presented a challenge: demonstrating the platform’s ability to withstand an unanticipated surge in user traffic.

According to Mills, the company anticipated a substantial influx of new players due to increased time spent at home. Existing capacity forecasts hadn’t accounted for this level of demand.

Roblox successfully navigated this traffic increase through a combination of DNS traffic steering, provided by NS1, and a robust infrastructure developed internally over several years.

The evolution of the game application and platform began as early as 2017, when Roblox initiated the construction of its own infrastructure platform, a project spearheaded by Williams. Mills joined around the same time and focused on leading the traffic engineering initiatives.

Traffic engineering is a crucial function for any online gaming enterprise. Numerous factors contribute to game speed, including server processing power, connection speeds, and graphics capabilities. However, the fundamental requirement is establishing a connection.

“A direct correlation exists between website loading speed and player engagement time,” Mills stated. “Faster website loading translates to extended user engagement.”

Maintaining the attention of a nine-year-old player is a primary driver for Mills and his team. Roblox’s user base predominantly consists of younger players, and Mills himself acknowledged his own children’s enjoyment of the platform.

“A nine-year-old, like any user, will quickly abandon a slow-loading experience in favor of an alternative game,” he explained. “Reliability is paramount, even for younger audiences.”

NS1’s Pulsar Active Traffic Steering technology plays a vital role in Roblox’s traffic optimization, as previously detailed.

Roblox actively collects latency metrics directly from players’ browsers during game asset retrieval and platform connection. This data, combined with broader internet connection speed information from NS1, provides a precise assessment of performance for each user. Pulsar then utilizes this comprehensive data to route users to the optimal server based on their geographic location.

The exploration of NS1’s capabilities, particularly Pulsar, involved considerable experimentation. Mills admitted a preference for hands-on learning over extensive documentation. He also commended NS1’s responsiveness to feedback and willingness to implement improvements based on Roblox’s needs.

“When we identified areas for improvement, NS1 was proactive in addressing them,” Mills said. “They would often release updated code within weeks to accommodate our requirements.”

Beyond understanding functionality, Mills also identified areas that required refinement. In 2017, Roblox initially utilized NS1’s Managed DNS service. After three years, the team determined the need for a secondary DNS provider to enhance resilience. “Our decision wasn’t a reflection of dissatisfaction with NS1, but rather a recognition that outages can occur in any region.”

This foresight proved valuable. In 2019, Roblox experienced an outage in São Paulo, Brazil, with NS1. While NS1 responded to the issue, the resolution process took time.

DNS redundancy is essential, and NS1’s Dedicated DNS service provides this. It utilizes a separate network and a distinct deployment cycle from the Managed DNS service. Mills emphasized that this architecture minimizes the likelihood of simultaneous failures in both NS1 services.

Roblox chose not to pursue a different DNS provider for redundancy due to the lack of comparable traffic steering capabilities.

“While companies like Google possess extensive infrastructure, a startup like ours didn’t have that level of resources initially, and still doesn’t,” Mills explained. “NS1 effectively offsets the engineering effort required to accelerate connections and provide robust DNS services.”

Finally, Mills highlighted NS1’s compatibility with Roblox’s existing technology stack. DNS and traffic steering are just one component of a broader suite of tools that contribute to the platform’s functionality. Roblox also leverages open-source Ansible for configuration management and Hashicorp tools, including Nomad for application workload scheduling and Consul for service networking. Ansible automates deployments, while the Hashicorp technologies manage applications within Roblox’s infrastructure.

Charting NS1's Course for the Future

NS1 began with the premise that the capabilities of DNS were significantly underutilized. Since its founding, the company’s technology has undergone continuous development. It now leverages DNS to empower organizations – including prominent names like Pinterest and Roblox – to enhance application performance.

Currently, NS1’s founder, Kris Beevers, indicates no immediate plans to seek additional funding. The possibility of either an Initial Public Offering (IPO) or an acquisition remains uncertain at this time.

Raj Dutt, NS1’s earliest investor, previously CEO of Voxel and presently CEO of Grafana, expresses confidence in the startup’s trajectory, irrespective of a potential public offering. He stated, “My primary aspiration is for them to consistently make choices that guarantee their long-term viability and dominance within their sector.” Dutt further clarified, “I’m not focused on a quick return on my investment; I encourage Kris to prioritize sustained growth.”

Tyler Jewell of Dell Capital also hasn’t set a specific IPO timeline for NS1. However, he conveyed to TechCrunch his anticipation of an “unconventional outcome” for the company. This suggests a scenario where investors realize substantial returns through either an acquisition or an IPO.

Independent of NS1’s eventual public or private status, opportunities for product diversification and capitalizing on prevailing market trends persist.

Last month, the company unveiled NS1 Connect, a new platform designed to integrate its services more cohesively. This platform will allow users to manage multiple NS1 services through a unified interface, creating opportunities for cross-selling.

Jewell anticipates growing demand driven by the increasing prevalence of 4K video collaboration, alongside the expansion of augmented reality (AR) and virtual reality (VR) technologies. All internet traffic, throughout its history, relies on DNS for connectivity, and NS1’s stakeholders are optimistic about the continued growth of the Internet of Things (IoT) as devices proliferate at the network edge.

The company’s future direction isn’t about radical change, but rather about continuous improvement, experimentation, and adaptation – a strategy it has employed since its inception.

Beevers emphasizes the importance of a data-driven approach, stating, “I firmly believe in trying something, gathering data, establishing a feedback loop, and then refining your approach.”

Over the past nine years, since initially conceptualizing NS1 on a napkin in a New York City bar, Beevers has witnessed the evolution of his company from a nascent idea to a fundamental component of the internet infrastructure.

“Looking back, there are countless decisions I would make differently, armed with the knowledge we’ve gained over the last seven or eight years. However, we lacked that data at the time,” Beevers acknowledged. “I am proud of our continued commitment to experimentation and innovation, mirroring the spirit with which I wrote the first 22 lines of code on my couch.”