How Outbrain gained unprecedented scalability with Cumulus Linux
Outbrain is the world’s leading premium discovery platform, reaching a highly engaged audience through distribution on the world’s biggest and best publisher sites, including CNN, ESPN, Time Inc., Le Monde, Fox News, The Guardian, SPH, The Telegraph, New York Post, India.com, Sky News, TF1, Condé Nast, Orange, Spiegel, L'Equipe, and more. Outbrain helps its customers understand their audiences through data resulting in increased website traffic and sales revenue.
Founded in 2006, the company is headquartered in New York with a presence in a growing number of locations globally, including the U.S., UK, Israel, France, Germany, Singapore, Japan and Australia.
Outbrain has grown into a business that serves more than 250 billion monthly content recommendations, reaching in the region of a billion users every month across the globe. With an upcoming network refresh, Outbrain began evaluating solutions that would help it optimize its infrastructure, so that it could continue to serve the most relevant content to readers at scale.
With complex, large-scale requirements and continued rapid growth at the forefront of its considerations, Outbrain wanted to adopt a next generation architecture that would provide standardization, scalability and predictability of the network behavior. They also wanted to enable automation and the ability to leverage the Network and Linux talent that existed in-house.
It was critical for the Outbrain team to select a solution that allowed zero-to-full deployment without any human touch. It also wanted the ability to use its existing agent-based monitoring (Prometheus) and automation tool (Chef) across the stack.
To reduce Layer 2 complexity, Outbrain decided on Layer 3 Clos architecture. The team considered solutions from multiple vendors and settled on Cumulus Linux due to the flexibility and choice it delivered. A modern data center architecture with Cumulus Linux helped Outbrain to reduce the infrastructure complexity, standardize on the OS and gain operational efficiency. With a Linux OS across the stack and ability to leverage its existing talent, Outbrain was able to reduce its OpEx and streamline operations.
A combination of Cumulus Routing on the Host and Redistribute Neighbor completely eliminated Layer 2 within Outbrain’s environment, providing full equal cost load sharing on all links and a highly resilient architecture.
The main criteria for selecting Cumulus Linux and Cumulus Routing on the Host included:
- Operational Efficiency: Linux across the stack helped to unify the infrastructure. With single OS, Outbrain was able to:
- Automate its compute and network with Chef
- Standardize on Prometheus for monitoring
- Leverage the existing Linux talent and knowledge to architect and operate the complete environment as one.
- Scalability and Reliability: Outbrain adopted a Layer 3 Clos (leaf/spine) architecture that is highly manageable and allows rapid growth. A Clos design is simple to troubleshoot, automate, scale and evolve, and it provides a fully non-blocking architecture. Because of only a single hop between any two leaf nodes, it allows for a predictable latency through the network while making it easy to add additional capacity. Further, there are multiple redundant paths between any two leaf switches — redundancy and scale is inherent in the design.
- Manageability: Leveraging BGP unnumbered helped Outbrain to reduce the number of IP addresses that are required to be configured and managed. BGP unnumbered also allows flexibility as IP addresses do not need to be configured on the physical ports — connections and cables can be moved around at will. Outbrain’s automation tool, Chef, is used to configure both applications and BGP on the host. Additionally, the ability to leverage Linux skills and server side tool sets with automation and monitoring made the infrastructure more programmatic and manageable.
In the end, Outbrain built an infrastructure free of the complexities usually faced in Layer 2 networking. The result is a fully streamlined, uniform network and compute using the same networking (BGP), monitoring (Prometheus) and automation (Chef) technology everywhere.
Outbrain leveraged Cumulus Networks to deploy an all-Linux, modern network architecture, capitalizing on its existing in-house Network and Linux expertise and tools. With the goal to adopt web-scale principles to achieve operational efficiency, Outbrain built a highly scalable, reliable and manageable network, all while eliminating Layer 2 complexities.