NTT Builds a Flexible and Simple Network with Cumulus Linux

Overview

NTT Software Innovation Center (SIC) is undertaking promotion of open innovation centering on the development of open source platforms, collaborative service development with NTT operating companies, and other R&D. NTT SIC is playing an active part in various open source projects including OpenStack, Hadoop, PostgreSQL and Linux kernel.

As the technology thought leader, NTT SIC required a flexible network that allowed rapid innovation and was simple to operate while controlling their TCO.

Challenges

The team at NTT SIC was experiencing growing layer 2 complexities like network loop, rogue DHCP servers, mis-configuration of IP addresses, and wanted to adopt a network architecture that addressed these challenges. The ability to build and rapidly deploy custom applications was of paramount importance. The solution had to be able to easily integrate with their existing software stack and leverage custom applications like GoBGP

As multiple operational teams depend on NTT SIC, the R&D team was working on tight SLAs and could not spend cycles provisioning and managing their network. They needed a reliable yet simple network infrastructure that allowed collaboration and rapid innovation.

Solution

To reduce layer 2 complexities, NTT SIC decided on a Clos layer 3 fabric. After conducting thorough research and an internal evaluation, the team decided on Open Networking with Cumulus Linux. With Ubuntu on the compute nodes and Cumulus Linux on the leaf and spine switches, NTT SIC was able to leverage their existing Ansible knowledge and automation experience to automate their network similar to their compute environment. For users who wanted to continue using the layer 2 network, the team used EVPN/VXLAN.

The main criteria for selecting Cumulus Linux included:

• Network Flexibility: Cumulus Linux provided flexibility to leverage existing custom applications like GoBGP to run directly on the switch OS. The team made no modifications to the application to run their Cumulus Linux ToR switches.
  NTT SIC was also able to leverage their existing tool to monitor all BGP neighbors, share switch matrices with InfluxDB, and get a visual report from Grafana, none of which was previously possible with their incumbent switch vendor.
• Network Simplification: Leveraging ONIE to install Cumulus Linux allowed NTT SIC to institute zero touch provisioning (ZTP), which simplified network operations and enabled rapid service delivery. Without a provisioning server, the team utilized a custom REST API to invoke Ansible for provisioning. Because it was all Linux, NTT SIC was able to develop and run a boot script for the switches to automatically trigger Ansible provisioning.

The network flexibility and simplification gained by using layer 3 and Cumulus Linux enabled NTT SIC to bring best practices and custom applications and tools from the compute side to their network. NTT SIC reduced their CapEx by half and lowered OpEx by:

• Eliminating the need to build new applications and tools for their network.
• Cutting troubleshooting time by 50% with the same custom monitoring tools across the compute and network.
• Reducing deployment time of additional servers and hosts which used to be managed manually by administrator from several days to a few minutes.
• Leveraging ZTP to expedite time to production.

Recap

NTT SIC worked with Cumulus Networks to build a flexible, simplified and cost-effective network for their data center and offices. Cumulus Linux enabled the team to develop and deploy custom applications across compute and network, helping expedite time to production. The resulting system allows NTT SIC to focus on rapid innovation and the support needs of their operational teams.