How Yahoo! JAPAN is preparing for its new machine learning application with Cumulus and Backpack

"

75% Reduction in CapEx. "Backpack passed all the tests we conducted without any issue"

-- Kenya Murakoshi, Yahoo! JAPAN Corp

Industry

Internet services

Business Objective

Scale network to meet streaming services demand and rollout of new applications

Partners

Celestica / Facebook

Yahoo! JAPAN is preparing for the deployment of Cumulus Linux in its Backpack chassis for their new application with machine learning infrastructure. These videos and script are from Yahoo! Japan's story on who they are, their basic network designs, how they evaluated Backpack, and the advantages and results they found from the architecture testing.

Yahoo! JAPAN video script

My name is Kenya Murakoshi from Yahoo! JAPAN. I am the General Manager of our entire production network infrastructure.

Yahoo! JAPAN operates a major web portal site in Japan.  We have more than 100 web services, such as Search, News, Weather, Auction, and Shopping.  In addition, we offer about 50 apps each for both iOS and Android devices.

Our company was founded in 1996 and its revenue has grown for more than 20 consecutive years since then. We have successfully navigated the rapid shift of our users to smartphones during the last several years, and we are now working hard to make the best use of Big Data.

We have vast amounts of traffic every day.  A steadily growing part of our Internet traffic is North to South traffic, which is due to our video streaming and video chat service we offer. We are continuously scaling our network equipment to deal with this demand. 

At the same time, the East-West traffic caused by applications like Hadoop, is growing rapidly as well.  The amount of this type of traffic is significant. In order to keep up with it, we are improving our network in several ways.

To handle this East - West traffic growth effectively, we have started deploying a Clos network architecture (or leaf/spine), and we use Cumulus Linux as a part of it for commercial basis.

To build a Clos network, size matters. If the size of network reaches a certain level, you need to choose either a Layer 3 design, which requires a spine layer in the Clos design with only box switches.  Otherwise, you use a Layer 2 design, which collapses the spine layer with chassis switches.

The advantage of a Layer 3 design is no single point failure.  But this results in a lot more cabling and optics, which increases CAPEX and also OPEX. As a result, we decided on a Layer 2 design with a chassis switch from the traditional vendor, at least as of right now.

By leveraging Backpack from Cumulus and Celestica as a spine switch, we can build a Clos network, yet have the advantages of a Layer 3 design, but with less cables and optics to manage.  This was the reason why we ultimately selected Backpack.

We evaluated Backpack in our lab environment. The underlay network of the test environment was configured with BGP, and the network inside of Backpack was with i-BGP.

We leveraged BGP unnumbered from Cumulus for the network inside Backpack.

We also configured an overlay network with EVPN and VXLAN and tested the functionality and performance with IXIA.  We checked the functionality of EVPN in Single home and Dual home configurations, and we confirmed that the packets were forwarded in wire rate.

In addition, we captured the EVPN/VXLAN packets and were able to analyze and inspect each of them.

Backpack passed all the tests we conducted without any issue.

We already have a project planned to deploy this new infrastructure for our science team this summer. We plan to build the Clos network with Backpack, which we know will be able to support large amounts of East-West traffic.