Nine months ago, I wrote about how advances in silicon designs and technologies were going to create a product set that will democratize the networking components in modern data centers. Specifically, that the Trident II family of products will perform the role that Xeon did on the compute side and provide the fulcrum on which open networking OSes would flip the industry.

Today, I am happy to add to that story and talk a little about the secondary effects that that tipping point has set in motion. We had set about to make the networking space an open, agile and innovation-laden environment akin to the compute space, but we are finding a tremendous appetite for the story to be moved further forward and have networking and compute be treated as complete equals. The drive to manage compute and networking in a harmonious way, in the way that a bus and CPUs operate inside a single box, will have a familiar lynchpin – x86 processors.

Let’s look at a little historical context around the progression of CPUs that sit inside data center switching systems. Traditionally, the CPU that sat inside a networking box operated its control plane. The calculus used to select the CPU was the lowest power profile you can use to run the relatively constrained use cases of networking protocol engines (although beefier CPUs were used for route processing and chassis-like systems). Since each box was unique and special, an inordinate amount of effort needed to be spent to build up a new device, so squeezing every value out of it was worth the effort.

PowerPC: The Incumbent King of the Hill

In this space, for historic software reasons, the PowerPC architecture was a great fit. Delivering great flops per watt, with a simple, clean pipeline, it rapidly established itself as the king of the hill in the embedded processor space. At that point, PowerPC proponents like Freescale Semiconductor looked for the natural progression to their processors. Linux had very good infrastructure to handle PowerPC systems and the P2020-class processors had many mainline distributions supporting it.

Enter the current protagonist, the P2041-class processors.

Cumulus Networks | Communication Processors

To address the “other” needs of a
 data center control plane, they added features that obviated the need for special function processors often found on network devices that had been customized by OEMs to provide added value.

The P2041, for example, has 5 1GbE and 1 10GbE SerDes, allows for 128B regular expression searches, and has packet policers, lossless flow control and 1588 support.

All in all, it’s an awesome product for the hyper-featured networking device that someone may want to build to provide value through hardware differentiation.

There was an associated cost, however. Every bit of innovation in the PowerPC processor is something that makes it stand out, requires a special Freescale fork of the Linux kernel for support, and requires Freescale-provided runtime libraries. Moreover, any investment in software stays confined to the network control plane. Over time this has led to deprecated support in the mainline kernel and a move to the fringes for the PowerPC processors.

Enter Atom-based x86: The Contender

Intel has been steadily chipping away at the embedded systems space with an increasingly relevant series of CPUs in the Atom family. The latest incarnation looks like this:

Intel EP80579

As you can see from the picture, there are none of the specialized technologies that the PowerPC processors boast of. Instead, the blocks look remarkably similar to a server-class CPU.

On the face of it, this would make it a very unsuitable control processor for the use case outlined above. However, we observe the opposite effect.

Where the Intel processors do provide the state of the art is in supporting the HT, VT-x and amd64 instruction sets, allowing for applications to “just run” on the network control plane. There are improvements in the x86 ISA to help with cryptographic algorithms, improved DMA engines and integrated high performance 10GbE SerDes, but those features also make for a good server CPU.

So, which processor holds the key to the future for the control plane of networking devices? In the data center, the answer is the Intel x86-compatible processor. And the reasons it is has nothing to do with the discussion above. It has to do with the evolutionary path the modern data centers are running on. It has to do with economic principles.

As cloud-based application/services permeate all forms of computing we interact with, the need to manage things at scale is becoming more and more the cornerstone of technology planning. Problems that are mere annoyances at small unit count can become crippling at a scale of a few thousand. The problem resembles the macro-economic concept of operating at economies of scale.

 Cost per mips

And scale is where the selection of the x86 processor really shines through. This is because the processor architecture and its full suite of applications happen to be running in large portions of the data center already.

This is where having a kernel that just works, with compilers and debuggers that just work begins to provide economies of scale on the operational aspects.

Furthermore, with an x86-based solution, it means that you can compile an application on your laptop (or in a Linux VM) and just copy it over to your production environment without the need for a cross-compile setup. Thus, you get investment protection for your IT infrastructure and all the tools that you develop for it.

There are numerous instances of historical precedence for it too. The highly popular PS2/PS3 MIPS technology-based game engines from Sony were eventually replaced by the PS4, based on x86 architecture to address the “too hard to develop for” complaint. Apple switched from the PowerPC architecture to x86 precisely because they needed to get on the economy of scale curve.

Bottom line: While history may not repeat itself, it does rhyme, and all indications are that we will hear that rhyme coming from the data center. The wave of open, innovative platforms that overtook the compute side of cloud computing will not just take over the networking side of the house, but will also drive to blur the lines between the compute and the networking sides of the data center. This move has already begun, as we are rolling out support for all the x86-based switches from all the partners that we have in our portfolio.

Operationally, it means that orchestration tools like Puppet, Chef, Salt, Ansible and the like will treat each component as interchangeable, and can configure and manage each layer without additional functional constraints. The dynamic operations model will be consistent from binary management to interacting with operating system-provided services and devices. Monitoring tools and their associated agents will just operate in networking and compute components seamlessly and cross-correlating events will become almost trivial.

As a colleague recently put it, in this model, the distinction between network and server is truly reduced to a couple of lines in your orchestration tool’s configuration section.