Modern web-scale data centers are thirsty for bandwidth. Popular applications such as video and virtual reality are increasing in demand, causing data centers to require higher and higher bandwidths — both within data centers and between data centers. In this blog post, we will briefly discuss the current challenges in the optics space as well as some of the key technical aspects of the Voyager’s DWDM transponders. In part two of this series, we will cover why Voyager is a unique, powerful and robust solution.
The challenges to accommodate longer distances
Within a data center, organizations are adding higher and higher bandwidth ports and connections to accommodate the need for more bandwidth. However, connections that accommodate longer distances between data centers may be limited and expensive. Therefore, a critical requirement for businesses with this challenge is how to support longer distance spans at higher bandwidths over a small amount of fiber pairs.
The optical industry solves the bandwidth problem using Dense Wave Division Multiplexing (DWDM). DWDM allows many separate connections on one fiber pair by sending them over different wavelengths. Although the wavelengths are sent on the same physical fiber, they act as “ships in the night” and don’t interact with each other, similar to VLANs on a trunk. Each wavelength can transport very high speeds (hundreds of Gigabits per second) over very long distances. While this is an incredible feat, today’s DWDM systems are typically closed and expensive. The transponders (which I’ll be explaining in more detail below) are generally the most expensive part of the closed DWDM network.
Announcing Voyager early access
Back in November, we announced the partnership between Cumulus and the open packet DWDM platform Facebook brought to the Telecom Infra Project (TIP), called Voyager, bringing the first open packet optical product to the industry. In just a few weeks, Voyager will officially be available for early access, and we’ll be rolling out a variety of resources for you to get to know the solution in more detail. Voyager is a Broadcom Tomahawk-based switch, similar to Facebook’s Wedge 100, but with added DWDM ports that can connect to another switch tens, hundreds or thousands of kilometers away by adding transponders.
By running Cumulus Linux, Voyager brings all the functionality of Cumulus with it, including BGP, EVPN, OSPF, Layer 2, native network automation and advanced monitoring — right to the optical world in just 1RU. We have also added typical DWDM transponder features, such as configuring power, wavelength, FEC, speed and performance monitoring. And of course, being Cumulus Linux, it’s extremely cost effective too. Finally, integrating L1/L2 and L3 all on Linux may enable you to reduce the number of nodes in the network and unify the entire data center from the hosts to the switches and even to the optical devices!
What is a DWDM transponder?
For reference, a typical active DWDM network is depicted below. Depending upon the use case, all elements below are not required for Voyager deployment. For example, Voyager can also be deployed over dark fiber with no ROADMs.
Voyager is the transponder (TPDR) in the below scenario and is also a Layer2/3 switch with all the functionality of a Broadcom Tomahawk switch with Cumulus Linux.
The transponder lives primarily at the edge of the DWDM network (with some exceptions, like when back-to-back transponders are used as a regenerator) and has a port to connect to a switch or router and a port to connect to the DWDM line system facing the remote end. In some cases, such as Voyager, the transponder could be located within a switch or router (i.e. the same box does switching, routing and transponding).
A muxponder is similar to a transponder, only it also performs time division multiplexing (TDM) over a pre-specified wavelength. For example, with a muxponder, you can send ten 10GigE links over a single 100G wavelength. Voyager will support this functionality as well.
What does a transponder do?
A transponder performs an optical-electrical-optical conversion and is primarily responsible for three tasks:
- Converting the “grey” wavelength (850nm, 1310nm or 1550nm) to a pre-specified C-band wavelength and back
- Encapsulating/decapsulating the ethernet frame into a layer 1 OTN or OTN-like frame and providing performance monitoring and forward error correction (FEC)
- Modulating, transmitting, receiving, demodulating and controlling signal power
After the wavelength leaves the transponders line side port, it is typically multiplexed with other wavelengths and travels through the network over single mode fiber. Any wavelength could be dropped off or added at any site with a ROADM. At the remote end, the signal is de-multiplexed before being handed back to the transponder. The transponder provides performance monitoring, fixes any transmission errors and decapsulates the OTN frame before handing the ethernet frame back to a switch or router or an upper layer.
Voyager combines the transponder with L2/L3 — doing the “ethernet handoff” internally to Voyager itself.
What is a C-band wavelength?
The ITU-T divides the fiber optic communication spectrum (part of the infrared section of the full electromagnetic spectrum) into 6 bands: O, E, S, C, L and U. The attenuation across a single mode fiber optic cable is lowest (0.20-0.25dB/km) at the C-band (or conventional band), so it is primarily used for DWDM communications. Also, low cost Erbium Doped Fiber Amplifiers (EDFAs) help boost signals operating at this range. Voyager will support transmitting/receiving on the C-band.
The C-band wavelength spectrum used by Voyager is shown below:
The Voyager line port can be tuned to use a pre-specified C-band wavelength. It will support tuning at 50GHz spacing and flex spacing at 12.5GHz increments. The channels are fixed and determined at every 12.5GHz or 50GHz along the spectrum, identified by ITU-T G.694.1. With flex grid, they can be chosen by the operator at every 12.5GHz, with the operator being careful not to let the channels sidebands overlap. The sidebands can be different depending on the speed and modulation type of the channel.
What is an OTN or OTN-like frame?
An Optical Transport Network (OTN) frame (or the like) is used with DWDM. An ethernet frame is run inside it for this application.
It looks similar to the following:
As you can see, an OTN frame (which is in the electrical domain) consists of three overhead layers and they are analogous to SONET Line, Section and Path overheads:
- Optical Transport Unit (OTU): Between optical network elements
- Optical Data Unit (ODU): Network level
- Optical Path Unit (OPU): Responsible for end-to-end
It also offers Forward Error Correction (FEC) and OAM&P (performance monitoring) capabilities within the overheads. Using amplifiers (specifically EDFAs) in a network can increase noise, thereby reducing signal quality and creating potential errors. FEC is used to correct the errors.
Several FEC algorithms exist in the industry today. Generally, the Soft Decision (SD) FECs can correct more errors than Hard Decision FECs, so the optical network can push longer distances at faster speeds with an SD-FEC. Voyager will run an SD-FEC.
Performance monitoring consists of reporting and keeping track of when there are errors in the network. For example, it keeps track of the pre-FEC bit error rate (BER), which is how many errors the FEC had to correct to be able to deliver a clean frame up to the higher layers. All of these characteristics can tell an operator if the network has issues, such as a degrading amplifier.
What is modulation and transmitting?
A transponder/muxponder is also responsible for modulating the signal and transmitting the signal at the appropriate power level. Transmitting at too little power won’t drive very far, and transmitting at too much power can distort the signal. Voyager will be able to effectively transmit up to +2.5dBm.
Different modulations are typically used with different bit rates. For example, lower bit rates at or below 10Gbps may use on-off keying (OOK). OOK is a simple modulation method that turns light on to signify a binary “1” and turns light off to signify a binary “0”. In this scenario, the bit rate and the symbol (baud) rate (signals through the fiber) are the same. Phase Shift Keying (PSK) uses the same idea, but instead of changing the power of the signal, it changes its phase to denote a binary “0” or “1”.
As we get to higher and higher bit rates, we want to keep the baud rate low to minimize distortions while increasing the bit rate we send. This leads to more complex modulation types, such as QPSK (Quadrature Phase Shift Keying) and 8 & 16 QAM (Quadrature Amplitude Modulation). QPSK transmits 2 bits per baud, 8 QAM transmits 3 bits per baud and 16 QAM transmits 4 bits per baud. Additionally, when we modulate 2 orthogonal polarizations separately (sometimes called Dual Polarization or Polarization Division Multiplexing), we double the throughput. Voyager will support QPSK with 100Gbps, 8QAM with 150Gbps, and 16QAM with 200Gbps with PDM.
As more and more bits per symbol are transmitted, more symbols are needed to represent the different bit patterns. The symbols are then represented closer together and thus more difficult to distinguish from each other. Therefore, the signal needs to be “cleaner” and have less noise in order to read it effectively. This leads to shorter supported distances and the need for more complex FEC algorithms, such as SD-FEC.
Of course, a transponder also receives and demodulates the signal on the remote end.
At this point, you hopefully understand the basic technical challenges and needs with DWDM. This blog post covered the technical aspects of a transponder within Voyager. In part two, we cover the question “Why and how to deploy Voyager?” and discuss Voyager functionality and use cases in greater detail. We couldn’t be more excited to be releasing Voyager for early access in just a few weeks, along with several supporting resources. If you’re dying to check it out in more detail in the meantime, visit Adva’s website, as they will be supporting and selling the product with Cumulus Linux, or contact TIP. Stay Tuned for more!