The next big step on the relentless march forward for Ethernet is 400 Gbit/s. This week's OFC in Los Angeles brought a flurry of activity around 400 Gbit/s and finally we are seeing efforts come together from across a broad ecosystem of module vendors, chip suppliers, test and measurement companies, network equipment suppliers, telecom companies and Internet content providers. As with previous generations, the 400 Gigabit Ethernet will appear in next-generation switches and routers used in high-density cloud data centres and in the core of carrier networks. Meanwhile, work continues on 200 Gbit/s as an interface speed for optical transport, especially in data centre interconnect (DCI) and metro networks, but the building momentum behind 400 Gbit/s seems likely to assure its place as the 'new 100 Gbit/s' as the common denominator for high-speed networks.
While there are always prototype implementations of any new technology years before a standard is ready, this time it looks as if legitimate 400 Gbit/s transceivers will hit the market this year, and switch designs capable of driving this bandwidth may soon follow. The recent Ethernet Switch - Data Centre 5-year forecast report from Dell'Oro predicts that cloud data centres will drive the transition toward 400 Gbit/s by 2019. However, it must first be shown that 400 Gbit/s works as specified, that multi-vendor interoperability is ensured, and that the famous Ethernet price curve is maintained.
Where the technology stands today
At this year's OFC, the first-generation of 400 Gigabit Ethernet pluggable transceiver modules, offered in the MSA based CFP8 form-factor, were shown. Some designs are sampling and some promise to do so soon. Testing tools for 400 Gbit/s developers are on the market and specification work has progressed and MSAs are being completed, while multi-vendor interoperability events are being held, and even the first carrier field trials are occurring.
In February, the Optical Internetworking Forum (OIF) approved an Implementation Agreement (IA) for an optical integrated polarisation multiplexed (PM) quadrature modulator for coherent applications with nominal symbol rates up to 64 Gbaud. The agreement supports the 16QAM modulation format for 400 Gbit/s applications.
In March, a new MSA group released a specification for a new quad small form factor pluggable double density (QSFP-DD), which is a next generation high-density, high-speed pluggable module with a QSFP28-compatible double-density interface. QSFP-DD pluggable modules can quadruple the bandwidth of networking equipment while remaining backwards compatible with existing QSFP form factors used across Ethernet, Fibre Channel and InfiniBand for 40 and 100 Gbit/s network applications. To have significant market impact, the 400 Gbit/s transceiver will need to plug into switches and routers that are 400 Gbit/s ready, and this requires switching silicon ready to support these rates.
AT&T has announced that it completed the first of a multi-phase trial testing 400 Gigabit Ethernet data speeds, using optical gear from Coriant to carry a true 400 Gigabit Ethernet service across a long-distance span of AT&T global backbone from New York to Washington. This demonstrated that AT&T's nationwide software-centric network is 400 Gbit/s-ready. A second phase of the trial will carry 400 Gigabit Ethernet on a single wavelength across its OpenROADM metro network. This phase is expected to use optical gear from Ciena. A third phase is expected to test the first instance of a 400 Gigabit Ethenet open router platform. The 'disaggregated router' platform uses merchant silicon and open source software and is expected to be another industry first.
In a post deadline paper at OFC, researchers at AT&T Labs described the trial, which encompassed an end-to-end 400 Gigabit Ethernet circuit, inclusive of 400 Gbit/s client cards with a CPF8 interface and dual-carrier 16QAM line-side. The trial demonstrated the feasibility of SDN-enabled creation, deletion and re-routing of the 400 Gbit/s service.
In addition, the Ethernet Alliance conducted a live 400 Gbit/s demo carrying traffic from its stand to four additional company booths on the show floor, namely Finisar, Xilinx, Broadcom and Viavi. All the connections were made with LR8 CFP8 transceivers over single mode fibre using 8 x 50 Gbit/s PAM4. Notably, three test equipment companies also participated - Ixia, Spirent and Viavi.
Other 400 Gbit/s announcements from OFC
Broadcom announced a comprehensive portfolio of N x 56 Gbit/s PAM4 PHY devices enabling 50, 100, 200 and 400 Gigabit Ethernet. The new 16 nm N x 56 Gbit/s PAM4 PHY enables low power PAM4 PHYs with high performance DSP-based equalisation to address both copper and optical cabling systems, including multi-mode fibre (MMF) and single-mode fibre (SMF). The PHYs are protocol agnostic to support various interconnect interfaces, including IEEE 802.3bs/cd 50/100/200/400 Gigabit Ethernet for backplane and copper cabling, CDAUI-8 chip-to-module and chip-to-chip, and OIF 56G-LR-PAM-4 for PCB and cabled backplanes.
Inphi introduced the Polaris 16 nm PAM4 DSP PHY ICs, which provide a full bi-directional interface with host ASICs that have 28 GBaud PAM4 and NRZ electrical interfaces and support bridging to 28 Gbaud PAM4 optics. The products also support retiming and gearbox functionality. The line-up includes: Polaris-400G, 8 x 56 Gbit/s PAM4, 16 x 28 Gbit/s NRZ gearbox for CFP8; and Polaris-400G (NG), 8 x 56 Gbit/s, 8 x 56 Gbit/s PAM4 retiming for OSFP/QSFP-DD.
Ixia, Cisco and Foxconn Interconnect Technology (FIT) demonstrated 400 Gigabit Ethernet traffic with QSFP-DD via PAM4 electrical lanes, and Lumentum showcased several 400 Gbit/s transceivers, including QSFP-DD FR4, DR4 and OSFP FR4. The QSFP-DD and OSFP transceivers comply with the recently announced QSFP-DD MSA and OSFP MSA, respectively. Lumentum was an early participant and contributor to both MSAs. The new transceivers use the company’s high-performance EML (electro-absorption modulator laser) technology.
MACOM announced a complete PAM4 technology chipset for 100 Gbit/s data rates over a single wavelength enabling single fibre and four-lane parallel fibre connectivity for 100, 200 and 400 Gigabit Ethernet applications. The chipset features a new TIA, transmit and receive clock and data recovery (CDR) devices and linear EML driver module.
In addition, NeoPhotonics demonstrated a pluggable coherent CFP2-ACO module for 400 Gbit/s over data centre interconnect and metro/regional distances. The ClearLight CFP2-ACO platform uses the company's high bandwidth Class 40 coherent receiver, capable of 64 Gbaud with 16QAM to 64QAM modulation.
Oclaro announced sampling of its 400 Gbit/s CFP8 transceiver for core routers/transport applications. The CFP8 leverages Oclaro's EML laser and receiver technology and is compliant with 400GBASE-LR8 optical and 400GAUI-16 electrical interface specifications under final standardisation by the IEEE 802.3bs task force. Oclaro expects commercial production in the second half of 2017.
Finally, Viavi Solutions showcased its ONT-600 400 Gbit/s test platform, which aims to accelerate 400G deployment, from the early stages of developing and validating CFP8 FR8/LR8 modules to fully comprehensive Ethernet performance needed for system level and service validation. The company said it has recently released several applications, including enhanced FEC testing, enabling validation of performance of 400 Gbit/s elements against IEEE P802.3 bs (draft) FEC. Viavi also announced new advanced error analysis applications: all types of errors including bursts, individual bit errors and slips are characterized to quickly establish the root cause.
Watch for bigger switching silicon
One further area to keep an eye on is the progression of terabit-class switching silicon. Broadcom dominates in switching silicon but several start-ups are looking at this next step function in Ethernet bandwidth as the ideal opportunity to make a market entrance. One such upstart is San Jose-based Innovium, which has unveiled its 12.8 Tbit/s TERALYNX scalable Ethernet silicon for data centre switches. The design is powerful enough to deliver 128 ports of 100 Gigabit Ethernet, 64 ports of 200 Gigabit Ethernet or 32 ports of 400 Gigabit Ethernet in a single device. Innovium, in partnership with Inphi, has already produced a single switch chip-based reference design for a platform supporting 12.8 Tbit/s (128 x 100 Gigabit Ethernet) QSFP28 deployments. The reference design uses Innovium's 12.8Tbit/s TERALYNX Ethernet switch silicon and Inphi's PAM4 chipset. Versions with 400 Gbit/s ports should be just over the horizon.
Where the technology stands today
At this year's OFC, the first-generation of 400 Gigabit Ethernet pluggable transceiver modules, offered in the MSA based CFP8 form-factor, were shown. Some designs are sampling and some promise to do so soon. Testing tools for 400 Gbit/s developers are on the market and specification work has progressed and MSAs are being completed, while multi-vendor interoperability events are being held, and even the first carrier field trials are occurring.
In February, the Optical Internetworking Forum (OIF) approved an Implementation Agreement (IA) for an optical integrated polarisation multiplexed (PM) quadrature modulator for coherent applications with nominal symbol rates up to 64 Gbaud. The agreement supports the 16QAM modulation format for 400 Gbit/s applications.
In March, a new MSA group released a specification for a new quad small form factor pluggable double density (QSFP-DD), which is a next generation high-density, high-speed pluggable module with a QSFP28-compatible double-density interface. QSFP-DD pluggable modules can quadruple the bandwidth of networking equipment while remaining backwards compatible with existing QSFP form factors used across Ethernet, Fibre Channel and InfiniBand for 40 and 100 Gbit/s network applications. To have significant market impact, the 400 Gbit/s transceiver will need to plug into switches and routers that are 400 Gbit/s ready, and this requires switching silicon ready to support these rates.
AT&T has announced that it completed the first of a multi-phase trial testing 400 Gigabit Ethernet data speeds, using optical gear from Coriant to carry a true 400 Gigabit Ethernet service across a long-distance span of AT&T global backbone from New York to Washington. This demonstrated that AT&T's nationwide software-centric network is 400 Gbit/s-ready. A second phase of the trial will carry 400 Gigabit Ethernet on a single wavelength across its OpenROADM metro network. This phase is expected to use optical gear from Ciena. A third phase is expected to test the first instance of a 400 Gigabit Ethenet open router platform. The 'disaggregated router' platform uses merchant silicon and open source software and is expected to be another industry first.
In a post deadline paper at OFC, researchers at AT&T Labs described the trial, which encompassed an end-to-end 400 Gigabit Ethernet circuit, inclusive of 400 Gbit/s client cards with a CPF8 interface and dual-carrier 16QAM line-side. The trial demonstrated the feasibility of SDN-enabled creation, deletion and re-routing of the 400 Gbit/s service.
In addition, the Ethernet Alliance conducted a live 400 Gbit/s demo carrying traffic from its stand to four additional company booths on the show floor, namely Finisar, Xilinx, Broadcom and Viavi. All the connections were made with LR8 CFP8 transceivers over single mode fibre using 8 x 50 Gbit/s PAM4. Notably, three test equipment companies also participated - Ixia, Spirent and Viavi.
Other 400 Gbit/s announcements from OFC
Broadcom announced a comprehensive portfolio of N x 56 Gbit/s PAM4 PHY devices enabling 50, 100, 200 and 400 Gigabit Ethernet. The new 16 nm N x 56 Gbit/s PAM4 PHY enables low power PAM4 PHYs with high performance DSP-based equalisation to address both copper and optical cabling systems, including multi-mode fibre (MMF) and single-mode fibre (SMF). The PHYs are protocol agnostic to support various interconnect interfaces, including IEEE 802.3bs/cd 50/100/200/400 Gigabit Ethernet for backplane and copper cabling, CDAUI-8 chip-to-module and chip-to-chip, and OIF 56G-LR-PAM-4 for PCB and cabled backplanes.
Inphi introduced the Polaris 16 nm PAM4 DSP PHY ICs, which provide a full bi-directional interface with host ASICs that have 28 GBaud PAM4 and NRZ electrical interfaces and support bridging to 28 Gbaud PAM4 optics. The products also support retiming and gearbox functionality. The line-up includes: Polaris-400G, 8 x 56 Gbit/s PAM4, 16 x 28 Gbit/s NRZ gearbox for CFP8; and Polaris-400G (NG), 8 x 56 Gbit/s, 8 x 56 Gbit/s PAM4 retiming for OSFP/QSFP-DD.
Ixia, Cisco and Foxconn Interconnect Technology (FIT) demonstrated 400 Gigabit Ethernet traffic with QSFP-DD via PAM4 electrical lanes, and Lumentum showcased several 400 Gbit/s transceivers, including QSFP-DD FR4, DR4 and OSFP FR4. The QSFP-DD and OSFP transceivers comply with the recently announced QSFP-DD MSA and OSFP MSA, respectively. Lumentum was an early participant and contributor to both MSAs. The new transceivers use the company’s high-performance EML (electro-absorption modulator laser) technology.
MACOM announced a complete PAM4 technology chipset for 100 Gbit/s data rates over a single wavelength enabling single fibre and four-lane parallel fibre connectivity for 100, 200 and 400 Gigabit Ethernet applications. The chipset features a new TIA, transmit and receive clock and data recovery (CDR) devices and linear EML driver module.
In addition, NeoPhotonics demonstrated a pluggable coherent CFP2-ACO module for 400 Gbit/s over data centre interconnect and metro/regional distances. The ClearLight CFP2-ACO platform uses the company's high bandwidth Class 40 coherent receiver, capable of 64 Gbaud with 16QAM to 64QAM modulation.
Oclaro announced sampling of its 400 Gbit/s CFP8 transceiver for core routers/transport applications. The CFP8 leverages Oclaro's EML laser and receiver technology and is compliant with 400GBASE-LR8 optical and 400GAUI-16 electrical interface specifications under final standardisation by the IEEE 802.3bs task force. Oclaro expects commercial production in the second half of 2017.
Finally, Viavi Solutions showcased its ONT-600 400 Gbit/s test platform, which aims to accelerate 400G deployment, from the early stages of developing and validating CFP8 FR8/LR8 modules to fully comprehensive Ethernet performance needed for system level and service validation. The company said it has recently released several applications, including enhanced FEC testing, enabling validation of performance of 400 Gbit/s elements against IEEE P802.3 bs (draft) FEC. Viavi also announced new advanced error analysis applications: all types of errors including bursts, individual bit errors and slips are characterized to quickly establish the root cause.
Watch for bigger switching silicon
One further area to keep an eye on is the progression of terabit-class switching silicon. Broadcom dominates in switching silicon but several start-ups are looking at this next step function in Ethernet bandwidth as the ideal opportunity to make a market entrance. One such upstart is San Jose-based Innovium, which has unveiled its 12.8 Tbit/s TERALYNX scalable Ethernet silicon for data centre switches. The design is powerful enough to deliver 128 ports of 100 Gigabit Ethernet, 64 ports of 200 Gigabit Ethernet or 32 ports of 400 Gigabit Ethernet in a single device. Innovium, in partnership with Inphi, has already produced a single switch chip-based reference design for a platform supporting 12.8 Tbit/s (128 x 100 Gigabit Ethernet) QSFP28 deployments. The reference design uses Innovium's 12.8Tbit/s TERALYNX Ethernet switch silicon and Inphi's PAM4 chipset. Versions with 400 Gbit/s ports should be just over the horizon.
