Broadcom describes its Tomahawk 3 as a quantum leap in switch power and cost efficiency for next-gen hyperscale data centres. There are four arguments for this: (1) Tomahawk3 offers scalable forwarding for container networking based on rich tunneling support and support for segment routing (2) Scale-Out, Configurable Load Balancing & Multipathing, which provide traffic load adaptive capabilities in the switch (3) Broadview Gen3 network instrumentation based on in-band telemetry with per-packet timestamping and mirroring (4) 40% reduced Watt/Gbps thanks to the chip’s 16nm geometry.
Broadcom’s BroadView software generates network analytics directly from the switching silicon, providing a way for network telemetry to collected.
Tomahawk3 also brings a high-performance integrated SerDes on chip, specifically, up to 256 x 50G-PAM4/25G-NRZ Dual-Mode SerDes. This enables 50G PAM-4 Based Ethernet Speeds, including 50GbE (1-lane), 100GbE (2-lane), 200GbE (4-lane), and 400GbE (8-lane). High-density port configurations for hyperscale spine switches based on the 12.8 Tbps Tomahawk3 could be:
- 128 x 100GE
- 64 x 200GE
- 32 x 400GE
Broadcom is also offering an 8.0 Tbps version of the Tomahawk3 that could be used for 100G Top-of-rack switches with the following port combinations:
- 80 x 100GE
- 48 x 100GE + 8 x 400GE or 16 x 200GE
- 96 x 50GE + 8 x 400GE or 16 x 200GE
The Broadcom universe
Finally, its worth considering the wide scope of partners and customers in the Broadcom switching ecosystem.
Till this point, pretty much all the major switch OEMs/ODMs are aboard. The big cloud vendors are building their own switches in-house but with Broadcom silicon. If there has been competitive pressure for this type of switch, mostly it would be from manufacturers opting to build switching ASICs in-house rather than rely on Broadcom solution. But there are a few start-ups attempting to enter the market.
Innovium will soon launch a 12.8 Tbps switch
In March 2017, Innovium, a start-up based in San Jose, California and backed by $90 million in venture funding, announced plans for a new line of TERALYNX scalable Ethernet silicon for data centers switches with aggregate capacity options at 12.8Tbps, 9.6Tbps, 6.4Tbps and 3.2Tbps performance points. Though the chips have yet to begin shipping to our knowledge at least, in an updated blog posting this week, Innovium revealed that initial its initial systems will be available in early part of 2018 and that the company expects its customer and partners to disclose further details at that time. Innovium says it has active engagements with leading customers across all categories: Cloud, OEMs & ODMs.
Innovium’s TERALYNX silicon promises support for 10/25/40/50/100/200/400GbE Ethernet standards. It will deliver 128 ports of 100GbE, 64 ports of 200GbE or 32 ports of 400GbE in a single device.
From the Innovium Ethernet switching silicon spec sheet:
- 12.8Tbps, 9.6Tbps, 6.4Tbps and 3.2Tbps single chip performance options at packet sizes of 300B or smaller
- Single flow performance of 400Gbps at 64B minimum packet size, 4x vs alternatives
- 70MB of on-chip buffer for superior network quality, fewer packet drops and substantially lower latency compared to off-chip buffering options
- Up to 128 ports of 100GbE, 64 ports of 200GbE or 32 ports of 400GbE, which enable flatter networks for lower Capex and fewer hops
- Support for cut-through with best-in-class low latency of less than 350ns
- Programmable, feature-rich INNOFLEX forwarding pipeline
- Comprehensive layer 2/3 forwarding and flexible tunneling including MPLS
- Large table resources with flexible allocation across L2, IPv4 and IPv6
- Line-rate, standards-based programmability to add new/custom features and protocols
- FLASHLIGHT telemetry and analytics to enable autonomous data center networks
- Extensive visibility and telemetry capabilities such as sFlow, FlexMirroring along with highly customizable extra-wide counters
- P4-INT in-band telemetry and extensions to dramatically simplify end to end analysis
- Advanced analytics enable optimal resource monitoring, utilization and congestion control allowing predictive capabilities and network automation
Innovium was co-founded by Rajiv Khemani, formerly COO of Cavium; Puneet Agarwal, former Senior Director and Distinguished Engineer at Broadcom; and Mohammad Issa, previously VP Engineering at Broadcom. Its investors include Greylock Partners, Walden International, Capricorn Investment Group, S-Cubed Capital, Redline, and Qualcomm Ventures.
Barefoot is shipping a 6.5 Tbps switching chip
Barefoot Networks, a start-up based in Palo Alto, California and backed by over $130 million in venture funding including an investment from Google, is already shipping a 6.5 Tbps version of its “Tofino” user programmable switching silicon. The silicon is designed for user programmability via the open-source P4 programming language.
Barefoot has previously stated that its technology is being adopted by large enterprises and telecommunications providers to increase network performance and efficiency through leveraging programmable forwarding plane technology.
One publicly disclosed example is AT&T, which has tested Barefoot’s Tofino and In-band Network Telemetry (INT) to gain deep insight into the network down to packet-level. Barefoot stated that it has recently worked with AT&T and SnapRoute to deliver what it believes is the first real-time path and latency visualisation.
Earlier this month, Barefoot introduced “Deep Insight” software, which can run on commodity servers and in a network powered by switches based on the "Tofino" programmable switch chip to interpret, analyze and pinpoint packet telemetry. Using stateful baselining of a network's performance, the company says its software automatically filters out irrelevant data, detecting only anomalies at any time scale and with nanosecond resolution. The Deep Insight software can track the sequence of switches the packet visited along its path, the set of rules it matched upon at every switch along the way, the time it spent buffered in every switch, to the nanosecond, and the packets, flows and application that the packet shared each queue with.
Barefoot Networks was co-founded by Nick McKeown, a Stanford professor and co-founder of Nicira (acquired by VMware), Martin Izzard, Pat Bosshart, and Dan Lenoski VP Engineering. In February 2017, Barefoot named Craig Barratt as President and Chief Executive Officer. Barratt joined Barefoot from Alphabet and Google, where he was Senior Vice President at Google and Chief Executive Officer of Alphabet’s Access business, which includes the Google Fiber broadband internet service. Prior to Google, he served as President of Qualcomm Atheros.
