by Arnold Jansen, Senior Product Marketing Manager, Nokia
The introduction of 400 Gigabit Ethernet (GE) pluggable digital coherent optics (DCOs) has stirred up considerable debate about innovative approaches to metro access and metro/regional network design that blend IP and optical networking technologies in more optimal ways. The key question is how this technology can be leveraged to more cost-efficiently meet service-level requirements without adding more complexity to network operations.
Routers, ROADMs and rings
Due to their lower cost, power and space requirements, pluggable 400GE DCOs are generally more economical for shorter reaches than connecting routers to dense wavelength division multiplexing (DWDM) transponders in optical line systems. With in-line amplification, 400ZR+ coherent optics can interconnect 400GE router ports over hundreds of kilometers of fiber, and up to 1,000 km at lower bit rates.
This capability is certainly adequate for most point-to-point fiber applications, but dedicated fiber may not be readily available where it is needed because laying new fiber is costly and time consuming. At least initially, new 400GE router-to-router connections will be implemented over existing fiber plant in metro/regional access and aggregation networks. This fiber is typically laid as interconnected and overlapping rings that aggregate traffic from multiple central offices.
An IP-centric way to look at metro/regional access and aggregation rings is as a distributed leaf-spine fabric with access routers on the ring (the leaves) connecting into a centralized aggregation router at the ring head-end (the spine). Ideally, each access leaf directly connects with the spine in a logical hub-and-spoke IP topology, especially since the volume and growth of ingress traffic on individual access leaves can differ greatly in urban and regional settings. This allows transport efficiency and latency to remain low and deterministic, as access traffic on the ring is passed to the hub router in a single hop.
The first option to implement this target architecture leverages 2-degree ROADMs and optical transponders (OTs) to connect access routers to the centralized hub router over protected, point-to-point wavelengths. This is the present mode of operation (PMO) for most communication service provider networks, and is depicted at the left side of Figure 1. Routers connect using 400GE gray optics to performance-optimized OTs that can operate over long fiber spans with many ROADM hops, allowing single-hop connections from each access router to the hub node at full 400G rates.
Figure 1. IP aggregation over metro/regional fiber rings
In the center, future mode of operation 1 (FMO1) leverages 400ZR+ pluggable router optics instead of WDM transponders to save space, power and cost through IP-optical integration. Through 2-degree ROADMs, all access routers connect to the hub router in a single hop over dedicated wavelengths. For large rings with many nodes, it may be necessary to reduce the line rate of 400ZR+ pluggable DCOs for an increased reach when and where it’s needed.
On the right side of Figure 1, FMO2 bypasses ROADMs altogether and interconnects routers in a daisy chain over point-to-point WDM line systems between each node on the ring. Each router aggregates local ingress traffic with transit traffic from other nodes, and passes it hop-by-hop along the ring until it reaches the hub router at the ring head-end. With wavelengths having to travel only one hop to the adjacent routers, 400ZR/ZR+ DCOs can connect at the full 400 Gb/s line rate for most router-to-router distances.
Figure 2. Comparing scaling properties of PMO, FMO1 and FMO2 in aggregation rings
Figure 2 compares the scaling properties of the PMO with FMO1 and FMO2 for a single aggregation ring with varying numbers of access nodes and amounts of ingress traffic. Although this is a simple modeling exercise, it illustrates several points:
- The PMO scales the best with increasing traffic and ring sizes and makes the most efficient use of available 400GE router ports. Although DWDM transponders cost more than pluggable DCOs, fewer of them are required on each ring and current investments in ROADMs and fiber can be fully leveraged. Also note that pluggable 400G Multihaul DCOs will become available for routers to enable higher capacity-reach over 400ZR+, and as a lower-cost alternative to DWDM transponders.
- The FMO1 can be deployed as an overlay to offload the PMO over an additional fiber pair. While FMO1 consumes slightly more 400GE router ports for larger rings and traffic volumes, this upfront cost is offset by incremental savings created by using router pluggable 400ZR+ DCOs instead of WDM transponders.
- FMO2 has a marginally lower upfront cost than FMO1 due to minor savings on ROADM capabilities, but its incremental scaling costs are much higher. There is a sweet spot for small rings and low initial ingress traffic volumes where access routers can aggregate all ring traffic over one or two wavelengths, but the number of 400G DCOs required will quickly surpass the 4–6 QSFP-DD network ports that are typically available on a 1 RU leaf aggregation router.
Key Takeaways
From an end-to-end cost perspective, a critical objective of any network architect is to minimize the number of hops required to transport traffic between source and destination. Each router hop adds cost, latency and power consumption that must be offset by statistical multiplexing gains of packet aggregation, and these gains have diminishing returns with each hop.
400GE pluggable coherent optics are a new and powerful technology that can be used to cost-optimize IP networks, provided that the number of router hops in the end-to-end data path does not increase significantly as a result. There are two ways to achieve this:
- Deploy dedicated point-to-point fiber routes where available and feasible.
- Provision dedicated point-to-point wavelengths over shared fiber using ROADMs.
When point-to-point wavelengths must traverse several ROADMs, it is generally more economical to either de-rate the capacity of 400ZR+ optics or deploy higher-performance optics such as 400G Multihaul DCOs or 400GE OTs than to bridge the distance with additional router hops or back-to-back DCOs. Importantly, this new generation of compact and modular line systems with ROADM capabilities can ensure that the capacity-reach and cost benefits of 400GE pluggable coherent optics can be maximized for all applications.
