Tutorial: MPLS Network Reliance and Recovery

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

December 17, 2001

This series of tutorials has defined MPLS into two operations: data flow and signaling. The previous tutorials have addressed these subjects with special attention given to signaling protocols CR-LDP and RSVP-TE. To complete this series, this article will cover the failure recovery process.

Vocabulary

• Back-up Path: the path that traffic takes if there is a failure on the primary path.
• Fast ReRoute (FRR): a protection plan in which a failure can be detected without a need for error notification or failure signaling (Cisco).
• Link Protection: a backup method that replaces the entire link or path of a failure.
• Make Before Break: a procedure in which the back-up path is switched in before the failed path is switched out. For a small period of time, both the primary and back-up paths carry the traffic.
• Node Protection: a backup procedure in which a node is replaced in a failure.
• Pre-provisioned Path: a path in the switching database on which traffic engineering has been performed in order to accommodate traffic in case of a failure.
• Pre-qualified Path: a path that is tested prior to switchover that meets the quality of service (QoS) standards of the primary path.
• Primary Path: the path through which the traffic would normally progress.
• Protected Path: a path for which there is an alternative back-up path.
• Rapid ReRoute (RRR): a protection plan in which a failure can be detected without a need for error notification or failure signaling (Generic).

Introduction

Around the country you will find highways under repair.  A good many of these highways have bypass roads or detours to allow traffic to keep moving around the construction or problem areas.  Traffic rerouting is a real challenge for highway departments, but they have learned that establishing detour paths before construction begins is the only way they can keep traffic moving (Figure 1).

Figure 1: Traffic Detour

The commitment to keeping traffic moving has been a philosophy in voice and telephone communications since its inception. In a telephony network, not only are detour paths set-up before a circuit is disconnected (make before break), but the back-up or detour paths must have at least the same quality as the links that are to be taken down for repair. These paths are said to be pre-qualified (tested) and pre-provisioned (already in place).

Historically in IP networking, packets would find their own detours around problem areas; there were no pre-provisioned bypass roads. The packets were in no particular hurry to get to the destination. However, with the convergence of voice onto data networks, the packets need these bypass roads to be pre-provisioned so that they do not have to slow down for the construction or road failures.

The Need for Network Protection

MPLS has been primarily implemented in the core of the IP network. Often, MPLS competes head-to-head with ATM networks; therefore, it would be expected to behave like an ATM switch in case of network failure.

With a failure in a routed network, recovery could take from a few tenths of a second to several minutes. MPLS, however, must recover from a failure within milliseconds – the most common standard is 60 ms. To further complicate the recovery process, an MPLS recovery must ensure that traffic can continue to flow with the same quality as it did before the failure. So, the challenge for MPLS networks is to detect a problem and switch over to a path of equal quality within 60ms.

Failure Detection

There are two primary methods used to detect network failures: heartbeat detection (or polling) and error messaging. The heartbeat method (used in fast switching) detects and recovers from errors more rapidly, but uses more network resources.  The error-message method requires far less network resources, but is a slower method.  Figure 2 shows the tradeoffs between the heartbeat and error-message methods.

Figure 2: Heartbeat vs. Error Message

The heartbeat method (Figure 3) uses a simple solution to detect failures. Each device advertises that it is alive to a network manager at a prescribed interval of time. If the heartbeat is missed, the path, link, or node is declared as failed, and a switchover is performed.  The heartbeat method requires considerable overhead functions - the more frequent the heartbeat, the higher the overhead. For instance, in order to achieve a 50ms switchover, the heartbeats would need to occur about every 10ms.

Figure 3: Heartbeat Method

The other failure detection system is called the error-message detection method (Figure 4).  When a device on the network detects an error, it sends a message to its neighbors to redirect traffic to a path or router that is working.  Most routing protocols use adaptations of this method. The advantage of the error message is that network overhead is low. The disadvantage is that it takes time to send the error-and-redirect message to the network components. Another disadvantage is that the error messages may never arrive at the downstream routers.

Figure 4: Error Message

If switchover time is not critical (as it has historically been in data networks), the error-message method works fine; however, in a time-critical switchover, the heartbeat method is often the better choice.

Reviewing Routing

Remember that, in a routed network (Figure 5), data is connectionless, with no real quality of service (QoS). Packets are routed from network to network via routers and routing tables. If a link or router fails, an alternative path is eventually found and traffic is delivered. If packets are dropped in the process, a layer-4 protocol such as TCP will retransmit the missing data.

Figure 5: Standard Routing

This works well when transmitting non-real time data, but when it comes to sending real-time packets, such as voice and video, delays and dropped packets are not tolerable. To address routing-convergence problems, the OSPF and IGP working groups have developed IGP rapid convergence, which reduces the convergence time of a routed network down to approximately one second.  

The benefits of using IGP rapid convergence include both increased overhead functions and traffic on the network; however, it only addresses half of the problem posed by MPLS.  The challenge of maintaining QoS parameter tunnels is not addressed by this solution.

Network Protection

In a network, there are several possible areas for failure. Two major failures are link failure and node failure (Figure 6). Minor failures could include switch hardware, switch software, switch database, and/or link degradation.

Figure 6: Network Failures

The telecommunication industry has historically addressed link failures with two types of fault-tolerant network designs: one-to-one redundancy and one-to-many redundancy.  Another commonly used network protection tactic utilizes fault-tolerant hardware. 

To protect an MPLS network, you could pre-provision a spare path with exact QoS and traffic-processing characteristics.  This path would be spatially diverse and would be continually exercised and tested for operations. However, it would not be placed online unless there were a failure on the primary protected path.  

This method, known as one-to-one redundancy protection (Figure 7), yields the most protection and reliability, but its cost of implementation can be extreme.

Figure 7: One-to-One Redundancy

A second protection scheme is one-to-many redundancy protection (Figure 8).

In this method, when one path fails, the back-up path takes over. The network shown in the Figure 8 can handle a single path failure, but not two path failures.

Figure 8: One-to-Many Redundancy

A third protection method is having fault tolerant switches (Figure 9).  In this design, every switch features inbuilt redundant functions – from power supplies to network cards. The drawing shows redundant network cards with a back-up controller.  Take note that the one item in common, and not redundant, is the cross-connect tables. If the switching data becomes corrupt, the fault tolerant hardware cannot address this problem.

Figure 9: Fault Tolerant Equipment

Now that you know the three network protection designs (one-to-one, one-to-many, and fault-tolerant hardware) and two methods for detecting a network failure (heartbeat and error message), we need to talk about which layers and protocols are responsible for fault detection and recovery.

Remembering that the further the data progresses up the OSI stack, the longer the recovery will take, it makes sense to attempt to detect failures at the physical level first.

MPLS could rely on the layer-1 or layer-2 protocols to perform error detection and correction.  MPLS could run on a protected SONET ring, or it could use ATM and Frame Relay fault-management programs for link and path protection. In addition to the protection MPLS networks could experience via SONET, ATM or Frame Relay, IP has its recovery mechanism in routing protocols, such as OSPF or IGP.

With all these levels of protection already in place, why does MPLS need additional protection? Because there is no protocol that is responsible for ensuring the quality of the link, tunnel, or call placed on an MPLS link. The MPLS failure-recovery protocol must not only perform rapid switching, but it must also ensure that the selected path is pre-qualified to take the traffic loads while maintaining QoS conditions. If traffic loads become a problem, MPLS must be able to offload lower-priority traffic to other links.

Knowing that MPLS must be responsible for sending traffic from a failed link to a link of equal quality, let’s look at the two error-detection methods as they apply to MPLS.

MPLS Error Detection

The LDP and CR-LDP protocols contain an error message type-length value (TLV) in their protocols to report link and node errors.  However, there are two main disadvantages to this method: (1) it takes time to send the error message, and (2) since LDP is a connection-oriented message, the notification message may never arrive if the link is down.

An alternative approach to error detection is to use the heartbeat method that is found at the heart of the RSVP-TE protocol. RSVP has features that make it a good alternative for an error- message model. RSVP is a soft-state protocol that requires refreshing – i.e., if the link is not refreshed, then the link is torn down. No error messages are required, and rapid recovery (rapid reroute) is possible if there is a pre-provisioned path.  If RSVP-TE is already used as a signaling protocol, the additional overhead needed for rapid recovery is insignificant.

Rapid reroute is a process in which a link failure can be detected without the need for signaling. Because RSVP-TE offers soft-state signaling, it can handle a rapid reroute.

Many vendors are using the RSVP-TE for rapid recovery of tunnels and calls, but in doing so, other MPLS options are restricted. For example, labels are allocated per switch, not per interface. Another restriction is that RSVP-TE must be used for a signaling protocol.

RSVP-TE Protection

In RSVP-TE protection, there are two methods used to protect the network: link protection and node protection. 

In link protection, a single link is protected with a pre-provisioned backup link. If there is a failure in the link, the switches will open the pre-provisioned path (Figure 10).

Figure 10: RSVP-TE with Link Protection

In a node failure, an entire node or switch could fail, and thus, all links attached to the node will fail.  With node protection, a pre-provisioned tunnel is provided around the failed node (Figure 11).

Figure 11: RSVP-TE with Node Protection

Thrashing Links

A discussion about fault-tolerant networks would not be complete without mentioning thrashing links. Thrashing is a phenomenon that occurs when paths are quickly switched back and forth.  For example: In a network with two paths (primary and back-up), the primary path fails and the back-up path is placed in service. The primary path self-heals and is switched back into service, only to fail again.

Thrashing is primarily caused by intermittent failures of primary paths and pre-programmed switchback timers. In order to overcome thrashing, the protocols and the switches must use hold-down times. For example, some programs allow one minute for the first hold-down time and set a trigger so that on the second switchback, operator intervention is required to perform a switchover and to prevent thrashing.

Summary

Building a fault-tolerant, rapid recovery network is new to the data world. One reason is that data communications philosophy is that the data will get there or it will be retransmitted. This philosophy does not work well in voice networks. To make MPLS competitive with ATM and on par with voice networks, rapid recovery must be implemented.

There are several methods under study to provide network protection. Vendors recommend approaches that are supported by their overall design concepts and specifications; therefore, failure recovery is not necessarily interoperable among different vendors. Design teams must carefully select vendors with interoperable recovery methods.

The failure recovery method that has received much favorable press lately is RSVP-TE. The soft-state operations of RSVP-TE make it very suitable for failure recovery.  One reason is that the polling (reservation/path) functions are already in place for signaling.  If RSVP-TE is already used for a signaling protocol, it makes a logical selection to protect your MPLS tunnels.

MPLS Resource Sites:

A Framework for MPLS Based Recoveryhttp://www.ietf.org/internet-drafts/draft-ietf-mpls-recovery-frmwrk-03.txt

Surviving Failures in MPLS Networkshttp://www.dataconnection.com/download/mplsprotwp.pdf

MPLS Links Pagehttp://www.rickgallaher.com/   (click on the MPLS Links tab)

Network Traininghttp://www.globalknowledge.com

Special Thanks

I would like to thank Ben Gallaher, Susan Gallaher, and Amy Quinn for their assistance, reviewing, and editing.

A special thank you to all those who assisted me with information and research on the MPLSRC-OP /mail list, especially: Robert Raszuk.

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

Read More

Tutorial: Advanced MPLS Signaling

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

December 10, 2001

In previous tutorials, we talked about data flow (Tutorial #1) and label distribution (Tutorial #2). This article discusses MPLS signaling and the ongoing conversations regarding signaling choices.

Vocabulary

• Soft State – A link, path, or call that needs to be refreshed to stay alive.
• Hard State – A link, path, or call that will stay alive until it is specifically shut down.
• Explicit Route – A path across the Internet wherein all routers are specified. Packets must follow this route, and they cannot detour.
• CR-LDP – Constraint-based Routing over Label-Distribution Protocol.
• RSVP-TE – The Resource ReSerVation Protocol (RSVP), modified to handle MPLS traffic-engineering requirements.
• IntServ – Integrated Service; allows traffic to be classified into three groups: guaranteed, controlled load, and best effort.  IntServ works together with RSVP protocol.

Your commute to work every day is a long one, but with all the congestion it seems to take forever. New lanes have been added to the highway, but they are reserved as express lanes – sure, they will cut your travel time in half, but you will have to carry extra passengers in order to use them. You decide, finally, to try it; you decide to carry four additional passengers in order to use the express lane.  You are permitted to pass through the express-lane gate and scurry on your way to and from work.

The four passengers do not cost much more to transport than yourself alone, and they really allow you to increase the speed and lower the rate of interference from the unpredictable and impossible-to-correct behavior of the routine traffic. (Figure 1)

Figure 1: Backed Up Express Lane

One day you enter the express lanes and find that they are all in a state of bumper-to-bumper congestion. You look around and find routine traffic in the express lanes. You are angry, of course, because you had guaranteed express lanes, and the routine traffic is required to stay off the express lanes unless they are carrying extra passengers.   As you slowly progress down your road, you see that construction has closed down the routine lanes and diverted the traffic to your express lanes. So, what good is it to be special if regular traffic is diverted to your express lanes?

Traffic Control in MPLS Networks

In networking, MPLS is express traffic that carries four (4) additional bytes of payload.  For taking that effort, it gets to travel the express lanes.  But, as is too often the case in the actual freeway, your nice, smooth-running express lane is subjected to routine traffic being rerouted onto it, causing congestion and slowdowns.

Remember that MPLS is an overlay protocol that overlays MPLS traffic on a routine IP network. The self-healing properties of IP may cause congestion on your express lanes. There is no accounting for the unforeseen traffic accidents and reroutes of routine traffic onto the express lanes. The Internet is self-healing with resource capabilities, but the problem becomes this: how does one ensure that paths and bandwidth that are reserved for their packets do not get overrun by rerouted traffic? (Figures 2 –4)

Figure 2: MPLS with Three Paths

In Figure 2, we see a standard MPLS network with three different paths across the Wide- Area Network.  Path A is engineered to the 90^thpercentile of bandwidth of peak busy hour; Path B is engineered to the 100^th percentile bandwidth of peak busy hour; finally, Path C is engineered to the 125^th percentile of peak busy hour.  In theory, Path A will never have to contend with congestion, owing to sound network design (including traffic engineering). In other words, the road is engineered to take more traffic than it will receive during rush hour. The C network, however, will experience traffic jams during rush hour, because it is designed not to handle peak traffic conditions.

The Quality of Service (QoS) in Path C will have some level of unpredictability regarding both jitter and dropped packets, whereas the traffic on Path A should have consistent QoS measurements.

Figure 3: MPLS with a Failed Path C

In Figure 3, we see a network failure in Path C, and the traffic is rerouted (Figure 4) onto an available path – Path A.  Under these conditions, Path A is subjected to a loss of QoS criteria. To attain real QoS, there must be a method for controlling both traffic on the paths and the percentage of traffic that is allowed onto every engineered path.

Figure 4: MPLS with Congestion Caused by a Reroute

To help overcome the problems of rerouting congestion, the Internet Engineering Task Force (IETF) and related working groups have looked at several possible solutions.  This problem had to be addressed both in protocols and in the software systems built into the routers.

In order to have full  QoS, a system must be able to mark, classify, and police traffic. From previous articles, we see how MPLS can classify and mark packets with labels, but the policing function has been missing. Routing and label distribution establishes the Label Switch Paths, but still it does not police traffic and control the load factors on each link.

New software engines, which add management modules between the routing functions and the path selector, allow for the policing and management of bandwidth. These functions, along with the addition of two protocols, allow for traffic policing.

Figure 5: MPLS Routing State Machines

The two protocols that give MPLS the ability to police traffic and control loads are RSVP-TE and CR-LDP.

RSVP-TE

The concept of a call set-up process, wherein resources are reserved before calls are established, goes back to the signaling-theory days of telephony.  This concept was adapted for data networking when QoS became an issue.

An early method designed by the IETF in 1997, called Resource ReSerVation Protocol (RSVP), was designed for this very function.  The protocol was designed to request required bandwidth and traffic conditions on a defined or explained path.  If the bandwidth was available under the stated conditions, then the link would be established.
The link was established with three types of traffic that were similar to first-class, second-class and standby air travel – the paths were called, respectively: guaranteed load, controlled load and best-effort load. 

RSVP, with features added to accommodate MPLS traffic engineering, is called RSVP-TE. The traffic-engineering functions allow for the management of MPLS labels or colors.

Figure 6:  RSVP-TE Path Request

In Figures 6 and 7, we see how a call or path is set up between two endpoints. The target station requests a specific path, with detailed traffic conditions and treatment parameters included in the path-request message.  This message is received, and a reservation message, reserving bandwidth on the network, is sent back to the target. After the first reservation message is received at the target, the data can start to flow in explicit paths from end to end.

Figure 7: RSVP-TE Reservation

This call set-up, or signaling, process is called “soft state,” because the call will be torn down if it is not refreshed in accordance with the refresh timers. In Figure 8, we see that the path-request and reservation messages continue for as long as the data is flowing.

Figure 8: RSVP-TE Path Set Up

Some early arguments against RSVP included the problem of scalability: the more paths that were established, the more refresh messages would be created, and the network would soon become overloaded with refresh messages. Methods of addressing this problem include not allowing the traffic links and paths to become too granular, and aggregating paths.

To view an example of an RSVP-TE path request for yourself, you can download a protocol analyzer and sample file from www.ethereal.com.

Protocol Analyzer: http://www.ethereal.com/download.html

Sample file: Go to http://www.ethereal.com/sample/ and click on "MPLS-TE.cap" (sample 15).

After downloading, install ethereal and open the MPLS-TE.Cap file.

In the sample below (Figure 9), MPLS captures MPLS-TE files. In the capture, we can see the traffic specifications (TSPEC) for the controlled load.

Figure 9: RSVP-TE Details

CR-LDP

With CR-LDP (Constraint-based Routing over Label Distribution Protocol), modifications were made to the LDP protocol to allow for traffic specifications. The impetus for this design was to use an existing protocol LDP and give it traffic-engineering capabilities.  A major effort by Nortel Networks was made to launch the CR-LDP protocol.

The CR-LDP protocol adds fields to the LDP protocol. They are called peak, committed, and excess-data rates – very similar to terms used for ATM networks. The frame format is shown in Figure 10.

Figure 10: CR-LDP Frame Format

The call set-up procedure for CR-LPD is a very simple two-step process: a request and a map, as shown in Figure 11.  The reason for the simple set-up is that CR-LPD is a hard-state protocol – meaning that the call, link, or path, once established, will not be broken down until it is requested that it be done.

Figure 11: CR-LDP Call Set Up

The major advantage of a hard-state protocol is that it should be more scaleable, because there is less “chatter” needed in order to keep the link active.

Comparing CR-LDP to RSVP-TE

The technical comparisons of these two protocols are listed in Figure 12.  We see that CR-LDP uses the LDP protocol as its carrier, where RSVP-TE uses the RSVP protocol.  RSVP is typically paired with IntServ’s detection of QoS, while the CR-LDP protocol uses ATM’s traffic-engineering terms to map QoS.

Comparison	CR-LDP	RSVP-TE
Vendors	Nortel	Cisco, Juniper, Foundry
State	Hard State	Soft State
QoS Type	ATM	IntServ
Recovery Time	A little slower	Faster
Chat Overhead	Low	High
Transported on	LDP over TCP	RSVP on IP
Path Modifications	Make before break	Make before break

Figure 12:  CR-LDP vs. RSVP-TE

In the industry today, we find that while Cisco and Juniper favor the RSVP-TE model and Nortel favors the CR-LDP model, both signaling protocols are supported by most vendors.

The jury is still very much as out as to the scalability, recovery, and interoperability between the signaling protocols.  However, it appears from the sidelines that the RSVP-TE protocol may be in the lead. This is not because it is less “chatty” or more robust of the two, but is due more to the fact that RSVP was an established protocol, with most of its bugs removed, prior to the inception of MPLS.  Both protocols remain the topics of study by major universities and vendors.  In the months to come, we will see test results and market domination affect these protocols.  Stay tuned…

Special thanks to:

I would like to thank Ben Gallaher, Susan Gallaher, and Amy Quinn for their assistance, reviewing, and editing.

A special thank you to all those who assisted me with information and research on the MPLSRC-OP mail list, especially Senthil Kumar Ayyasamy (mplsgeek@yahoo.com) and Javed A Syed (Jjsyed@aol.com)

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

Read More

Tutorial: Introduction to MPLS Label Distribution and Signaling

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

November 1, 2001

In the first tutorial, we discussed the data flow and the foundational concepts of MPLS networks. In this section, we will introduce the concepts and application of MPLS label distribution and introduce MPLS signaling. Moving forward, there will be a tutorial on Advanced MPLS Signaling.

Vocabulary

• Border Gateway Protocol (BGP)
• Binding
• Constrained Router Label Distribution Protocol (CR-LDP)
• Down Stream on Demand (DOD)
• Down Stream Unsolicited (DOU)
• Explicit Routing
• Independent Control
• Implicit Routing
• Intermediate System to Intermediate System (IS-IS)
• Label Distribution Protocol (LDP)
• Next Hop Label Forward Entry (NHLFE)
• Ordered Control
• Open Shortest Path First with Traffic Engineering (OSPF-TE)
• Resource Reservation Setup Protocol with Traffic Engineering (RSVP-TE)

The Early Days of Switching

Circuit switching by label is not new.  A quick look back at telephony shows us how signaling was done in the “old days.”  A telephone switchboard had patch cables and jacks; each jack was numbered to identify its location.  When a call came in, an operator would plug in a patch cord into the properly numbered jack.  This is a relatively simple concept.

Recalling these days, we find that although the process seemed simple enough, it was really hard work. Telephone operators would attend school for weeks and go through an apprenticeship before qualifying to operate a switchboard because the rules for connecting, disconnecting, and prioritizing calls were complex and varied from company to company.

Figure 1 Label Switching in the Early Days

Some of the rules included:

• Never disconnect the red jacks – these are permanent connections.
• Connect only the company executives to the jacks labeled for long distance.
• Never connect an executive to a noisy circuit.
• If there are not enough jacks when an executive needs to make a call, disconnect the lower priority calls.
• When “Mr. Big’s” secretary calls up at 9 a.m. to reserve a circuit for 10 a.m.–noon, make sure that the circuit is ready and that and you’ve placed the call by 9:50 a.m.
• In an emergency, all circuits can be controlled by the fire department.

So one operator had to know the permanent circuits (red jacks), the switched circuits, the prioritization scheme, and the reservation protocols.  When automatic switching came along, the same data and decision-making processes had to be loaded into a software program.

The MPLS switches must also be trained – they must learn all the rules and when to apply them. Two methods are used to make these switches.  One method uses hard programming; it is similar to how a router is programmed for static routing.   Static programming eliminates the ability to dynamically reroute or manage traffic.

Modern networks change on a dynamic basis.  To accommodate this need, many network engineers have chosen to use the second method: dynamic signaling and label distribution.  Dynamic label distribution and signaling can use one of several protocols, with each its given advantages and disadvantages. Because this is an emerging technology, we have not seen the dust fully settle on the most dominant label and signaling protocols.  

Yet despite the selection of protocols and their tradeoffs, the basic concepts of label distribution and signaling remain consistent across the protocols.

At a minimum, MPLS switches must learn how to process packets with incoming labels. Sometimes this is called a cross-connect table.   For example, label 101 in at port A will go out port B with a label swapped for 175.  The major advantage of using cross-connect tables instead of routing is that cross-connect tables can be processed at the “data link” layer, where processing is considerably faster than routing.

We will start our discussion using a simple network (figure 2) with four routers.  Each router has designated ports. For the sake of illustration, the ports have been given a simple letter a, b, s, h, a, and e. These port identifications are router specific.  The data flows from the input a of r1 to the input of r4.  This basic network diagram will be enhanced as we progress through MPLS signaling.

Figure 2:  Basic MPLS Network with 4 Routers

CONTROL OF LABEL DISTRIBUTION

There are two modes used to load these tables.  Each router could listen to routing tables, make its own cross-connect tables, and inform others of its information.  These routers would be operating independently. Independent control occurs when there is no designated label manager, and when every router has the ability to listen to routing protocols, generate cross-connect tables, and distribute them.  (Figure 3)

Figure 3:  Independent Control

The other model is ordered control, as shown in Figure 4.  In the ordered control mode, one router – typically the egress LER – is responsible for distributing labels.

Each of the two models has its tradeoffs. Independent control provides for faster network convergence.  Any router that hears of a routing change can relay that information to all other routers.  The disadvantage is that there is not one point of control making traffic, which makes engineering more difficult.

Ordered control has the advantages of better traffic engineering and tighter network control; however, its disadvantages are that convergence time is slower and the label controller is the single point of failure.

Figure 4: Ordered Control (pushed)

The Triggering of Label Distribution

Within ordered control, there are two major methods to trigger the distribution of labels.
These are called down-stream unsolicited and down-stream on demand.

DOU

In figure 4, we saw the labels “pushed” to the down-stream routers.  This push is based upon the decisions of the label manager router. When labels are sent out unsolicited by the label manager, it is known asdown-stream unsolicited (DOU).

For example: The label manager may use a trigger point (such as a time interval) to send out labels or label refresh messages every 45 seconds.   Or, a label manager may use the change of standard routing tables as a trigger – when a router changes, the label manager may send out label updates to all affected routers.


OD

When labels are requested, they are “pulled” down or demanded, so this method has been called pulled or down-stream on demand (DOD).  Note in Figure 5, that in the first step   the labels are requested and in the second step the labels are sent.

Figure 5:  Down-stream on Demand  (DOD)

Whether the labels arrive via independent or ordered control, or via DOD or DOU, the label switch router (LSR) creates a cross-connect table similar to the one shown in Figure 6.

The connect tables are sent to router r3 to r1. The tables heading read: label-in, port-in, label-out, port-out, and instruction (I).  In this case, the instruction is to swap (s).  It is important to note that the labels and cross-connect tables are router specific.

After the cross-connect tables are loaded, the data can flow from router 1 to router 4 with each router following its instructions to swap the labels.

Figure 6: LSR with Cross-connect Tables Populated

After the cross-connect tables are loaded, the data can now follow a designated LSP (label switch path) and flow from route 1 to router 4, as shown in Figure 7.

Figure 7:  Data Flow on LSP

REVIEW

As a brief review, we learned that routers need cross-connect tables in order to make switching decisions.  The routers can receive these tables from their neighbors via independent control or from a label manager via ordered control.

A label manger can send labels on demand (called down-stream on demand) or it can send labels when it decides to, even though it has not been requested by the down-stream routers, by using down-stream unsolicited (DOU).

With these basic concepts understood, there are some more advanced concepts to consider.  For instance, just how are labels sent to routers? What vehicle will be used to carry these labels?  How is the quality of service information relayed or sent to the routers? 

Reviewing from the first article, MPLS packets carry labels; however, the packets do not have an area that tells routers how to process the packet for quality of service (QoS).

Recalling that traffic can be separated into groups called forward equivalence classes (FECs), and that FECs can be assigned to label switch paths (LSP), we can perform traffic engineering to force high-priority FECs on to high-quality LSP and lower priority FECs on to lower-quality LSP.  The mapping of traffic using different QoS standards will cause the distribution of label and maps to be more complex.

Figure 8 shows a drawing of what goes on inside a LSR. There are two planes: the data plane and the control plane.  Labeled packets enter at input a with a label of 1450 and exit port b with a label of 1006. This function takes place in the cross-connect table. This table can also be called the next hop label forwarding entry table (NHLFE).  

Figure 8: A Closer Look at the Router

This database is not a stand-alone database.  It connects to two additional databases in the control plane: the FEC data and the FEC-to-NHLFE database.  The FEC database contains, at a minimum, the the destination IP address, but it can also contain traffic characteristics and packet processing requirements. Data in this database must be related to a label; the process of relating an FEC to a label is called binding.
Here is an example of how labels and FECs are set-up:

FEC Database

FEC	Protocol	Port
192.168.10.1	06	443	guaranteed no packet loss
192.168.10.2	11	69	best efforts
192.168.10.3	06	80	controlled load

Free Label Table
100-10,000 are not in use at this time
FEC to NHLFE Table

FEC	Label in	Label out
192.168.10.1	1400	100
192.168.10.2	500	101
192.168.10.3	107	103

NHLFE Table

Label in	Label out
1400	100
500	101
107	103

So we see that packets with labels can be quickly processed when entering the data plane, if the labels are bound to an FEC.  However, a lot of background processing must be done to the data traffic off line before a cross-connect table can be established.


Protocols

Finding a transport vehicle to build these complex tables is of the utmost concern to network designers.  What is needed is a protocol that can carry all of the necessary data while, at the same time, be fast, self-healing, and maintain very high reliability.

The MPLS workgroup and design engineers created the Label Distribution Protocol.
(LDP). This protocol works like a telephone call. When labels are bound, they stay bound until there is a command to tear down the call. This hard-state operation is less “chatty” than a protocol that requires refreshing. The LDP protocols provide implicit routing.

Other groups argue against using a new untested label distribution protocol when there exist routing protocols that can be modified or adapted to carry the bindings. Thus, some existing routing protocols have been modified to carry information for labels.  The Border Gateway Protocol (BPG) and IS-IS work well for distributing label information along with routing information.

The LDP, BGP and IS-IS protocols establish the Label Switch Path (LSPs), but do little for traffic engineering, because routed traffic could be redirected onto a high priority LSP, causing congestion.

To overcome this problem, the signaling protocols were established to create traffic tunnels (explicit routing) and allow for better traffic engineering.  They are Constraint Route Label Distribution Protocol (CR-LDP) and Resource Reservation Setup Protocol (RSVP-TE). In addition, the Open Shortest Path First (OSPF) routing protocol has undergone modifications to handle traffic engineering (OSPF-TE); however, it is not currently widely used.

Protocol	Routing	Traffic engineering
LDP	Implicit	NO
BGP	Implicit	NO
IS-IS	Implicit	NO
CR-LPD	Explicit	YES
RSVP-TE	Explicit	YES
OSPF-TE	Explicit	YES

Summary

In this article, we learned that one of several protocols could be used to dynamically program switches to build the cross-connect tables.  In the next article we will further explore the details and tradeoffs of the label distribution and signaling protocols.

Suggested URLs:

CD-LDP VS RSVP-TE   http://www.dataconnection.com/download/crldprsvp.pdf

George Mason University
   http://www.gmu.edu/news/release/mpls.html

Network Training
   http://www.globalknowledge.com/

MPLS Links Page
   http://www.rickgallaher.com/mplslinks.htm

MPLS Resource Center
   http://MPLSRC.COM

RSVP
   http://www.juniper.net/techcenter/techpapers/200006-08.html

Special thanks to:

I would like to thank Uyless Black, Susan Gallaher, and Amy Quinn for their assistance, reviewing, and editing.

A special thank you to all those who assisted me with information and research on the MPLSRC OP mail list, especially: Syed Ali, Adithya Bhat, Krishna Kishore, Irwin Lazar, Christopher Lewis, Vic Nowoslawski, Mario Puras, Mehdi Sif, and Geoff Zinderdine.

Rick Gallaher, CISSP, is owner of Dragonfly Associates LLC http://dragonfly-associates.com and author of  Rick Gallaher's MPLS Training Guide

Read More