Previous Section  < Free Open Study >  Next Section

Load Sharing

Equal-Cost Load Sharing

Load sharing can be done based on the packet's source and destination IP addresses or on a per-packet basis. Load sharing on the packet's source and destination addresses is often called (somewhat accurately) per-flow or per-destination load sharing, but it is more correctly called per-src-dest load sharing.

With MPLS and per-src-dest load sharing, when a packet enters the router, the packet underneath the label stack is examined. If the underlying data is an IP packet, it is load shared using the source and destination IP addresses of the underlying data, just like the router would do if it had received an unlabeled IP packet in the first place. If the packet underneath the received label stack is not an IP packet, load sharing is based on the bottommost label in the label stack.

Per-packet load sharing sends all packets in a round-robin fashion to the different next hops for a given route, without regard for the packet's contents. Per-packet load sharing is often undesirable, because the receiving end could receive packets out of order. For the following discussion, assume per-flow (also known as per-destination) load sharing. The next sections describe equal-cost load sharing between TE tunnels and IGP paths and between multiple TE paths leading to a destination.

Load Sharing Between the TE Tunnel Path and IGP Path

In the preceding section, you saw that the TE tunnel is used as the next hop as soon as you determine the cost of the shortest path to the tunnel tail. This approach has two ramifications:

  • You will never load share between an IGP route and a TE route for the tunnel tail.

  • You might load share between an IGP route and a TE route for nodes behind the tunnel tail.

Because the reasons behind these points might not be obvious, they deserve further discussion.

Load Sharing to the Tunnel Tail

Assume that you will never load share between an IGP route and a TE route for the tunnel tail. Why not? Because if you did, you'd lose the ability to explicitly route traffic down a tunnel that takes a suboptimal path.

Consider the network topology shown in Figure 5-3.

Figure 5-3. Load Sharing to a Tunnel Tail Through the Tunnel and the IGP Route

graphics/05fig03.gif

Suppose you have a TE tunnel between Router A and Router C. By default, this tunnel has a path cost of 20 because that's the IGP shortest-path cost between Routers A and C.

Two paths exist between Routers A and C, however, and they both have the same cost. There's ABC and AC. If you didn't drop the other paths in favor of the TE tunnel, you would load share across these paths, which means that you can't steer traffic around your network the way you want to.

Depending on how you did things, you would share either between the TE tunnel and the AC path (so that only 50 percent of the traffic destined for C goes down the TE tunnel) or between the TE tunnel, the AC path, and the ABC path (so that only 33 percent of the traffic destined for Router C goes down the TE tunnel).

Neither of these scenarios is desirable because it's much harder to traffic-engineer your network if you don't have complete control over all your traffic.

Suppose, however, that you wanted to send traffic across the AC path and the ABC path. To accomplish this, you would build two TE tunnels, one across each path, and then load share between them, as illustrated in Figure 5-4.

Figure 5-4. Load Sharing Across Two TE Tunnels

graphics/05fig04.gif

NOTE

You will never load share traffic to the TE tunnel tail between an IGP path and a TE tunnel path, but you will load share between multiple TE tunnels to the same tail.


When the Cisco IOS Software load shares IP packets between multiple paths to the same destination, it almost always does equal-cost load sharing between these paths. If no TE tunnels existed in 5-4, you'd share traffic in a 1:1 ratio between the top path and the bottom path from AC. However, MPLS TE provides additional flexibility when you load share. See the upcoming section "Unequal-Cost Load Sharing" for details.

NOTE

EIGRP has something called variance that lets you use a limited form of unequal-cost load sharing. However, MPLS TE's load sharing is more flexible than EIGRP's. Also, because you can't do TE using EIGRP, you will see no more mention of variance.


Load Sharing to Nodes Behind the Tunnel Tail

Although you will never share between an IGP path and a TE tunnel to get to the tunnel tail, you will sometimes share between a TE tunnel path and an IGP path to get to destinations downstream of the tunnel tail.

You can end up load sharing between an IGP route and a TE tunnel to a node behind the TE tunnel if you have multiple paths to a given node and the TE tunnel is not on one or more of these paths.

This makes more sense when you consider the network shown in Figure 5-5.

Figure 5-5. Load Sharing to Nodes Behind a Tunnel Tail

graphics/05fig05.gif

The following text walks you through all of Router A's SPF calculations. The gist of it is that if you find multiple paths to a given node, and that node is not the tunnel tail, you load share between these paths up to the max-paths limit in your IGP. Some of the "Move one node to the PATH list and then move its neighbors to the TENT list" stuff has been condensed into one step because by now you are either well-acquainted with how SPF works, or you're ignoring the SPF examples altogether because they don't make any sense!

Router A has a TE tunnel to Router C, across the top path.

As always, you start with Router A in the PATH list, as follows:

PATH List TENT List
{A,0,0} {B,10,B}
  {C,20,C}

Move Router B to the PATH list, and put its neighbors on TENT, with the following results:

PATH List TENT List
{A,0,0} {C,20,C}
{B,10,B} {D,30,B}

Move Router C to the PATH list, change the next hop to Router C to Tunnel0, and put Router C's neighbors on TENT, with the following results:

PATH List TENT List
{A,0,0} {D,30,B}
{B,10,B} {D,30,Tunnel0}
{C,20,Tunnel0}  

At this point, there are two paths to Router D—one through Router B and one through Tunnel0. If two paths exist with the same cost in the TENT list, you use them both; this rule is the same whether you're doing equal-cost forwarding with IP or with MPLS. So, leave both entries for Router D in the TENT list.

Next, move Router D to the PATH list. Router D has no new neighbors, so you don't add anything to the TENT list, as indicated in this table:

PATH List TENT List
{A,0,0} <empty>
{B,10,B}  
{C,20,Tunnel0}  
{D,30,B}  
{D,30,Tunnel0}  

Table 5-4 shows the routing table resulting from this SPF process.

Table 5-4. Router A's Routing Table After IGP SPF
Node Next Hop Cost
A Self 0
B B 10
C Tunnel0 20
D Tunnel0 30
  B  

That's how you can load share between TE tunnels and IGP paths to a destination behind a tunnel tail. Load sharing is equal-cost; you cannot get unequal-cost load sharing between a TE tunnel and a non-TE tunnel path. The next section tells you why.

Unequal-Cost Load Sharing

Normal IP routing has equal-cost load sharing. If there's more than one path between two nodes, and those two paths have equal cost, traffic is shared equally between those paths. Routing protocols currently are limited to installing either six or eight maximum parallel paths, depending on the level of code you're running.

Equal-cost load sharing is all well and good, but it's not very flexible. If you've ever tried to balance traffic on your network by changing link costs, you're probably aware that it's not as easy as you might think. Changing a link cost in one corner of your network often affects traffic in various other parts of your network in ways you didn't anticipate.

There's also the issue of unequal-cost load sharing. This is difficult to do while guaranteeing a loop-free topology. Consider the network topology shown in Figure 5-6.

Figure 5-6. Unequal-Cost Load Sharing with IP

graphics/05fig06.gif

The following example shows what can happen if a router is allowed to compute arbitrary non-shortest paths to send traffic over.

This example ignores many of the implementation-specific details of CEF. Those come later in this chapter, in the discussion of how load sharing works in real life.

Assume that unequal-cost paths are calculated based on path cost, with the amount of traffic forwarded down a particular path being inversely proportional to the cost of the path—the lower the path cost, the more traffic is forwarded down that path.

For example, if a router has three possible paths to a destination—two with a cost of 20 and one with a cost of 40—traffic is shared between these paths in a 40:40:20 ratio. This means that 40 percent of traffic to a destination goes down the first path, 40 percent goes down the second, and 20 percent goes down the third.

In Figure 5-6, Router A sends traffic roughly as shown in Table 5-5 and Table 5-6.

Table 5-5. Router A's Routing Table
Path Cost Amount of Traffic
AC 20 40%
AB(link1)C 20 40%
AB(link2)C 40 20%

Table 5-6. Router B's Routing Table
Path Cost Amount of Traffic
B(link 1)C 10 60%
B(link 2)C 30 20%
BAC 30 20%

This example becomes more concrete when you add real traffic to the mix. Suppose that Router A has 100 Mbps of traffic to send to Router C. Forty Mbps of this 100 Mbps goes through path AC and gets where it needs to go. The remaining 60 Mbps is given to Router B, with Router A assuming that Router B forwards the traffic along the path that Router A wants the traffic to follow.

But Router B has other ideas. Of the 60 Mbps received from Router A, Router B forwards 80 percent of that (approximately 48 Mb) over either link 1 or link 2, and the traffic arrives at Router C. But Router B then forwards the remaining 20 percent (approximately 12 Mb) back to Router A!

What happens next depends on the load sharing that Routers A and B are doing. If both routers make their packet forwarding decision based on some combination of source and destination IP addresses, the aforementioned 12 Mb of traffic loops indefinitely between Router A and Router B until its TTL times out.

If both routers are doing per-packet load sharing, odds are good that most of the traffic will eventually get to Router C, although the route it takes to get there might be time-consuming. 12 Mb goes from Router B to Router A, Router A sends 60 percent of this (approximately 7.2 Mb) back to Router B, Router B sends 20 percent of this (approximately 1.4 Mb) back to Router A, and so on until no more traffic is left.

This suboptimal situation occurs because Router A couldn't tell Router B what to do with the packet. Router A must assume that Router B shares in a way that makes sense for Router A, and Router B has other ideas.

What Router A needs is some way to identify a path that traffic needs to follow. Router A needs to be able to tell Router B which traffic should be forwarded across link 1 and which traffic should go across link 2, without giving Router B the chance to do an IP lookup on the packet and do something other than what Router A wants. Some sort of tag or label is needed to indicate the direction in which the traffic should flow.

This is why MPLS TE is so beneficial. Although the MPLS TE configuration model is currently less dynamic than the model in the preceding example, the important point is that it doesn't matter how Router A decides to send traffic. If a label is applied to a packet going from Router A to Router B, and that label came from an RSVP LSP setup, the traffic is forwarded to Router C without fear of potential looping at Router B.

This example might seem a bit contrived. Load sharing can be set up in this picture so that no loops occur. But the point is that you can't guarantee that a next hop will be as intelligent as you'd like it to be, so the odds of a loop are high.

How Unequal-Cost Load Sharing Works

Now that you've seen how unequal-cost load sharing is more feasible with MPLS than with IP, you need to know how it works.

It's simple, actually. It has two parts:

  1. Setting up the load-sharing ratios

  2. Installing these ratios in the FIB for CEF to use

MPLS TE load sharing works between multiple tunnels to the same destination. If there is more than one tunnel to a destination, you share between those tunnels. You decide what unequal-cost ratios to use through one of the following:

  • Bandwidth (configured with tunnel mpls traffic-eng bandwidth 0-4294967295)

  • Load-share value (configured with tunnel mpls traffic-eng load-share 0-1000000)

If the load-share value is configured, it is used as the ratio with which to share. Otherwise, you share traffic between multiple parallel tunnels in accordance with the configured bandwidth. Traffic share is directly proportional to bandwidth—the more bandwidth a tunnel has, the more traffic it receives. The load-share value works the same way. It's just a knob that lets you reserve the same bandwidth on multiple tunnels but share differently between them. For the following discussions, bandwidth refers to tunnel bandwidth or load-share value. In other words, the two tunnel configurations in Example 5-3 are treated exactly the same as far as load sharing goes; the only difference is that in the load-sharing case, no bandwidth is reserved in the network.

Example 5-3 Tunnel Configurations withbandwidth and load-share Values Allocated

interface Tunnel1

 ip unnumbered Loopback0

 no ip directed-broadcast

 tunnel destination 192.168.1.8

 tunnel mode mpls traffic-eng

 tunnel mpls traffic-eng bandwidth 20

 tunnel mpls traffic-eng path-option 10 dynamic

____________________________________________________________________________

interface Tunnel1

 ip unnumbered Loopback0

 no ip directed-broadcast

 tunnel destination 192.168.1.8

 tunnel mode mpls traffic-eng

 tunnel mpls traffic-eng path-option 10 dynamic

 tunnel mpls traffic-eng load-share 20

If you like, you can configure both tunnel mpls traffic-eng bandwidth and tunnel mpls traffic-eng load-share. The configured bandwidth value is reserved through RSVP, and the configured load-share value is used for load sharing on the tunnel headend.

Consider the network topology shown in Figure 5-7.

Figure 5-7. Unequal-Cost Load Sharing with MPLS TE

graphics/05fig07.gif

This is the same topology as shown in Figure 5-6, but with TE tunnels set across the three paths from Router A to Router C. Table 5-7 shows the bandwidth and paths for each of the three tunnels in Figure 5-7.

Table 5-7. Tunnel Bandwidth/Path Information
Tunnel Configured Bandwidth Path
Tunnel1 20 AB(link1)C
Tunnel2 20 AC
Tunnel3 40 AB(link2)C

NOTE

It is important to understand that the amount of bandwidth you reserve for a tunnel is purely a control plane number. TE deals with reservations at the control plane only. No policing, fancy queuing, or shaping are established on an interface as a result of a TE bandwidth reservation. This means that, no matter how much traffic is sent from Router A to Router C, it is shared correctly between all three tunnels. You could have just as easily set the tunnel bandwidths to 2, 2, and 4, or 90, 90, and 180; it makes no difference.


Example 5-4 shows the routing table with autoroute enabled on these tunnels.

Example 5-4 Unequal-Cost Load Sharing with MPLS TE Tunnels

RouterA#show ip route 192.168.1.11 

Routing entry for 192.168.1.11/32

  Known via "isis", distance 115, metric 40, type level-2

  Redistributing via isis

  Last update from 192.168.1.11 on Tunnel13, 00:01:55 ago

  Routing Descriptor Blocks:

  * 192.168.1.11, from 192.168.1.11, via Tunnel11

      Route metric is 40, traffic share count is 1

    192.168.1.11, from 192.168.1.11, via Tunnel12

      Route metric is 40, traffic share count is 1

    192.168.1.11, from 192.168.1.11, via Tunnel13

      Route metric is 40, traffic share count is 2

Even though the bandwidths are configured to share in the ratio 20:20:40, the routing table has boiled these down to 1:1:2. The routing table follows two rules when generating these traffic share counts:

  1. Reduce until the lowest metric is 1.

  2. Allow only whole numbers as traffic share counts.

The actual algorithm is more complex than that, but that's the basic idea.

A consequence of this algorithm is that sometimes bandwidth ratios aren't preserved exactly as they're inserted into the routing table. What if the bandwidth ratios were 20:20:30 instead of 20:20:40? The routing table would look the same as it does in Example 5-4—traffic share counts of 1, 1, and 2. Why? Because in 20:20:30, the steps are as follows:

  1. Reduce 20 to 1.

  2. Realize that 30:20 is 1.5:1, which isn't allowed.

  3. Round 1.5 up to 2.

  4. End up with traffic share counts of 1:1:2.

This has a subtle impact on your load sharing. If you have a tunnel with a bandwidth of 7 and a tunnel with a bandwidth of 9, 7:9 is reduced to 1:1 as it enters the RIB.

The routing table hands the traffic share count values down to CEF, which is the mechanism that actually does all the load sharing.

Chapter 2, "MPLS Forwarding Basics," covered the basics of how CEF works. When CEF is load sharing, it keeps an array of 16 possible next hops, in a data structure called a CEF picks one of these loadinfo entries. CEF picks the loadinfo entry based on a combination of the source and destination IP addresses, so the default load sharing has no out-of-order packets. CEF can also be configured to load share per packet rather than in a source/destination hash. See Appendix B,"CCO and Other References," for more information on CEF and its load sharing.

The show ip cef exact-route command is helpful in that it lets you give CEF a source and destination IP address pair, and it tells you which way the router switches the packet. With a routing table like the one shown in Example 5-4, several routes exist for 192.168.1.11. Example 5-5 shows the output from show ip cef exact-route src dst, which shows you which way CEF switches packets of a given source/destination pair.

Example 5-5 show ip cef exact-route Command Output

RouterA#show ip cef exact-route 1.2.3.4 192.168.1.11

1.2.3.4         -> 192.168.1.11   : Tunnel11 (next hop 192.168.1.11)

RouterA#show ip cef exact-route 1.2.3.5 192.168.1.11   

1.2.3.5         -> 192.168.1.11   : Tunnel11 (next hop 192.168.1.11)

RouterA#show ip cef exact-route 1.2.3.6 192.168.1.11   

1.2.3.6         -> 192.168.1.11   : Tunnel12 (next hop 192.168.1.11)

RouterA#show ip cef exact-route 1.2.3.4 192.168.1.11   

1.2.3.4         -> 192.168.1.11   : Tunnel11 (next hop 192.168.1.11)

Example 5-5 shows which path packets from three different sources (1.2.3.4, 1.2.3.5, and 1.2.3.6) take to get to 192.168.1.11. Example 5-5 shows the output for source 1.2.3.4 twice to emphasize that the same source/destination pair always takes the same interface, as long as all these tunnel interfaces are up.

You can see the load-sharing information by looking at the output of show ip cef destination ip address internal, as demonstrated in Example 5-6.

Example 5-6 show ip cef Command Output

RouterA#show ip cef 192.168.1.11 internal

192.168.1.11/32, version 80, per-destination sharing

...

  Load distribution: 0 1 2 0 1 2 0 1 2 0 1 2 0 0 0 0 (refcount 2)



  Hash  OK  Interface                 Address         Packets  Tags imposed

  1     Y   Tunnel13                  point2point           0    {50}

  2     Y   Tunnel11                  point2point           0    {49}

  3     Y   Tunnel12                  point2point           0    {48}

  4     Y   Tunnel13                  point2point           0    {50}

  5     Y   Tunnel11                  point2point           0    {49}

  6     Y   Tunnel12                  point2point           0    {48}

  7     Y   Tunnel13                  point2point           0    {50}

  8     Y   Tunnel11                  point2point           0    {49}

  9     Y   Tunnel12                  point2point           0    {48}

  10    Y   Tunnel13                  point2point           0    {50}

  11    Y   Tunnel11                  point2point           0    {49}

  12    Y   Tunnel12                  point2point           0    {48}

  13    Y   Tunnel13                  point2point           0    {50}

  14    Y   Tunnel13                  point2point           0    {50}

  15    Y   Tunnel13                  point2point           0    {50}

  16    Y   Tunnel13                  point2point           0    {50}

The output in Example 5-6 shows the load sharing for CEF, populated in an 8:4:4 ratio. How do you know it's an 8:4:4 ratio? If you count all the occurrences of each tunnel in Example 5-6, Tunnel13 shows up eight times, and Tunnel11 and Tunnel12 each show up four times.

Unequal-cost load sharing works only when all the paths you're sharing between have a nonzero bandwidth.

Attempting to load share in the following situations forces reversion to equal-cost load sharing between these paths:

  • Between a TE tunnel with nonzero bandwidth and a TE tunnel with zero bandwidth

  • Between a TE tunnel and an IGP path (for load sharing to a node behind a TE tunnel tail)

This reversion occurs because in order to figure out the proper percentages to load share among n paths, you have to do something like the following:

  1. Add up all the traffic-share counts to get the value T (in the preceding example, T = 1 + 1 + 2, or 4).

  2. Divide each tunnel's traffic share count by the total tunnel traffic share count to get the percentage of load sharing each tunnel should receive. If you have a zero-bandwidth tunnel, this means you end up doing something like 0 / 4, which is 0.

  3. Convert the percentage to 16ths by multiplying by 16. 16 * 0 is, as you know, 0.

This means that if you try to follow the load-sharing algorithm and include a path with a zero-traffic-share count, you'll have a path that you never end up using. The logic is that if there's a zero-traffic-share path anywhere in the set of paths to a given destination, you revert to equal-cost load sharing. So, if you change the sample network shown in Figure 5-7 to have four tunnels to 192.168.1.11, but you give the fourth tunnel (Tunnel14) zero bandwidth, the route for 192.168.1.11 looks like that shown in Example 5-7.

Example 5-7 Router A's Route for 192.168.1.11 Sharing Across Four Tunnels, One with Zero Bandwidth

RouterA#show ip route 192.168.1.11

Routing entry for 192.168.1.11/32

  Known via "isis", distance 115, metric 40, type level-2

  Redistributing via isis

  Last update from 192.168.1.11 on Tunnel14, 00:00:12 ago

  Routing Descriptor Blocks:

  * 192.168.1.11, from 192.168.1.11, via Tunnel13

      Route metric is 40, traffic share count is 1

    192.168.1.11, from 192.168.1.11, via Tunnel12

      Route metric is 40, traffic share count is 1

    192.168.1.11, from 192.168.1.11, via Tunnel11

      Route metric is 40, traffic share count is 1

    192.168.1.11, from 192.168.1.11, via Tunnel14

      Route metric is 40, traffic share count is 1

The output in Example 5-8 shows what the CEF loadinfo looks like.

Example 5-8 Router A's CEF Entry for 192.168.1.11 Sharing Across Four Tunnels, One with Zero Bandwidth

RouterA#show ip cef 192.168.1.11 internal

...

  Load distribution: 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 (refcount 4)

  Hash  OK  Interface                 Address         Packets  Tags imposed

  1     Y   Tunnel13                  point2point           0    {22}

  2     Y   Tunnel12                  point2point           0    {21}

  3     Y   Tunnel11                  point2point           0    {20}

  4     Y   Tunnel14                  point2point           0    {19}

  5     Y   Tunnel13                  point2point           0    {22}

  6     Y   Tunnel12                  point2point           0    {21}

  7     Y   Tunnel11                  point2point           0    {20}

  8     Y   Tunnel14                  point2point           0    {19}

  9     Y   Tunnel13                  point2point           0    {22}

  10    Y   Tunnel12                  point2point           0    {21}

  11    Y   Tunnel11                  point2point           0    {20}

  12    Y   Tunnel14                  point2point           0    {19}

  13    Y   Tunnel13                  point2point           0    {22}

  14    Y   Tunnel12                  point2point           0    {21}

  15    Y   Tunnel11                  point2point           0    {20}

  16    Y   Tunnel14                  point2point           0    {19}

Because one tunnel has zero bandwidth, all tunnels now share equally between each other rather than in proportion to any bandwidths.

The IGP path is treated as though it has a bandwidth of 0, because you don't know the amount of available bandwidth along the path, just the path cost. In order to take advantage of unequal-cost load sharing, the following must be true:

  • All paths to a destination have to be TE tunnels.

  • All these paths have to have a nonzero bandwidth (or nonzero load-share metric).

How to Play with the TE Tunnel Metric

As mentioned earlier in the autoroute SPF example, the metric assigned to a tunnel interface when that interface is used in the IGP SPF is the shortest-cost path the IGP takes to get to the tunnel tail. Any routes behind the tunnel (such as the routes to Routers H and I in Figure 5-2) have the cost to transit the tunnel plus the cost to get from the tunnel tail to that node—just like your IGP.

However, you might want to change the metric. You can do so using the tunnel mpls traffic-eng autoroute metric command, as demonstrated in Example 5-9.

Example 5-9 Changing the Autoroute Metric

vxr15(config-if)#tunnel mpls traffic-eng autoroute metric ?

  <1-4294967295>  Set tunnel metric for autoroutes

  absolute        Set metric for all autoroutes over tunnel

  relative        Adjust tunnel metric for autoroutes relative to IGP

Changing the tunnel metric with the tunnel mpls traffic-eng autoroute metric command influences only the tunnel headend; other routers in the network don't know about the changed metric. You might want to advertise the tunnel into the IGP in some situations; see the later section "Forwarding Adjacency."

You would want to change the tunnel metric mainly if you want the IGP shortest-cost route to be preferred over the TE tunnel routes in some cases. Changing the metric isn't all that common, but it's worth understanding.

Fixing the Tunnel Metric

The first way you can tweak the tunnel metric is with the command tunnel mpls traffic-eng autoroute metric 1-4294967295. Doing this overrides the IGP shortest-path cost and uses the configured cost instead. Setting this number influences the cost assigned to the tunnel interface in the IGP SPF, and thus the routing.

You typically do this if you want to use a TE tunnel only if the IGP shortest path is unavailable. Consider the network shown in Figure 5-8.

Figure 5-8. Setting the Tunnel Metric to 25

graphics/05fig08.gif

Based on Figure 5-2, Figure 5-8 adds Routers J, K, and L in the middle. Again, all links have a cost of 10.

For the sake of simplicity, Table 5-8 lists the three different paths Router A has to Router E.

Table 5-8. Router A's Paths to Router E
Path Name Cost
ABE Top path 20
AJKLE Middle path 40
ACDE Bottom path 30

By default, if the top path goes away, the next-best path to Router E is the bottom path. Assume that instead you want to use the middle path (cost: 40) if the top path goes away. You can do this by building a TE tunnel across the middle path and setting its metric to 25 with the command tunnel mpls traffic-eng autoroute metric 25. Table 5-9 lists the new path costs after you change this metric.

Table 5-9. New Path Costs from Router A to Router E After Changing the Tunnel Metric and Path
Path Name Cost
ABE Top path 20
AJKLE Middle path with a TE tunnel installed 25
ACDE Bottom path 30

If the top (IGP) path exists, it wins because the top path cost is the lowest at 20. However, if that goes away for some reason, the TE tunnel with a cost of 25 beats the bottom path (cost of 30) into the routing table. Because the TE tunnel terminates on Router E, Router A's cost is the cost to reach Router E (25, if the tunnel is used) plus 10 (the cost of EG), or 35.

In all these examples, the cost to nodes behind the TE tunnel tail is increased by the cost to get from the tunnel tail to these nodes. So, if the middle path is used for the TE tunnel, and it has a cost of 25, the cost to get to Router F is 25 + 10, or 35. The cost to Router G is also 35, and the cost to Routers H and I is 45.

Absolute Tunnel Metric

Assume that you have a network topology that looks like Figure 5-9.

Figure 5-9. Network Topology to Use with Absolute Metrics

graphics/05fig09.gif

Suppose that you configure a TE tunnel from Router A to Router E with tunnel mpls traffic-eng autoroute metric 17.

Table 5-10 shows the resulting routing table for Router A after this configuration.

Table 5-10. Router A's Routing Table After the TE Tunnel from Router A to Router E Is Configured with a Metric of 17
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 17
F Tunnel0 27
G Tunnel0 27
H Tunnel0 37
I Tunnel0 37

Another thing you can do is set the absolute tunnel metric, which assigns the same cost to the tunnel and all nodes behind the tunnel. Table 5-11 shows Router A's routing table if you configure tunnel mpls traffic-eng autoroute metric absolute 17 on Tunnel0.

Only IS-IS supports tunnel mpls traffic-eng autoroute metric absolute; if you configure absolute metrics on an OSPF network, it's just ignored.

Table 5-11. Router A's Routing Table After Tunnel0 Is Configured with an Absolute Metric of 17
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 17
F Tunnel0 17
G Tunnel0 17
H Tunnel0 17
I Tunnel0 17

Assigning an absolute metric is one way around the problem of sharing between IGP paths and TE paths. If every node behind the tunnel tail has the same cost as the tunnel tail, no matter what the topology looks like behind the tunnel tail, you'll always use the tunnel to forward traffic, no matter how many back doors exist. As long as the absolute cost on the tunnel is equal to or lower than the IGP cost to the tunnel tail, the tunnel carries all traffic for or behind the node.

Relative Tunnel Metric

The relative metric is yet another tool in the tunnel metric toolbox. Rather than setting the tunnel metric to a particular cost, you can set the tunnel metric relative to the IGP cost that the tunnel would normally have.

The CLI command to do this is as follows:


tunnel mpls traffic-eng autoroute metric relative    -10 - 10

That is, tunnel mpls traffic-eng autoroute metric relative X, where X is any integer from –10 to 10.

Table 5-12 shows Router A's routing table if you set tunnel mpls traffic-eng autoroute metric relative -8 on the tunnel interface on Router A's Tunnel0 in 5-9.

Table 5-12. Router A's Routing Table After Tunnel0 Is Configured with a Relative Metric of –8
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 12
F Tunnel0 22
G Tunnel0 22
H Tunnel0 32
I Tunnel0 32

The cost to get to Router E through Tunnel0 is 12 because the IGP shortest-cost path to get to E is 20, and 20–8 (or 20 + –8, if you want to be precise) is 12.

If Router B goes away, the IGP shortest path to Router E becomes 30 (ACDE). Configuring tunnel mpls traffic-eng autoroute metric relative -8 means that the tunnel to Router E has a cost of 22, and all nodes behind Router E inherit this cost as they are accounted for in the IGP SPF.

Conversely, if you use a positive offset (tunnel mpls traffic-eng autoroute metric relative 8), the tunnel gets installed with a metric of 28 (ABE has a cost of 20+8 for the tunnel offset). If Router B goes away, the tunnel is given a cost of 38.

Applying Negative Offsets to Tunnel Costs

If you try to apply a negative offset to a TE tunnel path, and that offset would make the cost to get to the TE tunnel tail negative, the cost to the TE tunnel tail is instead set to 1. So if the IGP shortest cost to get to Router E is 6, and you apply tunnel mpls traffic-eng autoroute metric relative -8, the cost used for the AE tunnel is 1 rather than –2.

From this discussion, you might note that, if a tunnel is given a positive offset metric, it is never installed in the routing table. So there's not any reason to use the positive offset.

Just like setting the TE tunnel to a different cost or an absolute cost, setting a negative relative metric allows you to set a preferred TE tunnel path in some cases when you wouldn't otherwise.

To better clarify this concept, read through the following example of how the different metric types work. Figure 5-10 shows a single tunnel and the routing tables if different techniques are used to change the metric. All link costs are 10, and there's a TE tunnel from Router A to Router E.

Figure 5-10. Network Topology to Demonstrate Different Metric Types

graphics/05fig10.gif

Table 5-13 shows Router A's routing table with no special metrics applied.

Table 5-13. Router A's Routing Table with No Metric Changes
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 20
F Tunnel0 30
G Tunnel0 30
H Tunnel0 40
I Tunnel0 40

Table 5-14 shows Router A's routing table after tunnel mpls traffic-eng autoroute metric 12 is applied to Tunnel0.

Table 5-14. Router A's Routing Table After Tunnel0 Is Configured with an Autoroute Metric of 12
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 12
F Tunnel0 22
G Tunnel0 22
H Tunnel0 32
I Tunnel0 32

Table 5-15 shows Router A's routing table after tunnel mpls traffic-eng autoroute metric absolute 4 is applied to Tunnel0 instead.

Table 5-15. Router A's Routing Table After Tunnel0 Is Configured with an Autoroute Absolute Metric of 4
Node Next Hop Cost
A Self 0
B B 10
C C 10
D C 20
E Tunnel0 4
F Tunnel0 4
G Tunnel0 4
H Tunnel0 4
I Tunnel0 4

Finally, Table 5-16 shows Router A's routing table after tunnel mpls traffic-eng autoroute metric relative-3 is applied to Tunnel0.

Table 5-16. Router A's Routing Table After Tunnel0 Is Configured with an Autoroute Relative Metric of –3
Node Next Hop Cost
A Self 0
B B 7
C C 7
D C 17
E Tunnel0 17
F Tunnel0 27
G Tunnel0 27
H Tunnel0 37
I Tunnel0 37

As with most knobs in the Cisco IOS Software, don't play with this one unless you know that you must. Most of the time, leaving the metrics at their defaults (where they track the IGP shortest cost) is fine.

How Changing the Metric Works

No matter which mechanism you use to change the metric, they all work the same in SPF. The key thing to remember is that any metrics are changed after SPF is run. Figure 5-11 shows a scenario in which changing the metric after SPF, rather than before, makes a difference.

Figure 5-11. Sample Network for the SPF Processing Example

graphics/05fig11.gif

Links AB and AC have a cost of 30; all other links have a cost of 10. The tunnel from Router A to Router E has been configured with tunnel mpls traffic-eng autoroute metric 1. This means that the cost to get to Router E is 1, the cost to get to Router F is 11, and the cost to get to Router G is also 11.

But what about the costs to Routers B, C, and D? If the tunnel from Router A to Router E had a metric of 1 when the tunnel was put in the TENT list, the SPF would look something like this:

PATH List TENT List
{A,0,0} {E,1,Tunnel0}
  {B,30,B}
  {C,30,C}

PATH List TENT List
{A,0,0} {D,11,Tunnel0}
{E,1,Tunnel0} {F,11,Tunnel0}
  {G,11,Tunnel0}
  {B,30,B}
  {C,30,C}

In the next table, {B,30,B} and {C,30,C} are removed from the TENT list because, as far as SPF is concerned, {B,21,Tunnel0} and {C,21,Tunnel0} are the shortest paths to Routers B and C.

PATH List TENT List
{A,0,0} {F,11,Tunnel0}
{E,1,Tunnel0} {G,11,Tunnel0}
{D,11,Tunnel0} {B,21,Tunnel0}
  {C,21,Tunnel0}
  {B,30,B}
  {C,30,C}

PATH List TENT List
{A,0,0} <Empty>
{E,1,Tunnel0}  
{D,11,Tunnel0}  
{B,21,Tunnel0}  
{C,21,Tunnel0}  
{F,11,Tunnel0}  
{G,11,Tunnel0}  

Table 5-17 shows the resulting routing table.

Table 5-17. Router A's Routing Table After Tunnel0 Is Configured with an Autoroute Metric of 1 if Metric Modification Happened Before SPF
Node Next Hop Cost
A Self 0
E Tunnel0 1
D Tunnel0 11
F Tunnel0 11
G Tunnel0 11
B Tunnel0 21
C Tunnel0 21

This is extremely suboptimal. To get from Router A to Router B, the packets go across Router A's Tunnel0 to Router E, back to Router D, and then to Router B. This is a physical path of ABDEDB. Nodes B and D show up twice in the path—once carrying the TE tunnel, and once carrying the IP packet after it reaches Router E.

Because you can give the tunnel a cost of less than the physical path to get to the tunnel tail, putting the tunnel metric in the SPF means you can easily have problems like this. Just think about how much worse it could become on a larger network if you used tunnel mpls traffic-eng autoroute metric absolute 1instead! If you did that, every node more than halfway across the network would be suboptimally routed down the TE tunnel instead of on the physical path.

It's important to realize that these problems can happen in real life. In order to avoid these problems, metrics are changed after the SPF run is complete. This means that you don't end up with extremely silly routes like those shown in Table 5-17. It also means that changing the tunnel metric doesn't influence what routes are installed through the tunnel, only the cost to get to those routes. The SPF run for the network in Figure 5-11 actually looks like this:

PATH List TENT List
{A,0,0} {B,30,B}
  {C,30,C}

PATH List TENT List
{A,0,0} {C,30,C}
{B,30,B} {D,40,B}

PATH List TENT List
{A,0,0} {D,40,B & C}
{B,30,B}  
{C,30,C}  

PATH List TENT List
{A,0,0} {E,50,B & C}
{B,30,B}  
{C,30,C}  
{D,40,B & C}  

PATH List TENT List
{A,0,0} {F,60,Tunnel0}
{B,30,B} {G,60,Tunnel0}
{C,30,C}  
{D,40,B & C}  
{E,50,Tunnel0}  

PATH List TENT List
{A,0,0} <Empty>
{B,30,B}  
{C,30,C}  
{D,40,B & C}  
{E,50,Tunnel0}  
{F,60,Tunnel0}  
{G,60,Tunnel0}  

Then the routing table is built, and any metric changes are made. With a configuration of tunnel mpls traffic-eng autoroute metric 1, the routing table looks like Table 5-18.

Table 5-18. Router A's Routing Table After Tunnel0 Is Configured with an Autoroute Metric of 1
Node Next Hop Cost
A Self 0
B B 30
C C 30
D B,C 40
E Tunnel0 1
F Tunnel0 11
G Tunnel0 11

As you can see, this is far more sensible than the previous example.

The final thing to consider about SPF metric processing is that it might not always do what you want. Consider the network topology shown in Figure 5-12.

Figure 5-12. Sample Topology for Autoroute Metric

graphics/05fig12.gif

In Figure 5-12, the AB link has a cost of 30, and the AC link has a cost of 20. Without any TE tunnels, the routing table for this network looks like Table 5-19.

Table 5-19. Routing Table Without TE Tunnels
Node Next Hop Cost
A Self 0
B B 30
C C 20
D C 30
E C 40

Table 5-20 shows what you might expect the routing table to look like if you add a TE tunnel between Routers A and B and configure it with tunnel mpls traffic-eng autoroute metric 1.

Table 5-20. Expected Routing Table After a TE Tunnel Is Configured Between Routers A and B with a Metric of 1
Node Next Hop Cost
A Self 0
B Tunnel0 1
C C 20
D Tunnel0 11
E Tunnel0 21

But this is not what happens. The SPF looks like this:

PATH List TENT List

{A,0,0}

{B,30,B}

{C,20,C}

{A,0,0}

{C,20,C}

{B,30,B}

{D,30,C}

{A,0,0}

{C,20,C}

{B,30,Tunnel0}

{D,30,C}

{A,0,0}

{C,20,C}

{B,30,Tunnel0}

{D,30,C}

{E,40,C}

After the SPF process, the routing table looks like Table 5-21.

Routers D and E can't be reached through the TE tunnel to Router B because when the IGP SPF is finished, the TE tunnel from Routers A to B has a cost of 30, not 1, so the shortest paths to Routers D and E are still through Router C. Remember, it's only after the SPF is done that any autoroute metric modifications are applied.

Table 5-21. Routing Table After the SPF Process
Node Next Hop Cost
A Self 0
B Tunnel0 1
C C 20
D C 30
E C 40

The moral of the story is that you should follow a few rules when deploying MPLS TE, including

  • Don't change tunnel metrics unless you know you need to.

  • Become familiar with how SPF works.

  • If routing isn't doing what you expect it to, work it out by hand and see if you can figure out what's happening.

    Previous Section  < Free Open Study >  Next Section