|
|
< Free Open Study > |
|
Finding the Root Cause of the ProblemThe section "Common Configuration Mistakes" discussed the effects of configuring certain things incorrectly. The section "Tools for Troubleshooting MPLS TE Problems" provided a quick overview of commands that are useful for troubleshooting MPLS TE problems. Sometimes, the problem's not so easy to spot. This section goes into detail about what you should look for when something goes wrong, how to look for it, and what you can do to fix it. If you have an MPLS TE problem, you generally start looking at the problem from the headend of the TE tunnel. From the headend, you find only two possible categories of problems—your tunnel is either down or up. MPLS TE problems can broadly be broken up into the following:
If your tunnel is down, you need to focus on the section "Tunnel-Down Problems"; if your tunnel is up, you need to focus on the "Tunnel-Up Problems" section. Generally speaking, the most interesting MPLS TE problems are the ones in which the tunnel doesn't come up. If the tunnel's up and things still aren't working, you probably have either a routing problem or a forwarding problem. Tunnel-Down Problems"My TE tunnel won't come up" is the most common MPLS TE problem. Of course, if the tunnel does not come up, no traffic can be forwarded down that tunnel. There are three basic reasons why a TE tunnel won't come up:
The steps for fixing a tunnel-down problem are shown in Figure 11-3. Figure 11-3. MPLS TE Troubleshooting Flow Chart
Tunnel Admin DownExample 11-13 shows how you can determine if the tunnel is administratively shut down and then fix it. Example 11-13 Admin Down Tunnel7200a#show ip interface brief | include Tunnel1 Tunnel1 4.4.4.4 YES unset administratively down down 7200a#show mpls traffic-eng tunnels brief Signalling Summary: LSP Tunnels Process: running RSVP Process: running Forwarding: enabled Periodic reoptimization: every 3600 seconds, next in 137 seconds Periodic FRR Promotion: every 300 seconds, next in 137 seconds Periodic auto-bw collection: disabled TUNNEL NAME DESTINATION UP IF DOWN IF STATE/PROT Primary tunnel 7200a->12008a... 12.12.12.12 - unknown admin-down Displayed 1 (of 1) heads, 0 (of 0) midpoints, 0 (of 0) tails 7200a#show mpls traffic-eng tunnels tunnel 1 | include Admin Admin: admin-down Oper: down Path: not valid Signalling: Down Then you unshut the tunnel. If the only reason the tunnel was down was because it was administratively shut down, the tunnel comes up. As you can see in Example 11-14, the tunnel on 7200a comes up when you use no shutdown on the tunnel interface. Example 11-14 Doing no shutdown on the Tunnel1 Interface of 7200a Brings the Tunnel Up7200a#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 7200a(config)#interface Tunnel 1 7200a(config-if)#no shutdown 7200a(config-if)#end 23:53:27: %SYS-5-CONFIG_I: Configured from console by console 23:53:28: %LINK-3-UPDOWN: Interface Tunnel1, changed state to up 23:53:29: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up Remember that TE tunnels are treated much like any other interface on the router—if you shut them down, they won't be used. PCALC ProblemsIf the tunnel is not administratively shut down but is still not up, the next thing to check for is to see if the path is valid. You can again use the show mpls traffic-eng tunnel [tunnel] command to see if the path is valid. Example 11-15 shows the output of show mpls traffic-eng tunnel tunnel1 under normal circumstances. Example 11-15 show mpls traffic-eng tunnel tunnel1 on 7200a Under Normal Circumstances7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: up Path: valid Signalling: connected path option 5, type explicit primary (Basis for Setup, path weight 3) path option 6, type dynamic Config Parameters: Bandwidth: 100 kbps (Global) Priority: 7 7 Affinity: 0x0/0xFFFF Metric Type: TE (default) AutoRoute: enabled LockDown: disabled Loadshare: 100 bw-based auto-bw: disabled InLabel : - OutLabel : POS3/0, 12305 RSVP Signalling Info: Src 4.4.4.4, Dst 12.12.12.12, Tun_Id 1, Tun_Instance 260 RSVP Path Info: My Address: 4.4.4.4 Explicit Route: 10.0.3.5 10.0.5.11 10.0.17.12 12.12.12.12 Record Route: NONE Tspec: ave rate=100 kbits, burst=1000 bytes, peak rate=100 kbits RSVP Resv Info: Record Route: NONE Fspec: ave rate=100 kbits, burst=1000 bytes, peak rate=Inf Shortest Unconstrained Path Info: Path Weight: 3 (TE) Explicit Route: 10.0.3.5 10.0.5.11 10.0.17.12 12.12.12.12 History: Tunnel: Time since created: 1 days, 1 minutes Time since path change: 1 minutes, 9 seconds Current LSP: Uptime: 1 minutes, 9 seconds Prior LSP: ID: path option 5 [259] Removal Trigger: tunnel shutdown In this case, the path is valid. The text at the bottom of the output is highlighted to call your attention to the History section of the output. The History section is a good place to start looking if path is invalid. As you can see, it has the reason why the tunnel was last removed—because of tunnel shutdown. You'll find several cases that might cause the path to be invalid:
NOTE Constraints are things configured on the headend that can cause the TE tunnel path to diverge from the IGP shortest path. Specifically, the available TE constraints are bandwidth, affinity/link attributes, priority, and path option. An unconstrained path is the path that the TE tunnel would take with no constraints—essentially, the IGP path. Case A: Removing Some Constraints Brings Up the TunnelConsider Example 11-16, in which the path is invalid. Example 11-16 Path on tunnel1 on 7200a Is Invalid7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: down Path: not valid Signalling: Down path option 5, type explicit primary Config Parameters: Bandwidth: 100 kbps (Global) Priority: 7 7 Affinity: 0x0/0xFFFF Metric Type: TE (default) AutoRoute: enabled LockDown: disabled Loadshare: 100 bw-based auto-bw: disabled Shortest Unconstrained Path Info: Path Weight: 3 (TE) Explicit Route: 10.0.3.5 10.0.5.11 10.0.17.12 12.12.12.12 History: Tunnel: Time since created: 1 days, 44 minutes Time since path change: 23 minutes, 5 seconds Prior LSP: ID: path option 5 [260] Removal Trigger: path verification failed Last Error: PCALC:: Can't use link 10.0.3.5 on node 5.5.5.5 You need to notice a few facts from the highlighted output of Example 11-16:
If the shortest unconstrained path is the same as the specified explicit path and the path is invalid, this can mean only one thing—the constraints placed on the tunnel cannot be satisfied along the explicitly specified path. You can do one of the following things to fix this problem:
NOTE Using an explicit path option is also a type of constraint, because you are asking the headend to build the tunnel using an explicitly specified path list. Removing Bandwidth Constraints While Keeping the Explicit Path OptionTunnel1 comes up after you remove the bandwidth constraint but use the same explicit path option. Example 11-17 demonstrates this. Example 11-17 Removing Bandwidth Constraints on tunnel17200a#show running-config interface tunnel1 Building configuration... Current configuration : 378 bytes ! interface Tunnel1 description Primary tunnel 7200a->12008a->12008c->7200c ip unnumbered Loopback0 no ip directed-broadcast tunnel destination 12.12.12.12 tunnel mode mpls traffic-eng tunnel mpls traffic-eng autoroute announce tunnel mpls traffic-eng priority 7 7 tunnel mpls traffic-eng bandwidth 100 tunnel mpls traffic-eng path-option 5 explicit name primary end 7200a#show ip explicit-paths name primary PATH primary (strict source route, path complete, generation 6) 1: next-address 10.0.3.5 2: next-address 10.0.5.11 3: next-address 10.0.17.12 4: next-address 12.12.12.12 7200a#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 7200a(config)#interface tunnel1 7200a(config-if)#no tunnel mpls traffic-eng bandwidth 100 7200a(config-if)#end 7200a# 1d05h: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up 1d05h: %SYS-5-CONFIG_I: Configured from console by console Keeping Bandwidth Constraints While Removing the Explicit Path OptionInstead of removing the bandwidth, as in Example 11-17, if you add a second path option (dynamic), tunnel1 comes up. This is demonstrated in Example 11-18. Example 11-18 Adding a Dynamic Path Option to tunnel1 Brings Up the Tunnelmpls-7200a#configure terminal Enter configuration commands, one per line. End with CNTL/Z. mpls-7200a(config)#interface tunnel1 mpls-7200a(config-if)#tunnel mpls traffic-eng path-option 6 dynamic 3d02h: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up NOTE The reason for adding another path option is because the tunnel has only one path option (refer to Example 11-17), and removing it would leave the tunnel with no path options, which certainly would not bring up the tunnel. Also, there is no need to remove the explicit path option, because Cisco IOS Software attempts the next path option in sequence when the current one fails. If the tunnel does not come up after one of the two constraints (bandwidth or explicit path option) is removed, you can conclude that either there's a problem along the specified explicit path, or you're requesting too much bandwidth. This means you've got a problem that falls into either Case B or Case C. If the tunnel comes up after you remove both the explicit path and the bandwidth constraints, you have a Case B problem; if it continues to stay down, you have a Case C problem. As you can see from the highlighted text in Example 11-18, the tunnel comes up the moment you add another path-option while retaining the bandwidth. This means that somewhere along the explicitly specified path there is insufficient bandwidth to satisfy a 100 kbps reservation. The output of show mpls traffic-eng tunnels tunnel1 in Example 11-16 shows this: Last Error: PCALC:: Can't use link 10.0.3.5 on node 5.5.5.5 This is confusing, because it seems as though 7200a is complaining about not being able to use 10.0.3.5 (POS2/1) on 5.5.5.5 (12008a). In reality, 7200a is trying to tell you that it can't use the hop that comes after 10.0.3.5, which is 10.0.5.11. Think of this error as indicating the last known usable link in the configured explicit path; something past this link is not acceptable, given the headend configuration. Because for RSVP the nodes do accounting for downstream links, you need to look at the interface of POS1/0 (10.0.5.5) on 12008a. You can check the TE database to see what is being advertised for the next hop that comes after 10.0.3.5 by 12008a. Example 11-19 shows you how to do this. Example 11-19 Checking the Contents of the TE Database for 10.0.5.5 Advertisement7200a#show mpls traffic-eng topology 11.11.11.11 link[3]: Point-to-Point, Nbr IGP Id: 11.11.11.11, nbr_node_id:4, gen:26 frag_id 1, Intf Address:10.0.5.5, Nbr Intf Address:10.0.5.11 TE metric:1, IGP metric:1, attribute_flags:0x0 physical_bw: 622000 (kbps), max_reservable_bw_global: 0 (kbps) max_reservable_bw_sub: 0 (kbps) Global Pool Sub Pool Total Allocated Reservable Reservable BW (kbps) BW (kbps) BW (kbps) --------------- ----------- ---------- bw[0]: 0 0 0 bw[1]: 0 0 0 bw[2]: 0 0 0 bw[3]: 0 0 0 bw[4]: 0 0 0 bw[5]: 0 0 0 bw[6]: 0 0 0 bw[7]: 0 0 0 As you can see from the highlighted text in Example 11-19, the maximum reservable bandwidth for link 10.0.5.0/24 on 12008a is 0. Because the tunnel needs 100 kbps, PCALC is failing. Example 11-20 shows the configuration of interface POS1/0 on 12008a, which confirms that the bandwidth statement is missing from the interface configuration. Adding it back brings up tunnel1. Example 11-20 Checking and Fixing the Configuration of Interface POS1/0 on 12008a12008a#show running-config interface pos 1/0 Building configuration... Current configuration : 181 bytes ! interface POS1/0 ip address 10.0.5.5 255.255.255.0 no ip directed-broadcast mpls label protocol ldp mpls traffic-eng tunnels crc 32 clock source internal pos ais-shut end ! Case B: Removing All Constraints Brings Up the TunnelExample 11-21 fits the description of Case B. Only the unconstrained path is available, but not along the explicitly specified path. Example 11-21 Case B: Removing All Constraints Brings Up the Tunnel7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: down Path: not valid Signalling: Down path option 5, type explicit primary Config Parameters: Bandwidth: 100 kbps (Global) Priority: 7 7 Affinity: 0x0/0xFFFF Metric Type: TE (default) AutoRoute: enabled LockDown: disabled Loadshare: 100 bw-based auto-bw: disabled Shortest Unconstrained Path Info: Path Weight: 4 (TE) Explicit Route: 10.0.3.5 10.0.13.16 10.0.7.11 10.0.17.12 12.12.12.12 History: Tunnel: Time since created: 1 hours, 4 minutes Time since path change: 31 seconds Prior LSP: ID: path option 5 [127] Removal Trigger: path verification failed Last Error: PCALC:: No addresses to connect 10.0.3.5 to 10.0.5.11 For cases involving an explicit path option, you can try to narrow down the problem by first checking every hop in the explicit hop list. You can also try to back down the tunnel: Move the tail one hop back each time to see if the tunnel comes up. If the tunnel comes up when you move the tail to a previous hop, you can conclude that there is a problem between that hop and the next hop, and you can start scrutinizing that hop carefully. Case C: The Tunnel Remains Down Even After All Constraints Are RemovedFinally, for Case C, consider Example 11-22. Example 11-22 Case C: The Tunnel Remains Down Even After All Constraints Are Removed7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: down Path: not valid Signalling: Down path option 6, type dynamic Config Parameters: Bandwidth: 0 kbps (Global) Priority: 7 7 Affinity: 0x0/0xFFFF Metric Type: TE (default) AutoRoute: enabled LockDown: disabled Loadshare: 0 bw-based auto-bw: disabled Shortest Unconstrained Path Info: Path Weight: UNKNOWN Explicit Route: UNKNOWN History: Tunnel: Time since created: 1 hours, 14 minutes Time since path change: 4 minutes, 3 seconds Prior LSP: ID: path option 5 [147] Removal Trigger: path verification failed Path Option 6: Last Error: PCALC:: No path to destination, 12.12.12.12 Example 11-22 provides little information as to what might be wrong. The Shortest Unconstrained Path Info: shows the explicit route as UNKNOWN, and the last error says No path to destination, 12.12.12.12. This output shows only one active path option, and it is dynamic. With an explicitly specified path, it is easy to check for MPLS TE information at each hop, because you specify the hops that should be used. If something goes wrong, it would have to be somewhere along this path. However, with the dynamic path option, if there are multiple ways to get to the tail from the head, there is no obvious path where you can start looking for errors. To troubleshoot this type of problem, you must know the topology of your network very well! For example, you need to know the IGP path between the headend and the tail. As soon as you know that, you should remove the constraints that are placed on the tunnel and see if the tunnel comes up. If you remove all constraints, the headend should follow the IGP best path to get to the tail. If the tunnel does not come up, you can start looking for information in the TE-DB, check the information distribution, and check the configuration of each node along that path, just as you'd do with the explicit path option. NOTE One thing you need to be aware of is what forwarding adjacency can do to your shortest path. If you advertise forwarding adjacencies into your IGP, the path that IP packets will follow across the network is not necessarily the path that unconstrained TE tunnels would follow. One way to determine the IGP shortest path to the tail is to turn off autoroute and forwarding adjacency (FA) at the headend and at all the midpoints so that traffic truly uses the IGP path to the tunnel's tail if you, for example, run traceroute from the headend to the tail. But, turning off FA or autoroute at the midpoints might affect other traffic in your network and thus is not a viable option. Consider Example 11-23. 7200a has only one tunnel that is currently down. No other tunnels downstream will sway traffic from 7200a to 7200c in a non-IGP path. Example 11-23 IGP Shortest Path from 7200a to 7200c7200a#show ip route 12.12.12.12 Routing entry for 12.12.12.12/32 Known via "ospf 100", distance 110, metric 4, type intra area Last update from 10.0.3.5 on POS3/0, 04:39:01 ago Routing Descriptor Blocks: * 10.0.3.5, from 12.12.12.12, 04:39:01 ago, via POS3/0 Route metric is 4, traffic share count is 1 7200a#traceroute 12.12.12.12 Type escape sequence to abort. Tracing the route to 12.12.12.12 1 10.0.3.5 0 msec 0 msec 0 msec 2 10.0.5.11 0 msec 0 msec 0 msec 3 10.0.17.12 0 msec * 0 msec Notice in the highlighted text of the traceroute output that there is no label information for each hop. Not having labels in the traceroute output is not always a guarantee that there are no tunnels on the way to the destination. You could have many one-hop tunnels from 7200a to 7200c that make the traffic from 7200a Of course, you also see labels in a traceroute if you have LDP/TDP turned on somewhere in the path. Example 11-24 shows a traceroute from 7200a to 7200c when the traffic is going down a TE tunnel that has autoroute configured on 12008a, causing the packets from 7200a to take the non-IGP shortest path. Traceroute provides you with information about the path that the packet takes to reach the destination. Traceroute is safe to use only if you know that nothing downstream will make the traceroute packets take the non-IGP best path. Although show mpls traffic-eng topology path destination tunnel-dest shows you what CSPF thinks is the shortest unconstrained path to the destination, there is no substitute in such cases for knowing what path a packet should take in the absence of MPLS TE. Example 11-24 traceroute from 7200a to 7200c When a Tunnel Is Present on 12008a with Autoroute7200a#traceroute 12.12.12.12 Type escape sequence to abort. Tracing the route to 12.12.12.12 1 10.0.3.5 0 msec 0 msec 0 msec 2 10.0.11.10 [MPLS: Label 12334 Exp 0] 0 msec 0 msec 0 msec 3 10.0.9.16 [MPLS: Label 12332 Exp 0] 4 msec 0 msec 0 msec 4 10.0.86.13 [MPLS: Label 18 Exp 0] 0 msec 0 msec 0 msec 5 10.0.87.12 4 msec * 0 msec Going back to Case C (Example 11-23), you should start checking the MPLS TE state at each hop in the traceroute. Example 11-25 shows that the global MPLS settings are fine because the LSP and RSVP processes are running, and forwarding is enabled. Example 11-25 Checking the MPLS TE State at First Hop 12008a12008a#show mpls traffic-eng tunnels summary Signalling Summary: LSP Tunnels Process: running RSVP Process: running Forwarding: enabled Head: 4 interfaces, 2 active signalling attempts, 2 established 49 activations, 47 deactivations Midpoints: 1, Tails: 0 Periodic reoptimization: every 3600 seconds, next in 1347 seconds Periodic FRR Promotion: every 300 seconds, next in 147 seconds Periodic auto-bw collection: disabled Example 11-26 checks the link management information for inbound interface POS2/1 and outbound interface POS1/0. The output of show mpls traffic-eng link-management interface tells you that MPLS TE is on, RSVP is on, the interface is admin-up, and TE information has been flooded on this interface. Example 11-26 Checking Inbound and Outbound Interface Link Management States12008a#show mpls traffic-eng link-management interface pos 2/1 12008a#show mpls traffic-eng link-management interface pos 1/0 System Information:: Links Count: 5 Link ID:: PO1/0 (10.0.5.5) Link Status: Physical Bandwidth: 622000 kbits/sec Max Res Global BW: 466500 kbits/sec (reserved: 0% in, 0% out) Max Res Sub BW: 0 kbits/sec (reserved: 100% in, 100% out) MPLS TE Link State: MPLS TE on, RSVP on, admin-up, flooded Inbound Admission: allow-all Outbound Admission: allow-if-room Admin. Weight: 1 (IGP) IGP Neighbor Count: 1 IGP Neighbor: ID 11.11.11.11, IP 10.0.5.11 (Up) Flooding Status for each configured area [1]: IGP Area[1]: ospf 100 area 0: flooded However, no output is generated for interface POS2/1. This means that you need to check the configuration of interface POS2/1. This is shown in Example 11-27. As you can see, the mpls traffic-eng tunnels command is missing from this interface, causing all the grief. Example 11-27 Checking the Configuration of Inbound Interface POS2/1
12008a#show running-config interface pos2/1
Building configuration...
Current configuration : 199 bytes
!
interface POS2/1
ip address 10.0.3.5 255.255.255.0
no ip directed-broadcast
encapsulation ppp
crc 16
clock source internal
pos ais-shut
pos report prdi
ip rsvp bandwidth 155000 155000
end
Example 11-28 shows mpls traffic-eng tunnels being added back to interface POS2/1. Example 11-28 Adding mpls traffic-eng tunnels on Interface POS2/112008a#configure terminal Enter configuration commands, one per line. End with CNTL/Z. 12008a(config)#interface pos2/1 12008a(config-if)#mpls traffic-eng tunnels This causes tunnel1 on 7200a to come back up, as shown in Example 11-29. Example 11-29 Tunnel1 on 7200a Comes Back Up
7200a#
12:39:03: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
When the unconstrained path is invalid, as in Case C, you might have to make sure that the information distribution—OSPF/IS-IS flooding—population of the TE-DB is correct. The next section describes how you go about doing this. Validating Information DistributionChapter 3, "Information Distribution," provided in-depth coverage of how MPLS TE resource information is flooded using OSPF and IS-IS. This section goes over how you verify that the information distribution has taken place properly and that the TE-DB has the correct information. This can be broken up into five pieces:
Reading the TE-DBThe output of show mpls traffic-eng topology can be rather cumbersome if you don't use the command in conjunction with output modifiers. Here are some ways to limit the output produced:
Example 11-30 TE-DB Entries Built on LSAs from 12008a (5.5.5.5)7200a#show mpls traffic-eng topology 5.5.5.5 ...output omitted... link[2]: Point-to-Point, Nbr IGP Id: 10.10.10.10, nbr_node_id:7, gen:52 frag_id 0, Intf Address:10.0.11.5, Nbr Intf Address:10.0.11.10 TE metric:1, IGP metric:1, attribute_flags:0x0 physical_bw: 622000 (kbps), max_reservable_bw_global: 466500 (kbps) max_reservable_bw_sub: 0 (kbps) Global Pool Sub Pool Total Allocated Reservable Reservable BW (kbps) BW (kbps) BW (kbps) --------------- ----------- ---------- bw[0]: 0 466500 0 bw[1]: 0 466500 0 bw[2]: 0 466500 0 bw[3]: 0 466500 0 bw[4]: 0 466500 0 bw[5]: 0 466500 0 bw[6]: 0 466500 0 bw[7]: 0 466500 0 ...output omitted... Example 11-30 examines the TE-DB for just the entries built from 12008a LSAs, but it has been trimmed to show the entry for only one link. Focus on the highlighted output. The Intf Address and Nbr Intf Address fields give the IP addresses of the advertising router and its IGP neighbor on that link, respectively. The physical bw field gives you the link's actual physical bandwidth, or whatever you've configured with the bandwidth command (not the ip rsvp bandwidth command) on the interface. In Example 11-30, it is 622000 (kbps) for the 12008a The output in Example 11-30 is useful when you want to check the bandwidth, affinity, and TE metric information. Also keep in mind that the bandwidth is advertised on a per-priority basis. So, if your tunnel is requesting bandwidth of 100 Kb and the headend's tunnel setup/holding priority is 5, you should check the bw[5] row of the output. Let us say you suspect that MPLS TE is not enabled on a certain link on 12008a, you can use output modifiers, as shown in Example 11-31, rather than combing through all the output by hand. This allows you to quickly check if a given link has even been advertised. Example 11-31 Checking for All MPLS-TE Links Advertised by 12008a7200a#show mpls traffic-eng topology 5.5.5.5 | include link link[0]: Broadcast, DR: 100.100.100.100, nbr_node_id:-1, gen:50 link[1]: Point-to-Point, Nbr IGP Id: 3.3.3.3, nbr_node_id:8, gen:50 link[2]: Point-to-Point, Nbr IGP Id: 10.10.10.10, nbr_node_id:7, gen:53 link[3]: Point-to-Point, Nbr IGP Id: 11.11.11.11, nbr_node_id:4, gen:50 link[4]: Point-to-Point, Nbr IGP Id: 4.4.4.4, nbr_node_id:9, gen:50 Example 11-32 shows how to use the path keyword to get destination-based information from the TE-DB. Example 11-32 Using the path destination Keywords to Get Destination-Based Information from TE-DB
7200a#show mpls traffic-eng topology path destination 12.12.12.12
Query Parameters:
Destination: 12.12.12.12
Bandwidth: 0
Priorities: 0 (setup), 0 (hold)
Affinity: 0x0 (value), 0xFFFFFFFF (mask)
Query Results:
Min Bandwidth Along Path: 116250 (kbps)
Max Bandwidth Along Path: 466500 (kbps)
Hop 0: 10.0.3.4 : affinity 00000000, bandwidth 116250 (kbps)
Hop 1: 10.0.5.5 : affinity 00000000, bandwidth 466500 (kbps)
Hop 2: 10.0.17.11 : affinity 00000000, bandwidth 116250 (kbps)
Hop 3: 12.12.12.12
This is a quick way to check the path from 7200a to 12.12.12.12, as determined by what's in the TE-DB. If you want to find any paths from 7200a to 7200c that have 120,000 Kb bandwidth, you can add constraints using the keywords bandwidth, priority, and affinity. Example 11-33 demonstrates this for bandwidth. Example 11-33 Using the path destination bandwidth Keywords to Get Destination-Based Information from TE-DB with Bandwidth Constraints7200a#show mpls traffic-eng topology path destination 12.12.12.12 bandwidth 120000 Query Parameters: Destination: 12.12.12.12 Bandwidth: 120000 Priorities: 0 (setup), 0 (hold) Affinity: 0x0 (value), 0xFFFFFFFF (mask) Query Results: % No matching path to destination, 12.12.12.12 As you can see, asking for 120,000 Kb yields no paths to destination 12.12.12.12. On the other hand, if you want to apply the attributes of tunnel1 to see what paths are available, you can use the tunnel keyword, as shown in Example 11-34. Example 11-34 Checking the TE-DB Against Tunnel1 Attributes7200a#show mpls traffic-eng topology path tunnel1 Query Parameters: Destination: 12.12.12.12 Bandwidth: 100 Priorities: 7 (setup), 7 (hold) Affinity: 0x0 (value), 0xFFFF (mask) Query Results: Min Bandwidth Along Path: 116150 (kbps) Max Bandwidth Along Path: 466400 (kbps) Hop 0: 10.0.3.4 : affinity 00000000, bandwidth 116150 (kbps) Hop 1: 10.0.5.5 : affinity 00000000, bandwidth 466400 (kbps) Hop 2: 10.0.17.11 : affinity 00000000, bandwidth 116150 (kbps) Hop 3: 12.12.12.12 As you can see from the highlighted output of Example 11-34, the attributes of tunnel1 are used to filter the content of the TE-DB. Looking for TE Information in the OSPF DatabaseIf you find an incorrect entry in the TE-DB, the first thing you should check is the IGP database. If the IGP database and the TE-DB show different information, you have found the problem. For OSPF, you can check the received OSPF opaque-area LSAs. This is shown in Example 11-35. Example 11-35 OSPF Opaque-Area Database Contents7200a#show ip ospf database opaque-area adv-router 5.5.5.5 OSPF Router with ID (4.4.4.4) (Process ID 100) Type-10 Opaque Link Area Link States (Area 0) LS age: 1252 Options: (No TOS-capability, DC) LS Type: Opaque Area Link Link State ID: 1.0.0.0 Opaque Type: 1 Opaque ID: 0 Advertising Router: 5.5.5.5 LS Seq Number: 8000012F Checksum: 0xD6A Length: 132 Fragment number : 0 MPLS TE router ID : 5.5.5.5 Link connected to Point-to-Point network Link ID : 10.10.10.10 Interface Address : 10.0.11.5 Neighbor Address : 10.0.11.10 Admin Metric : 1 Maximum bandwidth : 77750000 Maximum reservable bandwidth : 58312500 Number of Priority : 8 Priority 0 : 58312500 Priority 1 : 58312500 Priority 2 : 58312500 Priority 3 : 58312500 Priority 4 : 58312500 Priority 5 : 58312500 Priority 6 : 58312500 Priority 7 : 57062500 Affinity Bit : 0x0 The highlighted text in the output of Example 11-35 corresponds to that shown in Example 11-30, except that the bandwidth numbers from the OSPF database are in bytes per second, whereas the numbers in the TE-DB are in kilobits per second. 77,750,000 bytes per second is 622,000 kilobits per second, and 58,312,500 bytes per second is 466,500 kilobits per second. Looking for TE Information in the IS-IS DatabaseThis is similar to what you saw in the OSPF database. Because the sample network is configured for OSPF and not IS-IS, the output shown in this section was taken from a different sample network. You can obtain the TE information in the IS-IS database using the show isis database verbose command, which is similar to the show ip ospf database opaque-area command. It displays the TE TLVs received, as demonstrated in Example 11-36. Example 11-36 show isis database verbose gsr2.00-00 Command Outputgsr1#show isis database verbose gsr2.00-00 IS-IS Level-2 LSP gsr2.00-00 LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL gsr2.00-00 0x000000E5 0xA79C 802 0/0/0 Area Address: 47 NLPID: 0x81 0xCC Hostname: gsr2 Router ID: 192.168.1.2 IP Address: 192.168.1.2 Metric: 10 IP 192.168.6.0/24 Metric: 10 IP 192.168.6.0 255.255.255.0 Metric: 10 IS gsr4.02 Metric: 10 IS gsr1.00 Metric: 10 IS-Extended gsr4.02 Affinity: 0x00000000 Interface IP Address: 192.168.6.2 Physical BW: 622000000 bits/sec Reservable Global Pool BW: 466500000 bits/sec Global Pool BW Unreserved: [0]: 466500000 bits/sec, [1]: 466500000 bits/sec [2]: 466500000 bits/sec, [3]: 466500000 bits/sec [4]: 466500000 bits/sec, [5]: 466500000 bits/sec [6]: 466500000 bits/sec, [7]: 466500000 bits/sec The IS-Extended text refers to the IS-IS extension TLV for MPLS TE. Inside the IS-Extended TLV is the same information you'd find in the TE-DB. Checking the Link Manager for What Was FloodedIn addition to checking the TE-DB for the information it holds (based on the information it received), you can see what was actually advertised by a given router. Example 11-37 shows this output. Example 11-37 Link Management Advertisements on 12008a12008a#show mpls traffic-eng link-management advertisements Flooding Status: ready Configured Areas: 1 IGP Area[1] ID:: ospf 100 area 0 System Information:: Flooding Protocol: OSPF Header Information:: IGP System ID: 5.5.5.5 MPLS TE Router ID: 5.5.5.5 Flooded Links: 6 ...output omitted... Link ID:: 2 Link Subnet Type: Point-to-Point Link IP Address: 10.0.11.5 IGP Neighbor: ID 10.10.10.10, IP 10.0.11.10 TE metric: 1 IGP metric: 1 Physical Bandwidth: 622000 kbits/sec Res. Global BW: 466500 kbits/sec Res. Sub BW: 0 kbits/sec Downstream:: Global Pool Sub Pool ----------- ---------- Reservable Bandwidth[0]: 466500 0 kbits/sec Reservable Bandwidth[1]: 466500 0 kbits/sec Reservable Bandwidth[2]: 466500 0 kbits/sec Reservable Bandwidth[3]: 466500 0 kbits/sec Reservable Bandwidth[4]: 466500 0 kbits/sec Reservable Bandwidth[5]: 466500 0 kbits/sec Reservable Bandwidth[6]: 466500 0 kbits/sec Reservable Bandwidth[7]: 456500 0 kbits/sec Attribute Flags: 0x00000000 ...output omitted... As you can see, the highlighted output in Example 11-37 matches what you saw in the TE-DB for the link2 entry from 12008a (5.5.5.5) in Example 11-30. The show mpls traffic-eng link-management advertisements command is basically used to get an idea of whether a certain interface was advertised at all and, if so, what resources (bandwidth and link attributes) are associated with each link. Note that link management output is only relevant on the router that controls those links, i.e. the router that does admission control and flooding for those links. Checking RSVP Bandwidth InformationAnother way to get a snapshot of all the interface bandwidth on a router is to use the show ip rsvp interface command. Example 11-38 shows this output. Example 11-38 Snapshot of Link Bandwidth on 12008a
12008a#show ip rsvp interface
interface allocated i/f max flow max sub max
PO0/0 0G 1866M 1866M 0G
PO1/0 100K 466500K 466500K 0G
PO1/1 10M 466500K 466500K 0G
PO2/0 0G 116250K 116250K 0G
PO2/1 0G 155M 155M 0G
Fa4/0 0G 0G 0G 0G
Tu1 0G 0G 0G 0G
Tu2 0G 0G 0G 0G
Tu3 0G 0G 0G 0G
Several show commands related to TE information distribution were presented in this section. The show commands essentially provide the current state information. However, if you want to catch the LSAs as they leave a router or as they arrive at a router, you can use debug commands. Table 11-4 gives you the show commands and the equivalent debug commands. As with any debug commands, be aware that it is not wise to turn them on in a production environment unless you know it produces a small amount of data. Turning off logging console and turning on logging buffered is also advised when you use debugs, especially in a production network.
This concludes the PCALC section. The next section assumes that the path is valid, but that you are having an RSVP signalling problem. RSVP Signalling ProblemsWhen the headend shows that the path is valid, but the tunnel is down, it is because of a signalling problem. How do you identify a problem in signalling? What are the symptoms of a signalling problem? The first sign, of course, is that the tunnel is down. As with troubleshooting PCALC problems, the output of show mpls traffic-eng tunnels [tunnel] plays a key role in determining the cause of a tunnel's being down. Example 11-39 shows you a portion of the output from Example 11-22. Example 11-39 RSVP Signalling Status in the show mpls traffic-eng tunnels tunnel1 Command7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: down Path: not valid Signalling: Down path option 6, type dynamic The Signalling field can have one of the following values:
In all the output shown earlier in this chapter, you saw only signalling being Down or Connected. If the tunnel is either admin down or the path is invalid, the signalling state shows Down. When a Path message is sent out, the signalling state goes into RSVP signalling proceeding. When the corresponding Resv message is received, the signalling state goes to Connected. Most of the time, you don't see the signalling proceeding state, because signalling happens relatively quickly. However, there are certainly cases in which you see it—if there's lots of signalling traffic, or if a problem downstream is causing Path or Resv messages to be dropped. For example, if you see a tunnel in RSVP signalling proceeding, and it stays in that state for more than a second or two, it's probably worth investigating. Example 11-40 shows tunnel1 on 7200a in RSVP signalling proceeding. Example 11-40 7200a in Signalling Proceeding State7200a#show mpls traffic-eng tunnels tunnel1 Name: Primary tunnel 7200a->12008a->12... (Tunnel1) Destination: 12.12.12.12 Status: Admin: up Oper: down Path: valid Signalling: RSVP signalling proceeding path option 5, type explicit primary (Basis for Setup, path weight 3) path option 6, type dynamic ...output omitted... As you can see from the highlighted output, the tunnel is operationally down, the path is valid, and signalling shows RSVP signalling proceeding. You can check the output of show ip rsvp counter to see how many RSVP messages were sent and received across all RSVP-enabled interfaces. Of course, if you are troubleshooting an RSVP problem and you want to start looking at the counters, it is a good idea to clear the counters first using the clear ip rsvp counters command. This is shown in Example 11-41. Example 11-41 Clearing RSVP Counters Before Checking Them
7200a#clear ip rsvp counters
Clear rsvp counters [confirm]
Example 11-42 shows the RSVP counters immediately after they are cleared. Example 11-42 RSVP Counters Right After They Are Cleared
7200a#show ip rsvp counters interface pos 3/0
POS3/0 Recv Xmit Recv Xmit
Path 0 0 Resv 0 0
PathError 0 0 ResvError 0 0
PathTear 0 0 ResvTear 0 0
ResvConfirm 0 0 ResvTearConfirm 0 0
UnknownMsg 0 0 Errors 0 0
Example 11-43 shows the RSVP counters a few minutes after they are cleared. Example 11-43 RSVP Counters a Few Minutes After They Are Cleared
7200a#show ip rsvp counters interface pos 3/0
POS3/0 Recv Xmit Recv Xmit
Path 0 6 Resv 0 0
PathError 0 0 ResvError 0 0
PathTear 0 0 ResvTear 0 0
ResvConfirm 0 0 ResvTearConfirm 0 0
UnknownMsg 0 0 Errors 0 0
As you can see from the highlighted output, 7200a has sent six Path messages but has not received any Resv messages. This is what is causing the signalling state to remain in proceeding. The way to troubleshoot this type of problem is to methodically follow Path messages as they leave the headend and go all the way to the tail. If the Path messages are making it to the tail, follow the Resv messages upstream to the headend. To see the RSVP Path and Resv messages, you need to use the debug ip rsvp command. Example 11-44 shows the Path message leaving the headend 7200a. NOTE We recommend that you use service timerstamps debug datetime msec, as this gives debug messages millisecond resolution, which makes it easier to tell one RSVP message from the next. Example 11-44 Path Message Leaving 7200a7200a#debug ip rsvp path RSVP debugging is on 7200a# *Apr 5 23:40:57.421: RSVP: Outgoing path message 4.4.4.4_1614- >12.12.12.12_1_4.4.4.4 (on POS3/0) *Apr 5 23:40:57.421: RSVP session 12.12.12.12_1_4.4.4.4: send path multicast on POS3/0 *Apr 5 23:40:57.421: RSVP: version:1 flags:0000 type:PATH cksum:0EAA ttl:254 reserved:0 length:236 *Apr 5 23:40:57.421: SESSION type 7 length 16: *Apr 5 23:40:57.421: Tun Dest: 12.12.12.12 Tun ID: 1 Ext Tun ID: 4.4.4.4 *Apr 5 23:40:57.421: HOP type 1 length 12: *Apr 5 23:40:57.421: Hop Addr: 10.0.3.4 LIH: 0x0 *Apr 5 23:40:57.421: TIME_VALUES type 1 length 8 : *Apr 5 23:40:57.421: Refresh Period (msec): 30000 *Apr 5 23:40:57.421: EXPLICIT_ROUTE type 1 length 36: *Apr 5 23:40:57.421: 10.0.3.5 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:40:57.421: 10.0.5.11 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:40:57.421: 10.0.17.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:40:57.421: 12.12.12.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:40:57.421: LABEL_REQUEST type 1 length 8 : 00000800 *Apr 5 23:40:57.421: SESSION_ATTRIBUTE type 7 length 52: *Apr 5 23:40:57.421: Setup Prio: 7, Holding Prio: 7 *Apr 5 23:40:57.421: Flags: SE Style *Apr 5 23:40:57.421: Session Name: Primary tunnel 7200a->12008a->12008c->7200c *Apr 5 23:40:57.421: SENDER_TEMPLATE type 7 length 12: *Apr 5 23:40:57.421: Tun Sender: 4.4.4.4 LSP ID: 1614 *Apr 5 23:40:57.421: SENDER_TSPEC type 2 length 36: *Apr 5 23:40:57.421: version=0, length in words=7 *Apr 5 23:40:57.421: service id=1, service length=6 *Apr 5 23:40:57.421: parameter id=127, flags=0, parameter length=5 *Apr 5 23:40:57.421: average rate=12500 bytes/sec, burst depth=1000 bytes *Apr 5 23:40:57.421: peak rate =12500 bytes/sec *Apr 5 23:40:57.421: min unit=0 bytes, max unit=0 bytes *Apr 5 23:40:57.421: ADSPEC type 2 length 48: *Apr 5 23:40:57.421: version=0 length in words=10 *Apr 5 23:40:57.421: General Parameters break bit=0 service length=8 *Apr 5 23:40:57.421: IS Hops:1 *Apr 5 23:40:57.421: Minimum Path Bandwidth (bytes/sec):19375000 *Apr 5 23:40:57.421: Path Latency (microseconds):0 *Apr 5 23:40:57.421: Path MTU:4470 *Apr 5 23:40:57.421: Controlled Load Service break bit=0 service length=0 *Apr 5 23:40:57.421: As you can see from the output of Example 11-44, the path message left 7200a. Before trying to check if it made it into next hop 12008a, you should know that the debug ip rsvp messages produce large amounts of output. This is because RSVP is a soft-state protocol—refresh messages keep going back and forth. If you have many tunnels going through a midpoint, it becomes hard to sift through the output to try to find the Path or Resv message you are looking for. Fortunately, you can simplify this by using IP extended access lists. The extended access list is used differently in debugging RSVP messages than when you use it to filter packets in the forwarding path. Here is the syntax if you want to use an extended access list to filter out unwanted RSVP debug output: access-list {100-199} permit udp host {tunnel source} [eq {tunnel-id}] host {tunnel dest} [eq {LSP-ID}] The access list has to be 100 through 199 for it to be an IP extended access list. Even though RSVP messages have their own protocol type, the access list you specify is listed as UDP. This is because the Cisco IOS Software allows you to specify a port number for UDP after the source or destination address. When a UDP access list is used in RSVP debugs, the source and destination port numbers are interpreted to mean tunnel-id and LSP ID, respectively. The tunnel source is the headend's TE-ID. For a Path message coming from 7200a for tunnel1, the tunnel source needs to be 4.4.4.4, and the tunnel destination should be the tail's TE-ID—12.12.12.12. The [eq {tunnel-id}] and the [eq {LSP-ID}] are useful if you have many tunnels going from the same headend to the same tail. Example 11-45 shows extended access list 101 being defined to be used in the RSVP debug. Example 11-45 Checking if the Path Message Made It to First Hop 12008a12008a#configure terminal 12008a(config)#access-list 101 permit udp host 4.4.4.4 host 12.12.12.12 debug ip rsvp path 101 detail is turned on after that. It uses IP extended access list 101, as shown in Example 11-46. Example 11-46 Turning on debug ip rsvp path 10112008a#debug ip rsvp path 101 RSVP debugging is on *Apr 5 23:30:35.576: RSVP: version:1 flags:0000 type:PATH cksum:0000 ttl:254 reserved:0 length:236 *Apr 5 23:30:35.576: SESSION type 7 length 16: *Apr 5 23:30:35.576: Tun Dest: 12.12.12.12 Tun ID: 1 Ext Tun ID: 4.4.4.4 *Apr 5 23:30:35.576: HOP type 1 length 12: *Apr 5 23:30:35.576: Hop Addr: 10.0.3.4 LIH: 0x0 *Apr 5 23:30:35.576: TIME_VALUES type 1 length 8 : *Apr 5 23:30:35.576: Refresh Period (msec): 30000 *Apr 5 23:30:35.576: EXPLICIT_ROUTE type 1 length 36: *Apr 5 23:30:35.576: 10.0.3.5 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:35.576: 10.0.5.11 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:35.576: 10.0.17.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:35.576: 12.12.12.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:35.576: LABEL_REQUEST type 1 length 8 : 00000800 *Apr 5 23:30:35.576: SESSION_ATTRIBUTE type 7 length 52: *Apr 5 23:30:35.576: Setup Prio: 7, Holding Prio: 7 *Apr 5 23:30:35.576: Flags: SE Style *Apr 5 23:30:35.576: Session Name: Primary tunnel 7200a->12008a->12008c>7200c *Apr 5 23:30:35.576: SENDER_TEMPLATE type 7 length 12: *Apr 5 23:30:35.576: Tun Sender: 4.4.4.4 LSP ID: 1618 *Apr 5 23:30:35.576: SENDER_TSPEC type 2 length 36: *Apr 5 23:30:35.576: version=0, length in words=7 *Apr 5 23:30:35.576: service id=1, service length=6 *Apr 5 23:30:35.576: parameter id=127, flags=0, parameter length=5 *Apr 5 23:30:35.576: average rate=12500 bytes/sec, burst depth=1000 bytes *Apr 5 23:30:35.576: peak rate =12500 bytes/sec *Apr 5 23:30:35.576: min unit=0 bytes, max unit=0 bytes *Apr 5 23:30:35.576: ADSPEC type 2 length 48: *Apr 5 23:30:35.576: version=0 length in words=10 *Apr 5 23:30:35.576: General Parameters break bit=0 service length=8 *Apr 5 23:30:35.576: IS Hops:1 *Apr 5 23:30:35.576: Minimum Path Bandwidth (bytes/sec):19375000 *Apr 5 23:30:35.576: Path Latency (microseconds):0 *Apr 5 23:30:35.576: Path MTU:4470 *Apr 5 23:30:35.576: Controlled Load Service break bit=0 service length=0 *Apr 5 23:30:35.576: *Apr 5 23:30:35.576: RSVP 4.4.4.4_1618-12.12.12.12_1: PATH message arrived from PHOP 10.0.3.4 on POS2/1 *Apr 5 23:30:45.320: RSVP: Outgoing path message 4.4.4.4_1618->12.12.12.12 _1_4.4.4.4 (on POS1/0) *Apr 5 23:30:45.320: RSVP: version:1 flags:0000 type:PATH cksum:3BC1 ttl:253 reserved:0 length:228 *Apr 5 23:30:45.320: SESSION type 7 length 16: *Apr 5 23:30:45.320: Tun Dest: 12.12.12.12 Tun ID: 1 Ext Tun ID: 4.4.4.4 *Apr 5 23:30:45.320: HOP type 1 length 12: *Apr 5 23:30:45.320: Hop Addr: 10.0.5.5 LIH: 0x0 *Apr 5 23:30:45.320: TIME_VALUES type 1 *Apr 5 23:30:45.320: Refresh Period (msec): 30000 *Apr 5 23:30:45.320: EXPLICIT_ROUTE type 1 length 28: *Apr 5 23:30:45.320: 10.0.5.11 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:45.320: 10.0.17.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:45.320: 12.12.12.12 (Strict IPv4 Prefix, 8 bytes, /32) *Apr 5 23:30:45.320: LABEL_REQUEST type 1 length 8 : 00000800 *Apr 5 23:30:45.320: SESSION_ATTRIBUTE type 7 length 52: *Apr 5 23:30:45.320: Setup Prio: 7, Holding Prio: 7 *Apr 5 23:30:45.320: Flags: SE Style *Apr 5 23:30:45.320: Session Name: Primary tunnel 7200a->12008a->12008c->7200c *Apr 5 23:30:45.320: SENDER_TEMPLATE type 7 length 12: *Apr 5 23:30:45.320: Tun Sender: 4.4.4.4 LSP ID: 1618 As you can see from the highlighted output of Example 11-46, the Path message arrives on interface POS2/1 and goes out on interface POS1/0. You know these messages belong to the same session because both the incoming and outgoing message carry the same Tunnel Sender, Tunnel Destination, Tunnel ID, and LSP ID. You might have noted that the same RSVP message showed up twice in Example 11-46. The first RSVP message is the message the router received; the second one is the message it sent. There are a few differences between the two messages, such as the HOP object and the contents of the EXPLICIT_ROUTE object. This sample network has only a single tunnel, whose LSP ID is 1618. Tracking RSVP messages that carry this LSP ID means that they all belong to the same tunnel. Now you need to check the same thing at the next hop 12008c. Example 11-47 shows this on 12008c. Example 11-47 Checking on 12008c to See if the Path Message Made It12008c#show clock *23:31:18.236 UTC Fri Apr 5 2002 12008c#debug ip rsvp 101 RSVP debugging is on ... ... 12008c#show clock *23:32:25.396 UTC Fri Apr 5 2002 You can see from the highlighted output of Example 11-47 that 12008c is not receiving the Path message. Path messages by default arrive every 30 seconds, plus or minus 50 percent jitter. This is what is preventing the tunnel from being set up. After the signalling problem is fixed, the Path message arrives on the tail 7200c, which sends a Resv message that arrives on 7200a, and tunnel1 on 7200a finally comes up. Example 11-48 shows the Resv message arriving at 7200a and tunnel1 coming up. Example 11-48 Resv Message7200a#debug ip rsvp resv RSVP debugging is on *Apr 6 00:26:10.781: RSVP: version:1 flags:0000 type:RESV cksum:0000 ttl:255 reserved:0 length:108 *Apr 6 00:26:10.781: SESSION type 7 length 16: *Apr 6 00:26:10.781: Tun Dest: 12.12.12.12 Tun ID: 1 Ext Tun ID: 4.4.4.4 *Apr 6 00:26:10.781: HOP type 1 length 12: *Apr 6 00:26:10.781: Hop Addr: 10.0.3.5 LIH: 0x0 *Apr 6 00:26:10.781: TIME_VALUES type 1 length 8 : *Apr 6 00:26:10.781: Refresh Period (msec): 30000 *Apr 6 00:26:10.781: STYLE type 1 length 8 : *Apr 6 00:26:10.781: RSVP_SE_OPTION *Apr 6 00:26:10.781: FLOWSPEC type 2 length 36: *Apr 6 00:26:10.781: version = 0 length in words = 7 *Apr 6 00:26:10.781: service id = 5, service length = 6 *Apr 6 00:26:10.781: tspec parameter id = 127, tspec flags = 0, tspec length= 5 *Apr 6 00:26:10.781: average rate = 12500 bytes/sec, burst depth = 1000 bytes *Apr 6 00:26:10.781: peak rate = 2147483647 bytes/sec *Apr 6 00:26:10.781: min unit = 0 bytes, max unit = 0 bytes *Apr 6 00:26:10.781: FILTER_SPEC type 7 length 12: *Apr 6 00:26:10.781: Tun Sender: 4.4.4.4, LSP ID: 1618 *Apr 6 00:26:10.781: LABEL type 1 length 8 : 00003025 *Apr 6 00:26:10.781: *Apr 6 00:26:10.781: RSVP 4.4.4.4_1623-12.12.12.12_1: RESV message arrived from 10.0.3.5 on POS3/0 *Apr 6 00:26:10.781: RSVP session 12.12.12.12_1_4.4.4.4: Reservation is new 1d03h: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up If your signalling state shows connected, you no longer have a tunnel-down problem. The next section talks about problems related to forwarding after the TE tunnel is up. Tunnel-Up ProblemsAs discussed earlier, tunnel-up problems are either routing problems or forwarding problems. For troubleshooting MPLS forwarding problems, it is important to know what labels are exchanged in the control plane so that you know what labels each hop expects to see in the forwarding plane. Figure 11-4 shows the labels exchanged (from right to left) and packets being forwarded over the tunnel from left to right. 7200a imposes a label of 12325 on the packet as it switches it down the TE tunnel. 12008a label-swaps it to 12330, and 12008c pops the label and sends it to 7200c, the tunnel's tail. Figure 11-4. Labels Exchanged Over the Tunnel and Packet Forwarding Over the Tunnel
Forwarding problems can be categorized as follows:
Traffic Doesn't Enter the TunnelJust because your tunnel is up doesn't mean traffic will flow over it. After the tunnel is up, traffic might not go through the tunnel if you don't have the following configured on the headend:
If you don't have ip unnumbered loopback0 under the tunnel interface, the tunnel does not have a source IP address. As a result, it will not be installed in the routing table as an outgoing interface. If you remember from Chapter 5, "Forwarding Traffic Down Tunnels," you have to use either static routes, autoroute announce, FA, or PBR, or no traffic will use the tunnel, even if it is signalled up. Verifying Routing and Forwarding TablesFor static routes, autoroute, or FA, the easiest way to check what prefixes are mapped over the tunnel is to use show ip cef {tunnel}. This is shown in Example 11-49. Example 11-49 Using CEF to See What Is Mapped onto the Tunnel
7200a#show ip cef tunnel1
Prefix Next Hop Interface
1.1.1.1/32 0.0.0.0 Tunnel1
2.2.2.2/32 0.0.0.0 Tunnel1
10.0.18.0/24 10.0.3.5 POS3/0
0.0.0.0 Tunnel1
10.0.87.0/24 10.0.3.5 POS3/0
0.0.0.0 Tunnel1
10.1.1.1/32 0.0.0.0 Tunnel1
10.1.1.2/32 0.0.0.0 Tunnel1
12.12.12.12/32 0.0.0.0 Tunnel1
66.66.66.0/24 0.0.0.0 Tunnel1
171.68.0.0/16 12.12.12.12 Tunnel1
171.68.1.0/24 0.0.0.0 Tunnel1
It is important to look at the CEF table and not the routing table, because the forwarding might have been recursively resolved to the tunnel interface, whereas the routing table does not actually point down the tunnel interface for a certain prefix. Example 11-50 shows the routing entry for prefix 171.68.0.0. The match is highlighted in the output. Example 11-50 Routing Table Entry for 171.68.0.07200a#show ip route 171.68.0.0 Routing entry for 171.68.0.0/16, 2 known subnets Variably subnetted with 2 masks O E2 171.68.1.0/24 [110/20] via 10.0.3.5, 00:02:04, POS3/0 B 171.68.0.0/16 [200/0] via 12.12.12.12, 00:01:09 Example 11-51 shows the CEF table entry for 171.68.0.0. Example 11-51 Checking the CEF Table for 171.68.0.07200a#show ip cef 171.68.0.0 171.68.0.0/16, version 432, epoch 0 0 packets, 0 bytes tag information from 12.12.12.12/32, shared, unshareable local tag: 37 fast tag rewrite with Tu1, point2point, tags imposed {12325} via 12.12.12.12, 0 dependencies, recursive next hop 12.12.12.12, Tunnel1 via 12.12.12.12/32 valid adjacency tag rewrite with Tu1, point2point, tags imposed {12325} The reason that the routing table entry for 171.68.0.0 doesn't point down Tunnel1 is because only its next hop 12.12.12.12 is mapped onto Tunnel1. As a result, the CEF entry recursively resolves 171.68.0.0 to Tunnel1, whereas the routing table does not. Example 11-52 shows that 12.12.12.12 is mapped onto Tunnel1 statically. Example 11-52 12.12.12.12 Is Statically Routed Over Tunnel17200a#show ip route 12.12.12.12 Routing entry for 12.12.12.12/32 Known via "static", distance 1, metric 0 (connected) Routing Descriptor Blocks: * directly connected, via Tunnel1 Route metric is 0, traffic share count is 1 If you have autoroute turned on for Tunnel1, you can also check to see if it is being announced using show mpls traffic-eng autoroute, as shown in Example 11-53. Example 11-53 Checking if Autoroute Is Enabled for the Tunnel7200a#show mpls traffic-eng autoroute MPLS TE autorouting enabled destination 12.12.12.12 has 1 tunnels Tunnel1 (load balancing metric 20000000, nexthop 12.12.12.12) (flags: Announce) The same applies with FA. If you have FA configured on Tunnel1 instead of autoroute, you can use show mpls traffic-eng forwarding-adjacency, as shown in Example 11-54. Example 11-54 Checking if FA Is Enabled for the Tunnel
pls-7200a#show mpls traffic-eng forwarding-adjacency
destination 12.12.12.12 has 1 tunnels
Tunnel1 (load balancing metric 20000000, nexthop 12.12.12.12)
(flags: Forward-Adjacency, holdtime 0)
Verifying That Traffic Is Really Going Down the TunnelIf everything checks out, how do you know that traffic is actually entering the tunnel? You can check this by following these two steps:
Before you check the interface tunnel accounting, it is a good idea to clear the counters, as shown in Example 11-55. Example 11-55 Clearing Tunnel1 Counters
mpls-7200a#clear counters tunnel 1
Clear "show interface" counters on this interface [confirm]
3d04h: %CLEAR-5-COUNTERS: Clear counter on interface Tunnel1 by console
Next, send 1000 pings to the tail 12.12.12.12, as shown in Example 11-56. Example 11-56 Sending 1000 Pings to 7200c (12.12.12.12)7200a#ping ip Target IP address: 12.12.12.12 Repeat count [5]: 1000 Datagram size [100]: Timeout in seconds [2]: Extended commands [n]: Sweep range of sizes [n]: Type escape sequence to abort. Sending 1000, 100-byte ICMP Echos to 12.12.12.12, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!! Success rate is 100 percent (1000/1000), round-trip min/avg/max = 1/1/4 ms The next step is to check the interface accounting for Tunnel1 to see how many packets went over the tunnel. This is shown in Example 11-57. Example 11-57 Checking the Accounting for Tunnel17200a#show interfaces tunnel1 accounting Tunnel1 Primary tunnel 7200a->12008a->12008c->7200c Protocol Pkts In Chars In Pkts Out Chars Out IP 0 0 1000 108000 As you can see from the highlighted output, the Pkts Out field for the Tunnel1 accounting shows 1000 packets. The same is reflected on interface POS3/0, as shown in Example 11-58. Notice for interface POS3/0 that the packets out are Tag or MPLS packets. Example 11-58 Checking the Accounting for Interface POS3/0 (the Outbound Physical Interface)7200a#show interfaces pos 3/0 accounting POS3/0 Protocol Pkts In Chars In Pkts Out Chars Out IP 1036 107881 45 3804 CDP 2 696 3 1047 Tag 0 0 1000 108000 Also, if a prefix is being forwarded down a tunnel, the CEF entry for that destination should look as shown in Example 11-59. Example 11-59 CEF Entry for the Destination Mapped onto the Tunnel7200a#show ip cef 12.12.12.12 12.12.12.12/32, version 521, epoch 0 0 packets, 0 bytes tag information set, shared local tag: 37 fast tag rewrite with Tu1, point2point, tags imposed {12325} via 0.0.0.0, Tunnel1, 1 dependency next hop 0.0.0.0, Tunnel1 valid adjacency tag rewrite with Tu1, point2point, tags imposed {12325} Traffic Enters the Tunnel But Gets LostIf the traffic left the headend, but does not make it to the destination properly, it is getting blackholed somewhere. To troubleshoot this, you have to determine where traffic is getting lost. You can determine the location of the black hole by
Example 11-60 shows the output of traceroute from 7200a to 7200c (12.12.12.12). Example 11-60 traceroute from 7200a to 7200c7200a#traceroute 12.12.12.12 Type escape sequence to abort. Tracing the route to 12.12.12.12 1 10.0.3.5 [MPLS: Label 12325 Exp 0] 4 msec 0 msec 0 msec 2 10.0.5.11 [MPLS: Label 12330 Exp 0] 0 msec 0 msec 0 msec 3 10.0.17.12 0 msec * 0 msec From the traceroute results, you can conclude that the packet should reach 12008a (the first hop) with a label of 12325 and should reach 12008c (the second hop) with a label of 12330. The output of Example 11-60 is what you see when things are normal. For example, if 12008a were blackholing the packets, you would only see the output in row 1, followed by rows of asterisks. If you see asterisks in the second row, does this mean that the packet left 12008a and did not reach 12008c, or does it mean that the packets did not even leave 12008a? Traceroute does not help you answer this question. You have to resort to looking at the interface accounting for incoming and outgoing interfaces of each hop. The recommended steps are as follows:
As always, clear the counters first, as shown in Example 11-61. Example 11-61 Clearing MPLS Counters
12008a#clear mpls counters
Clear "show mpls forwarding-table" counters [confirm]
12008a#
1w0d: %TFIB-5-CLEAR_COUNTERS: Clear tagswitch forwarding counters by console
From Example 11-59, you know that packets forwarded down the tunnel have a label value of 12,325 as they leave 7200a and go toward 12008a. So, as shown in Example 11-62, you have to check the MPLS forwarding table for entry 12325. As you can see from the high-lighted output, Bytes tag switched is 108,000, which equals 1000 packets of size 108 (bytes). Example 11-62 Checking MPLS Forwarded Packets at 12008a for Tunnel1 on 7200a12008a#show mpls forwarding-table labels 12325 detail Local Outgoing Prefix Bytes tag Outgoing Next Hop tag tag or VC or Tunnel Id switched interface 12325 12330 4.4.4.4 1 [1623] 108000 PO1/0 point2point MAC/Encaps=4/8, MTU=4470, Tag Stack{12330} 0F008847 0302A000 No output feature configured NOTE If you are using pings to test a tunnel that you just provisioned, just because you don't see the pings succeeding does not mean that there is a problem with the tunnel. Tunnels are unidirectional, and forwarding might be broken in the return path, but not in the forward path. Turn on debug ip icmp on the tail to see if ICMP packets are received or not. |
|
|
< Free Open Study > |
|
7200c follow the non-IGP best path and still don't show labels in the output of a traceroute from 7200a to 7200c. However, if you see labels in a traceroute, and you see the traceroute taking a path that does not come across as the IGP shortest path, this should make you suspicious of midpoint tunnels that have autoroute or FA configured on them.