WAN Aggregator QoS Design

This section discusses WAN QoS considerations and designs, including the following:
• Slow-speed (≤ 768 kbps) WAN link design
• Medium-speed (768 kbps to T1/E1 speed) WAN link design
• High-speed (> T1/E1 speed) WAN link design

Additionally, these designs are applied to specific Layer 2 WAN media, including the following:
• Leased lines
• Frame Relay
• ATM-to-Frame Relay Service Interworking

WAN Edge QoS Design Considerations:

Traffic is assumed to be correctly classified and marked (at Layer 3) before WAN aggregator ingress. Remember, Layer 3 markings (preferably DSCP) are media independent and traverse the WAN media, whereas Layer 2 CoS is lost when the media switches from Ethernet to WAN media.

Software QoS: WAN edge QoS is performed within Cisco IOS Software. WAN topologies and QoS policies should be designed to limit the average CPU utilization of the WAN aggregator to 75 percent (or lower) especially when the WAN aggregator is homing several hundred remote branches.

Bandwidth Provisioning for Best-Effort Traffic: It is recommended that at least 25 percent of a WAN link’s bandwidth be reserved for the default Best-Effort class.

Bandwidth Provisioning for Real-Time Traffic: Cisco Technical Marketing testing has shown a significant decrease in data application response times when Real-Time traffic exceeds one-third of a link’s bandwidth capacity. Extensive testing and production network customer deployments have shown that limiting the sum of all LLQs to 33 percent is a conservative and safe design ratio for merging real-time applications with data applications.

Serialization Delay: Serialization delays depends not only on the line rate of the link speed, but also on the size of the packet being serialized. Lower link speeds can cause sufficient serialization delay to adversely affect real-time streams. Because the end-to-end one-way jitter target has been set as 30 ms, the typical per-hop serialization delay target is 10 ms (which allows for up to three intermediate hops per direction of VoIP traffic flow). This 10 ms per-hop target leads to the recommendation that a link fragmentation and interleaving (LFI) tool (either MLP LFI or FRF.12) be enabled on links with speeds at or below 768 kbps.

When deploying LFI tools, it is recommended that the LFI tools be enabled during a scheduled downtime. Naturally, LFI tools need to be enabled on both ends of the link. It is recommended that LFI be enabled on the branch router first (which is on the far end of the WAN link) because this generally takes the WAN link down. Then the administrator can enable LFI on the WAN aggregator (the near end of the WAN link), and the link will come back up.

Additionally, since traffic assigned to the LLQ escapes fragmentation, it is recommended that Interactive-Video not be deployed on slow-speed links; the large Interactive-Video packets (such as 1500-byte full-motion I-Frames) could cause high serialization delays.

IP RTP Header Compression: CRTP is one of the most CPU-intensive features within the Cisco IOS Software QoS toolset. Therefore, it is recommended that cRTP be used primarily on slow-speed (≤ 768 kbps) links with a careful eye on CPU levels (especially for WAN aggregators that home a large number of remote branches).

Tx-ring Tuning: Newer versions of Cisco IOS Software automatically size the final interface output buffer (Tx-ring) to optimal lengths for Real-Time applications. Using command show controller, the value following the tx_limited keyword reflects the value of the Tx-ring. Normally, the Tx-ring is set to 64 packets. This value can be tuned to the recommended setting of 3 on T1/E1 (or slower) links.

PAK_priority: PAK_priority is the internal Cisco IOS mechanism for protecting routing and control traffic. On heavily congested links, it might be necessary to explicitly provision a CBWFQ bandwidth class for routing/control traffic, as identified by either IPP or CS6.

Although BGPs (both eBGPs and iBGPs) are marked to IPP6/CS6, they do not receive PAK_priority treatment within the routers. Therefore, it may be necessary to provision a separate bandwidth class to protect BGP sessions. Although IS-IS traffic receives PAK_priority within the router, it cannot be marked to IPP6/CS6 because IS-IS uses a CLNS protocol. (It does not use IP, so there are no IPP or DSCP fields to mark). Thus, NBAR can be used within a class map to match IS-IS traffic.

Link Speeds: There are three main groupings of link speeds. These link speeds and their respective design implications are summarized below:

• Slow (link speed ≤ 768 kbps):
– Deployment of Interactive-Video generally is not recommended on these links because of serialization implications.
– These links require LFI to be enabled if VoIP is to be deployed over them.
– cRTP is recommended (with a watchful eye on CPU levels).
– Check Tx-ring sizes (especially on slow-speed ATM PVCs); tune to 3, if needed.
– Three- to five-class traffic models are recommended.

• Medium (768 kbps ≤ link speed ≤ T1/E1):
– VoIP or Interactive-Video can be assigned to the LLQ (usually, there is not enough bandwidth to do both and still keep the LLQ provisioned at less than 33 percent—alternatively, Interactive-Video can be placed in a CBWFQ queue).
– LFI is not required.
– cRTP is optional.
– Three- to five-class traffic models are recommended.

• High (Super T1/E1 link speeds):
– LFI is not required.
– cRTP generally is not recommended (because the cost of increased CPU levels typically offsets the benefits of the amount of bandwidth saved).
– Five- to 11-class traffic models are recommended.

WAN Edge Classification and Provisioning Models:

Slow/Medium Link-Speed QoS Class Models: It includes Three-Class (Voice and Data) Model and Five-Class (Voice and Data) Model.



High Link Speed QoS Class Models: High-speed links (such as multiple T1/E1 or above speeds) allow for the provisioning of Voice, Interactive-Video, and multiple classes of data, according to the design rules presented in this chapter (for example, 25 percent for Best Effort class and < 33 percent for all LLQs). It includes Eight-Class Model and QoS Baseline (11-Class) Model.



WAN Edge Link-Specific QoS Design:

The most popular WAN media in use today are leased lines, Frame Relay, and ATM (including ATM-to-Frame Relay Service Interworking). Each of these media can be deployed in three broad categories of link speeds: slow speed (≤ 768 kbps), medium speed (≤ T1/E1), and high speed (multiple T1/E1 or greater).

Leased Lines: Leased lines, or point-to-point links, can be configured with HDLC, PPP, or MLP encapsulation. MLP offers the network administrator the most flexibility and deployment options. For example, MLP is the only leased-line protocol that supports LFI on slow-speed links (through MLP LFI). Additionally, as bandwidth requirements grow over time, MLP requires the fewest modifications to accommodate the
addition of multiple T1/E1 lines to a WAN link bundle. Furthermore, MLP supports all of the security options of PPP (such as CHAP authentication).

Slow-Speed (≤768 kbps) Leased Lines: Use MLP LFI and cRTP. For slow-speed leased lines, LFI is required to minimize serialization delay.
MLP, therefore, is the only encapsulation option on slow-speed leased lines because MLP LFI is the only mechanism available for fragmentation and interleaving on such links.

Medium-Speed (≤ T1/E1) Leased Lines: MLP LFI is not required; cRTP is optional. Medium-speed leased lines can use HDLC, PPP, or MLP encapsulation. An advantage of using MLP encapsulation is that future growth (to multiple T1/E1 links) will be easier to manage.

High-Speed (Multiple T1/E1 or Greater) Leased Lines: Use MLP bundling. MLP bundling attained the best overall SLA (Service-Provider Service-Level Agreement) values for delay and jitter, but it required more CPU resources than IP CEF per-packet load balancing. If CPU levels are kept under the recommended 75 percent, it is recommended to use MLP bundling for multiple T1/E1 links.

Frame Relay: use class-based Frame Relay traffic shaping whenever possible. Frame Relay networks are the most popular WANs in use today because of the low costs associated with them. Frame Relay is a nonbroadcast multiaccess (NBMA) technology that frequently utilizes
oversubscription to achieve cost savings (similar to airlines overselling seats on flights to achieve maximum capacity and profitability).

To manage oversubscription and potential speed mismatches between senders and receivers, a traffic-shaping mechanism must be used with Frame Relay. Either Frame Relay traffic shaping (FRTS) or class-based FRTS can be used. The primary advantage of using class-based FRTS is management because shaping statistics and queuing statistics are displayed jointly with the show policy interface verification command and are included in the SNMPv2 Cisco class-based QoS Management Information Base (MIB).

Slow-Speed (≤ 768 kbps) Frame Relay Links: Enable FRF.12 and set the fragment size for 10 ms maximum serialization delay. Enable cRTP.

Medium-Speed (≤ T1/E1) Frame Relay Links: FRF.12 is not required. cRTP is optional.

High-Speed (Multiple T1/E1 and Greater) Frame Relay Links: Use IP CEF per-packet load balancing for load sharing across multiple physical
Frame Relay links.

ATM: Two options exist for carrying voice traffic over slow-speed ATM PVCs: either Multilink PPP over ATM (MLPoATM), in conjunction with MLP LFI, or ATM PVC bundling. ATM PVC bundling is a legacy technique that has drawbacks such as inefficient bandwidth utilization and classification limitations. But sometimes service providers make ATM PVC bundles economically attractive to enterprise customers.

Slow-Speed (≤ 768 kbps) ATM Links: MLPoATM: Use MLP LFI. Tune the ATM PVC Tx-ring to 3. cRTP can be used only in Cisco IOS Release 12.2(2)T or later.

Slow-Speed (≤ 768 kbps) ATM Links: ATM PVC Bundles: Queuing policies for voice are not required (because voice uses a dedicated ATM
PVC). Tune the ATM PVC Tx-ring to 3.

Medium-Speed (≤ T1/E1) ATM Links: Use ATM inverse multiplexing over ATM (IMA) to keep future expansion easy to manage. No LFI is required. cRTP is optional.

High-Speed (Multiple T1/E1) ATM Links: Use ATM IMA and add members to the IMA group, as needed.