Quality of Service configuration for the traffic entering/leaving a VPN tunnel may require some special considerations. In this article, I am going to focus on interactions between QoS and IPSec on IOS and the ASA.
There are two methods of deploying QoS for VPNs – you can match the original (Clear-text/ unencrypted) traffic flows or the actual VPN (Aggregate traffic). This second option can be useful when you want to apply a single QoS policy to all packets leaving a tunnel, no matter what are the original sources and destinations protected by the VPN.
We have got a VPN tunnel built between R1 and ASA. R6 and 10.1.1.0/24 are protected networks
Let’s start on IOS (R1). The VPN tunnel is already up – we will configure a basic QoS Policy to enable LLQ for delay-sensitive traffic, such as Voice (I assume these are all packets with DSCP of EF). Note that this configuration would normally match all EF-colored packets (including non-VPN EF traffic), but since we won’t have any clear-text EF flows in this network we don’t really care:
class-map match-all VOICE
match dscp ef
policy-map QOS
class VOICE
priority
int f0/0
service-policy output QOS
Voice traffic will be now emulated by our switches CAT2 and CAT3 using Echo packets with DSCP value of EF (decimal value 46 = ToS decimal value 184 = ToS in hex 0xB8) :
CAT2#ping
Protocol [ip]:
Target IP address: 192.1.6.6
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface:
Type of service [0]: 0xb8
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.1.6.6, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/4/9 ms
CAT3#ping
Protocol [ip]:
Target IP address: 192.1.6.6
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface:
Type of service [0]: 0xb8
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.1.6.6, timeout is 2 seconds:
.!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/7/17 ms
Note that 9 packets went through the tunnel and they were all matched by our VOICE class, so prioritized :
R1#sh cry sess det
Crypto session current status
Code: C - IKE Configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal, T - cTCP encapsulation
X - IKE Extended Authentication, F - IKE Fragmentation
Interface: FastEthernet0/0
Uptime: 00:01:20
Session status: UP-ACTIVE
Peer: 192.1.4.31 port 500 fvrf: (none) ivrf: (none)
Phase1_id: 192.1.4.31
Desc: (none)
IKEv1 SA: local 192.1.14.1/500 remote 192.1.4.31/500 Active
Capabilities:(none) connid:1004 lifetime:23:58:39
IPSEC FLOW: permit ip 10.1.1.0/255.255.255.0 192.1.6.0/255.255.255.0
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 9 drop 0 life (KB/Sec) 4565014/3519
Outbound: #pkts enc'ed 9 drop 1 life (KB/Sec) 4565014/3519
R1#sh policy-map int f0/0
FastEthernet0/0
Service-policy output: QOS
queue stats for all priority classes:
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 10/1660
Class-map: VOICE (match-all)
9 packets, 1494 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: dscp ef (46)
Priority: Strict, burst bytes 1500, b/w exceed drops: 0
Class-map: class-default (match-any)
101 packets, 9305 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 101/9584
OK, looks like we don’t have any issues here. All EF-marked packets entering the VPN were properly classified and prioritized. How about if we change our policy? We will still be matching EF packets, but, let’s say, only sourced by CAT2 :
access-list 160 permit ip host 10.1.1.120 host 192.1.6.6
class-map match-all VOICE
match access-group 160
R1#sh class-map VOICE
Class Map match-all VOICE (id 1)
Match dscp ef (46)
Match access-group 160
Packets will be now first sent from CAT3, then from CAT2. Here are the results :
After CAT3 :
R1#sh policy-map int f0/0
FastEthernet0/0
Service-policy output: QOS
queue stats for all priority classes:
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 9/1494
Class-map: VOICE (match-all)
9 packets, 1494 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: dscp ef (46)
Match: access-group 160
Priority: Strict, burst bytes 1500, b/w exceed drops: 0
Class-map: class-default (match-any)
441 packets, 41417 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
After CAT2 :
R1#sh policy-map int f0/0
FastEthernet0/0
Service-policy output: QOS
queue stats for all priority classes:
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 9/1494
Class-map: VOICE (match-all)
9 packets, 1494 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: dscp ef (46)
Match: access-group 160
Priority: Strict, burst bytes 1500, b/w exceed drops: 0
Class-map: class-default (match-any)
475 packets, 41417 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
R1#sh cry sess det
Crypto session current status
Code: C - IKE Configuration mode, D - Dead Peer Detection
K - Keepalives, N - NAT-traversal, T - cTCP encapsulation
X - IKE Extended Authentication, F - IKE Fragmentation
Interface: FastEthernet0/0
Uptime: 00:28:39
Session status: UP-ACTIVE
Peer: 192.1.4.31 port 500 fvrf: (none) ivrf: (none)
Phase1_id: 192.1.4.31
Desc: (none)
IKEv1 SA: local 192.1.14.1/500 remote 192.1.4.31/500 Active
Capabilities:(none) connid:1004 lifetime:23:31:20
IPSEC FLOW: permit ip 10.1.1.0/255.255.255.0 192.1.6.0/255.255.255.0
Active SAs: 2, origin: crypto map
Inbound: #pkts dec'ed 19 drop 0 life (KB/Sec) 4565013/1880
Outbound: #pkts enc'ed 19 drop 1 life (KB/Sec) 4565013/1880
R1#sh access-list 160
Extended IP access list 160
10 permit ip host 10.1.1.120 host 192.1.6.6 (5 matches)
Wow… hold on. We are no longer matching the Voice class, even from CAT2 (though we have a match in the ACL). What’s the problem??
This behavior is perfectly normal on IOS. The problem we run into here is that IOS devices apply IPSec services BEFORE QoS by default. So our QoS engine sees already encrypted traffic (from R1 to the ASA), and not the original flow (from 10.1.1.120 to 192.1.6.6). The reason we had the packets matched in our first test (without an ACL), is that ToS field is always copied to the transport header of the IPSec packet (or in other words IPSec always copies the ToS from the original packet). This means that pure ToS-based matching, including DSCP/IP Precedence which are part of the ToS field, will always work, and this is how you can match the traffic as an Aggregate (another Aggregate option would be to match the VPN tunnel itself – so ESP/AH).
Anything we can do to allow for more granular policies, like what we tried to accomplish in our second test (Clear-text flow)? Yes. This is known as QoS Pre-Classification. Simply enable it in the crypto map to make sure that QoS will see the original traffic flows, before encryption/IPSec kicks in :
crypto map MAP10 10 ipsec-isakmp
qos pre-classify
R1#sh crypto map int f0/0
Crypto Map "MAP10" 10 ipsec-isakmp
Peer = 192.1.4.31
Extended IP access list 157
access-list 157 permit ip 10.1.1.0 0.0.0.255 192.1.6.0 0.0.0.255
Current peer: 192.1.4.31
Security association lifetime: 4608000 kilobytes/3600 seconds
Responder-Only (Y/N): N
PFS (Y/N): N
Transform sets={
SET4: { esp-3des esp-md5-hmac } ,
}
QOS pre-classification
Interfaces using crypto map MAP10:
FastEthernet0/0
Another five EF Echos from CAT2 (traffic from CAT3 of course will not be a match) and better :
R1#sh policy-map int f0/0
FastEthernet0/0
Service-policy output: QOS
queue stats for all priority classes:
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 14/2324
Class-map: VOICE (match-all)
14 packets, 2064 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: dscp ef (46)
Match: access-group 160
Priority: Strict, burst bytes 1500, b/w exceed drops: 0
Class-map: class-default (match-any)
957 packets, 90525 bytes
5 minute offered rate 0 bps, drop rate 0 bps
Match: any
queue limit 64 packets
(queue depth/total drops/no-buffer drops) 0/0/0
(pkts output/bytes output) 957/91962
Now let’s move on to the ASA. Are we going to see any differences from IOS? I can tell you now – yes. On the ASA there are no special commands needed to match the Original flows – QoS happens before and even after IPSec. Quick test to show you how to police packets to CAT2 :
access-list 150 permit ip host 192.1.6.6 host 10.1.1.120
class-map R6TOCAT2
match access-list 150
policy-map QOS
class R6TOCAT2
police output 16000
service-policy QOS interface outside
R6#ping 10.1.1.120 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.120, timeout is 2 seconds:
.!!!.!!!.!
Success rate is 70 percent (7/10), round-trip min/avg/max = 4/5/8 ms
R6#ping 10.1.1.130 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.130, timeout is 2 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 4/6/8 ms
A1(config)# show vpn-sessiondb l2l
Session Type: LAN-to-LAN
Connection : 192.1.14.1
Index : 4 IP Addr : 192.1.14.1
Protocol : IKEv1 IPsec
Encryption : 3DES Hashing : MD5
Bytes Tx : 17000 Bytes Rx : 17000
Login Time : 20:07:33 UTC Tue Jan 20 2015
Duration : 0h:03m:14s
A1(config)# show service-policy police
Interface outside:
Service-policy: QOS
Class-map: R6TOCAT2
Output police Interface outside:
cir 16000 bps, bc 1500 bytes
conformed 7 packets, 7448 bytes; actions: transmit
exceeded 2 packets, 2128 bytes; actions: drop
conformed 728 bps, exceed 208 bps
Now let’s change this ACL to match ESP between the ASA and R1 and observe the results :
no access-list 150 ext permit ip host 192.1.6.6 host 10.1.1.1
access-list 150 ext permit esp host 192.1.4.31 host 192.1.14.1
R6#ping 10.1.1.130 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.130, timeout is 2 seconds:
!!!.!!!.!!
Success rate is 80 percent (8/10), round-trip min/avg/max = 4/6/12 ms
R6#ping 10.1.1.120 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.120, timeout is 2 seconds:
!!!.!!!.!!
Success rate is 80 percent (8/10), round-trip min/avg/max = 4/5/8 ms
So here we are matching the Aggregate for this single VPN – ASA was also able to catch traffic after encryption. ToS-based classification would also work – again, IPSec copies ToS field to the transport header in the protected packet and ASA would see it.
That’s not the end. There is also another way to match Aggregate traffic on the ASA that is not available on IOS. We can match a Tunnel Group and all traffic leaving it via “match tunnel-group”, or even do this per-VPN peer when multiple devices terminate a tunnel on a single ASA’s Tunnel Group (just add “match flow ip destination-address” to the class-map that matches a Tunnel Group). This second option would be useful for Remote Access VPNs.
Let’s look at that – I am just going to switch from Policing to LLQ, since Policing only works with “match flow ip destination-address” – and this command is pretty much useless for L2L tunnels because there is always just one peer for a single Tunnel Group. Here are the changes :
priority-queue outside
policy-map QOS
class R6TOCAT2
priority
class-map R6TOCAT2
no match access-list 150
match tunnel-group 192.1.14.1
R6#ping 10.1.1.120 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.120, timeout is 2 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 4/6/8 ms
R6#ping 10.1.1.130 size 1000 rep 10
Type escape sequence to abort.
Sending 10, 1000-byte ICMP Echos to 10.1.1.130, timeout is 2 seconds:
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 4/6/8 ms
A1# sh service-policy priority
Interface outside:
Service-policy: QOS
Class-map: R6TOCAT2
Priority:
Interface outside: aggregate drop 0, aggregate transmit 20
If you were to configure destination flow –based matching and rate-limit in the policy for this Tunnel Group, you would see that our 16kbps limit applies to all clear-text flows together, no matter what are the sources and destinations. Again, the purpose of “match flow ip destination-address” is to apply QoS per VPN peer and not clear-text destinations.
To finish that discussion, let me recap on our classification options for IPSec VPNs :
1. IOS
- Aggregate traffic can be matched based on ToS field or using an ACL for ESP/AH
- Clear-text flows can be only matched with QoS Pre-Classification turned on
2. ASA
- Aggregate traffic can be matched based on ToS field or using an ACL for ESP/AH
- Aggregate traffic can be matched for the entire Tunnel Group or per VPN peer terminating its tunnel on this Tunnel Group (note that some QoS tools are not supported unless you enable this per-peer matching)
- Clear-text flows can be always classified without any special considerations
