Share this post on:


Very often when IPSec tunnel is used, throughput is affected or users are experiencing fragmentation issues. This is caused by incorrect MTU size and encapsulation overhead. Common example is when icmp ping works both way without any issues, or manual telnet to www port is open but the actual page won’t open or opens intermittently.


MTU size for Ethernet is 1500 (1514 if we count 802.1 Ethernet header).When original IP packet gets encrypted by IPSec, there’s an overall increase in packet size. Any encapsulation that takes place, adds overhead to the original packet size:

  • IPSec encryption performed by the DMVPN adds 73 bytes for ESP-AES-256 and ESP-SHA-HMAC overhead (overhead depends on transport or tunnel mode and the encryption/authentication algorithm and HMAC).
  • MPLS adds 4 bytes for each label in the stack.
  • IEEE 802.1Q tag adds 4 bytes (Q-in-Q would add 8 bytes)

Most networking devices use Path MTU to calculate proper MTU size on the entire path. This works by setting DF-bit to 1 and forcing MTU size. If MTU size along the way to destination is too small, router/firewall will inform the host and drops the packet and sends an ICMP Fragmentation Needed Type 3 Code 4 packet back to the sending device with its MTU size.


To fix the issue, we need to determine our MTU size in non-VPN enivorment:

[root@smtp ~]# ping -M do -s 1472
PING ( 1472(1500) bytes of data.
76 bytes from icmp_seq=1 ttl=111 (truncated)
76 bytes from icmp_seq=2 ttl=111 (truncated)
76 bytes from icmp_seq=3 ttl=111 (truncated)
76 bytes from icmp_seq=4 ttl=111 (truncated)
76 bytes from icmp_seq=5 ttl=111 (truncated)
76 bytes from icmp_seq=6 ttl=111 (truncated)
76 bytes from icmp_seq=7 ttl=111 (truncated)

As we can see in test above, our interface MTU is default 1500 bytes (1500-28bytes for IP+ICMP overhead = 1472). Now, let’s calculate the IPSec overhead based on encryption used:

IPSec Transform SetIPSec Overhead, Maximum Bytes
esp-AES-(256 or 192 or 128) esp-SHA-hmac or md5  73
esp-AES (256 or 192 or 128)61
esp-3des, esp-DES45
esp-(DES or 3des) esp-SHA-hmac or md5 57
esp-null esp-SHA-hmac or md5 45
ah-SHA-hmac or md5 44

Let’s calculate our proper MTU size using the formula:

MTU size – encapsulation overhead = interface MTU

1500 – 61 = 1439 definitive MTU

Depending on the vendor used, we can update our MTU size to calculated value.

Share this post on:

Leave a Comment

Your email address will not be published. Required fields are marked *