doc: Document MTU features in User Manual and example config files

Related: OS#6298
Related: SYS#7122
Change-Id: Ib6e974b38107fe48072380f768e1881f0fc95e80

doc/examples/osmo-ggsn-kernel-gtp.cfg

@@ -44,6 +44,7 @@ ggsn ggsn0
   gtpu-mode kernel-gtp
   tun-device tun4
   type-support v4
+  mtu default apply
   ip prefix dynamic 172.16.222.0/24
   ip dns 0 8.8.8.8
   ip dns 1 8.8.4.4

doc/examples/osmo-ggsn.cfg

@@ -44,6 +44,7 @@ ggsn ggsn0
   gtpu-mode tun
   tun-device tun4
   type-support v4
+  mtu default apply
   ip prefix dynamic 172.16.222.0/24
   ip dns 0 8.8.8.8
   ip dns 1 8.8.4.4
@@ -53,6 +54,7 @@ ggsn ggsn0
   gtpu-mode tun
   tun-device tun6
   type-support v6
+  mtu default apply
   ipv6 prefix dynamic 2001:780:44:2000:0:0:0:0/56
   ipv6 dns 0 2001:4860:4860::8888
   ipv6 dns 1 2001:4860:4860::8844
@@ -62,6 +64,7 @@ ggsn ggsn0
   gtpu-mode tun
   tun-device tun46
   type-support v4v6
+  mtu default apply
   ip prefix dynamic 172.16.46.0/24
   ip dns 0 8.8.8.8
   ip dns 1 8.8.4.4

doc/manuals/chapters/configuration.adoc

@@ -1,3 +1,4 @@
+[[osmoggsn_configuring]]
 == Configuring OsmoGGSN
 
 All configuration of OsmoGGSN is performed using the VTY. For more
@@ -22,6 +23,7 @@ ggsn ggsn0
   gtpu-mode tun
   tun-device tun4
   type-support v4
+  mtu default apply
   ip prefix dynamic 176.16.222.0/24
   ip dns 0 192.168.100.1
   ip dns 1 8.8.8.8
@@ -162,18 +164,20 @@ out of shutdown.
 ----
 OsmoGGSN(config-ggsn-apn)# gtpu-mode tun <1>
 OsmoGGSN(config-ggsn-apn)# type-support v4 <2>
-OsmoGGSN(config-ggsn-apn)# ip prefix dynamic 176.16.222.0/24 <3>
+OsmoGGSN(config-ggsn-apn)# mtu 1420 apply <3>
-OsmoGGSN(config-ggsn-apn)# ip dns 0 192.168.100.1 <4>
+OsmoGGSN(config-ggsn-apn)# ip prefix dynamic 176.16.222.0/24 <4>
-OsmoGGSN(config-ggsn-apn)# ip dns 1 8.8.8.8 <5>
+OsmoGGSN(config-ggsn-apn)# ip dns 0 192.168.100.1 <5>
-OsmoGGSN(config-ggsn-apn)# ip ifconfig 176.16.222.0/24 <6>
+OsmoGGSN(config-ggsn-apn)# ip dns 1 8.8.8.8 <6>
+OsmoGGSN(config-ggsn-apn)# ip ifconfig 176.16.222.0/24 <7>
 ----
 <1> Use the userspace GTP-U handling using a TUN device
 <2> Support (only) IPv4 Addresses
-<3> Specify the pool of dynamic IPv4 addresses to be allocated to PDP
+<3> Specify MTU to announce to MS. Apply the MTU on the tunnel interface.
+<4> Specify the pool of dynamic IPv4 addresses to be allocated to PDP
     contexts
-<4> Specify the primary DNS server to be provided using IPCP/PCO
+<5> Specify the primary DNS server to be provided using IPCP/PCO
-<5> Specify the secondary DNS server to be provided using IPCP/PCO
+<6> Specify the secondary DNS server to be provided using IPCP/PCO
-<6> Request OsmoGGSN to configure the `tun4` device network/netmask
+<7> Request OsmoGGSN to configure the `tun4` device network/netmask
 
 NOTE:: If you use the optional `ip ifconfig` command to set the network
 device address/mask, OsmoGGSN must run with root or `CAP_NET_ADMIN`
@@ -271,6 +275,7 @@ to match the `ip prefix dynamic` config item, and activate the link, for example
 
 ----
 # ip addr add 192.168.7.1/24 dev apn0
+# ip link set mtu 1420 dev apn0
 # ip link set apn0 up
 ----
 
@@ -298,13 +303,18 @@ Group=username <3>
 [Match]
 Name=apn0 <1>
 
+[Link]
+MTUBytes=1420 <2>
+
 [Network]
-Address=192.168.7.1/24 <2>
+Address=192.168.7.1/24 <3>
-IPMasquerade=yes <3>
+IPMasquerade=yes <4>
 ----
 <1> The network device name, which must match the one in the apn0.netdev unit file above
-<2> The local IP address configured on the device
+<2> Requesting systemd to set the MTU for this interface. The MTU of the tun
-<3> Requesting systemd to configure IP masquerading for this interface.  Depending on your needs,
+    interface should be lower than regular, since it must accommodate the extra IP/UDP/GTPv1U headers.
+<3> The local IP address configured on the device
+<4> Requesting systemd to configure IP masquerading for this interface.  Depending on your needs,
     You may not want this if you have proper end-to-end routing set up, and want to have transparent
     inbound IP access to your GPRS-attached devices.
 
@@ -326,6 +336,7 @@ ggsn ggsn0
   gtpu-mode tun
   tun-device apn0
   type-support v4
+  mtu 1420
   ip prefix dynamic 192.168.7.0/24
   ip dns 0 192.168.100.1
   ip dns 1 8.8.8.8

doc/manuals/chapters/mtu.adoc

@@ -0,0 +1,171 @@
+=== MTU considerations
+
+When running OsmoGGSN, the user may want to take network Maximum Transmission
+Unit (MTU) into consideration, and configure it based on network specific setup.
+
+Applying and announcing a proper MTU provides, for the MS employing it, reduced
+transmission overhead (ie. due to IP fragmentation) and avoids potential
+problems due to misconfigured nodes in the path (e.g. ICMP packet filtering).
+
+In OsmoGGSN, the MTU can be configured per APN through the VTY, see
+<<osmoggsn_configuring>>. If told so by the config, osmo-ggsn will apply the MTU
+on the APN network interface.
+
+==== MTU announced to MS
+
+The configured MTU is also announced to the MS through:
+
+* IPv4 APN: GTPv1C Create PDP Context Response, PCO IE "IPv4 Link MTU", 3GPP TS
+  24.008 Table 10.5.154.
+* IPv6 APN: ICMPv6 Routing Advertisement during IPv6 SLAAC procedure, RFC 4861.
+
+NOTE: It is up to the MS to request and use the link MTU size provided by the
+network. Hence, providing an MTU size does not guarantee that there will be no
+packets larger than the provided value.
+
+==== GTP-U tunnel overhead
+
+OsmoGGSN is encapsulating traffic over GTP-U, it means the packets being received,
+encapsulated and transmitted over the tunnel get their size increased by the sum of
+IP/UDP/GTPv1U headers being prepended:
+
+* IP: IPv4 headers can take up to 60 bytes (due to IPv4 options). IPv6 headers
+  can take up to 40 bytes (assuming no extension headers for IPv6 in general,
+  since they are uncommon). Hence, the 60 bytes of IPv4 are picked since that's
+  greater than the IPv4.
+* UDP: The UDP header takes 8 bytes.
+* GTPv1U: The GTPv1U header takes 12 bytes, assuming here no extensions headers
+  are used (OsmoGGSN doesn't use them).
+
+Hence, these headers add an overhead of up to `80`` bytes, as per the below formula:
+
+----
+GTPv1U_OVERHEAD = 60 + 8 + 12 = 80 bytes
+----
+
+==== Figuring out optimal MTU value
+
+There is no golden MTU value, since it really depends on the local (and remote)
+networks where traffic is routed. The idea is finding out a value that:
+
+* Is as big as possible, to avoid need to split big chunks of data into lots of
+  small packets, hence affecting performance due to processing overhead: extra
+  headers being trnasmitted, plus processing of extra packets.
+* Is small enough so that it can be transported over the lower layers of the
+  links involving the communication, avoiding IP fragmentation, which again hurts
+  performance.
+
+OsmoGGSN, by default, assumes that traffic is transported over an Ethernet
+network, which has a standarized maximum MTU of 1500 bytes. Hence, by default it
+announces an MTU of of `1420` bytes as per the following formula:
+
+----
+TUNNEL_MTU = ETH_MTU - GTPv1U_OVERHEAD = 1500 - 80 = 1420 bytes
+----
+
+Under certain networks, the base MTU may already be smaller than Ethernet's MTU
+(1500 bytes), due to, for instance, existence of some sort of extra tunneling
+protocol in the path, such as a VPN, ipsec, extra VLAN levels, etc. Under this
+scenario, the user must take care of figuring out the new base MTU value to use
+for the calculations presented above. This can be accomplished by packet
+inspection (eg. `wireshark`) or with tools such as `ping`, running it with a
+certain packet size and the IPv4 DF bit set, and see up to which packet size the
+networks is able to forward the message.
+
+.Example: Test if packets of 1420 bytes can reach peer host 176.16.222.4
+----
+$ ping -M probe 176.16.222.4 -s 1420
+----
+
+=== Increasing outer MTU
+
+Specifications at IEEE 802.3 establish that standard Ethernet has a maximum MTU
+of `1500` bytes.
+However, many Ethernet controllers can nowadays overcome this limit and allow
+the use of so called _jumbo frames_. The _jumbo frames_ maximum MTU varies
+depending on the implementation, with `9000` bytes being a commonly used limit.
+
+Note that using MTUs over the standarized `1500` bytes by means of _jumbo frames_
+can create interoperability problems with networks not supporting such frames
+(eg. forcing of IP packet fragmentation), plus the fact that larger frames
+consume more Ethernet link transmission time, causing greater delays and
+increasing latency.
+
+Nevertheless, if the operator:
+
+* is in control of the whole GTP-U path between OsmoGGSN and the MS, and
+* has Ethernet NICs supporting MTUs bigger than 1500 or uses any other link
+  layer supporting as well bigger MTUs.
+
+Then, it may be wise for the operator to configure such links with an increased
+outer MTU so that they can end up transporting GTP-U inner payload of 1500 bytes
+without fragmentation ocurring.
+
+Hence, following the examples presented on the above sections, one could
+configure *all the links* which are part of the GTP-U path to use an outer MTU
+of `1580` bytes, as per the following formula:
+
+----
+TUNNEL_MTU = ETH_MTU + GTPv1U_OVERHEAD = 1500 + 80 = 1580 bytes
+----
+
+.Example: Setting an MTU of `1580` to network interface `eth0` under Linux
+----
+ip link set mtu 1580 dev eth0
+----
+
+==== TCP MSS Clamping
+
+Usually endpoints use Path MTU Discovery (PMTUD) to determine the maximum MTU to
+reach the peer. However, this technique may sometimes not be optimal for all
+users of OsmoGGSN:
+
+* MS may not support requesting and/or configuring the MTU OsmoGGSN announced.
+* MS may not support PMTUD on its network stack, or may not have it enabled or
+  may be buggy.
+* Network may be misconfigured or some middlebox may be buggy (eg. not
+  forwarding ICMP `Packet Too Big` packets).
+
+Furthermore, PMTUD takes time to figure out the maximum MTU to use, since it
+relies on sending data and checking if it got lost, and adapting to the fact,
+reducing efficiency (throughput) of connections or even stalling them completely
+when big packets are generated.
+
+Hence, it may become useful for the operator of OsmoGGSN to, on top of MTU
+configuration, also configure its network to tune TCP Maximum Segment Size (MSS)
+option of TCP connections being established over the GTPv1U tunnel. This will
+make sure at least TCP connections can use the full capacity of the path MTU
+without passing its imit.
+
+The MSS TCP option is an optional parameter in the TCP header sent during TCP
+initial handshake (`SYN,SYN/ACK`) that specifies the maximum amount of bytes of
+TCP payload a TCP chunk may transport. The MSS value doesn't count the
+underlaying IP/TCP headers.
+
+Hence, following up on MTU size calculations from previous section, with a
+sample GTPv1U MTU of 1420 bytes and IP header of 60 bytes, plus taking into
+account that TCP header can span up to 56 bytes, we'd get to an MSS value of:
+
+----
+MSS = TUNNEL_MTU - IP_HDR - TCP_HDR = 1420 - 60 - 56 = 1304
+----
+
+In linux, the MSS of TCP connections can be clamped using iptables:
+
+----
+iptables -t nat -A PREROUTING -p tcp --tcp-flags SYN,RST SYN -i apn0 -j TCPMSS --set-mss 1304
+iptables -t nat -I POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o apn0 -j TCPMSS --set-mss 1304
+ip6tables -t nat -A PREROUTING -p tcp --tcp-flags SYN,RST SYN -i apn0 -j TCPMSS --set-mss 1304
+ip6tables -t nat -I POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o apn0 -j TCPMSS --set-mss 1304
+----
+
+==== Further Reading
+
+Check the following specs regarding MTU in 3GPP mobile networks:
+
+* 3GPP TS 29.061 section 11.2.1.5
+* 3GPP TS 290.060 section 13.2 IP Fragmentation
+* 3GPP TS 25.414 section 6.1.3.3
+* 3GPP TS 23.060 section 9.3, Annex C
+* 3GPP TS 24.008 (PCO IPv4 MTU)
+* RFC 4861 (IPv6 Router Advertisement)

doc/manuals/chapters/running.adoc

@@ -52,6 +52,8 @@ There are various ways to enable these settings persistently, please refer to
 your distribution's documentation -- e.g. look for @net.ipv4.ip_forward=1@ in
 @/etc/sysctl.d/@, and https://wiki.debian.org/iptables for masquerading.
 
+include::{srcdir}/chapters/mtu.adoc[]
+
 === Multiple instances
 
 Running multiple instances of `osmo-ggsn` is possible if all GGSN instances