Jumbo packets and hardware checksumming

Jumbo packets are packets that carry more payload than the normal 1500 bytes. Even the definition of a jumbo packet is unclear; different people use different lengths.

The io-pkt (hardware-independent) stack and the drivers support jumbo packets, but not all network hardware does (generally, newer GiGE NICs do).

If you can use jumbo packets, you can see substantial performance gains because more data can be moved per packet header processing overhead.

To configure a driver to operate with jumbo packets, do this (for example):

# ifconfig wm0 ip4csum tcp4csum udp4csum
# ifconfig wm0 mtu 8100
# ifconfig wm0 10.42.110.237

For maximum performance, we also turned on hardware packet checksumming (for both transmit and receive) and we've arbitrarily chosen a jumbo packet MTU of 8100 bytes. A little detail: io-pkt by default allocates 2 KB clusters for packet buffers. This works well for 1500 byte packets, but for example when an 8 KB jumbo packet is received, we end up with 4 linked clusters. We can improve performance by telling io-pkt (when we start it) that we're going to use jumbo packets, like this:

# io-pkt-v6-hc -d e1000 -p tcpip pagesize=8192,mclbytes=8192

If we pass the pagesize and mclbytes command-line options to the stack, we tell it to allocate contiguous 8 KB buffers (which may end up being two adjacent 4 KB pages, which works fine) for each 8 KB cluster to use for packet buffers. This reduces packet processing overhead, which improves throughput and reduces CPU utilization.