The TCP/IP parameters for tweaking a Linux-based machine for fast internet connections are located in /proc/sys/net/... (assuming 2.1+ kernel). This location is volatile, and changes are reset at reboot. There are a couple of methods for reapplying the changes at boot time, ilustrated below.
Locating the TCP/IP related parameters
All TCP/IP tunning parameters are located under /proc/sys/net/... For example, here is a list of the most important tunning parameters, along with short description of their meaning:
/proc/sys/net/core/rmem_max - Maximum TCP Receive Window
/proc/sys/net/core/wmem_max - Maximum TCP Send Window
/proc/sys/net/ipv4/tcp_timestamps - timestamps (RFC 1323) add 12 bytes to the TCP header...
/proc/sys/net/ipv4/tcp_sack - tcp selective acknowledgements.
/proc/sys/net/ipv4/tcp_window_scaling - support for large TCP Windows (RFC 1323). Needs to be set to 1 if the Max TCP Window is over 65535.
Keep in mind everything under /proc is volatile, so any changes you make are lost after reboot.
There are some additional internal memory buffers for the TCP Window, allocated for each connection:
/proc/sys/net/ipv4/tcp_rmem - memory reserved for TCP rcv buffers (reserved memory per connection default)
/proc/sys/net/ipv4/tcp_wmem - memory reserved for TCP snd buffers (reserved memory per connection default)
The tcp_rmem and tcp_wmem contain arrays of three parameter values: the 3 numbers represent minimum, default and maximum memory values. Those 3 values are used to bound autotunning and balance memory usage while under global memory stress.
Applying TCP/IP Parameters at System Boot
You can edit /etc/rc.local, or /etc/boot.local depending on your distribution so the parameters get automatically reapplied at boot time. The TCP/IP parameters should be self-explanatory: we're basically setting the TCP Window to 256960, disabling timestamps (to avoid 12 byte header overhead), enabling tcp window scaling, and selective acknowledgements:
echo 256960 > /proc/sys/net/core/rmem_default
echo 256960 > /proc/sys/net/core/rmem_max
echo 256960 > /proc/sys/net/core/wmem_default
echo 256960 > /proc/sys/net/core/wmem_max
echo 0 > /proc/sys/net/ipv4/tcp_timestamps
echo 1 > /proc/sys/net/ipv4/tcp_sack
echo 1 > /proc/sys/net/ipv4/tcp_window_scaling
Change the values above as desired, depending on your internet connection and maximum bandwidth/latency. There are other parameters you can change from the default if you're confident in what you're doing - just find the correct syntax of the values in /proc/sys/net/... and add a line in the above code analogous to the others. To revert to the default parameters, you can just comment or delete the above code from /etc/rc.local and restart.
Another method to reapply the values upon boot is to include the following in your /etc/sysctl.conf (adjust RWIN values as needed):
net.core.rmem_default = 256960
net.core.rmem_max = 256960
net.core.wmem_default = 256960
net.core.wmem_max = 256960
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
Notes:
Execute sysctl -p to make these new settings take effect.
To manually set the MTU value under Linux, use the command: ifconfig eth0 mtu 1500 (where 1500 is the desired MTU size)
Changing Current Values without rebooting
The current TCP/IP parameters can be edited without the need for reboot in the following locations:
/proc/sys/net/core/
rmem_default = Default Receive Window
rmem_max = Maximum Receive Window
wmem_default = Default Send Window
wmem_max = Maximum Send Window
/proc/sys/net/ipv4/
You'll find timestamps, window scalling, selective acknowledgements, etc.
Keep in mind the values in /proc will be reset upon reboot. You still need to add the code in /etc/rc.local or /etc/boot.local in order to have the changes applied at boot time as described above.
Other TCP Parameters to consider
TCP_FIN_TIMEOUT
This setting determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. During this TIME_WAIT state, reopening the connection to the client costs less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, making more resources available for new connections. Addjust this in the presense of many connections sitting in the TIME_WAIT state:
# echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout
(default: 60 seconds, recommended 15-30 seconds)
Notes:
You can use any of the earlier described methods to reapply these settings at boot time.
Here is a quick way to view the number of connections and their states:
netstat -tan | grep ':80 ' | awk '{print $6}' | sort | uniq -c
TCP_KEEPALIVE_INTERVAL
This determines the wait time between isAlive interval probes. To set:
echo 30 > /proc/sys/net/ipv4/tcp_keepalive_intvl
(default: 75 seconds, recommended: 15-30 seconds)
TCP_KEEPALIVE_PROBES
This determines the number of probes before timing out. To set:
echo 5 > /proc/sys/net/ipv4/tcp_keepalive_probes
(default: 9, recommended 5)
TCP_TW_RECYCLE
It enables fast recycling of TIME_WAIT sockets. The default value is 0 (disabled). The sysctl documentation incorrectly states the default as enabled. It can be changed to 1 (enabled) in many cases. Known to cause some issues with hoststated (load balancing and fail over) if enabled, should be used with caution.
echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
(boolean, default: 0)
TCP_TW_REUSE
This allows reusing sockets in TIME_WAIT state for new connections when it is safe from protocol viewpoint. Default value is 0 (disabled). It is generally a safer alternative to tcp_tw_recycle
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
(boolean, default: 0)
Note: The tcp_tw_reuse setting is particularly useful in environments where numerous short connections are open and left in TIME_WAIT state, such as web servers. Reusing the sockets can be very effective in reducing server load.
Kernel Recompile Option
There is another method one can use to set TCP/IP parameters, involving kernel recompile... If you're brave enough. Look for the parameters in the following files:
/LINUX-SOURCE-DIR/include/linux/skbuff.h
Look for SK_WMEM_MAX & SK_RMEM_MAX
/LINUX-SOURCE-DIR/include/net/tcp.h
Look for MAX_WINDOW & MIN_WINDOW
References
See also the complete ip-sysctl parameters reference -here-
TCP Variables
Articles
- 2008.03.09 18:44
by
by
- 2009.05.15 11:14
