The TCP Window, Latency, and the Bandwidth Delay productThe relation between the TCP Window, packet delay (pings) and maximum bandwidth
2008-09-15 10:06 by Philip
Tags: BDP, RFC, TCP Window, latency, bandwidth, MSS, MTU, packet
This article is intended as a primer on some TCP/IP networking concepts and factors that determine an optimal TCP Receive Window.
The TCP Window
The TCP Window is the amount of outstanding data (unacknowledged by the recepient) that can remain in the network. After sending that amount of data, the sender stops and waits for acknowledgement back from the receiver that it has gotten some of it. As such, this value is probably the single most important setting in tuning broadband internet connections. The TCP Window is negotiated at the beginning of every connection during the TCP "handshake" stage.
In the original DARPA TCP/IP standard, the TCP Receive Window (RWIN) was limited to 64K (65535), since there are only 16-bits in the TCP headers for the RWIN value, and 2^16=64K. This limitation needed to be addressed, and in 1992 RFC 1323 added a "TCP Options" header extension, which allowed for expanding the maximum TCP Window size by adding another byte to act as a "scale factor" to the RWIN value. The RFC1323 RWIN byte can contain any value between 0 and 14, as follows:
For example, let's assume an unscaled RWIN value of 64240, and a scale factor is 3. the actual RWIN value then would be: 64240 * 2^3 = 513920.
Note that the scale factor is limited to 14; 2^14=16384, and the maximum unscaled RWIN is 65535. 16384 * 65536 = 1,073,725,440 (a gigabyte). Thus, RFC1323 allows for a maximum TCP Receive Window of up to one gigabyte.
See also: TCP Header structure
The speed of every data transfer, like TCP is of course largely determined by the line speed. In addition, however, let's consider the delay, or RTT(round trip time) of each data packet.
Any time a client computer asks a server a question, there is a RTT delay until it receives a response. Data packets have to thravel through a number of high-traffic (sometimes congested) routers, and there is always the speed of light (or electricity for copper lines) as limitation, considering the huge distances of internet communication.
Let's examine a client computer communicating with a server over a geosynchronous satellite link. The client's request (every packet) has to travel 22,300 miles to the satellite, then 22,300 miles down to the server. Then, when the server sends its response, it has to travel the same distance back to the client, adding another 22,300 miles up + 22,300 miles down. Thus, that simple packet of data traveled at least 89,200 miles. Considering the speed of light (186,000 miles per second), we can conclude that there is a minimum round-trip delay on a satellite connection of about half a second (500ms).
The Bandwidth * Delay Product
The Bandwidth*Delay product, or BDP for short, determines the amount of data that can be in transit in the network (just as RWIN). It is the product of the available bandwidth and the latency (RTT). BDP is a very important concept in a window-based protocol such as TCP, as throughput is bound by the BDP ! The BDP states that:
BDP (bits) = bandwidth (bits/second) * latency (seconds)
What does it mean ? The BDP, and the TCP Receive Window limit our connection to the product of the latency and the bandwidth. A transmission can not exceed the RWIN / latency value.
See also: SG BDP calculator
Optimizing the TCP Receive Window
When calculating an optimal RWIN value, one should try to use as high as possible unscaled RWIN values (usually the highest MSS multiple under 65535) and a smaller scale factor. It is a much better method accounting for older routers and some wireless networks that don't work well with TCP Options (RFC1323), or large scale factors.
To determine the optimal TCP Receive Window, you can simply use one of the SG TCP Analyzer recommended values, or perform the following calculations:
TCP Window in Vista / Windows 7 / 2008 Server
In Windows Vista and 2008 server, Microsoft introduced a new TCP/IP stack with a number of improvements. It also includes a concept called TCP Window "Auto-Tuning" that's been used in Linux for years. The idea is, a small initial RWIN value is advertised, which is then adjusted on the fly depending on the current line speed and latency. This new implementation works much better by default, compared to previous Windows versions. In theory, the new automatic RWIN algorithm adjusts the TCP Window size based on three main factors:
The algorithm has the ability to control the TCP Window value per connection. Also, by default, Vista/2008 will not allocate RWIN values larger than 16Mb.
There are still a couple of downsides to the new approach:
For additional information on tunning TCP/IP under Vista, see our Windows Vista/2008 tweaks article.