Latency vs. Bandwidth: The Key to Faster Applications

Whether you are building a one-page static site or a product containing hundreds of microservices, speed is a critical feature. A slow-performing application can never survive without adequate performance. In fact, performance is directly associated with retaining end users and keeping them engaged. A slow response time will allow the user to start thinking about something else than engaging with your application.

Delivering optimal speed has many factors and limitations including how well the underlying network is performing and to dictate the performance of all network traffic, we have two critical components: latency and bandwidth.

Latency

Latency is the measurement of time from the source (origin server) sending a packet to the destination (client) receiving it. This is a very simple definition of the Latency but is it really that simple? What are the factors that contribute to the total latency? For example, let’s have a look at a simple traceroute that tries to connect to google.com.

Huntize-MacBook-Pro ~ % traceroute google.com
traceroute to google.com (142.250.217.110), 64 hops max, 40 byte packets
 1  10.0.0.1 (10.0.0.1)  15.418 ms  21.467 ms  6.141 ms
 2  70.77.224.1 (70.77.224.1)  22.795 ms  19.908 ms  25.696 ms
 3  rc3so-be123-1.cg.shawcable.net (64.59.134.233)  21.476 ms  22.205 ms  19.291 ms
 4  24.244.57.126 (24.244.57.126)  22.565 ms  19.428 ms  18.690 ms
 5  24.83.252.70 (24.83.252.70)  20.447 ms  26.846 ms  20.773 ms
 6  * * *
 7  rc1wt-be82.wa.shawcable.net (66.163.76.9)  54.979 ms  49.593 ms  39.414 ms
 8  72.14.221.102 (72.14.221.102)  43.261 ms  42.312 ms  43.431 ms
 9  * * *
10  209.85.254.248 (209.85.254.248)  40.401 ms
    sea09s30-in-f14.1e100.net (142.250.217.110)  37.555 ms
    216.239.43.120 (216.239.43.120)  39.869 ms

If you carefully look at the output of the traceroute google.com command above, the very first line of the output says 64 hops max and there have been 10 hops (first column). The command above exposes that simply to connect to the google.com server, the request was routed to 10 intermediaries (or 10 hops were performed). Column 2nd is the router IP or Host name and column 3rd (and subsequent columns) is the RTT.

RTT is the round trip time measured in MS means sending a packet to a destination and receiving a packet back from it. Latency is the one-way trip time also measured in MS.

After looking at the output of the traceroute, we can surely say that Latency is not just a business of Origin Server and Client. There are many other factors affecting the Latency i.e. the number of intermediaries routers, the protocol used (TCP, UDP, etc), congestion on the network, and data link (physical medium like cables, routers, etc). So, to understand Latency we need to understand how these factors are affecting the performance of the network and ultimately our application performance.

Let’s look at some common delays that contribute to the total latency time.

Propagation Delay

This is the amount of time required for a message to travel from an origin server to the destination. This is directly a function of the distance and the speed over which a packet is traveling.

For example, sending a packet from a server to the client should be faster on a 4000km distance than 10000km. Again this distance is affected by the medium on which the packet travels. Now let’s consider, that 10000km distance packet travels over a fiber optic and the same travels over copper wire.

A network is a mix of several mediums to transfer data. For example, usually, the long-distance mediums are fiber optics but cables that usually reach the end user can still be copper wires. Within a household, mostly we are connected through a wireless connection. So, a varying amount of distances and the different speeds of the mediums involved to propagate a packet would affect Latency

Transmission Delay

Transmission delay refers to the amount of time it takes to push the packet’s bits into the link. A link is a physical or logical connection between two network devices. Transmission delay is a function of data rate of the link and size of a packet. Data rate refers to the amount of bits that can be transmitted through the link.

Below are the examples of the data links and their rates.

1. Ethernet 10 Gbps (Gigabits per second).
2. WiFi4 (802.11n) 600 Mbps (Megabits per second).
3. Wifi6 (802.11ax) 9.6 Gbps

For example, if the intermediary router is set up to receive a maximum of 1 Gbps data rate and the source is able to transmit at a rate of 10Gbps but the actual transmission would be 1 Gbps even though the source is able to send more data.

Processing Delay

This is the amount of time required to process the packet header, checking the error codes, and determine the destination address.

Queuing Delay

Lastly, queuing delay is the time a packet is spent in the buffer waiting for its time to be processed at a router. If the network router is busy processing other requests or may be due to network congestion, a packet can be queued into the router’s buffer and will be processed only once the router is available to process the packet. Bufferbloat is a term used to refer to this situation.

At any given time during the packet is in transition, any of the above delay might effect the the Latency.

Luckily, nowadays most of the network’s physical medium (mostly over long distances) is using fiber optic. Fiber optic uses light signals to transmit the bits. This means we can utilize the fastest available speed of transmission which is called speed of light. However, even fiber optics cannot use the full potential of the speed of light because of the material used. The full potential of the light speed is only possible in a vacuum. Material can affect the light speed by a certain ratio and this ratio is called refractive index. So, there is an upper hard limit on the Latency based on the speed of light and the material used.

The speed of light in a vacuum is approx 299792km per second or we can say 299.792km per milliseconds. If we assume that a packet is traveling at the full potential of light speed we can say that for 15993km (approx New York to Sydney) it would take around 53ms one way and 106ms RTT. Actual Latency time is bigger than this because fiber optics cannot match the actual speed of light, intermediaries, different data rates, congestion on the network, and the protocol used.

For example, most browser communication happens over TCP/IP and TCP does a three-way handshake which means in order to actually be able to transmit the real application data there is 1.5 RTT for SYN, SYN-ACK, and ACK.

We can say that Latency is affected by a number of factors mentioned above and there is a hard limit on the speed of data that can be achieved with the latest technologies today. Considering that the upper limit on the speed at which data can be transferred, the number of intermediaries processing the data, and the distance affect the performance of the application.

One of the promising ways of reducing the latency, apart from performance tuning the application and protocols, is to place the application processing locations near the end users. This is exactly what CDNs and Cloud computing are trying to achieve by placing the source and data processing units as near as possible to the end user. Another popular technology we are hearing of these days is edge networks, much like CDN what with the compute power.

Bandwidth

Bandwidth is the maximum capacity of the channel through which data is transferred. In other words, we can say that Bandwidth is the maximum amount of data that we can transfer at once through a medium.

If we take an example of a water pipeline then a pipeline connecting cities might have bigger pipes than the pipes connecting different taps in a household. A much larger body of water can flow from a city water pipelines than the pipeline within the household.

Similarly, each data link has a different Bandwidth. For example, the latest Ethernet has the capacity to process 10 Gbps (Gigabits per second). This doesn’t mean that all Ethernet links are able to transfer data at this rate. There are other Ethernet with the capacity of 1Gbps. Another medium we use daily is the WiFi. WiFi 4 has a max capacity of 600Mbps whereas WiFi 6 has a capacity of 9.6 Gbps.

Data Rate

Data rate and Bandwidth are sometimes used interchangeably but there is a bit of difference between them. While Bandwidth is the maximum capacity of the channel but Data rate may and may not be the maximum capacity of a channel.

For example, let’s say in a household a user has a phone with WiFi 6 channel, and the router in the same household is using a WiFi 5. Now the communication between the phone accessing the internet through this router cannot utilize the full capability of the user’s phone because the user’s phone can accept a maximum of 9.6 Gbps data but the actual data being transferred from the router is only 3.5 Gbps (WiFi 5’s full capacity.)

Multiplexing

Moreover, we can divide the bandwidth to do parallel computing. For example, if a channel has a maximum capacity of 10 Gbps and a network needs to utilize only 1 Gbps of the maximum available bandwidth. Then we might be able to send 10 different network requests in parallel to utilize the full capacity of the bandwidth. This concept is called Multiplexing

If we come back to the example of fiber optics, fiber optics can carry many different wavelengths without conflicting with each other this is called wavelength division multiplexing (WDM). This means many different connection can travel through the same fiber optic without impacting each other.

But the bandwidth of different channels in the intermediaries can be different so the maximum data rate of a network can be affected by a low data link rate somewhere in the network.

In summary, we can roughly say that Latency is related to the time it takes and Bandwidth is related to the maximum throughput of the channel.