What Is Low Latency and Who Needs It?July 9, 2020
Here’s a dirty secret: When it comes to streaming media, it’s rare that “live” actually means live. Say you’re at home watching a live streamed concert, and you see an overly excited audience member jump onstage. The audience at the concert venue saw that happen at least 30 seconds before you did.
Why? Because it takes time to pass chunks of data from one place to another. That delay between when a camera captures video and when the video is displayed on a viewer’s screen is called latency.
What Is Latency?
Latency describes the delay between when a video is captured and when it’s displayed on a viewer’s device. Passing chunks of data from one place to another takes time, so latency builds up at every step of the streaming workflow. The term glass-to-glass latency is used to represent the total time difference between source and viewer. Other terms, like ‘capture latency’ or ‘player latency,’ only account for lag introduced at a specific step of the streaming workflow.
What Is Low Latency?
So, if several seconds of latency is normal, what is low latency?
It’s a subjective term. The popular Apple HLS streaming protocol defaults to approximately 30 seconds of latency (more on this below), so when people talk about low latency, they are often speaking of whittling that down to five seconds or less: similar to what you’d expect in traditional broadcast viewing.
However, some people need even faster delivery. For this reason, separate categories like ultra low latency and near real-time have emerged, coming in at under one second. This group of streaming applications is usually reserved for interactive use cases, two-way chat, and real-time device control (think live streaming from a drone).
To explore current trends and technologies around low-latency streaming, Wowza’s Justin Miller, video producer, sits down with Barry Owen, vice president of solutions services, in the video below.
When Is Low Latency Important?
No one wants notably high latency, of course — but in what contexts does low latency truly matter?
For most streaming scenarios, the typical 30- to 45-second delay isn’t problematic. In fact, the majority of broadcasters surveyed in our 2019 Video Streaming Latency Report indicated that they were experiencing latency in the 10-45 second range.
How much latency are you currently experiencing?
Returning to our concert example, it’s often irrelevant if 36 seconds pass before viewers find out that the lead guitarist broke a string. But for some streaming use cases — especially those requiring interactivity — latency is a business-critical consideration.
What Category of Latency Fits Your Scenario?
You’ll want to differentiate between the multiple categories of latency and determine which is best suited for your streaming scenario. These categories include:
- Near real time for video conferencing and remote devices
- Low latency for interactive content
- Reduced latency for live premium content
- Typical HTTP latencies for linear programming and one-way streams
You’ll notice that the more passive the broadcast, the higher it’s acceptable for latency to climb.
You’ll also notice in the chart above that the two most common HTTP-based protocols, HLS and MPEG-DASH, are at the high end of the spectrum. So why are the most popular streaming protocols also sloth-like when it comes to measurements of glass-to-glass latency?
Due to their ability to scale and adapt, HTTP-based protocols ensure reliable, high-quality experiences across all screens. This is achieved with technologies like buffering and adaptive bitrate streaming — both of which improve the viewing experience by increasing latency.
High resolution is great. But what about scenarios where a high-quality viewing experience requires lightning-fast delivery? For some use cases, getting the video where it needs to go quickly is more important than 4K resolution.
Who Needs Low Latency?
Let’s take a look at a few streaming use cases where minimizing video lag is undeniably important.
If you’re watching a live event on a second-screen app (such as a sports league or official network app), you’re likely running several seconds behind live TV. While there’s inherent latency for the television broadcast, your second-screen app needs to at least match that same level of latency to deliver a consistent viewing experience.
For example, if you’re watching your alma mater play in a rivalry game, you don’t want your experience spoiled by comments, notifications or even the neighbors next door celebrating the game-winning score before you see it. This results in unhappy fans and dissatisfied (often paying) customers.
This is where ultra low latency “real-time” streaming comes into play. We’ve all seen televised interviews where the reporter is speaking to someone at a remote location, and the latency in their exchange results in long pauses and the two parties talking over each other. That’s because the latency goes both ways. Maybe it takes a full second for the reporter’s question to make it to the interviewee, but then it takes another second for the interviewee’s reply to get back to the reporter. These conversations can turn painful quickly.
When true immediacy matters, about 500 milliseconds of latency in each direction is the upper limit. That’s short enough to allow for smooth conversation without awkward pauses.
Betting and Bidding
Activities such as auctions and sports-track betting are exciting because of their fast pace. And that speed calls for real-time streaming with two-way communication.
For instance, horse-racing tracks have traditionally piped in satellite feeds from other tracks around the world and allowed their patrons to bet on them online. Ultra low latency streaming eliminates problematic delays, ensuring that everyone has the same opportunity to place their bets in a time-synchronized experience. Similarly, online auctions and trading platforms are big business, and any delay can mean bids or trades aren’t recorded properly. Fractions of a second can mean billions of dollars.
Video Game Streaming and Esports
Anyone who has yelled “this game cheats!” (or more colorful invectives) at a screen knows that timing is critical for gamers. Sub-100-millisecond latency is a must. No one wants to play a game via a streaming service and discover that they’re firing at enemies that are no longer there. In platforms offering features for direct viewer-to-broadcaster interaction, it’s also important that viewer suggestions and comments reach the streamer in time for them to beat the level.
How Does Low-Latency Streaming Work?
Now that you know what low latency is and when it’s important, you’re probably wondering, how can I deliver lightning-fast streams?
As with most things in life, low-latency streaming involves tradeoffs. You’ll have to balance three factors to find the mix that’s right for you:
- Encoding efficiency and device/player compatibility.
- Audience size and geographic distribution.
- Video resolution and complexity.
The streaming protocol you choose makes a big difference, so let’s dig into that.
Low-Latency Streaming Protocols
Apple HLS is among the most widely used streaming protocols due to its reliability — but it wasn’t originally designed for true low-latency streaming. As an HTTP-based protocol, HLS streams chunks of data, and video players need a certain number of chunks (typically three) before they start playing. If you’re using the default chunk size for traditional HLS (6 seconds), that means you’re already lagging significantly behind. Customization via tuning can cut this down, but your viewers will experience more buffering the smaller you make those chunks.
Luckily, emerging technologies for speedy delivery are gaining traction. While HLS traditionally delivers latencies of 6-30 seconds, the Low-Latency HLS extension has now been incorporated as a feature set of HLS, promising to provide sub-2-second video delivery at scale. Additionally, broadcasters are now implementing alternatives such as WebRTC, SRT, and CMAF for DASH. Here’s a look at how various technologies compare:
- RTMP delivers high-quality streams efficiently, but is no longer supported due to the impending death of Flash. This protocol remains in use for speedy video contribution, but will disappear from the publishing end of most workflows.
- WebRTC is growing in popularity as an HTML5-based solution that’s well-suited for creating browser-based applications. WebRTC allows for low-latency delivery in an browser-based, Flash-free environment; however, it’s limited in scale without leveraging a media server like Wowza Streaming Engine.
- SRT is popular for use cases involving unstable or unreliable networks. As a UDP-like protocol, SRT is great at delivering high-quality video over long distances, but it suffers from player support without a lot of customization. For that reason, it’s more commonly used for transporting content to the ingest point, where it’s transcoded into another protocol for playback.
- Low-Latency CMAF for DASH is an emerging alternative to traditional HTTP-based video delivery supported by Akamai. While still in its infancy, the technology shows promise of delivering super-fast video at scale.
- Low-Latency HLS, which is supported by Wowza Streaming Engine™ software, is the next big thing when it comes to low-latency video delivery. The spec promised to achieve sub-two-second latencies at scale — while also offering backward compatibility to existing clients. Large-scale deployments of this HLS extension require integration with CDNs and players, and vendors across the streaming ecosystem are working to add support.
For whatever low-latency protocol you choose to employ, you’ll want streaming technology that provides you with fine-grained control over latency and video quality and offers the greatest flexibility. That’s where Wowza comes in.
With a wide variety of features, projects and protocol-compatibilities, there’s a Wowza low-latency solution for every use case. Whether you’re broadcasting with Wowza Streaming Engine or Wowza Streaming Cloud, we have the technology to get your streams from camera to screen with unmatched speed, reliability, quality, and resiliency.