How Artificial Intelligence Is Transforming Live Streaming

 

Artificial intelligence (AI), an umbrella term that encompasses machine learning (ML) and deep learning (DL) technologies, promises to transform every facet of human life — including live streaming.

For tech giants like Facebook, artificial intelligence has already become commonplace. The site uses it to automatically identify users’ faces each time a photo is uploaded. And as unsettling as it may sound, Facebook’s facial recognition software (called DeepFace) is far more accurate than the FBI’s. Why? Because Facebook’s extensive photo database grows each day, and as a result, the algorithm improves.

Thanks to Hollywood and Philip K. Dick, most of us have a healthy skepticism of AI. But what about the positive ways that it can revolutionize live streaming?

Streaming now makes up a considerable share of all the data floating around, with video forecast to account for 82 percent of all internet traffic by 2022. Leveraging AI to find more efficient ways to encode, distribute, and organize that data will streamline the digital landscape. From regulating illicit content to preventing copyright infringement, AI promises to play a critical role in the streaming industry.

 

Artificial Intelligence vs. Machine Learning vs. Deep Learning

While sometimes used interchangeably, machine learning and artificial intelligence refer to nuanced concepts. The broader term artificial intelligence describes machines performing tasks in a way that’s considered smart. Any ability to execute work that usually requires human intelligence — sensing, reasoning, acting, and adapting — falls in the realm of artificial intelligence.

Machine learning takes this one step further, referring to machines that acquire knowledge autonomously via exposure to data. Computing that relies on patterns rather than explicit instructions is called machine learning. Rather than depending on pre-programmed rules, machine learning allows the computer to define its own rules.

Lastly, we arrive at deep learning. This is where artificial neural networks come into play. Modeled after the human brain, these networks are made up of thousands of interconnected nodes (neurons) designed to process and transport information throughout the network. They’re described as deep because they’re made up of multiple stacked layers, allowing for computationally intensive functions such as image recognition, sound recognition, and natural language processing.

Because AI covers all of these subsets, we’ll use that term for the remainder of this article.

 

Why Live Streaming Needs Artificial Intelligence (AI)

Because AI encompasses so many complex learning processes, its capabilities are vast. Industry leaders are beginning to leverage it to tackle some issues that have popped up in the live video streaming space.

The dangers of streaming are well documented. An increasing number of people have chosen to broadcast crimes and other horrific content. Everything from fatal car accidents and incidents of torture to sexual assault and suicide has found its way to viewers’ screens.

There’s also the problem of accessibility. Thanks to today’s live-streaming technology, virtually anybody can publish or view real-time video. Sexual predators, terrorists, you name it. If they have a recording device (such as a smartphone or laptop) and a connection to the internet, then broadcasting a live stream takes seconds.

 

Privacy Issues Caused by Live Streaming

The unedited nature of live videos and the share-everything mentality of social media can create an ill-founded sense of intimacy between viewers and streamers. Such was the case for one Twitch streamer whose fan showed up at the doorstep uninvited.

In her article titled When Fans Take Their Love for Twitch Streamers Too Far, reporter Cecilia D’Anastasio explains, “Twitch streamers are like digital-age geisha… Maybe it’s because they let viewers into their homes, or because the live-streaming format feels candid, or because of their unprecedented accessibility, but there’s something about being an entertainer on Twitch that blurs the line between viewer and friend. It can be hard to keep a healthy distance from fans. And, for fans, it can occasionally be hard to tell the difference between entertainer and companion.”

Beyond just streamers themselves, unwilling participants fall victim to privacy invasion. Individuals affected by crimes as varied as sexual assault, murder, and torture experience more trauma when these events are broadcast to the public via live streaming. This is especially problematic when Facebook and Twitter fail to stop this content from going viral.

 

Object Detection as a Method of Censorship

While often used for entertainment, object detection offers practical benefits. Artificial intelligence will pave the way to content regulation with this tech-enabled form of censorship, which promises to replace manual monitoring.

In the past, violent video streams went viral due to the poor infrastructure in place for censoring this content. Specifically, Facebook relied on a team of reviewers to interrupt any live stream breaching their community standards. But by the time a stream was flagged, there was no guarantee that it hadn’t been copied and reposted elsewhere on the internet.

With machine learning and deep learning, organizations like Facebook and Google will be able to act faster, smarter, and more effectively. We can expect AI to interpret streaming content and automatically extract metadata. From there, they’ll be able to monitor harmful content more effectively and protect the privacy of victims.

“These programs generate digital signatures known as hashes for images and videos known to be problematic to prevent them from being uploaded again,” writes Issie Lapowsky. “What’s more, Facebook and others have machine learning technology that has been trained to spot new troubling content, such as a beheading or a video with an ISIS flag.”

AI is poised to make our digital world much safer, but that’s not all.

 

Content Indexing to Improve User Experience

With people creating more video than ever before, streams require real-time cataloging. Popular social media apps like TikTok are leveraging AI to achieve just that, and enhancing this with content personalization. A representative from ByteDance, the app’s parent company, sheds light:

“We build intelligent machines that are capable of understanding and analyzing text, images, and videos using natural language processing and computer vision technology. This enables us to serve users with the content that they find most interesting, and empower creators to share moments that matter in everyday life to a global audience.”

As of November 2018, ByteDance was ranked as the world’s most valuable startup, indicating that they must be doing something right.

AI tools are also being applied to music streams to improve personalization. Algorithms can now take the pitch, tempo, chord progression, and vocal style of a song — as well as the user’s history — to suggest similar content.

 

Artificial Intelligence to Prevent Copyright Infringement

Copyright protection gets tricky when dealing with live content. Whether preventing copyright infringement of a sporting event or a popular song, regulators are looking to AI. Similar to the ways AI can improve content indexing and flag illicit content, learning-based video tools will increasingly be used to analyze live videos for copyrighted material.

That said, the degree to which AI can achieve this remains hazy. In terms of copyright infringement in the music industry, this application of AI begs the question: What’s considered plagiarism? Vanilla Ice certainly has some opinions on the subject.

 

Content-Aware Encoding

Video streaming giants like Netflix use AI to determine the appropriate encoding settings for each video based on complexity. This helps optimize resources and video quality.

It’s simple. Encoding low-bitrate streams whenever possible helps minimize costs and save bandwidth, whereas sporting events and action-packed footage require a high bitrate to accommodate the pace of movement.

Netflix relies on machine learning to tailor encoding for each type of video by using Video Multimethod Assessment Fusion, or VMAF. Likewise, YouTube uses a neural network to improve the encoding process.

 

Conclusion

We’re just beginning to toy with how artificial intelligence can be applied to live streaming. Anything that humans are employed to do — such as, in my case, writing — will someday fall under the realm of AI. Cataloging, targeting, subtitling — the list goes on. So, our best route is to take advantage of it… Or risk the repercussions of failing to adapt.

There are also risks involved. One example would be the time Google’s object-recognition tool mistakenly categorized a photograph of black people as gorillas. In an effort to prevent these types of offensive mistakes from occurring in the future, IBM recently released a “debiasing toolkit” that allows companies to scan their own AI systems for discriminatory behavior and adjust appropriately. Because AI is a human invention, human bias can be built into the systems.

These issues need to be solved. But with the proliferation of live streaming — something so powerful that it was science fiction just 20+ years ago — we require equally powerful computing power to encode, index, and censor it. The industry needs artificial intelligence.