IBC2023 Accelerators Project: Real Time Interactive Streaming

The Challenge

The specific challenge as stated on the IBC2023 Accelerator project site is:

As linear broadcasts convert to streaming, revenues are being lost. This project aims to demonstrate that additional revenue / ROI can be achieved through personalizing viewer engagement from live interactive experience streaming, for big or small sporting and other events.

The Problem

Andrew Lombardi, Chief Product Officer of Snipitz, described the problem the project is addressing: “Content owners are struggling with not only gaining new viewers, but they are trying to do it while changing traditional broadcast into a streaming experience,” he said. “On top of that, how do they monetise and gain insights that they normally can’t track? Snipitz is giving these content owners a way to not only create the next-gen broadcast, but also monetise the grey space that normally goes untouched in a traditional broadcast.”

Traditional broadcasting with its linear focus and adherence to a scheduling grid can only cover limited amounts of major events. “According to the PGA 70% of their events go unseen,” said Lombardi. “Now imagine that 100% is available to all viewers during every event, and not only is it monetised, but the single viewer journey is tracked so the content owners can see how each of their viewers are consuming the events. Each person views an event differently, even if they are sitting next to each other in-venue or at-home and that data will help the monetisation of the event to get closer to 100%.”

Chris Allen, Co-founder and CEO, Red5 Pro, explained the challenges they face in attempting to deliver an interactive experience using existing streaming technologies, “From a technical standpoint, we were attempting to stream video in stadiums to fans in a way that was perfectly synced with the game on the field. And at the same time providing a way for the user to control multiple camera angles. Then the next step would be to add in overlays with live stats and visuals that are perfectly synced with the video,” he said. “There’s no way that we would be able to use traditional broadcast streaming techniques as the latency would simply be too high. We instead took the core work we’ve been building with Red5 and used WebRTC as the delivery mechanism. We were able to achieve sub 150ms latency end to end, making it so that people at the game couldn’t perceive any delay. It actually takes humans longer to focus their eyes from up close with their phone to far away in the field. Usually that takes around 300 to 400ms, so as long as we were under this, we could achieve the goal. We’ve been calling this concept the speed of thought.”

Read more Inside the IBC Accelerator Media Innovation Programme for 2023

The Goal

Allen described the aims of the Challenge, “We started the project with the goal to find out how much of an impact interactivity in live sports video would impact fan engagement, and if this could lead to better monetisation,” he said.

“Currently, sports leagues across the spectrum are looking at ways to bring in new fans, particularly those in a younger demographic - those who are used to quick interactive experiences like video games, Tik Tok, and who grew up with this technology. Our hypothesis was that by adding the ability for the fan to control the camera view themselves, and having perfect sync with the live game, that this would lead to better engagement.

“The challenge here is creating a delay in the stream and matching it up perfectly with the live stream. We’ve come up with a pretty unique way to do this. Snipitz had been primarily focused on an at-home multi-view experience from the very start, but incorporating the sync with a standard broadcast was a new addition we had to solve.”

Snipitz were already working with Red5 prior to joining the Accelerator and the Challenge is a natural progression of the relationship, “We entered the project through Red5 who supply the engine that drives our multi-screen video platform,” said Lombardi.

“We have proven to people in the industry that giving viewers options without leaving the single screen is an important piece of the future of Live and OnDemand broadcast experiences as we move away from second screen technology at-home and in-venue. Whether it’s interactivity, gamification, real-time analytics, machine learning, social media, or ad personalisation, Snipitz is using Red5 to make sure there is zero opportunity for content owners to miss out on monetising their live events while gaining insights in real-time and giving their viewers a next-gen broadcast.”

Participants

This type of solution requires multiple elements to work in sync and this is where the collaborative nature of the Accelerator programme gives the participants a major advantage.

“Well, this project required a lot of different pieces that wouldn’t have been possible on their own,” said Allen. “We needed fast encoders that could keep up in real-time and send KLV (Key Length Value) data as part of the streams, we needed a solid 5G network, we needed front end specialists, and platforms like Unity to pull this off. Obviously being able to provide this low of latency with high quality 60 fps video means that we need an extremely reliable connection at the venue for all the fans using the app,” said Allen.

“This is where utilisation of Verizon 5G technology is essential. Being able to achieve this latency also required having an incredibly fast encoder. Osprey 4k Talon encoders have allowed us to do high quality 60 fps encoding in just one frame of latency contributing only 16.4ms to the overall time to deliver the video to the fan.”

Lombardi concurred: “In the live streaming marketplace, there is no one company that covers all the bases from camera to viewer,” he said. “There are pieces that need to be put together in order to get from traditional single stream broadcast to multi-screen interactive ad and data video platform. Working with experts like Red5 makes the cost and more importantly, the speed to market, invaluable.”

The Accelerator Process

IBC’s Accelerator programme is unique in the industry for providing a safe place for a variety of companies, sometimes competitors, to come together and collaborate on solutions that benefit the entire industry. Lombardi and Allen are in agreement on the value of the programme for each of their companies.

“We were working on a lot of this stuff already before the project had started,” said Allen, “But that said, the accelerator actually helped pull together all the partners in a way that would have been way more fragmented. So, I think this quite literally accelerated the work we were already doing. This project required a lot of different pieces that wouldn’t have been possible on their own. We needed fast encoders that could keep up in real-time and send KLV data as part of the streams, we needed a solid 5G network, we needed front end specialists, and platforms like Unity to pull this off.”

The accelerator program has helped us show off the product and has already given us a path to integrating with great partners like Red 5, said Lombardi. “We are hoping to add more through the program that can become part of the architecture. In the live streaming marketplace, there is no one company that covers all the bases from camera to viewer. There are pieces that need to be put together in order to get from traditional single stream broadcast to multi-screen interactive ad and data video platform. Working with experts like Red5 makes the cost and more importantly, the speed to market, invaluable.”

The Showcase at IBC

Each Accelerator programme culminates in a showcase at IBC2023 in September.

You can catch the RealTime Interactive Streaming Personalises Live Experiences project, showcasing its Proof of Concept and findings at IBC2023, Monday 18th of September, from 10:45-11:45 at the Innovation Stage.

Allen described what IBC attendees can expect to see at the showcase, “We really focused on a multi-view experience where the controls are in the hands of the fans. We will be doing a live demo at the show with the same in-stadium setup we used with the undisclosed major US based sports league we’ve been working with. We will also be showing off the work Snipitz is doing with the Alumni Basketball league and the Legends of the LPGA among others,” he said.

Benefits

The Accelerator programme is intended to create value for the participants as well as the end users. Allen explained the importance of Red5’s involvement, “Being a small team in a startup means you really have to focus,” he said.

“There are so many different things you can do with live real-time video that it can be daunting to pick one direction and stick with it. For our project we purposely picked an industry - sports, and an issue - fan engagement that had a huge pain point associated with it. The IBC accelerator has allowed us to work directly with key partners to bring it all together. We would have eventually gotten to this point, but certainly not as quickly and not with the same level of focus.”

Read more The best of the IBC Accelerators