High-quality camera-equipped sports stadiums and speedy processors help Replay Technologies give fans mind-boggling perspectives on every play.
Sports fans who regularly berate the television over blown calls and blind refs might not need to raise their voices thanks to new high-definition sports replay technology. Used by many professional sports leagues, freeD video captures 360-degree views, giving fans access to the same real-world angles as the pros on the sidelines.
At CES 2016, Intel showed exactly how these immersive replays can change the game for basketball fans.
— The Verge (@verge) January 6, 2016
“What you are seeing here is the power to redefine what it means to watch and experience sports,” said Brian Krzanich, Intel CEO in his keynote. “We look forward to seeing this technology come to the biggest playing fields in the world. It is going to change what it means to celebrate the thrill of victory.”
freeD replays not only allow viewers to see victory points and plays from every angle, it enables them to re-watch key moments and post a custom clip on their social networks. Although the technology is primarily used for sports replays, the idea first materialized from somewhat unexpected places.
“freeD was born when Industry experts from the defense, computer generated images and visual effects worlds came up with the ‘crazy’ notion that perhaps there is a way to view reality in a way that is unconstrained from where a physical camera is,” said Preston Philips, vice president of marketing and communications for Replay Technologies, the company behind freeD.
Replay Technologies was named one of the top 10 most innovative companies in sports by Fast Company in 2014. Today, freeD tech wizardry is currently used in dozens of sports arenas across Major League Baseball and the National Football League, and it was used to cover the Olympics in London and the US Open Tennis Finals in New York.
From the fan perspective, these 360-degree replays offer new insights into highlight-reel plays and potentially botched calls. If LeBron James makes an epic 3-pointer only to have a ref say he stepped out of bounds prior to shooting, fans can look at his foot from every angle instead of from whatever angle the cameraman captured.
Teledyne Dalsa Falcon 2 cameras make the 360-degree replays possible by creating a grid of 5k sensors, Philips explained. An Intel high-performance computing (HPC) system then converts the 2D data into 3D volumetric pixels or “voxels.”
“It captures the sports action you love and turns it into a true 3D experience that you can re-watch as if you are right in the game,” Krzanich said. “With this technology, you can control and change the view to any perspective you want. You become the director.”
What’s most impressive about the technology is the speedy turnaround from 2D video footage to 360-degree replays. Philips said this has as much to with the engineers who used general purpose graphics processing unit (GPGPU) techniques to develop the system’s algorithms as it does with the hardware.
“This data, called ‘freeD video,’ allows us to not only create ‘impossible views’ replays for broadcast, but also gives the home user complete interactivity on their 6th Generation Intel Core processor, ushering a new type of video format and visual language,” Philips said.
“We get speed gains from using an Intel-based HPC [high performance computer], fully optimized for efficient processing,” he said. “Additionally, the ‘freeD renderer’ allows for real-time rendering of novel views anywhere within the coverage of the sensor grid.”
With the technology already used by the Yankees, Dodgers, White Sox, Mavericks, Cavaliers, Cowboys, Ravens, 49ers—to name a few—the company plans to implement freeD in all venues across the various sports leagues. Look for the capabilities to extend to fans of UFC, rugby and other sports.
Philips imagines that once “free dimensional video” is unleashed, viewers won’t want to go back to the regular, flattened video of yesteryear.
“Eventually, perhaps sooner than we think, we will live in a world where video is a complete 3D representation of reality, merging existing concepts of entertainment,” he said. “Cinema will fuse with theater, interactive games will fuse with sports broadcasting and, perhaps most importantly, people will be able to digitally share a physical space with other people thousands of miles away using holographic or VR types of enabling technology.”
Philips gives the example of a yoga lesson where all the students are thousands of miles apart. As the instructor guides each person through a series of poses, the students would wear hardware that projects their avatar in the same room. The teacher could then note their form and make adjustments as necessary.
CCS Insight technology analyst Ben Wood sees 360-degree videos and replays impacting how people interact and control what they’re watching. When this technology is tied to TV remote controls that function with gesture and voice commands, Wood imagines it could be mind-blowing for fans.
“If you’re watching a football game on your TV, you can actually say, ‘pan left’ or ‘pan right’ like you would adjust the volume up and down,” he explained. “I don’t have to wait for the director to cut to it. I can just kind of swipe across on the screen and look at what’s around.”
While Wood’s example involves 360-degree-capable televisions, Philips foresees fans being able to jump into the game from any device—and get a truly close-up view of the action.
“We believe that in a not-so-far point in the future, people would be able to turn on their viewing devices, be it a 2D tablet or a 3D VR/AR device, and leap right inside the game,” he said. “They could run beside their favorite players, watch several feet away from the ball or simply hang around the sidelines.”
Armchair refs can get so close to the action they won’t need to yell at their screens. Unfortunately, the virtual players are likely to get an earful.
Feature image credit: Replay Technologies Facebook Page