Kiwi tech start-up HyperCinema was just celebrating its success with the world’s first live AI experience, when the big leagues came calling. Atlanta’s College Football Hall of Fame was looking for a partner to help turn visiting the national home of the sport into a truly groundbreaking, unforgettable experience unique to every guest. With Microsoft Azure AI, the typical experience of visiting a museum has been transformed from viewing exhibits to being the star.
For more than a century, museum managers have tried a host of things to attract crowds to their experiences – from P. T. Barnum’s hoaxes to modern holograms that try to capture that Night at the Museum feeling of seeing exhibits come to life.
Yet the challenge has remained largely the same. For all the clever technology used to capture imaginations and reinvent the “dusty” stereotype, the exhibits themselves don’t change much. Exhibitions will get swapped out every few months, or a splash of interactivity added via kinetic “lightning” balls that respond to nervous fingers…but essentially what you see today is exactly what you get tomorrow.
A new visitor experience launching at the College Football Hall of Fame in Atlanta is completely breaking the glass and putting guests inside its own smart displays. Thanks to groundbreaking use of Azure AI technology from right here in New Zealand, no two experiences are ever the same – and the whole entertainment sector might never be the same either.
From the stage to the football field
As with most museum stories, we have to go back a bit. To September 2022, in fact, when Miles Gregory and Tarver Graham, of New Zealand creative digital agency Gladeye, were talking about the opportunities of generative AI to create narratives that respond to each individual. As the entrepreneur behind the Pop-Up Globe that hosted Shakespeare’s plays in Australia and New Zealand until 2020, Miles had a passion for live theatre and storytelling that provides a unique, immersive experience at every performance.
In 2023, Gladeye spun HyperCinema into a new, separate business, with Miles and Tarver joined by third co-founder and Chief Technology Officer, Gareth Hordyk.
“People spend far too much time on their screens. We wanted to create an exhibition where AI could create personalised, in-person experiences and make the visitors the hero of their own story,” Miles explains. “We were expecting to have something ready in six months. But the very first venue we approached asked if we could open in six weeks!”
A month and a half of “white knuckle” development turned into the hugely successful HyperCinema show in Auckland’s Queen Street, billed as the world’s first live AI experience. Founder Geoff Thatcher of Savannah, Georgia experiential agency Creative Principals happened to attend as part of the global 7 Experiences Summit, for which HyperCinema was the opening session.
The next morning, he already had a business proposal drafted. Within 36 hours, the HyperCinema team were on a call with Geoff’s client – Kimberly Beaudin, CEO of the College Football Hall of Fame, America’s national college football museum.
That was in November 2023. Working closely with Creative Principals and leading theme park consultancy The Producers Group, HyperCinema’s team crafted a complex and rich personalised AI storytelling experience to put the Hall of Fame’s guests at the heart of the action. And on August 24, 2024, the new AI-driven heart of the museum, “Game On!” opened to its first visitors.
The “hyperengine” creating unique experiences
At the heart of that experience is a special “hyperengine” developed by the HyperCinema team, which Gareth Hordyk calls a “recipe management programme”. The whole show requires around 2,000 “recipes” to create the full visitor experience, each with dozens of steps.
When visitors arrive, a self-serve kiosk takes their photos from multiple angles, and asks 10 fun questions such as “What did you want to be when you grew up?” and “What’s your favourite food?” along with questions around favourite schools and rivals to help with personalising their visit.
This trains the Azure AI model, which then uses a visual FX processing platform to put them in shots and videos from throughout college football history, across every major national team. For authenticity, the engine adds effects such as sepia tones or scratches to make the images appear like a real photo from the 1920s, with lighting perfectly calibrated to capture the right images.
At 14 touchpoints around the museum, visitors use their RFID-enabled lanyard to trigger special personalised content, whether performing a team chant in their favourite team colours or featuring in documentary-style videos portraying the 1860s. That question around favourite food might appear as a picture of how much pizza or tacos you would need to eat to meet the calorific needs of a professional football player.
“It’s an incredibly rich history – it’s playful, serious and respectful all in the one experience. There’s even a quarter-size American football pitch where you can play, and you can also get to be a coach or a cheerleader during the big game. You’re part of the story of the whole college football world,” Miles says.
Because the College Football Hall of Fame was already a big user of Microsoft technology, the Kiwi team used Microsoft Azure AI to build the hyperengine, sense-checking ideas with Microsoft experts and receiving training on how to get the most out of the technology.
The general processing units (GPUs) that crunch the data sit on Microsoft Azure, enabling the museum to scale the number of visitors using the technology at peak times – up to 3,000 guests every day. This takes incredible processing power. As Gareth points out, visitors don’t want to get their personalised experiences an hour later.
“The experience needs to start within 10 minutes. And we’ve got a patent on how user information is collected, getting the model training down to under six minutes. The average elsewhere is 20,” he says.
As well as the images, Azure AI generates the written information that accompanies every image, based on the answers visitors have provided. The data is also fully encrypted, and only kept for as long as it’s needed to create the personalised football merchandise visitors get to take home at the end of the day.
“There’s a beauty about the transience of it. You can come back the next day and your experience will be totally different,” says Gareth. “But just to see the smiles on people’s faces…It’s a great reminder of the value of what we’re doing.”
Tapping a growth industry
This is just the beginning for HyperCinema. Its 20-strong team now looks set to grow rapidly thanks to Geoff and Kimberley’s championship Stateside, and for the help Microsoft provided with Azure credits and sponsorship to go from start-up to enterprise level at light speed.
“New Zealand has all of this amazing talent, but we’re not always brilliant at getting out there in the world and telling people what we’re doing. We see so many more opportunities for employment and storytelling,” Miles says.
Already, the company is in talks with others interested in exploring what the technology can do. Miles and Gareth see a massive future not just in museums, but theme parks, stadiums – everywhere that offers live experiences. Instead of the typical virtual reality or augmented reality headsets and green screens, theirs is a unique proposition that makes it possible for every member of the group to experience things simultaneously.
“From what we can tell, there’s no one else doing anything like this anywhere in the world. It really goes to show how generative AI can open up so many doors to businesses who are willing to take risks, to innovate and get creative – it doesn’t matter where you are anymore. We’re thrilled to have been able to support such an amazing Kiwi business to grow onto the world stage,” says Vanessa Sorenson, Managing Director of Microsoft New Zealand.
And Gareth says the opportunities keep growing as the technology evolves.
“Every week there’s a new technique that comes out. We’re already seeing some techniques that we’re saying: ‘We’ve got to get that into our next show’.”
Source: https://news.microsoft.com/
