Google’s DeepMind lab has recreated Pelé’s so-called lost goal — a strike that the iconic Brazilian soccer player said was his best-ever but was never captured on video.

It was August 2, 1959, and Pelé’s team, Santos, was playing against Clube Atlético Juventus at the Javari Stadium in São Paulo, Brazil. Much to the amazement of the gathered fans, Pelé performed three straight sombrero flicks past the defenders, before executing a knee flick to take it past the goalkeeper, and finally heading the ball into the empty net.

Pelé, who reportedly scored thousands of goals in his career, called this one his best-ever goal, which makes it all the more galling that it was never filmed. A photographer did manage to capture a shot of Pelé heading the ball as the opposition defenders looked on.

Google used that photo, taken by Raphael Herrera, as one of many prompts into a complex web of AI models. The team sought historical accuracy by speaking to people who had witnessed the goal. They used Gemini Omni, a multimodal AI model, to create an accurate picture of what it was like in 1959 São Paulo.

But to really recreate the goal Pelé scored that day, Google sent a film crew to Javari Stadium — which is still standing in São Paulo’s Mooca neighborhood — and hired local actors to don 1950s soccer jerseys.

For movies or video games, actors typically wear motion-cap suits. Instead of that, Google used a model called Performance Control, which takes an input video and produces 3D blue-mesh renderings of the people in the video. Performance Control then uses that data to transfer the motion of the players onto the AI video.

As well as wearing the same jerseys, the team also sourced a leather ball, which was the type of ball made out of heavier materials in the mid-20th century.

“All these AI tools, they’re such amplifiers of our imagination,” says Tom Murray, a research engineer at Google DeepMind. “It allows us to really explore and experiment in a way we’ve just never been able to.”

The resulting AI footage, which starts at 6:50 in the video above, is the best depiction of Pelé’s wonder goal that isn’t the Raphael Herrera photograph. It is now on display at the Pelé Museum in Santos.

Recreating events that were never recorded is an intriguing use case for AI; a few years back, photojournalist Michael Christopher Brown used AI to imagine pictures of Cubans fleeing to the U.S.

Image creditsGoogle