Watch artificial intelligence project a 3D soccer match on your kitchen table

Watch artificial intelligence project a 3D soccer match on your kitchen table

When the soccer World Cup kicks off this Thursday, imagine watching it as 3D “holograms” on your kitchen table. That may not be far off thanks to a new technique that turns YouTube videos into 3D reconstructions of matches. The key to the approach is a convolutional neural network – a type of artificial intelligence algorithm loosely modelled on the part of the brain that processes visual data – that researchers trained to estimate how far the surfaces of each player are from the camera that recorded the match. The network analyzed 12,000 2D images of players extracted from the soccer video game FIFA alongside the corresponding 3D data from the game engine to learn how the two correlate. That allowed it to estimate depth maps for players from unseen 2D images. When shown unseen videos, the system accurately predicted depth maps for each player and combined them with the color footage to reconstruct each player in 3D. The players were then superimposed on a virtual soccer pitch allowing the match to be viewed in any 3D content viewer. The researchers tested the approach with Microsoft’s HoloLens smart glasses, which let them overlay the 3D reconstruction onto a real-world tabletop. The end product is still glitchy, it can’t recreate the ball, doesn’t work in realtime, and only permits watching from the side of the pitch the video was recorded. But the technique could be more scalable than leading approaches for reconstructing sport in 3D, which require arrays of cameras around the pitch recording every angle. The researchers say the approach should also work for other events that happen in predefined arenas like music concerts or theatre.

Antonio Breitenberg

Related Posts

14 thoughts on “Watch artificial intelligence project a 3D soccer match on your kitchen table

  1. beepbeepcoyote says:

    This opens up many uses. Imagination endless.

  2. TechNomad says:

    Top 10 Things I Didn't Know I needed.

  3. Video Viewer says:


  4. cikif says:

    yine mi neural network

  5. multiYellowHorse says:

    Sports has to be viewed live … so thats too bad

  6. Hail Xenu says:

    Have it do a Dungeons & Dragons game or an RTS then I'll be happy

  7. samiam111 says:

    The 3D game models were necessary to estimate and create the depth maps for the neural network to approximate where the players were on the pitch. Wouldn't that mean 3D models for the performers are required to recreate said events like music concerts or theater?

  8. Vanitas VENTUS says:

    That was amazing.

  9. Daniel Mora says:

    We are yet another step closer to having yugioh duel disks

  10. nic w says:

    It's not nice to see Science Magazine, a usually very respected science magazine, casually assert in the year 2018 that convolutional neural networks "are loosely based on areas of the brain". The relationship is almost inexistent, and the name endure mostly for historical reasons. Furthermore, how is the reprojection of a hologram of a 3D scene akin to "intelligent behavior", thus making this a prime example of AI? Not every pattern recognition or machine learning application should be thrown in the AI bag like this. I would expect much more scientific accuracy and depth from Science. There is a difference between science communication and popularization, and insisting in broadcasting tired, shallow and innacurate clichés about technologies that we should instead be striving to attain a real understanding of, if not transcendence. Instead we remained at the 0x0 today. Good luck with the championship…

  11. Miguel Baptista says:


  12. Archiworks 992 says:

    Interesting 😎

  13. Guilt Azaour says:

    so you still think we don't live in a simulation

  14. ali khazai says:

    That's my idea
    I'm very happy
    Which is made

Leave a Reply

Your email address will not be published. Required fields are marked *