This room-sized LED egg captures amazing 3D models of the people inside it
1Reported today on TechCrunch
For the full article visit: https://tcrn.ch/358MsrU
This room-sized LED egg captures amazing 3D models of the people inside it
Capturing human performances in high-definition 3D is a complicated proposition, and among the many challenges is getting the lighting right. This impressive new project from Google researchers puts the subject in the center of what can only be described as an prismatic LED egg, but the resulting 3D models are remarkable - and more importantly, relightable.
What's called volumetric capture uses multiple cameras in a 360-degree setup to capture what can look like a photorealistic representation of a subject, including all the little details like clothing deformation, hair movement, and so on. It has two serious weaknesses: First, it's more like a 3D movie than a model, since you can't pose the person or change their attributes or clothing; The second is an extension of the first, in that you can't change the way the person is lit - whatever lighting they had when you captured them, that's what you get.
"The Relightables" is an attempt by a team at Google AI to address this second issue, since the first is pretty much baked in. Their system not only produces a highly detailed 3D model of a person in motion, but allows that model to be lit realistically by virtual light sources, making it possible to place it in games, movies, and other situations where lighting can change.
Images from the Google AI paper that show the capture process and resulting 3D model alone and in a lighted virtual environment.
It's all thanks to the aforementioned prismatic egg (and a couple lines of code, of course). The egg is lined with 331 LED lights that can produce any color, and as the person is being captured, those LEDs shift in a special structured pattern that produces a lighting-a