NVIDIA researchers present a rendering framework that can produce 3D objects from 2D images
3Reported today on TechSpot
For the full article visit: https://www.techspot.com/news/83096-nvidia-researchers-present-rendering-framework-can-produce-3d.html
NVIDIA researchers present a rendering framework that can produce 3D objects from 2D images
Could be used to give robots 'depth perception'
Forward-looking: Computers excel at taking a 3D model and rendering it on a 2D screen. What they are not so capable of is taking a 2D image and creating a 3D model. However, thanks to machine learning, this is now becoming possible.
Nvidia researchers have created a rendering framework that uses AI to take 2D information and transform it into a 3D object accurately. The system is called DIB-R, short for differentiable interpolation-based renderer and is built on PyTorch, a machine learning framework. The team will be presenting its findings at the annual Conference on Neural Information Processing Systems in Vancouver, Washington, this week.
Essentially the framework does the opposite of what GPUs usually do. It analyzes an image and then forms a high-fidelity 3D object, including shape, texture, color, and lighting. The encoder-decoder architecture starts with a polygonal sphere and morphs it using the given information in the 2D image.
The process only takes one-tenth of a second, but the neural network takes two days to train using a single Nvidia V100 GPU. Nvidia claims that training using other GPUs would take several weeks. After feeding it several datasets containing pictures of birds, DIB-R was able to accurately create 3D models when given a single image.
While birds were the focus of the experiment, co-author of the research paper on the subject, Jun Gao said that the system can render any 2D image into a 3D model.
"This is essentially the first time ever that you can take just about any 2D image and predict relevant 3D properties," said Gao.
The re