New NVIDIA Research Creates Interactive Worlds with AI

Monday, December 3, 2018

Invention Has Potential to Create Virtual Worlds for Gaming, Automotive, Robotics, VR

NeurIPS -- NVIDIA today introduced groundbreaking AI research that enables developers for the first time to render entirely synthetic, interactive 3D environments using a model trained on real-world videos.

Company researchers used a neural network to render synthetic 3D environments in real time. Currently, every object in a virtual world needs to be modeled individually, which is expensive and time consuming. In contrast, the NVIDIA research uses models automatically learned from real video to render objects such as buildings, trees and vehicles.

The technology offers the potential to quickly create virtual worlds for gaming, automotive, architecture, robotics or virtual reality. The network can, for example, generate interactive scenes based on real-world locations or show consumers dancing like their favorite pop stars.

“NVIDIA has been inventing new ways to generate interactive graphics for 25 years, and this is the first time we can do so with a neural network,” said Bryan Catanzaro, vice president of Applied Deep Learning Research at NVIDIA, who led the team developing this work. “Neural networks — specifically generative models — will change how graphics are created. This will enable developers to create new scenes at a fraction of the traditional cost.”

The result of the research is a simple driving game that allows participants to navigate an urban scene. All content is rendered interactively using a neural network that transforms sketches of a 3D world produced by a traditional graphics engine into video. This interactive demo will be shown at the NeurIPS 2018 conference in Montreal.

The generative neural network learned to model the appearance of the world, including lighting, materials and their dynamics. Since the scene is fully synthetically generated, it can be easily edited to remove, modify or add objects.

“The capability to model and recreate the dynamics of our visual world is essential to building intelligent agents,” the researchers wrote in their paper. “Apart from purely scientific interests, learning to synthesize continuous visual experiences has a wide range of applications in computer vision, robotics, and computer graphics,” the researchers explained.

For more information, read NVIDIA’s Developer News Center post.  

About NVIDIA
NVIDIA’s (NASDAQ: NVDA) invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots and self-driving cars that can perceive and understand the world. More information at http://nvidianews.nvidia.com/.

Certain statements in this press release including, but not limited to, statements as to: new NVIDIA research creating interactive virtual worlds with AI; the potential impact of NVIDIA’s AI research quickly creating virtual worlds for gaming, automotive robotics and VR; NVIDIA research enabling developers for the first time to render 3D environments using a model trained on real-world videos; the abilities, benefits, performance and impact of NVIDIA’s research technology, including its ability to generate interactive scenes based on real-world locations or consumers dancing like their favorite pop stars, its ability to model the appearance of the world and how it can be edited; NVIDIA creating new ways to generate interactive graphics with neural networks; neural networks changing how graphics are created and it enabling developers to create scenes at a fraction of the traditional cost; NVIDIA’s research being shown at the NeurIPS conference and its results in a simple driving game; the capability to model and recreate dynamics of our visual world being essential to building intelligent agents; and the wide range of applications learning to synthesize continuous visual experiences has are forward-looking statements that are subject to risks and uncertainties that could cause results to be materially different than expectations. Important factors that could cause actual results to differ materially include: global economic conditions; our reliance on third parties to manufacture, assemble, package and test our products; the impact of technological development and competition; development of new products and technologies or enhancements to our existing product and technologies; market acceptance of our products or our partners' products; design, manufacturing or software defects; changes in consumer preferences or demands; changes in industry standards and interfaces; unexpected loss of performance of our products or technologies when integrated into systems; as well as other factors detailed from time to time in the most recent reports NVIDIA files with the Securities and Exchange Commission, or SEC, including, but not limited to, its annual report on Form 10-K and quarterly reports on Form 10-Q. Copies of reports filed with the SEC are posted on the company's website and are available from NVIDIA without charge. These forward-looking statements are not guarantees of future performance and speak only as of the date hereof, and, except as required by law, NVIDIA disclaims any obligation to update these forward-looking statements to reflect future events or circumstances.

© 2018 NVIDIA Corporation. All rights reserved. NVIDIA and the NVIDIA logo are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated. Features, pricing, availability and specifications are subject to change without notice

Media Contacts

Ken Brown
+1 408 486 2626

Related Multimedia

NVIDIA AI research side by side comparison

NVIDIA AI research side by side comparison

Close Menu