NVIDIA Reveals its Latest AI-Powered ‘Omniverse' Platform in its Continued Push into Automotive Hardware & Software

author: Eric Walz   

Nvidia Corp kicked off its annual GTC 2021 technology conference on Tuesday with some major news and announcements from founder and CEO Jensen Huang during his opening keynote. This year's event, which normally draws thousands of developers, was held virtually once again due to the pandemic. 

Nvidia's new technology to support autonomous driving includes a virtual driving assistant that can autonomously park a standard car, as well as a system to speed up the computationally intensive training of machine learning algorithms for autonomous vehicles.

Among the new NVIDIA products are "DRIVE Concierge'' and "DRIVE Chauffeur", which are powerful AI-powered platforms designed to help humans remove the stress and hassles of everyday driving and make it much safer.

DRIVE Chauffeur is an AI-assisted driving platform based on the NVIDIA DRIVE AV software developer kit (SDK) that can handle both highway and urban traffic with the highest level of safety. It relieves the human driver from the burden of controlling the vehicle manually, while at the same time monitoring the surrounding environment at all times.

DRIVE Chauffeur is capable of driving autonomously from address to address. Nvidia says the entire DRIVE Chauffeur pipeline has been architected to perceive the world in 4D for robust and efficient autonomous operation. For those who would rather drive themselves, the system provides active safety features and can take over control of the steering or braking when necessary in dangerous situations.

During his keynote, Huang showed a video of the capabilities of a DRIVE Hyperion 8 equipped vehicle driving autonomously in traffic in Santa Clara, California near NVIDIA's headquarters.

Nvidia's DRIVE Hyperion 8 is a computer architecture and sensor set for full self-driving systems. This latest generation technology is designed for the highest levels of functional safety. and is supported by sensors from a wide range of leading suppliers including Continental, Hella, Luminar, Sony and Valeo. 

DRIVE Hyperion serves as a secure base layer for developers of autonomous vehicles. The production platform is designed to be open and modular, so customers can use only what they need for a particular autonomous driving application, such as SAE level 3 automated driving, level 4 automated parking and AI-powered cockpit capabilities. 

DRIVE Hyperion is available today for 2024 vehicle models, NVIDIA founder and CEO Jensen Huang announced at GTC.

DRIVE Concierge acts as a guardian, using interior cameras and interactions to ensure that the driver is always paying attention to the road ahead. It can perform functions such as summoning the vehicle, searching for a parking spot and reconstructing a realistic 3D surround view of the vehicle using neural graphics.

The new technologies announced on Tuesday are centered around its Nvidia's "Omniverse" computer simulation technology, which Huang said the company will begin offering to automakers as part of Drive Concierge. It uses Nvidia hardware and artificial intelligence software to provide a variety of new services.

With DRIVE Concierge, vehicle occupants have access to always-on intelligent services using NVIDIA's DRIVE IX, which is an open, scalable cockpit software platform.

The NVIDIA Omniverse Avatars serve as personal AI-powered assistants in a vehicle.

Omniverse Avatar 

Developers can also create their own custom NVIDIA Omniverse Avatar that supports real-time conversational AI in the vehicle. The Omniverse Avatar is also based on NVIDIA's powerful DRIVE technology and works directly with DRIVE Chauffeur to redefine the experience inside the car.

Omniverse Avatar is part of NVIDIA Omniverse, a virtual world simulation and collaboration platform for 3D workflows. It's currently in open beta with over 70,000 users.

Avatars created on the platform are interactive and customizable digital characters with realistic ray-traced 3D graphics that can see, speak, converse on a wide range of subjects, and understand natural human voices. The Omniverse Avatar connects AI, computer vision, natural language understanding, recommendation engines and simulation. 

When used in a vehicle, the Omniverse Avatar can help make recommendations, book restaurant reservations, make phone calls and provide alerts, such as if any personal belongings are left behind in the vehicle. 

Using the Omniverse avatar, drivers can have a natural conversation with the vehicle and use their voice to control many vehicle functions that previously required a driver to interact with physical buttons or touchscreens.

The avatar can be highly personalized to each driver and passenger, giving every vehicle occupant their own personal concierge. Nvidia says that these assistants will have "incredible intelligence."

Omniverse Avatars are not just for vehicles, they are easily customizable for virtually any industry, according to Nivida. These could help with the billions of daily customer service interactions, such as banking transactions, restaurant orders, making personal appointments and reservations, and many other customer service applications. The realistic avatars open up new business opportunities for companies by improving customer satisfaction.

"The dawn of intelligent virtual assistants has arrived," said Huang. "Omniverse Avatar combines NVIDIA's foundational graphics, simulation and AI technologies to make some of the most complex real-time applications ever created. The use cases of collaborative robots and virtual assistants are incredible and far reaching."

In a demo of the DRIVE Concierge AI platform during the GTC keynote, a digital assistant on the center dashboard screen helps a driver select the best driving mode to reach his destination on time, and then follows his request to set a reminder once the vehicle's range drops below 100 miles.

Another part of Omniverse is a system called "Omniverse Replicator" that will simplify and speed up the tasks required to create a virtual world for training automated vehicles, according to Nvidia's Vice President of Automotive, Danny Shapiro.

Nvidia is one of the world's biggest makers of graphics processors and chips that can run powerful AI-based software. More recently, the company has been making inroads in the auto industry with its NVIDIA DRIVE family of processors that act as the brains of a self-driving vehicle.

In addition to supplying its high-powered chips and software to automakers, Nvidia is developing a powerful computer simulation environment that provides developers an "artificial universe" to train robotaxis and self-driving vehicles how to drive in the real world, yet within the safe confines of a simulated environment that's built using real world data.

Shapiro said Nvidia has $8 billion in automotive revenue booked over the next six years, including business from automated trucking and robotaxi companies looking to deploy autonomous driving technology at scale.

With the centralized, high-performance architecture of Nvidia DRIVE, the power of AI can now enhance almost every part of the vehicle journey for a safer, more convenient experience for passengers.

Automakers building software-defined vehicles using Nvidia hardware and software, the DRIVE Concierge will support one AI assistant, while DRIVE Chauffeur will support a vehicle's autonomous driving tasks. 

NVIDIA's Automotive Partnerships

A growing number of automakers are partnering with Nvidia to supply the robust hardware and software to support fleets of self-driving vehicles. Among them are Mercedes-Benz parent company Daimler AG, Volvo Cars AB and Chinese EV startups XPeng and NIO Inc. All of the companies are racing to catch up with Tesla in the areas of autonomous driving technology. 

The electric Volvo XC90 SUV will use the powerful NVIDIA DRIVE Orin system-on-a-chip (SoC) to power the autonomous driving and safety systems. Nvidia's Orin processor is one of the world's most powerful SoCs and delivers the computing power required for computer vision and lidar sensor data processing.

Volvo says its next-generation safety technology will allow the vehicle to assist and improve the capabilities of a human driver in safety critical situations. Instead of just warning the driver of hazards, it can intervene automatically in some situations to protect occupants, such as controlling the steering and braking.        

In June 2020, Mercedes-Benz and Nvidia announced a partnership to develop a new generation of software-defined Mercedes-Benz vehicles using the Drive Orin AGX platform engineered to support autonomous driving and other functions like automated parking. 

The advanced software-based vehicle architecture will be introduced beginning with 2024 model year vehicles, eventually rolling out to the entire Mercedes Benz fleet globally. 

You can view Nvidia CEO Jensen Huang's entire keynote below:

Eric Walz

Originally hailing from New Jersey, Eric is a automotive & technology reporter covering the high-tech industry here in Silicon Valley. He has over 15 years of automotive experience and a bachelors degree in computer science. These skills, combined with technical writing and news reporting, allows him to fully understand and identify new and innovative technologies in the auto industry and beyond. He has worked at Uber on self-driving cars and as a technical writer, helping people to understand and work with technology.