BMW Debuts a ‘Natural Interaction' Assistant, Allowing Drivers to Use Voice, Gestures or Gaze to Control Vehicle Functions
【Summary】At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system it calls “Natural Interaction”, that allows driver to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system technology it calls "Natural Interaction", that allows drivers to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
Voice assistants are popular now with the widespread use of Apple's Siri and Amazon Alexa. However, BMW says its new system combines the most advanced voice command technology available with expanded gesture control and gaze recognition to enable multimodal operation for the first time.
BMW's Natural Interaction technology allows the driver to combine voice, gesture and gaze in various ways to interact with their vehicle. The driver decides how they want to interact with the vehicle by using either voice or gestures, based on their own personal preferences or the current situation.
For example, if a driver is having a conversation with a passenger, the system can be set up to only use gesture or gaze, so the voice conversation can continue uninterrupted. Other times when a driver need to keep their eyes on the road, the system can respond to voice controls instead of gestures or gaze for added safety.
Spoken commands are registered and processed using Natural Language Understanding (NLU). NLU is an intelligent learning algorithm, which is constantly being refined. NLU combines and interprets complex speech information, so the vehicle can responds accordingly. Other companies working on NLU include Amazon with Alexa, Apple's Siri, Google's Assistant and Microsoft's Cortana.
"Customers should be able to communicate with their intelligent connected vehicle in a totally natural way," Christoph Grote, Senior Vice President, BMW Group Electronics said in a statement.
"People shouldn't have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver's position and occupants will have more freedom."
The system's robust speech recognition technology combined with an optimized sensor suite, including a camera to monitor the driver's gestures. Through precise detection of hand and finger movements a driver can control most interior functions.
The driver can point to a control and ask what to do. This feature can be used to open or close the windows or the sunroof. The system uses a infrared light signal combined with the gesture camera to capture hand and finger movements of occupants in three dimensions.
Can Work Outside the Vehicle Also
With enhanced gesture recognition and the car's high level of connectivity to highly detailed maps, in the future, interactions may no longer be confined to just the vehicle's interior controls.
Passengers may one day be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command. For example a driver might point and say, "What's this building?", "What is this restaurant called?" or "How much does it cost to park here?" and the system will respond.
The system opens up a whole new way in interact with a BMW vehicle. One possible example presented by BMW is a user being able to buy movie tickets as they drive by a movie theatre. Extending this technology further, a user might be able to stream a movie trailer directly into the vehicle of what's playing nearby.
By connecting digital services, when the driver spots a parking space, they'll be able to find out whether they are allowed to park there and what it costs, and then reserve and pay for it directly via the vehicle's internet-connected infotainment system.
Pointing to a nearby building to get more information
The new system can be combined with BMW's Intelligent Personal Assistant, which learns the routines and habits of each driver with every voice command. As the next logical development, BMW Natural Interaction also offers the possibility of using gestures by simply pointing a finger.
For example, the driver can point a finger at objects in their field of vision and give related voice commands, such as asking for information about opening hours or customer ratings, or reserving a table at a restaurant, transforming the BMW vehicle into a well-informed, helpful passenger.
Attendees at this year's MWC2019 can try out the new features in a mixed-reality installation. A specially-designed spatial concept and virtual-reality goggles are used to create a thoroughly realistic experience that showcases the new possibilities during a virtual ride in the upcoming BMW Vision iNEXT.
BMW has even developed a "training mode" to try out the pointing gesture features of the system. For the installation at MWC, the pointing gestures are visualized by a dynamic light pulse on objects the driver can interact with by pointing their fingers at the object they want to control.
Pointing to a window highlights it and it can be moved up or down with gestures.
For example, if the driver points to a side window, the window is visually highlighted with a frame of light, by using the voice command "open" the window will open.
Development of driver-vehicle interaction will advance along with BMW's Intelligent Personal Assistant. In the future, the system might include enhanced sensor technology, which will be able to take occupants' emotions into account become even more personalized and tailored to the overall situation.
The first BMW Natural Interaction functions will be available in the upcoming fully-electric BMW iNEXT SUV starting in 2021.
Drivers will also be able to give their in-vehicle assistant a personalized name.
Lucid’s New ‘Stealth Look’ Appearance Package for the Electric Air Sedan Compliments its High Performance DNA
Toyota’s Redesigned Prius May Get More Drivers Behind the Wheel of a Hybrid Vehicle
Toyota to Collaborate With Texas-based Utility Provider Oncor to Accelerate a Vehicle-to-Grid EV Charging Ecosystem
Stellantis to Idle its Illinois Assembly Plant Indefinitely, Citing the High Costs of Electric Vehicles
Apple Delays its Long Rumored Electric ‘Apple Car’ Until 2026, According to Sources
The World’s First Level-4 Automated Parking Feature Developed by Mercedes-Benz and Bosch is Approved for Commercial Use
Hyundai Motor Group Signs MoU with SK On Co. to Secure Batteries for EVs Built in the U.S.
Tesla May Build its Next Factory in South Korea, According to the Country’s Presidential Office
- Ford Motor Co is Cutting 3,000 Jobs as it Transitions to Electric Vehicles, Software and Digital Services
- Tesla’s Model 3 is Reportedly Getting a Redesign to Make it More Appealing as Competition in the EV Segment Grows
- Electric Automaker Lucid Group Unveils the Lower Priced Air Pure and Air Touring Models
- Consumer Reports Survey Finds Roughly 28% of Respondents Don't Want to Buy an EV
- Toyota’s New ‘Intelligent Assistant’ Learns Voice Commands and Gets Smarter Over Time Using Machine Learning
- Federal Tax Credit on EVs Still Applies to American-Made Vehicles
- Volvo is Building a New EV Service and Training Facility at its U.S. Headquarters in New Jersey To Fast Track its Electrification Plans
- China Has Installed Around 4.7 Million Electric Vehicle Charging Poles as of October 2022
- Volvo Offers a Sneak Peak of its New Flagship EX90 Electric SUV Ahead of its Upcoming Reveal on Nov 9
- Toyota Offering Multiple Solutions for Non-Drivable bZ4X Electric SUVs