BMW Debuts a ‘Natural Interaction' Assistant, Allowing Drivers to Use Voice, Gestures or Gaze to Control Vehicle Functions
【Summary】At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system it calls “Natural Interaction”, that allows driver to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system technology it calls "Natural Interaction", that allows drivers to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
Voice assistants are popular now with the widespread use of Apple's Siri and Amazon Alexa. However, BMW says its new system combines the most advanced voice command technology available with expanded gesture control and gaze recognition to enable multimodal operation for the first time.
BMW's Natural Interaction technology allows the driver to combine voice, gesture and gaze in various ways to interact with their vehicle. The driver decides how they want to interact with the vehicle by using either voice or gestures, based on their own personal preferences or the current situation.
For example, if a driver is having a conversation with a passenger, the system can be set up to only use gesture or gaze, so the voice conversation can continue uninterrupted. Other times when a driver need to keep their eyes on the road, the system can respond to voice controls instead of gestures or gaze for added safety.
Spoken commands are registered and processed using Natural Language Understanding (NLU). NLU is an intelligent learning algorithm, which is constantly being refined. NLU combines and interprets complex speech information, so the vehicle can responds accordingly. Other companies working on NLU include Amazon with Alexa, Apple's Siri, Google's Assistant and Microsoft's Cortana.
"Customers should be able to communicate with their intelligent connected vehicle in a totally natural way," Christoph Grote, Senior Vice President, BMW Group Electronics said in a statement.
"People shouldn't have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver's position and occupants will have more freedom."
The system's robust speech recognition technology combined with an optimized sensor suite, including a camera to monitor the driver's gestures. Through precise detection of hand and finger movements a driver can control most interior functions.
The driver can point to a control and ask what to do. This feature can be used to open or close the windows or the sunroof. The system uses a infrared light signal combined with the gesture camera to capture hand and finger movements of occupants in three dimensions.
Can Work Outside the Vehicle Also
With enhanced gesture recognition and the car's high level of connectivity to highly detailed maps, in the future, interactions may no longer be confined to just the vehicle's interior controls.
Passengers may one day be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command. For example a driver might point and say, "What's this building?", "What is this restaurant called?" or "How much does it cost to park here?" and the system will respond.
The system opens up a whole new way in interact with a BMW vehicle. One possible example presented by BMW is a user being able to buy movie tickets as they drive by a movie theatre. Extending this technology further, a user might be able to stream a movie trailer directly into the vehicle of what's playing nearby.
By connecting digital services, when the driver spots a parking space, they'll be able to find out whether they are allowed to park there and what it costs, and then reserve and pay for it directly via the vehicle's internet-connected infotainment system.
Pointing to a nearby building to get more information
The new system can be combined with BMW's Intelligent Personal Assistant, which learns the routines and habits of each driver with every voice command. As the next logical development, BMW Natural Interaction also offers the possibility of using gestures by simply pointing a finger.
For example, the driver can point a finger at objects in their field of vision and give related voice commands, such as asking for information about opening hours or customer ratings, or reserving a table at a restaurant, transforming the BMW vehicle into a well-informed, helpful passenger.
Attendees at this year's MWC2019 can try out the new features in a mixed-reality installation. A specially-designed spatial concept and virtual-reality goggles are used to create a thoroughly realistic experience that showcases the new possibilities during a virtual ride in the upcoming BMW Vision iNEXT.
BMW has even developed a "training mode" to try out the pointing gesture features of the system. For the installation at MWC, the pointing gestures are visualized by a dynamic light pulse on objects the driver can interact with by pointing their fingers at the object they want to control.
Pointing to a window highlights it and it can be moved up or down with gestures.
For example, if the driver points to a side window, the window is visually highlighted with a frame of light, by using the voice command "open" the window will open.
Development of driver-vehicle interaction will advance along with BMW's Intelligent Personal Assistant. In the future, the system might include enhanced sensor technology, which will be able to take occupants' emotions into account become even more personalized and tailored to the overall situation.
The first BMW Natural Interaction functions will be available in the upcoming fully-electric BMW iNEXT SUV starting in 2021.
Drivers will also be able to give their in-vehicle assistant a personalized name.
Ford Smart Mobility Subsidiary Autonomic Announces Partnership with Compact EV Maker Ayro Inc
Trump Criticizes Automakers for Not Backing Fuel Economy Rollback
German Car Rental Company Nextmove Cancels its $5M Tesla Order Over Quality Issues
NIO Co-founder and Former VP Resigns from the Company
Volkswagen Reports a 3.3% Drop in Vehicle Deliveries for July
U.S. Automakers GM, Ford Planning for Possible Economic Downturn
Tesla Owner Files Lawsuit Claiming the Automaker Reduced the Battery Power of the Model S
Ride-Hailing Giant Uber Posts $5 Billion Loss as Growth Slows
- Silicon Valley Startup Aurora to Work with Fiat Chrysler on a Self-Driving Platform
- EVs Continue to Shine in Norway, Account for Nearly Half of Automobile Sales
- Chinese Automaker Geely Plans to Retain its Stake in Swedish Truckmaker AB Volvo
- Honda to Recall 222,674 Accord Models in China Over Engine Problems
- Driverless Software Startup Immense Simulations Receives $4.6 Million in Series A Funding
- Spy-Shots of a Lexus LS Hydrogen Fuel Cell Prototype Surface Online
- Car Rental Company Hertz Launching a Monthly Vehicle Subscription Service
- Waymo is Offering Free WiFi in its Robo-Taxis to Lure in Riders
- Lightyear One EV Boasts 450 Miles of Range, Solar Power Charging
- BMW's Hybrid cars to Switch to Electric-Only Mode in Polluted Cities