BMW Debuts a ‘Natural Interaction' Assistant, Allowing Drivers to Use Voice, Gestures or Gaze to Control Vehicle Functions
【Summary】At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system it calls “Natural Interaction”, that allows driver to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
At the MWC2019 (formerly Mobile World Congress) being held this week in Barcelona, BMW debuted a new in-vehicle control system technology it calls "Natural Interaction", that allows drivers to use their voice, hand gestures or gaze control to interact with their BMW vehicle.
Voice assistants are popular now with the widespread use of Apple's Siri and Amazon Alexa. However, BMW says its new system combines the most advanced voice command technology available with expanded gesture control and gaze recognition to enable multimodal operation for the first time.
BMW's Natural Interaction technology allows the driver to combine voice, gesture and gaze in various ways to interact with their vehicle. The driver decides how they want to interact with the vehicle by using either voice or gestures, based on their own personal preferences or the current situation.
For example, if a driver is having a conversation with a passenger, the system can be set up to only use gesture or gaze, so the voice conversation can continue uninterrupted. Other times when a driver need to keep their eyes on the road, the system can respond to voice controls instead of gestures or gaze for added safety.
Spoken commands are registered and processed using Natural Language Understanding (NLU). NLU is an intelligent learning algorithm, which is constantly being refined. NLU combines and interprets complex speech information, so the vehicle can responds accordingly. Other companies working on NLU include Amazon with Alexa, Apple's Siri, Google's Assistant and Microsoft's Cortana.
"Customers should be able to communicate with their intelligent connected vehicle in a totally natural way," Christoph Grote, Senior Vice President, BMW Group Electronics said in a statement.
"People shouldn't have to think about which operating strategy to use to get what they want. They should always be able to decide freely – and the car should still understand them. BMW Natural Interaction is also an important step for the future of autonomous vehicles, when interior concepts will no longer be geared solely towards the driver's position and occupants will have more freedom."
The system's robust speech recognition technology combined with an optimized sensor suite, including a camera to monitor the driver's gestures. Through precise detection of hand and finger movements a driver can control most interior functions.
The driver can point to a control and ask what to do. This feature can be used to open or close the windows or the sunroof. The system uses a infrared light signal combined with the gesture camera to capture hand and finger movements of occupants in three dimensions.
Can Work Outside the Vehicle Also
With enhanced gesture recognition and the car's high level of connectivity to highly detailed maps, in the future, interactions may no longer be confined to just the vehicle's interior controls.
Passengers may one day be able to interact with their direct surroundings, such as buildings or parking spaces. Even complex queries can be answered quickly and easily by pointing a finger and issuing a voice command. For example a driver might point and say, "What's this building?", "What is this restaurant called?" or "How much does it cost to park here?" and the system will respond.
The system opens up a whole new way in interact with a BMW vehicle. One possible example presented by BMW is a user being able to buy movie tickets as they drive by a movie theatre. Extending this technology further, a user might be able to stream a movie trailer directly into the vehicle of what's playing nearby.
By connecting digital services, when the driver spots a parking space, they'll be able to find out whether they are allowed to park there and what it costs, and then reserve and pay for it directly via the vehicle's internet-connected infotainment system.
Pointing to a nearby building to get more information
The new system can be combined with BMW's Intelligent Personal Assistant, which learns the routines and habits of each driver with every voice command. As the next logical development, BMW Natural Interaction also offers the possibility of using gestures by simply pointing a finger.
For example, the driver can point a finger at objects in their field of vision and give related voice commands, such as asking for information about opening hours or customer ratings, or reserving a table at a restaurant, transforming the BMW vehicle into a well-informed, helpful passenger.
Attendees at this year's MWC2019 can try out the new features in a mixed-reality installation. A specially-designed spatial concept and virtual-reality goggles are used to create a thoroughly realistic experience that showcases the new possibilities during a virtual ride in the upcoming BMW Vision iNEXT.
BMW has even developed a "training mode" to try out the pointing gesture features of the system. For the installation at MWC, the pointing gestures are visualized by a dynamic light pulse on objects the driver can interact with by pointing their fingers at the object they want to control.
Pointing to a window highlights it and it can be moved up or down with gestures.
For example, if the driver points to a side window, the window is visually highlighted with a frame of light, by using the voice command "open" the window will open.
Development of driver-vehicle interaction will advance along with BMW's Intelligent Personal Assistant. In the future, the system might include enhanced sensor technology, which will be able to take occupants' emotions into account become even more personalized and tailored to the overall situation.
The first BMW Natural Interaction functions will be available in the upcoming fully-electric BMW iNEXT SUV starting in 2021.
Drivers will also be able to give their in-vehicle assistant a personalized name.
NVIDIA to Acquire DeepMap, a Silicon Valley Startup Building HD Maps for Self-Driving Vehicles
Volvo to Offer Preferred Charging Rates For its Electric Vehicle Customers in Europe Starting July 1
Publicly Traded Electric Truck Startup Lordstown Motors Corp Warns it May Run Out of Cash
Tesla’s Electric Vehicle Sales Declined by Nearly Half Last Month in China
Tesla to Recall Roughly 6,000 U.S.-Built Vehicles to Inspect Brake Caliper Bolts
Honda and AT&T to Offer Expanded in-car Entertainment Options Via the WarnerMedia RIDE App
Ford is Boosting its EV Investments to More Than $30 Billion, Stock Reaches 5-Year High
Movandi Demonstrates its 5G mmWave Technology for Reliable & Fast Vehicle-to-Everything Communications
- Apple in Talks to Build an EV Battery Factory in the U.S. with Chinese Battery Makers, Reports Say
- Electric Truck Startup Rivian Applies for Zoning Amendment for a 380 Acre Site Near its Illinois Factory for Future Expansion
- BMW Isn’t Done Developing Internal-Combustion Engines, But EVs Are Still Coming
- Rivian Reveals Plans for Exclusive Charging Network, Public Charging Stations
- The Solterra SUV Will Be Subaru’s First Electric Model
- General Motors Faces Backlash From the UAW Over its Plans to Invest $1 Billion to Build EVs in Mexico
- Porsche is Offering Free OTA Software Updates for Early Taycan Owners
- Japan’s Honda Commits to Selling Only Electric Vehicles by 2040
- Toyota Trademarks BZ Name, Could Use Moniker for Future EVs
- President Biden Visits Ford Motor Co to Promote His $174 Billion Electric Vehicle Plan, Checks Out the F-150 Lightning Ahead of its Unveiling