NVIDIA Drone Flies Through the Woods Using AI Instead of GPS
【Summary】Researchers at Nvidia are working on a drone navigation system that relies on visual recognition and computer learning instead of GPS
GPS can be invaluable for driving and finding your way and a requirement for self-driving cars. Drones rely on GPS as well. A standard drone without GPS is essentially lost. However, researchers at Nvidia are working on a drone navigation system that relies on visual recognition and computer learning instead, to make sure drones do not get lost, even in the woods.
Nvidia's researchers outfitted an off-the-shelf drone with a navigation system using the company's Jetson TX1 machine learning module, which is fed visual data from two onboard cameras.
Although still at the experimental stage of development, Nvidia's drone was initially designed to make its way through forest trails on rescue missions – looking out for lost hikers or storm damage. However, the Nvidia team believes the low-flying drone could broaden its scope to anywhere GPS coverage is unreliable or not available at all, including canyons, crowded urban environments, or checking stock in an enclosed warehouse. The system could even be adapted to search for damaged cables underwater.
"This works when GPS doesn't," said the team's technical lead Nikolai Smolyanskiy. "All you need is a path the drone can recognize visually."
The drone's navigation system also boasts obstacle avoidance prowess, and has been taught to follow railroad tracks and has been installed in a wheeled robot for zipping around the halls of buildings.
In this case, the project's main proving ground has been forested paths, which can be more of a challenge than a relatively predictable urban setting where points of reference such as mailboxes, buildings and streets remain constant. Wooded areas, by comparison, offer little in the way of uniformity – varying light, lack of markings and trees of different shapes and sizes all help to push a camera-based navigation system to its limits.
"Our whole idea is to use cameras to understand and navigate the environment," Smolyanskiy said. "Jetson gives us the computing power to do advanced AI onboard the drone, which is a requirement for operating in remote environments."
The NVIDIA team isn't the first to pursue a drone that navigates without GPS, but the researchers achieved what they believe is the longest and most stable flight of its kind. Their fully autonomous drone flies along a forest trail for a kilometer (0.6 mile), avoiding obstacles such as trees and maintaining a steady position in the center of the trail.
Nvidia team member Alexey Kamenev developed the deep learning techniques that allowed the drone to smoothly fly along trails without sudden movements that would make it wobble. He also reduced the need for massive amounts of data typically needed to train such a deep learning system.
The drone machine learning training data was watching video captured by three GoPro cameras attached to a rig as Smolyanskiy walked through eight miles of forested trails in the Pacific Northwest. In addition to their own footage, researchers trained their neural network — called TrailNet — video of trails in the Swiss Alps recorded by the Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA) in Zurich.
Smolyanskiy's camera rig used to collect training data for the drone
The long term goal of the project is to be able to program coordinates into a drone's camera-based navigation system and then have it make the trip on its own. In the shorter term, software is being developed that drone or robot builders can download and use to get their own creations to navigate using only visual data.
The team now plans to create downloadable software for Jetson TX1 and Jetson TX2 so others can build drones and robots that navigate based on visual information alone.
Originally hailing from New Jersey, Eric is a automotive & technology reporter covering the high-tech industry here in Silicon Valley. He has over 15 years of automotive experience and a bachelors degree in computer science. These skills, combined with technical writing and news reporting, allows him to fully understand and identify new and innovative technologies in the auto industry and beyond. He has worked at Uber on self-driving cars and as a technical writer, helping people to understand and work with technology.
XPeng’s Urban Air Mobility Affiliate HT Aero to Raise $500+ Million in Series A Funding Round
iPhone Assembler Foxconn Unveils 3 New Electric Vehicle Prototypes as it Prepares to Diversify its Business by Building EVs
BMW i Ventures Invests in Michigan-based EV Battery Startup ‘Our Next Energy Inc.’
Commercial Vehicle Startup Electric Last Mile Solutions Inc. Announces Battery Supply Agreement with China’s CATL
Honda Unveils its New ‘SENSING 360’ Omnidirectional Safety & Advanced Driver Assist System
Aviation Startup Lilium and ABB Announce Plans to Build a MegaWatt eVTOL Charging Network
Lucid Reveals New Details of its ‘DreamDrive’ Advanced Driver Assist System, its Answer to Tesla’s Autopilot
Tesla Sold 56,000 China-made Vehicles in September, the Highest Since its Shanghai Factory Began Producing Cars in 2019
- Battery Maker LG Chem Loses $6 Billion in Market Value After GM Announces Chevy Bolt EV Recall
- Lidar Startup Quanergy Demonstrates an Industry-First Optical Phased Array Lidar Sensor with a 100 Meter Range
- Electric Vehicle Battery Swapping Startup Ample Raises Another $160 Million
- General Motors Invests $300 Million in Chinese Autonomous Driving Startup Momenta
- Rivian Reportedly Waiting for Government Approval To Deliver R1T
- Tesla Challenger XPeng Reports Record July Deliveries, its 2nd Consecutive Month of Record-Setting Growth
- EXCLUSIVE: ICONIQ Motors & Supercar Manufacturer W Motors Plan to Reinvent the Passenger Experience with the Sleek, Luxurious and Fully-Electric Seven ‘Smart Passenger Vehicle’
- Ford Motor Co to Invest $11.4 Billion in 2 New Mega Campuses to Lead the U.S. in Electric Vehicle Production
- Lucid Motors Debuts on the Nasdaq and Raises $4.4 Billion as its Looks to Take on Tesla
- HT Aero, an Aviation Company Backed by Chinese EV Startup XPeng, Details its Latest ‘Flying Vehicle’ Named the X2