Researchers at South Korea's Chung-Ang University Develop a ‘Meta-Reinforcement' Machine Learning Algorithm for Traffic Lights to Improve Vehicle Throughput
【Summary】Researchers from South Korea’s Chung-Ang University are experimenting using reinforcement learning (RL) algorithms to solve traffic signal control problems and reduce congestion. Recent advancements in artificial intelligence (AI) and machine learning have shown promise in optimizing traffic signal controls that can make driving in busy urban areas less frustrating.
No matter how advanced modern vehicles become, including those capable of autonomous driving, being stuck in busy urban traffic is not likely to go away anytime soon using traditional traffic signal technology to control the flow of vehicles.
But recent advancements in artificial intelligence (AI) and machine learning have shown promise in optimizing traffic signal control that can make driving in congested urban areas less frustrating.
Researchers from South Korea's Chung-Ang University are experimenting using reinforcement learning (RL) algorithms to solve non-stationary traffic signal control problems. The algorithm automatically customizes "reward functions" based on the classification of traffic regimes.
RL uses a "trial and error" problem-solving method for agent training in the field of machine learning.
RL is one of the three basic machine learning paradigms, along with supervised learning and unsupervised learning. It's an area of machine learning that's focused on how intelligent agents (the traffic light) should take actions in an environment in order to maximize the reward, which is vehicles traveling uninterrupted in urban traffic scenarios. The goal of agents using RL is to maximize the total rewards.
Basically, the reward function of a machine learning algorithm is an incentive that tells the agent what is correct and what is wrong using reward and punishment. But oftentimes RL algorithms must sacrifice immediate rewards (some drivers may get stuck at a red light) in order to maximize the total rewards (improved traffic flow).
Existing traffic signals rely on a "rule-based controller" (red means stop, and green means go). The objective is to reduce vehicle delays during light traffic conditions and maximize the vehicle throughput during periods of road congestion.
Sub-optimum traffic signal controllers like these affect the daily life of people living in urban areas that frequently deal with congestion and delays. Conventional traffic lights with fixed state times are not well equipped to ease traffic congestion.
In addition, existing traffic signal controllers cannot adapt to ever-changing and random traffic patterns throughout the day. Although a human traffic controller might be able to perform better than a fixed controller, they can only manage a few intersections at a time.
One of the biggest challenges for the researchers was to implement RL in a non-stationary environment, which are vehicles randomly passing through intersections. Current research has explored reinforcement learning (RL) algorithms as one possible solution for easing traffic woes. However, RL algorithms do not always achieve the best results due to the dynamic nature of traffic environments.
To help better address this, the researchers have developed what they call a "meta-RL model" that adjusts its goal based on the traffic environment. The meta-RL algorithm has a wide-area coverage and outperforms existing alternative algorithms, according to the researchers at Chung-Ang University.
The two main goals of the meta-RL machine learning model are to maximize vehicle throughput through intersections during peak hours and minimize delays during peak times, such as during rush hour. The researchers led by Prof. Keemin Sohn developed an extended deep Q-network (EDQN)-incorporated context-based meta-RL model for traffic signal control.
Sohn is a Professor in the school of Civil and Environmental Engineering at Chung Ang University, Korea. His research interests include data science, artificial intelligence with applications in transportation planning.
Here's how the meta-RL model works. First, it determines the traffic as either "saturated" or "unsaturated" using a latent variable that indicates the overall environmental condition. Based on the current traffic flow, the model either maximizes throughput or minimizes delays similar to a human controller. It performs this by implementing traffic signal cycles (action).
The action is controlled by the provision of a "reward." The reward function is set to be +1 or -1 corresponding to a better or worse performance in handling traffic relative to the previous traffic light interval. In addition, the EDQN acts as a decoder to jointly control traffic signals for multiple intersections.
"Existing studies have devised meta-RL algorithms based on intersection geometry, traffic signal phases, or traffic conditions," explained Prof. Sohn. "The meta-RL works autonomously in detecting traffic states, classifying traffic regimes, and assigning signal phases accordingly."
The researchers trained and tested their meta-RL algorithm using Vissim 21.0, which is a commercial traffic simulator used by engineers to model real-world traffic conditions.
The team set up a transportation network in southwest Seoul consisting of 15 intersections to serve as a real-world test environment. After meta-training, the RL model could adapt to new tasks without adjusting its parameters.
These experiments show that the proposed model could switch control tasks without any explicit traffic information. It could also differentiate between rewards according to the saturation level of traffic conditions.
The research team found that the EDQN-based meta-RL model outperformed the existing algorithms for traffic signal control. However, the researchers stressed the need for an even more precise algorithm that takes into account different saturation levels from intersection to intersection in an urban area.
"Existing research has employed reinforcement learning for traffic signal control with a single fixed objective. In contrast, this work has devised a controller that can autonomously select the optimal target based on the latest traffic condition. The framework, if adopted by traffic signal control agencies, could yield travel benefits that have never been experienced before," said Prof. Sohn.
The findings of the study were published in the "Computer-Aided Civil and Infrastructure Engineering" journal and were made available online on Sept 30.
Originally hailing from New Jersey, Eric is a automotive & technology reporter covering the high-tech industry here in Silicon Valley. He has over 15 years of automotive experience and a bachelors degree in computer science. These skills, combined with technical writing and news reporting, allows him to fully understand and identify new and innovative technologies in the auto industry and beyond. He has worked at Uber on self-driving cars and as a technical writer, helping people to understand and work with technology.
Ford is Testing a New Robotic Charging Station to Assist Drivers of EVs With Disabilities
Ford Raises the Prices of the F-150 Lightning Electric Pickup Due to Rising Raw Material Costs
The BMW 7-Series to Feature HD Live Maps From HERE Technologies for Hands-Free Highway Driving in North America at Speeds up to 80 MPH
AutoX to Use the 'Eyeonic Vision Sensor' from California-based SiLC Technologies for its Robotaxi Fleet in China
LG Develops ‘Invisible’ Speaker Sound Technology That Could Revolutionize In-Vehicle Audio
Zeekr’s New 009 Electric Passenger Van is the World’s First EV to Feature CATL’s Advanced ‘Qilin’ Battery With a Range of 510 Miles
Redwood Materials is Building an Electric Vehicle Battery Recycling Facility in South Carolina
Panasonic Announces Multi-Year Agreement to Supply Electric Vehicle Batteries to Lucid Group
- GM's Cruise Becomes First Company to Receive Permit to Charge for Autonomous Rides
- Volvo-backed EV Battery Maker Northvolt, Founded by Former Tesla Execs, Raises Another $1.1 Billion to Help Fund its Expansion
- Tesla's Battery Supplier CATL Unveils its New ‘Qilin’ Battery That Can Deliver 600+ Miles of Range to EVs
- EV Charging Provider Electrify America Raises $450 Million, Siemens to Become a Minority Shareholder
- Audi Rebrands its Premium Car Rental Service to ‘Audi on demand’
- Tesla Rival NIO Plans to Produce its Own EV Battery Packs to Improve Profitability
- Volkswagen Breaks Ground on the First of Six European Battery Cell Factories as Part of a $20 Billion Investment
- Vietnam-based Electric Vehicle Startup VinFast Ships its First Vehicles to the U.S.
- Shares of Volvo’s EV Brand Polestar to Begin Trading on the NASDAQ After Closing on its SPAC Deal With Gores Guggenheim
- Here's One Way Lucid Motors Made the Air Sedan More Spacious for Passengers