M.A. Zanger
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
5 records found
1
Effects of Partial Observability Solver Methods on Training and Final Policies in Autonomous Driver RL
How do different methods for dealing with partial observability in the environment influence training and the robustness of final policies under various testing conditions?
Autonomous driving is a complex problem that can potentially be solved using artificial intelligence. The complexity stems from the system's need to understand the surroundings and make appropriate decisions. However, there are various challenges in constructing such a sophisticated system. One of the main challenges is to make the agent learn from the environmental input and since the environment is not fully observable and constantly changing, the agent should be flexible enough to extract important information from the input. To create such an agent this paper focused on the 3 different methods, specifically the application of frame stacking, long short-term memory (LSTM), and a combination of both methods. To analyze these methods 3 Deep Q Network(DQN) based agents are created. These 3 agents have frame stacking, LSTM, and both combined methods to solve the partially observable problem of autonomous driving. Their training performances are analyzed, and the results revealed significantly different trends in the training and evaluation phase.
Especially the experiment resulted in LSTM being a more robust method but had lower performance than DQN with frame stack, which showed a trade-off between these 2 qualities of an agent. The agent with LSTM and frame stack was able to learn faster at the beginning, but as it was unstable, it got a lower return value from the training run at the end. This instability can be a problem in the real world, especially in autonomous driving, where it is very important to have robust implementation. ...
Especially the experiment resulted in LSTM being a more robust method but had lower performance than DQN with frame stack, which showed a trade-off between these 2 qualities of an agent. The agent with LSTM and frame stack was able to learn faster at the beginning, but as it was unstable, it got a lower return value from the training run at the end. This instability can be a problem in the real world, especially in autonomous driving, where it is very important to have robust implementation. ...
Autonomous driving is a complex problem that can potentially be solved using artificial intelligence. The complexity stems from the system's need to understand the surroundings and make appropriate decisions. However, there are various challenges in constructing such a sophisticated system. One of the main challenges is to make the agent learn from the environmental input and since the environment is not fully observable and constantly changing, the agent should be flexible enough to extract important information from the input. To create such an agent this paper focused on the 3 different methods, specifically the application of frame stacking, long short-term memory (LSTM), and a combination of both methods. To analyze these methods 3 Deep Q Network(DQN) based agents are created. These 3 agents have frame stacking, LSTM, and both combined methods to solve the partially observable problem of autonomous driving. Their training performances are analyzed, and the results revealed significantly different trends in the training and evaluation phase.
Especially the experiment resulted in LSTM being a more robust method but had lower performance than DQN with frame stack, which showed a trade-off between these 2 qualities of an agent. The agent with LSTM and frame stack was able to learn faster at the beginning, but as it was unstable, it got a lower return value from the training run at the end. This instability can be a problem in the real world, especially in autonomous driving, where it is very important to have robust implementation.
Especially the experiment resulted in LSTM being a more robust method but had lower performance than DQN with frame stack, which showed a trade-off between these 2 qualities of an agent. The agent with LSTM and frame stack was able to learn faster at the beginning, but as it was unstable, it got a lower return value from the training run at the end. This instability can be a problem in the real world, especially in autonomous driving, where it is very important to have robust implementation.
Evaluating robustness of deep reinforcement learning for autonomous driving
Effects of domain randomization on training and robustness
Deep reinforcement learning has been a topic of research in recent years and has been expanding into the domain of autonomous driving. As autonomous driving is likely to involve people, such as daily commuters, it is necessary to ensure the machine will perform well enough in real-life environments not to put anyone at risk. There exist possible approaches to make the transition from a simulation to real life easier, such as domain randomization. This paper uses OpenAI's CarRacing-v2 environment and the CARLA simulator to investigate the effect of domain randomization on training efficiency and robustness for a Deep Q-Network algorithm for autonomous driving. The results show a decrease in training efficiency and higher variance during training for both environments. CARLA also indicates an overestimation during training. As for robustness testing, while visual domain randomization in CarRacing-v2 does not suggest a significant influence on robustness, the dynamic domain randomization in CARLA offers a positive influence toward robustness at the expense of some reward.
...
Deep reinforcement learning has been a topic of research in recent years and has been expanding into the domain of autonomous driving. As autonomous driving is likely to involve people, such as daily commuters, it is necessary to ensure the machine will perform well enough in real-life environments not to put anyone at risk. There exist possible approaches to make the transition from a simulation to real life easier, such as domain randomization. This paper uses OpenAI's CarRacing-v2 environment and the CARLA simulator to investigate the effect of domain randomization on training efficiency and robustness for a Deep Q-Network algorithm for autonomous driving. The results show a decrease in training efficiency and higher variance during training for both environments. CARLA also indicates an overestimation during training. As for robustness testing, while visual domain randomization in CarRacing-v2 does not suggest a significant influence on robustness, the dynamic domain randomization in CARLA offers a positive influence toward robustness at the expense of some reward.
Effects of action space discretization and DQN extensions on algorithm robustness and efficiency
How do the discretization of the action space and various extensions to the well-known DQN algorithm influence training and the robustness of final policies under various testing conditions?
Reinforcement Learning (RL) has gained atten-tion as a way of creating autonomous agents for self-driving cars. This paper explores the adap- tation of the Deep Q Network (DQN), a popular deep RL algorithm, in the Carla traffic simulator for autonomous driving. It investigates the influ- ence of action space discretization and DQN ex-
tensions on training performance and robustness. Results show that action space discretization en- hances behaviour consistency but negatively af- fects Q-values, training performance, and robust- ness. Double Q-Learning decreases training per- formance and leads to suboptimal convergence, re- ducing robustness. Prioritized Experience Replay
also performs worse during training, but consis-tently outperforms in robustness testing, reward es-timation and generalization. ...
tensions on training performance and robustness. Results show that action space discretization en- hances behaviour consistency but negatively af- fects Q-values, training performance, and robust- ness. Double Q-Learning decreases training per- formance and leads to suboptimal convergence, re- ducing robustness. Prioritized Experience Replay
also performs worse during training, but consis-tently outperforms in robustness testing, reward es-timation and generalization. ...
Reinforcement Learning (RL) has gained atten-tion as a way of creating autonomous agents for self-driving cars. This paper explores the adap- tation of the Deep Q Network (DQN), a popular deep RL algorithm, in the Carla traffic simulator for autonomous driving. It investigates the influ- ence of action space discretization and DQN ex-
tensions on training performance and robustness. Results show that action space discretization en- hances behaviour consistency but negatively af- fects Q-values, training performance, and robust- ness. Double Q-Learning decreases training per- formance and leads to suboptimal convergence, re- ducing robustness. Prioritized Experience Replay
also performs worse during training, but consis-tently outperforms in robustness testing, reward es-timation and generalization.
tensions on training performance and robustness. Results show that action space discretization en- hances behaviour consistency but negatively af- fects Q-values, training performance, and robust- ness. Double Q-Learning decreases training per- formance and leads to suboptimal convergence, re- ducing robustness. Prioritized Experience Replay
also performs worse during training, but consis-tently outperforms in robustness testing, reward es-timation and generalization.
Comparative Analysis of Exploration Algorithms in Deep Reinforcement Learning for Autonomous Driving
How does epsilon-greedy, random network distillation, bootstrapped DQN affect training and the robustness of final policies under various testing conditions in autonomous driving?
Autonomous driving is a rapidly evolving field that aims to enhance road safety and reduce accidents through the use of advanced software and hardware technologies. Reinforcement learning (RL) combined with deep neural networks has emerged as a promising approach for training autonomous agents. This research paper investigates three exploration algorithms —Epsilon-Greedy, Random Network Distillation (RND), and Bootstrapped Deep Q-Network (DQN)— within the context of autonomous driving. Performance is assessed based on episodic returns in training and testing environments, as well as the time required to train the networks. The results show significant improvement in learning capability using Bootstrapped DQN without critical differences in training time. There also exists a potential to increase episodic returns further given an increase in the number of steps to train the models.
...
Autonomous driving is a rapidly evolving field that aims to enhance road safety and reduce accidents through the use of advanced software and hardware technologies. Reinforcement learning (RL) combined with deep neural networks has emerged as a promising approach for training autonomous agents. This research paper investigates three exploration algorithms —Epsilon-Greedy, Random Network Distillation (RND), and Bootstrapped Deep Q-Network (DQN)— within the context of autonomous driving. Performance is assessed based on episodic returns in training and testing environments, as well as the time required to train the networks. The results show significant improvement in learning capability using Bootstrapped DQN without critical differences in training time. There also exists a potential to increase episodic returns further given an increase in the number of steps to train the models.
Evaluating Robustness of Deep Reinforcement Learning for Autonomous Driving
How does entropy maximization affect the training and robustness of final policies under various testing conditions?
This research paper aims to investigate the effect of entropy while training the agent on the robustness of the agent. This is important because robustness is defined as the agent's adaptability to different environments. A self-driving car should adapt to every environment that it is being used in since a mistake could cost someone's life. Therefore, robustness is of great importance in self-driving cars. An increase in entropy would promote the exploration of different strategies and prevent convergence on local maximum results. In order to test entropy values in training, the Soft-Actor Critic algorithm is used. The algorithm is run on a simulated city environment called Carla. In the end, collected data shows that the agent which is trained with a higher entropy value adapts to environments it did not train on better than the low entropy agent. However, a low entropy agent performs better in the environment it is trained in. Therefore, increasing the entropy increases robustness but it lowers the performance in the training environment itself.
...
This research paper aims to investigate the effect of entropy while training the agent on the robustness of the agent. This is important because robustness is defined as the agent's adaptability to different environments. A self-driving car should adapt to every environment that it is being used in since a mistake could cost someone's life. Therefore, robustness is of great importance in self-driving cars. An increase in entropy would promote the exploration of different strategies and prevent convergence on local maximum results. In order to test entropy values in training, the Soft-Actor Critic algorithm is used. The algorithm is run on a simulated city environment called Carla. In the end, collected data shows that the agent which is trained with a higher entropy value adapts to environments it did not train on better than the low entropy agent. However, a low entropy agent performs better in the environment it is trained in. Therefore, increasing the entropy increases robustness but it lowers the performance in the training environment itself.