Reinforcement Learning AI for Self Driving in TORCS

By Victor Rahman

Faculty Mentor: Evan Coleman

Abstract

This project explores the application of reinforcement learning techniques to autonomous driving using The Open Racing Car Simulator (TORCS). We develop an agent that learns to drive by interacting with the environment and improving its driving policy over time. Over time we discover setbacks and new improvements to create a more comprehensive learning environment for the agent. The agent initially uses a discrete Q-learning approach with a tabular Q-table system, then scales to a more advanced model leveraging linear value function approximation to generalize across continuous state spaces creating a smoother and less resource intensive learning environment. The state representation includes spatial positioning, angle alignment, and velocity, while the action space utilizes simple pulse-based driving. A custom reward function is designed to encourage forward progress while heavily penalizing off-track behavior. We evaluate the agent’s performance across multiple training episodes and analyze how function approximation improves learning efficiency and scalability. The findings contribute to a better understanding of RL in continuous control environments and provide insight into designing intelligent driving systems for potential real-world applications.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php