Lightning McQueen Goes to School: Learning Based Autonomous Driving for F1TENTH Vehicles

Jamie Youn Author

07/28/2021 Added

45 Plays

Description

This project is centered around autonomous vehicles and F1TENTH cars. Two features are implemented in this project: lane following and advanced emergency braking. Aspects of deep learning, supervised learning, and reinforcement learning are explored.

Searchable Transcript

Search:

[00:00:00.390]Hello, my name is
[00:00:01.180]Jamie Youn and I'm a member of the Applied Unmanned Systems research
[00:00:05.550]group in the UNL NIMBUS lab.
[00:00:08.310]I conducted my research under the guidance of Dr.
[00:00:11.010]Tran and was mentored by Sung Woo Choi in the department of computer science and
[00:00:15.930]engineering. I worked with my partner,
[00:00:18.420]Ethan Clark, to create today's presentation: Lightning
[00:00:22.260]McQueen Goes to School.
[00:00:25.710]My presentation is about learning based autonomous driving for F1TENTH vehicles.
[00:00:31.110]And today, we will dive deep into the real life process of Lightning McQueen and
[00:00:35.820]his ability to be self-driven in the Cars movie franchise.
[00:00:41.910]Starting off with the motivation of this project, autonomous vehicles,
[00:00:46.020]including self-driving cars, have enormous potential in decreasing all vehicle
[00:00:50.970]related accidents. And with millions of people dying in road crashes each year,
[00:00:56.130]AVs have the potential to reduce crashes,
[00:00:59.040]prevent injuries and save lives, ultimately. However,
[00:01:03.690]safety has remained the key challenge for fully autonomous vehicles and
[00:01:08.580]AVs must first guarantee better safety approval ratings than human drivers
[00:01:13.440]before becoming available to consumers.
[00:01:18.870]Autonomous vehicles, or AVs, have been studied for decades,
[00:01:22.710]and there is yet to be a car that has fully reached this Level 5 Autonomy,
[00:01:27.900]or a fully autonomous driving vehicle.
[00:01:32.250]Even with extraordinary amounts of time, effort, and money,
[00:01:36.330]the high degree of reliability needed has yet to be attained.
[00:01:41.190]However, the emergence of neural networks
[00:01:44.880]catalyzed a resurgence in autonomous driving research.
[00:01:49.350]A neural network is a series of algorithms
[00:01:52.650]and it's honestly very similar to the way the human brain works to classify
[00:01:57.360]data and continuously learn.
[00:02:00.870]And through the combination of supervised learning,
[00:02:03.300]reinforcement learning, and the modern era of computing, autonomous driving
[00:02:08.130]research is ripe for advancement.
[00:02:11.850]The five logos at the bottom of the slide are some of the top companies with
[00:02:16.080]breakthroughs in the development of a fully autonomous car-- including Tesla,
[00:02:21.060]which many people have been following even outside of the research industry.
[00:02:27.560]Two features have been implemented on the F1TENTH cars in this project:
[00:02:32.990]lane following and advanced emergency braking. The code for lane following
[00:02:37.910]was created
[00:02:38.660]so the vehicles stays in position within the lane and does not go outside
[00:02:43.160]the lane markings.
[00:02:44.930]Advanced emergency braking was implemented so that the vehicle autonomously
[00:02:49.130]breaks when an obstacle in its line of sight passes a specified threshold
[00:02:54.110]of distance.
[00:02:56.000]These two features are demonstrated in this simulation, highlighting the
[00:03:00.340]vehicle's
[00:03:00.880]advanced emergency braking system that autonomously stops when approaching an
[00:03:05.770]obstacle in front of it.
[00:03:10.960]So as we can see, the car is autonomously driving.
[00:03:14.500]And once it gets close to this car, it'll brake,
[00:03:19.780]and the other car drives off. Quality isn't the best,
[00:03:23.140]but you get a general idea of what's going on.
[00:03:29.510]Moving on to the methods,
[00:03:31.340]this project placed more emphasis on the implementation of the advanced
[00:03:35.810]emergency braking system than the wall following algorithm.
[00:03:40.310]Deep Q networks are prediction-based neural networks that are used for
[00:03:44.720]training. Dueling DQNs are extensions to DQNs
[00:03:49.520]that separate the representation of state value and state-dependent action
[00:03:53.780]advantage into two separate streams.
[00:03:57.200]We chose to use Dueling DQNs for the project as the value of
[00:04:01.820]every action at each time
[00:04:03.320]step does not need to be known. Or in layman's terms, for the
[00:04:08.240]functionality of advanced emergency braking,
[00:04:11.660]previous actions of the autonomous vehicle are not relevant--
[00:04:16.100]only when a collision is imminent
[00:04:17.600]does the vehicle need to perform the advanced emergency braking function.
[00:04:23.570]The process of Dueling Deep Q Networks are as follows:
[00:04:26.990]it takes in the state of agent as an input, passes that information through
[00:04:31.580]linear transformations, applies an activation function, and
[00:04:36.350]outputs the Q values which represent the quality of taking each action.
[00:04:43.190]For this experiment,
[00:04:44.390]the Dueling DQN architecture was used to determine the most optimal braking
[00:04:49.250]action. The input for the Dueling DQN was a tensor,
[00:04:53.300]which is the primary data structure used by neural networks,
[00:04:56.240]consisting of the steering angle, velocity, and distance to the lead vehicle.
[00:05:01.490]This tensor was passed through multiple linear layers and had the activation
[00:05:06.110]function applied to each output layer.
[00:05:09.470]This process is also explained in the chart on the sides as well.
[00:05:15.460]And moving on to the results,
[00:05:17.740]the results of the training data is best summarized by this graph
[00:05:21.130]which represents the reward over 4,000 episodes.
[00:05:25.300]An episode is a sequence of states,
[00:05:27.910]actions and rewards that ends with a terminal state.
[00:05:32.050]So for example,
[00:05:33.790]playing a game until the terminal state of a player occurs would complete one
[00:05:38.680]episode, with the terminal state being the player,
[00:05:41.710]either winning, losing, or coming to a draw.
[00:05:45.940]The graph compares the number of episodes with the level of negative reward,
[00:05:50.470]also known as punishment.
[00:05:52.690]The blue lines in this graph represent the episodic reward for the 4000
[00:05:57.500]episodes,
[00:05:58.610]while the red lines illustrate the average reward for the last 50 episodes.
[00:06:04.460]One can see the greater range for the blue lines in comparison to the red lines
[00:06:09.230]in their level of reward.
[00:06:11.570]Also, the upward trend of the average reward line is significant as it
[00:06:15.980]portrays the agent minimizing its punishment over time,
[00:06:20.150]or getting a less negative reward, indicating the agent's ability to
[00:06:25.130]learn the optimal breaking policy.
[00:06:29.600]So in conclusion, over time and after thousands of episodes,
[00:06:33.650]the driving agent was able to learn the optimal policy for applying the brakes
[00:06:38.150]to ensure avoiding collision with the leading vehicle.
[00:06:41.510]The agent trailed the lead vehicle at a faster pace and thus it was forced
[00:06:46.430]to apply the brakes eventually. If the agent crashes,
[00:06:51.020]it is heavily punished with respect to its speed prior to the collision.
[00:06:55.760]A higher speed, a greater punishment.
[00:06:58.520]And while there are many extensions that can be made with this project,
[00:07:02.150]the results are significant as they provide the framework for fully autonomous
[00:07:06.320]driving.
[00:07:07.730]The findings portray the potential to achieve cyber-physical control tasks
[00:07:12.320]by combining reinforcement learning with supervised learning.
[00:07:18.080]The poster was also created to highlight the main components of the project
[00:07:22.280]discussed in this presentation.
[00:07:24.800]Feel free to spend some time looking at the poster for a better understanding
[00:07:28.340]with some visuals.
[00:07:32.330]Here are the references of this research project.
[00:07:35.450]I would like to acknowledge the National Science Foundation for sponsoring this
[00:07:39.080]project and would also like to acknowledge all members of the NIMBUS lab,
[00:07:43.460]including my mentor and supervisor
[00:07:45.440]and of course partner, for all of their help in the presentation and completion
[00:07:49.910]of this work. With that,
[00:07:52.160]I conclude my presentation and thank you for listening.

The screen size you are trying to search captions on is too small!

You can always jump over to MediaHub and check it out there.

Comments

0 Comments

Lightning McQueen Goes to School: Learning Based Autonomous Driving for F1TENTH Vehicles

Description

Searchable Transcript

Comments icon comment

Related Channels

Comments