AWS DeepRacer is an integrated learning system for users of all levels to learn and explore reinforcement learning and to experiment and build autonomous driving applications. This is the first AWS DeepRacer virtual community race dedicated for AWS DeepRacer beginners. This blog post serves to provide the essential comprehensive guides and resources to help the beginners to get started with this community race.
You will get the opportunity to learn and get hands-on with reinforcement learning, Python programming, AWS cloud services and more as you join and compete in the community race. To encourage more active participation from the new racers community, the top 50 racers in this community race will be rewarded with AWS credits. The winner will win $1,000 AWS credits. Refer here for more prize details. Terms and conditions apply.
Interested in Machine Learning? Check out the AWS Machine Learning NanoDegree Scholarship Program where AWS and Udacity give away 425 scholarships for a refreshed NanoDegree course teaching you the basics of ML and how to deploy models into production.
Registrations end 23rd of June 2021!
1. Join the Race
First of all, sign into your AWS account and join the AWS DeepRacer Beginner Challenge virtual community race from the race invitation link below:
https://console.aws.amazon.com/deepracer/home#raceToken/6JgHlXfoSRaEMu1OXW5RZA
If you don’t have an AWS account, visit aws.amazon.com and choose Create an AWS Account. For detailed instructions, see Create and Activate an AWS Account. After you have created an AWS account, see Set Up Account Resources for AWS DeepRacer to set up the required resources for AWS DeepRacer.
If you are using an AWS Educate Starter Account or AWS Educate Classroom, we are working on it and will provide an update in due course.
2. Race Home
When you click on the race invitation link above, you will come to this page as shown below. You will need to create and train a model for the 2019 DeepRacer Championship Cup track before you can submit the model for this challenge.
Before you submit a model which could complete at least one full lap around the track, your lap time will not appear in the race leaderboard.
Click on the AWS DeepRacer link at the top to go to the DeepRacer console home to create and train your model.
3. Get Started with DeepRacer
The AWS DeepRacer console is the platform to get started with to create, train, evaluate and submit your models for the community race. Note that AWS DeepRacer is only available in the US East (N. Virginia) region (us-east-1) currently. Click on Get Started to get started with reinforcement learning.
4. Get Started with Reinforcement Learning
Reinforcement Learning (RL) is the core which powers the autonomous driving, object avoidance as well as head-to-head racing in AWS DeepRacer.
Step 1: Learn the basics of reinforcement learning
Click on Start learning RL to open a new page in a new browser tab which provides a detailed, interactive and animated way of learning about the RL basics, how it applies to AWS DeepRacer, RL models training as well as the parameters of reward functions.
This is an essential guide for anyone who is new to RL and RL in AWS DeepRacer and useful for anyone who wish to revise and reinforce the concepts as well.
Step 2: Create a model and race (Required)
Click on Create Model to begin your journey to build, train and evaluate models for submission to the community race.
Step 3: Learn about sensors and new type of racing
You can learn about the different type of sensors and new type of racing. For this community race, the racing type is Time Trial and the sensor configuration of Front-facing single-lens camera only is sufficient.
5. Create and Train Your First Model in 45 Minutes
Step 1: Specify the model name and environment
Under Account Resources, make sure you have both valid IAM roles and valid AWS DeepRacer resources stack.
Under Training details, you’re required to name your model.
It is recommended you provide a model name which is distinctive and meaningful to avoid confusion when you submit the model to a race. For example, model name = reInvent-2019–45mins-v1
It is also recommended to provide description to each of the model you create to take note of important information or details about the model.
Under Environment simulation, choose The 2019 DeepRacer Championship Cup track to train your model on for this community race. Click Next at the bottom of the page to proceed.
Step 2: Choose training type and agent
Choose Time trial for the race type as the other race types are not supported for this community race.
For the Agent, choose the Original DeepRacer, which is the default vehicle in your AWS DeepRacer garage to get started with. You cannot modify the configuration for this default vehicle.
When you have become more familiar and want to try more advanced features, you can go to the AWS DeepRacer Garage to build a new vehicle that is customized to your needs and preferences on Sensor(s), Neural Network topologies, Action space, Vehicle trim and name.
Step 3: Customize reward function and training algorithm
You will need to write a reward function in Python 3.x programming language to define the reward strategy which guide the vehicle to move along the track to the destination as quickly as it could for time trial race type.
To build and train a minimum viable model for this community race in 45 minutes, the proposed reward strategies are a combination of Follow the Center Line, Stay Inside the Two Borders and Prevent Zig-Zag.
You can find the example codes for the three separate strategy above at the DeepRacer Reward Function Examples link below:
https://docs.aws.amazon.com/deepracer/latest/developerguide/deepracer-reward-function-examples.html
Combine the example codes from the three examples above into a single reward function as shown below:
def reward_function(params):
‘’’
Example of rewarding the agent to follow center line
‘’’
# Read input parameters
all_wheels_on_track = params[‘all_wheels_on_track’]
track_width = params[‘track_width’]
distance_from_center = params[‘distance_from_center’]
steering = abs(params[‘steering_angle’]) # Only need the absolute steering angle
# Calculate 3 markers that are at varying distances away from the center line
marker_1 = 0.1 * track_width
marker_2 = 0.25 * track_width
marker_3 = 0.5 * track_width
# Give a very low reward by default
reward = 1e-3
# Give a high reward if no wheels go off the track and
# the agent is somewhere in between the track borders
if all_wheels_on_track and (0.5*track_width — distance_from_center) >= 0.05:
reward = 1.0
# Give higher reward if the car is closer to center line and vice versa
if distance_from_center <= marker_1:
reward = 1.0
elif distance_from_center <= marker_2:
reward = 0.5
elif distance_from_center <= marker_3:
reward = 0.1
else:
reward = 1e-3 # likely crashed/ close to off track
# Steering penality threshold, change the number based on your action space setting
ABS_STEERING_THRESHOLD = 15
# Penalize reward if the agent is steering too much
if steering > ABS_STEERING_THRESHOLD:
reward *= 0.8
return float(reward)
Set the maximum time to 45 to set the training maximum time to 45 minutes for your first model, with the recommended reward function, training algorithm and hyperparameters as above.
Click Create Model to start the model training process. During the model training process, you can monitor the reward graph and simulated video stream on the training progress. They are important training metrics which could give you a high level overview of whether your reward function is on the right and expected direction.
6. Evaluate Your Model
You can evaluate your model after the training is completed. It is also important to review your reward graph before start the model evaluation.
For Average Reward, ideally it should increase over iterations until it reaches a point where it steadily remains or fluctuates slightly, which is a indicator that the model might have converged.
For Average percentage completion (Training & Evaluation), ideally they should also increase over iterations until they reach 100% and remain the same thereafter.
To begin evaluate your model, click Start new evaluation.
Choose The 2019 DeepRacer Championship Cup track, Time trial race type and 5 trials for number of trials.
Remember do not check on any checkbox for the Virtual race submission.
Click Start evaluation to start. The evaluation process could some time to complete depends on how fast your model runs around the track during evaluation.
After the evaluation is completed, if the number of successful trials is at least 3, you could submit the model to the community race for evaluations.
For the sample model built in this guide, it is able to completed all 5 evaluation trials successfully as shown below:
7. Submit Your Model
Once your model evaluation is completed and you are satisfied with the evaluation results, you are ready to submit your model to the Beginner Challenge Community Race for evaluation and entry into the race’s leaderboard.
Go to the race’s invitation link as below, choose your model which you have trained for this community race and click Submit model.
https://console.aws.amazon.com/deepracer/home#raceToken/6JgHlXfoSRaEMu1OXW5RZA
After you submitted your model, you will go to the Beginner Challenge community race home. In the status section at the left, you should see the status changed from Awaiting evaluation to Under evaluation. The evaluation process here should take a couple of minutes or more to complete.
When the evaluation is completed, you should see the status become Completed 1 lap and your rank and lap time should appear in the race’s leaderboard at the right. You can click on the Watch video link below Status to watch the video of your vehicle running around the race track based on your trained model.
If your model evaluation is successful, you should see your own racer name appear in the community race leaderboard as shown below.
If your model submission is unsuccessful as shown below, do review the evaluation logs for further root cause analysis and troubleshooting. If you are still having the same issue, feel free to reach out to the AWS Machine Learning and DeepRacer community. Refer to section 9: Ask the Community in the later part of this guide for more details.
8. Cost Management
Please kindly refer to AWS DeepRacing Pricing page below to understand its Free Tier and pricing examples. https://aws.amazon.com/deepracer/pricing/
Alternatively, you can also look for Local Training option to setup AWS DeepRacer training and simulation on your local machine or any cloud virtual machine instance e.g. Amazon EC2 which can lower your cost on AWS DeepRacer resources. Kindly note that the AWS DeepRacer local training project is contributed by the AWS DeepRacer/Machine Learning community.
9. Ask the Community
We welcome and encourage you to ask questions about this community race, AWS DeepRacer as well as AWS Machine Learning on the AWS Machine Learning Slack Community. There are many AWS DeepRacer and Machine Learning experts from the community or AWS who are passionate to share their knowledge and experience with other community members.
Please join the AWS Machine Learning Slack Community from this link and post your relevant questions on the relevant channels (e.g. #aws-deepracer, #dr-local-training-discussion) there.
There are other useful and public resources relevant to AWS DeepRacer and machine learning where you can find from the sections below.
Lastly, happy learning, racing and sharing! #AWSDeepRacer
10. Free Digital Training & Useful Resources
- AWS DeepRacer Developer Guide
- AWS DeepRacer Pit Stop
- AWS DeepRacer TV
- DeepRacer Expert Bootcamp at AWS re:Invent 2019
- AWS DeepRacer: Driven by Reinforcement Learning
- AWS DeepRacer community blog
- AWS DeepRacer — Free course on Udacity
- DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
- Using Reinforcement Learning and Simulation to Develop Autonomous Vehicle Control Strategies
11. GitHub Repositories