Thank you to all who have voted in October’s log analysis challenge. As the month has ended, it’s time to present the results.
What was the challenge about?
A quick reminder: October community challenge was about the log analysis tool. This tool was created and presented to users as a Jupyter Notebook as part of the Re:Invent 2018 DeepRacer workshop. I have used and improved it in the first months of the AWS DeepRacer Virtual League and then shared with the community. Now we have challenged our members to also contribute.
We have received ten pull requests with incredible changes to the tools, you can read about them in our previous post. The entries have been presented to the community to vote on. The voting rules were as follows:
- Each community member has a single vote to give and these votes are public,
- I have five votes to give as the main contributor so far, I have to vote at least 24 hours before the end and my votes get revealed after the challenge closes.
My votes
I decided to vote for following submissions:
- @evanca – her submission is a missing link in going from the console graph to log analysis,
- @Ahmed Shendy – I like all three log management solutions, but this one feels most complete, detecting evaluations vs trainings and using notebooks as templates which can save a lot of complications with git when you want to pull changes from git,
- @Mithilesh Hinge – I like his improvements that can be seen, but possibly even more the invisible ones which save a lot of time in track plotting and more,
- @Chris Thompson – The exit points aren’t something I thought about before and now I don’t know why I haven’t thought about it before, brilliant, it is something that can highlight a big problem in the training,
- @Daniel Morgan – This is something that many people have ask for before. Ray G plots track information in a spreadsheet and shares it in Slack, Daniel brings it to the log-analysis tool.
Community votes
Community voted as follows:
| Author | Votes | Comments from voters |
| GummyBear | 5 | “The display of steering decision and reward on the track is useful as hell. During my log analysis, I have to count step by step to check how my car turn or how my reward behave during the sharp corner, with this new log analysis I don’t have to” |
| evanca | 9 | “Some of the other ones were more impressive but think this one is the best for the community. Getting people a familiar starting point was a great idea.” “I think the simplicity of this entry combined with the familiar starting point make it a great contribution.” |
| Daniel Morgan | 2 | |
| Tom17 | 0 | |
| Finlay Macrae | 0 | |
| Tony Markham | 3 | “I’m voting on basis of what I’ll be personally using, and this is the one I find useful. This and Tom17. Had a hard time choosing between them, but I think this one will also help me build the auto-annealer script for learning rate, and also I’ll be using it more frequently in analysis than Tom17’s one. Both are awesome though.” |
| Ahmed Shendy | 2 | “This is everything I was trying/planning to do in mine, but apparently much better” |
| Chris Thompson | 0 | |
| Cahya Wirawan | 8 | “I think all the entries are great, but I can see this addition saving me the most time as I’ve been manually crunching until now!” “I’m pretty new here and this is really useful, thanks” |
| Mithilesh Hinge | 1 |
Final results
And so the results are as follows:
1st: evanca, 10 votes
2nd: Cahya Wirawan, 8 votes
3rd: GummyBear, 5 votes
4-6: Daniel Morgan, Tony Markham, Ahmed Shendy, 3 votes
7: Mithilesh Hinge, 2 votes
8: Chris Thompson, 1 vote
9-10: Tom17, Finlay Macrae, 0 votes
I would like to repeat once more that all the submissions were great and added something of value. I am especially pleased with the fact that many of the submitters prepared their first pull request, or even first contribution to somebody else’s code in the open. That is awesome. All ten contestants deserve a big round of high fives.
Let me remind the rewards:
1st place: $200 AWS credit
Places 2-3: $100 AWS credit
Places 4-10: $50 AWS credit
We will be sending them out in the upcoming days.
Next steps
Now comes the tricky part: merging. Because Jupyter notebook is written in a json format through a WYSIWYG editor, its changes can make it impossible to simply merge the code. We will have to go over them and manually apply the changes. Some of the submissions overlap or do similar things. I think we will have to work together to make them a single submission and then merge.
That’s it for the challenge. Thank you to all involved in it. As usual I encourage you to join the AWS DeepRacer Community if you haven’t yet done so. Now that the Virtual League of 2019 is over, we will be focusing on preparation for the finals at the Re:Invent 2019 in Las Vegas and for other exciting events, and for the AWS DeepRacer League 2020.