The semi-monthly Oakland DIY Robocars races have now moved next door to the American Steel Poplar Gallery, which has a shiny, slippery polished concrete floor, so we’ve recalculated the performance metrics just to compare races on that track (as opposed to the rougher track at Pacific Pipe next door). The data from the past four races are above (just the top five in each race), and you can see some clear trends:
1) An average race-to-race improvement of top finisher of about 10%:
2) Less of a gap between the top 5 cars, suggesting that people are starting to dial things in better
3) A machine learning car (Carputer) is currently in first place, but the next three are computer vision (most of them using OpenMV). That suggests that ML is potentially better, but harder to get working well, which agrees with our real-world experience.
4) At this pace, the autonomous cars will beat the best human time in the next 2-4 months. It’s worth noting we don’t actually know what the “best human time” is for this track because we don’t know any humans who can drive RC cars particularly well, so this is a guess based on the best driving we can do. Since the ML cars use “behavioral cloning” (the human drives while the car records the data, then the car attempts to do the same at 1.3-1.5x speed), the ML cars that use this technique are by definition faster than their masters. But that doesn’t mean that they could beat any human.
5) The really fun part is the “all cars demolition derby” at the end. Pay particular attention to the tiny car on the outside, which is a Minimum Viable Racer built by Eric. He was a good sport about following the Fight Club rules (“If this is your first night, you must race”) and we promised him we’d help him rebuild it if it got crushed. The race isn’t over until Eric is done!
