One of the things we’re hoping to do here is create a cheap, easy and powerful platform for people to get started with autonomous car technology. We start by asking the question: “What could we do for $100 in a weekend that is on the path to pro-level autonomous cars”? That price point pretty much points us to something based on RaspberryPi 3, which is the best way to do computer vision on a budget. Not only does it have enough computing power itself to run OpenCV, but it has built-in WiFi, so you can stream the data to a laptop to run even more challenging code, such as TensorFlow and neural networks.
The hardware side of this platform is easy: make this. But the software side is harder. We know we want to use Python if at all possible, since it’s become the standard for AI (it’s also easy to use on all platforms). But should we standardize on something based on OpenCV, like ForumulaPi and CompoundEye, which uses standard computer vision techniques for standard courses? Or should we move to the more flexible CNN neural network approach like TensorFlow/Keras, which requires training for each course?
Each has pros and cons. The CV approach is very predictable, and can be tuned to perform well on a standard track such as the RGB FormulaPi one. The CNN approach, on the other hand, is very flexible (it can work on any track/road/environment) but is very unpredictable. You can train it on a track so it nails it, and then the lighting changes or people gather around the track and it no longer knows where it is. It’s the classic blackbox problem and the debugging tools for neural networks are still in their infancy.
Will Roscoe is making good progress towards a framework that can take the best of both worlds. He’s already built a good RaspberryPi-based rover on the TensorFlow/Keras framework (called Donkey for some reason). Meanwhile the CompoundEye team has built an equally good RaspberryPi-based rover on the OpenCV framework (using C++ rather than Python for performance reasons, although that might not be necessary).
How to choose between the two? Will is working on a codebase that may allow you to use both. He’s porting the CompoundEye code to Python and merging it with his Donkey code.
If this works, you may be able to make a run-time decision to use a CNN or OpenCV on the same rover, to compare the performance of the two approaches head-to-head. That would be really cool, and if it works I’ll make this the standard DIY Robocars reference platform.
