Thanks to the astoundingly productive Tawn Kramer, I’ve had a chance to play around a bit with his Donkeycar fork that uses the impressive Intel Realsense T265 visual odometry “tracking camera” instead of a regular RaspberryPi camera. Since Tawn’s original work, the Intel Realsense team has released a number of performance improvements for the T265, so it tracks even better now.
I also decided to use a processor with more power than the Raspberry Pi 3 that Tawn used, so I went with the Jetson Nano, which is the only other non-X86 single-board computer I was able to get the Realsense SDK working on (it won’t work on a Raspberry Pi 4, since Ubuntu Mate, which is required, doesn’t run on that yet. Realsense will work on any X86-based SBC, such as an Up Board or LattePanda, but those are a bit more expensive).
It took a while to get all this working, so this post is just a summary of the steps needed.
Steps:
- Follow the Donkeycar installation on Jetson Nano. But when you get to the part about cloning the Donkeycar code, use my fork instead:
git clone https://github.com/zlite/donkey.git - Build Intel Librealsense on Jetson Nano
- Then build pyrealsense2 from source
- Go through this Donkeycar application setup and make sure you note the special Jetson Nano instructions
- Setup your Joystick, following these instructions. (update: see TCIII comment below for updated instructions) I used a PS4 controller (see TCIII’s comment below if you’re using an Xbox controller). Note that you’ll have to pair it via Bluetooth with the Nano. I did that in the Nano’s desktop interface by selecting the Bluetooth icon and adding the controller as a Bluetooth device when it was in pairing mode. The instructions say to set the driver to start at boot by editing the rc.local file, but you may find that the Nano doesn’t already have one. No worries, just create it from scratch and add the necessary line like this:
sudo nano /etc/rc.localpaste /home/pi/env/bin/ds4drv --led 00ff00 into the file. Save and exit.
If you’re using a PS3 controller, in manage.py change this line to say this:
cont_class = PS3JoystickController
Set bus num to 1 in myconfig.py:
PCA9685_I2C_BUSNUM = 1
After you’ve done that, set up the directory with this:
donkey createcar --path ~/follow --template path_follower
Running:
cd ~/follow
python3 manage.py drive
Once it’s running, open a browser on your laptop and enter this in the URL bar: http://<your nano’s IP address>:8887
The rest of the instructions from Tawn’s repo:
- When you drive, this will draw a red line for the path, a green circle for the robot location.
- Mark a nice starting spot for your robot. Be sure to put it right back there each time you start.
- Drive the car in some kind of loop. You see the red line show the path.
- Hit X on the PS3/4 controller to save the path.
- Put the bot back at the start spot.
- Then hit the “select” button (on a PS3 controller) or “share” (on a PS4 controller) twice to go to pilot mode. This will start driving on the path. If you want it go faster or slower, change this line in the myconfig.py file:
THROTTLE_FORWARD_PWM = 530 - Check the bottom of myconfig.py for some settings to tweak. PID values, map offsets and scale. things like that. You might want to start by downloading and using the myconfig.py file from my repo, which has some known-good settings and is otherwise a good place to start.
Some tips:
When you start, the green dot will be in the top left corner of the box. You may prefer to have it in the center. If so, change PATH_OFFSET = (0, 0) in the myconfig.py file to PATH_OFFSET = (250, 250)
For a small course, you may find that the path is too small to see well. In that case, change PATH_SCALE = 5.0 to PATH_SCALE = 10.0 (or more, if necessary)
If you’re not seeing the red line, that means that a path file has already been written. Delete “donkey_path.pkl” (rm donkey_path.pkl) and the red line should show up
It defaults to recording a path point every 0.3 meters. If you want it to be smoother, you can change to a smaller number in myconfig.py with this line: PATH_MIN_DIST = 0.3
Thank you for great work. I have following error on Step 1.
ERROR: Command errored out with exit status 1:
command: /home/jetson/env/bin/python3 /home/jetson/env/lib/python3.6/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmp1x9g2pht …..
Do you have any direction or comment to clear ?
This is a far superior method of compiling Intel’s Librealsense and pyrealsense2 at the same time for Jetpack 4.4 as long as you use LIBREALSENSE_VERSION=v2.32.1 and cuda=10.2 in the “./buildLibrealsense.sh” :https://brisbaneroboticsclub.id.au/compile-intel-realsense-for-nvidia-nano-d400-sr300-and-t265/
If Step 4 is followed to the letter, then donkey createcar –path ~/mycar will be created in the root directory. However later on donkey createcar –path ~/follow –template path_follower will be created in the root directory. I guess that there could be two “mycars” with different names in the root directory and one can select “follow” for this project?
For those of you who would like to use the XBox Bluetooth wireless controller with this project, the disabling of ertm will not work per the DC documentation for the XBox game controllers. To permanently disable ertrm do the following:
Step1: Install sysfsutils
sudo apt-get install sysfsutils
Step 2: Edit the config to disable bluetooth ertm
sudo nano /etc/sysfs.conf
Step 3: Append this to the end of the config
/module/bluetooth/parameters/disable_ertm=1
Step 4: Reboot your computer
sudo reboot
Step 5: Re-pair the Xbox One Bluetooth Controller
Unpair the controller first, then pair it again.
You should now have a solid light on the xbox button and a stable bluetooth connection.
Regards, TCIII
Presently it does not appear that any additional controller software (unlike the PS4 controller) needs to be installed for the path_follower software to communicate with the XBox BT controller as xpad is built into Ubuntu. You can find/test/map your XBox controller with Jstest by installing it with “sudo apt-get install jstest-gtk” as I did. However at this point I have yet to see if I have controller control over the steering and throttle. Doing this step by step!
A thorough review of the zlite/donkey/ master branch did not reveal a “manage.py” program that is to be used in setting “cont_class = PS3JoystickController”. There is however a “manage_remote.py”, but I do not think that would not work in this case. Should this have referenced some kind of configuration program/file?
This is a correction to my post above concerning “manage.py” Even though I did not find a “manage.py” in the master branch, it did appear after running the command “donkey createcar –path ~/follow –template path_follower” to create the follow directory. “Manage.py” appears in the /follow directory. However “manage.py” is not visible as a program until “donkey createcar –path ~/follow –template path_follower” has been run.
Since I am using a XBox One controller I assume that “cont_class = PS4JoystickController” should be changed to “cont_class = XboxOneJoystickController” in manage.py?
I caught a couple of other joystick controller changes that must be made to get correct operation of manage.py drive to function correctly.
“CONTROLLER_TYPE=” in myconfig.py also needs to be changed to whatever joystick controller is being use.
Also “cont_class =” has to be changed in “path_follower.py” to whatever joystick controller is being used as well as in manage.py as stated above.
After conversing with some friends on DC Discord, it turns out that “ds4drv” is not required for connecting manage.py drive to, in this case, the PS4 Joystick Controller. Just use the Nano Jetpack desktop to pair your joystick controller to the Nano though the desktop Bluetooth symbol negating the need for “ds4drv”. However make sure that your PS3/4 is connected (solid blue light) to the Nano before you run manage.py drive.
Also it should be noted that the command in rc.local to start “ds4drv” does not work at startup. If you need to run “ds4drv” for some reason at startup, you can use the CLI to send “ds4drv –led 00ff0 &” to start “ds4drv” manually in the background. Once “ds4drv has connected with your PS3/4 Joystick Controller you can use ^C to return to the CLI and run manage.py drive.
I have been able to install the latest version of Intel’s LibRealSense2, pyrealsense2 wrapper, and the original DC software that Tawn wrote and Chris modified for the T265 tracking camera on a Rpi 4B 4GB running the latest version of Buster.
Since my “car” is a three wheeled differential steering robot, that does not move that fast, I had to increase the path data sampling rate from 0.3 m to 0.1 m of travel to obtain accurate steering. Additionally I had to convert the DC steering and throttle output to differential steering commands by using a Dimension Engineering Sabertooth 2X10RC motor controller.
Since no training of the path data is required as in the normal DC process, I call this YODO, You only Drive once.
Hello, I’m trying to follow your process but it’s telling me that your GitHub page is no longer available, can you help me out, please?
Thank you