This repository is about CartoNaut, an AI-driven robot which uses vision and text-to-speech to explore the surroundings and attempt to reach goals and targets set by the user.
CartoNaut is currently powered by Gemini 2.0 Flash Thinking Experimental 01-21.
The name CartoNaut comes from "Carto" - from Cartographer, and "Naut" - from Astronaut. The name is designed to showcase the bot's visual mapping and exploration skills.
| Item | Price |
|---|---|
| Google Account | N/A |
| elevenlabs OR PlayHT account (you can sign up with google) | N/A |
| Another Computer (Windows, Mac, or Linux) | N/A |
| A Wi-Fi network | N/A |
| Raspberry Pi Zero 2WH | £16.98 |
| CamJam EduKit 3 - Robotics | £20 |
| Micro-USB OTG Adapter | £2 |
| microSD card | £8.00 |
| USB Webcam | £6 |
| Powerbank | £15 |
| Total | £67.98 |
-
Unpack the CamJam Edukit. Keep the box safe.
-
Set up your robot according to the instructions found here. Use the box as the chassis.
-
Download Raspberry Pi Imager on your computer. You can use Windows, Mac or Linux.
-
Plug in the microSD card to your computer
-
Open Raspberry Pi Imager and configure it as shown below:
-
Configure OS Customisation as shown below:
(set the password to "raspberry")
-
Make sure to enable SSH in the SERVICES tab.
-
Flash the SD Card.
-
Unplug the SD Card from your computer, and plug it in to the Pi.
-
Connect the USB Webcam to the Pi via the OTG cable.
-
Power the Pi using the powerbank and a USB cable.
-
On your computer, open a new CMD/Powershell/Terminal window and type: "ssh [email protected]"
-
If prompted, accept fingerprint warnings with "yes" and type "raspberry" as the password. You may not be able to see it as you are typing.
-
If not already installed on the Pi, install Python and Git.
sudo apt install python3 python-is-python3 gitThese should be installed on both the Pi and your local computer.
-
On both the existing command window and a brand new one, enter
git clone https://github.com/uukelele-scratch/CartoNaut.gitThen,cd CartoNaut. -
On the ssh-connected command window, run
pip install opencv-python flask flask-socketiothenpython server.py -
To verify everything is working correctly, in your browser on your local computer, go to
http://zero.local:5000. You should see the latest frame from the connected webcam. -
On your local PC, go to https://mpv.io/installation/ and follow the installation instructions. For Windows, download a version suitable for your PC (e.g. https://github.com/shinchiro/mpv-winbuild-cmake/releases/download/20250210/mpv-dev-x86_64-20250210-git-73b8459.7z) and unzip the file. Then, move the folder to a location such as C:\ and add it your PATH - type "path" in the Start menu and open "Edit the system environment variables", then click "Environment Variables" at the bottom. Double-click "Path" at the top, then click New and paste the location of the folder that the unzipped files are in (containing mpv.exe and mpv.com)
-
On your local PC, run
pip install google-generativeai python-socketio requests pillow elevenlabs pyht -
Grab yourself a Google API Key at https://aistudio.google.com/apikey. Also, get an ElevenLabs or PlayHT API Key at their respective websites.
-
In a text editor on your local PC, open
client.py. For example, on Windows, runnotepad client.py. -
Near the top, you will find a config area. Here, you can put config data. To start, put your Google API Key opposite "GEMINI_API_KEY" (enclosed in quotes), and do the same for either ElevenLabs or PlayHT (+ userid if using PlayHT), depending on which you are using. Set the "AUDIO" to either "ElevenLabs" or "PlayHT", or leave it as "" (empty) if you don't want your robot to talk.
-
Set your GOAL, TARGET, and SUCCESS_CRITERIA as you wish.
-
Run
python client.pyon your local computer. Then, make sure the system volume isn't muted, and enjoy!
If you find any issues with CartoNaut, or the building process, I'd love to know! Please file an issue at the "Issues" section near the top of this page.
This repo is open to pull requests!