[Update – Until there is correct compatibility of OPEN-CV with the new Raspberry Pi ‘Bullseye’ OS I highly recommend at this stage flashing and using the previous Raspberry Pi ‘Buster’ OS onto your Micro-SD for use with this guide – Official 'Buster' Image Download Link Here]
Here you will learn how to accurately measure the speed of an object (be it a car, person, or matchbox car) using only a Raspberry Pi and a camera. As the object travels in front of it machine learning will take over to determine both the moving object and its speed (in km/h or mp/h). The completely open-source software will then take a snapshot of the object, add a speed and timestamp to the picture of the object. Machine Learning and Raspberry Pi you’ve done it again.
Cameras in combination with artificial intelligence create arguably the most powerful sensor you can ever put on a Raspberry Pi. The variety of things you can do with this handheld technology is truly spectacular. In this tutorial, we will combine these two worlds to create a system that can determine the speeds of objects as they go past the camera. Below are the contents of the guide.
- What You Need
- Initial Set-Up and Install Process
- Demonstration
- Code Refinement and Calibration
- Where to Now
- Acknowledgement
- Setting up Open-CV on Raspberry Pi 'Buster' OS
Once set up you will be able to use the video data coming in to identify the speed of objects moving. The Open-CV will determine what is a moving object. The software will then compare two photos and (from some trigonometry and known distances) infer the speed of that object. Then it will take the second compared image, provide it with a timestamp, file location and stamp the speed of the object onto it, save it and upload it to a local network website. These photos can then be accessed through a website from any machine on your local network. Below you can see it has recorded a matchbox car going at the high speed of 1.1km/h (the Speed Stamp has been edited larger and green so you can see it better). The speed capturing location is defined by the red rectangular box.
Actual modern speed cameras use LIDAR to capture information of objects coming directly towards or away from them. This I find really interesting being able to do so with a camera facing perpendicular to the movement. This guide is taking a fully-fledged speed camera for monitoring cars and retrofitting it to all kinds of speed monitoring tasks.
This will use Open-CV which is a huge resource that helps solve real-time computer vision and image processing problems. This will be the third foray into the Open-CV landscape with Raspberry Pi and Facial Recognition being the first, Object and Animal Recognition with Raspberry Pi being the second. Also if you are interested in this kind of Computer Vision check out my other related tutorials QR codes with Raspberry Pi, Pose Estimation/Face Masking with Raspberry Pi, or Face and Movement Tracking with a Pan-Tilt System. Find at the bottom of this page the process to install Open-CV to your Raspberry Pi. As always if you have got any questions, queries, or things you'd like to see added please let us know your thoughts!
What You Need
Below is a list of the components you will need to get this system up and running real fast.
- Raspberry Pi 4 Model B (Having the extra computing power 'oomph' that the Pi provides is super helpful for this task but I'm informed that this set-up will work with a Raspberry Pi 3 Model B+)
- Raspberry Pi High-Quality Camera and Camera Lens (You can also use the Raspberry Pi Official Camera Module V2)
- Micro SD Card
- Power Supply
- Monitor
- HDMI Cord
- Mouse and Keyboard
Initial Set-Up and Install Process
The fast way to get up and running with object recognition on the Raspberry Pi is to do the following. Flash a micro-SD card with a fresh version of Raspberry Pi 'Buster' OS. Link on how to flash micro-SD with Raspberry Pi 'Buster' OS found here. With the Micro-SD Card flashed you can install it into your Raspberry Pi.
Then make sure to have the Raspberry Pi connected to a Monitor with peripheries and that a Pi Camera is installed in the correct slot with the ribbon cable facing the right way and start the Open-CV install process seen below. With that complete, you will have Open-CV installed onto a fresh version of Raspberry Pi OS. Then open up the Raspberry Pi Configuration menu (found using the top left Menu and scrolling over preferences) and enable the Camera found under the Interfaces tab. After enabling reset your Raspberry Pi. See the image below for the setting location.
Once that is complete the next step is to open up a new terminal using the black button on the top left of the screen. Then type and enter the following lines into it to install Docker. See further below for an image of this installation occurring in the terminal. The first line a quick method to download packages from all configured sources. This first line starts with | sudo | which means it will run with admin privileges. The next line is a client URL command line that pulls data from an online server to install the docker application and naming it | get-docker.sh |. The features that docker brings will be used here by the Speed camera. The last line is an admin command to install the docker package to the system.
sudo apt-get update && sudo apt-get upgrade
sudo apt-get install python-opencv python3-opencv opencv-data
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
Now we will install the Speed Camera to our Raspberry Pi system. Open up a new terminal command, then type and enter the following into it. See the image below of this occurring in the terminal. The first line is a change directory command which we will use to make sure that we are editing directly into the home folder. Then we will use a Website Get command to download the Speed Camera Installation files from a web server. The next line is a two-parter with the first section | chmod | used to change the access permissions of the files (so that anybody can alter the files, not just the admin), and then the | +x | is the method to access the files and folders within this directory. Then the last command line is used to run the speed camera installer.
cd ~
wget https://raw.github.com/pageauc/speed-camera/master/speed-install.sh
chmod +x speed-install.sh
./speed-install.sh # runs install script.
Demonstration
The best way to run this code is by typing and entering into a new terminal the following two lines. This will open up a window much like the one seen below. The first line is a change directory command so that we can focus on the freshly installed speed camera folder and its contents. The next line runs the Menu Box System which results in the image below.
cd ~/speed-camera
./menubox.sh
Old school graphics abound, but here you can everything you could want making this an amazing menu. Navigate through the options using the Arrow Keys on your Keyboard. Press Enter on Option A and Option B to run both the Speed Camera Script and the Webserver Script. Now, whenever a fast action occurs in front of the Camera inside the Red Capture Box it will capture the moment and upload it to the webserver. See the image below for this occurring. The webserver has the IP address of your Raspberry Pi followed by | :8080 | which when typed into any local network browser will let you access this information. For me, I type into my browser | 172.17.0.1:8080 |. Type and enter into the terminal | ifconfig | to find your IP address for your Raspberry Pi. At this stage the speed will not be accurate, we will need to adjust the code as described in the next section.
You can use this webpage to view all previous images and other installed software will even create reports and graphs from the gathered information.
Code Refinement and Calibration
There will be a couple of changes we can do to this software so you can get both accurate results and see what is occurring in the software if you want to troubleshoot (or just oogle the system in real-time). Access the Menu Box System like before, navigate down with the Arrow Keys, and Press Enter on Option C Settings. This will open up a new terminal page which you can navigate around using the arrow keys. When you have changed the settings by typing your preferences here press Ctrl-X, Y, and then Enter Key to save and lock in these changes.
Starting let's change this line | search_gui_on = True |. By doing this whenever the code operates we can now see a live stream video so we can see it judging speed in real-time. Keep in mind this will slow down the FPS of your device.
The second step will be adjusting the code so that it can accurately judge the speed. This software will judge speed accurately along a single plane perpendicular to the direction of the camera. It is important to calibrate this so that we can get good results. This is also a limitation of this particular system. It is important to realise that Raspberry Pi will be calibrated for particular distances, which means objects at different depths will end up getting inaccurate speed measurements. This system will allow you to calibrate the objects for a two-way street where one side is further away than the other which is a pretty nice feature to the code if you want to use it.
To do this we must do the following, change the setting | Calibrate = True |. When saved this will create images like the picture below with a grid of red lines across the image. This will also supply an uncalibrated speed result on the image too. For my matchbox, the uncalibrated speed was 33km/h. Now, use this image to measure how long the calibration object is in pixels knowing that the distance between each line is 10 pixels. In my case, the matchbox car was 110 pixels long. Now measure the object in millimeters in real life. In my case, the matchbox car was 85mm long. Now write down that actual value and the pixel size of the object into the system configuration setting as you can see in the image below. With that saved the system has been calibrated. In particular setup, any future images will now have accurate speed reading associated, see the very bottom of the below image for an accurate speed result of 1.1km/h.
If using as an actual speed camera take the time to test this using your own vehicle and log the actual speed for each run. Then fine adjust the results until you get exactly the corresponding speed values on your Raspberry Pi.
Another option worth talking about here is increasing or decreasing the resolution being captured by the camera. This will enable you to take excellent-looking photos but the software will run at a lower FPS. Depending on how long/how fast your object stays in the frame this can make for less reliable data. The flipside of this is running too low a resolution which can cause dimensional inaccuracies with a carry-on effect of garnering inaccurate speeds. This can be done by accessing the Option D Settings in the Menu Box Settings. Another option worth considering is adjusting the capturing box space. You can alter it so it captures the whole screen or so that it captures just a small slice of space. This setting can be found in the | speed-cam.py | python file.
Where to Now
Say a certain object goes past over a certain threshold speed. When this occurs I want to be able to control some physical hardware, in this case, I want my Raspberry Pi to turn on an LED. So let's dive into the code, make some simple adjustments and get this running. I will open up the | speed-cam.py | by right-clicking the file location and opening it up with Thonny.
Then I will add the following 7 lines of code to the top section of the code and far lower down in the | def speed_camera() | section. This is telling python to set up a Pin on the Raspberry Pi and for it to send power whenever the speed is faster than 0.5km/h. See the image below for all the code that I added highlighted.
Now as soon as you save this code and wire up a LED and resistor in series attached to a Ground Pin and the GPIO Pin 21, as you can see in the image below. So when you start the code the LED will start off and as soon as it catches something zipping past it faster than 0.5km/h the LED will turn on! Pretty nifty. GPIO (General Purpose Input and Output) is the interface that allows your Raspberry Pi to connect to the outside world. A great guide to clarify exactly why everything is set up as it is below can be found in Chapter 2 of the Raspberry Pi Workshop. This can be any manner of control
Acknowledgments
Huge thanks goes to Claude Pageau for writing this amazing, fully-fledged program, link to his Github here (which also has heaps of extra information on this. Honestly extraordinary how quickly you can have a fully-fledged speed camera set up using these components. He also has a great method to install Open-CV which I will experiment with and potentially create a stand-alone guide on as well. It could very well be a better process than my current labour of love method to install Open-CV via a long string of terminal commands that I explain below.
Setting Up Open-CV on Raspberry Pi 'Buster' OS
This is an in-depth procedure to follow to get your Raspberry Pi to install Open-CV that will work with Computer Vision for Object Identification. Soon I will create either a script/a separate tutorial to streamline this process. Turn on a Raspberry Pi 4 Model B running a fresh version of Raspberry Pi 'Buster' OS and connect it to the Internet.
Open up the Terminal by pressing the Terminal Button found on the top left of the button. Copy and paste each command into your Pi’s terminal, press Enter, and allow it to finish before moving onto the next command. If ever prompted, “Do you want to continue? (y/n)” press Y and then the Enter key to continue the process.
sudo apt-get update && sudo apt-get upgrade
We must now expand the swapfile before running the next set of commands. To do this type into terminal this line.
sudo nano /etc/dphys-swapfile
The change the number on CONF_SWAPSIZE = 100 to CONF_SWAPSIZE=2048. Having done this press Ctrl-X, Y, and then Enter Key to save these changes. This change is only temporary and you should change it back after completing this. To have these changes affect anything we must restart the swapfile by entering the following command to the terminal. Then we will resume Terminal Commands as normal.
sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev
sudo apt-get install libgtk2.0-dev libgtk-3-dev
sudo apt-get install libatlas-base-dev gfortran
sudo pip3 install numpy
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.4.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.4.0.zip
unzip opencv.zip
unzip opencv_contrib.zip
cd ~/opencv-4.4.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-4.4.0/modules \
-D BUILD_EXAMPLES=ON ..
make -j $(nproc)
This | make | Command will take over an hour to install and there will be no indication of how much longer it will take. It may also freeze the monitor display. Be ultra patient and it will work. Once complete you are most of the way done. If it fails at any point and you receive a message like | make: *** [Makefile:163: all] Error 2 | just re-type and enter the above line | make -j $(nproc) |. Do not fear it will remember all the work it has already done and continue from where it left off. Once complete we will resume terminal commands.
sudo make install && sudo ldconfig
sudo reboot
