Hey gang, Tim here at Core Electronics. Today we're building a face tracking system for Raspberry Pi. Using a Raspberry Pi camera to search for faces, it's going to lock onto them with the Pimoroni Picade pan-tailed hat, following them around the room. Then we're going to take it to the next level by adding a movement tracking feature and a patrol face. Locking your face into the center of the frame means always keeping the action in shot.

This video is going to provide code for the Pimaroni Pi K pan-tailed hat, but the fully commented code has been written with some logical alterations so that any pan and tilt system can be controlled using a Raspberry Pi single board computer. The intention here is to not only create an easy to use face tracking system with a pan-tailed hat, but to do so in a way that can readily be expanded upon no matter what system or features you choose to use. The backbone of the system used here is going to utilize OpenCV machine learning to identify faces and identify movement.

On the table before me is all the components you will need to get the system up and running real fast. This video makes extensive use of the Pimaroni Pi K pan-tailed hat. If you haven't set up one before, check out our online guide as it will provide you with all knowledge. Link down below. I'm going to be using the Raspberry Pi official camera module version 2, but you could use any camera module with a similar form factor. I am mounting the HAT to a Raspberry Pi 4 model B. The extra power that these boards have are uber valuable for computer vision systems like the one we're building towards here. You will also want a microSD card that has been flashed with the Raspberry Pi OS that has been connected to the Internet, is updated, and has the camera enabled. We have also installed a number of packages to the Raspberry Pi OS through the terminal.

We've jumped to everything connected, plugged in and with power to the system. The first code I'll demonstrate will get our system right into face tracking. Download link for all the codes talked about here in the description can be found at the bottom of the article page. Unzip them to your Raspberry Pi desktop and open the codes by right clicking and selecting Thonny IDE.

WinRun is going to actively identify faces and attempt to keep the found face in the center of the captured video by panning and tilting the servos. After a second or two on screen, you're going to see a live feed from the Raspberry Pi camera. Now I'm going to show up my face and hopefully it's going to draw a box around my face and use the pan tilt servos to keep my head centered in the middle of the frame. Oh, there we go. It's working straight away.

It is interesting here to test how well the system locks into your head and the different head orientations that it can or cannot pick up. For example, I found by cocking one's head, it makes the face invisible to the facial recognition. Of potential interest to those who have seen the video on facial recognition with the Raspberry Pi, you will notice that the machine learning used to identify faces here is different from the one used in that video. The system used here doesn't identify and label between different learned faces, but in exchange for that, it is much faster at finding human faces.

So after achieving this, the natural next step was to create a patrol phase for our pan tilt system. This will allow the system to scan the room and detect any faces that enter the frame.

The code is named facetrackpantilthatpimaroni.py, and I'm going to open it up the same way as before into Thonny IDE. The code can be downloaded at the bottom of the article page. When I press the green button, the system will revert to sentry mode whenever it's unable to see a person or movement. A live feed of this will be coming onto the desktop. When movement is identified, a green circle will appear on the video feed to approximate the center of the movement. The system will then attempt to make the center of movement the center of the camera frame. When a face is found, it will snap to it and paint a blue box around the face on the video stream. It will then do whatever it can to stick to that face. As soon as it cannot find a face, it will return to sentry mode.

There are a lot of settings that can be adjusted. The config.py file contains a small sample of the things that can be changed. Having opened up the config.py file, the system will be active whenever it doesn't see any faces on screen and will allow the pan tilt to move around its degrees of freedom logically in the search of one. To aid this patrol phase further, I'm going to add in the feature to automatically swivel the camera towards any objects that it identifies as moving. Whatever face is seen will lock into that face, and the two new features will be disregarded. These added features make finding elusive faces much more likely.

Before, here in this section, you can see that you can alter the default positions that the servers do in sentry mode. If you scroll down to the bottom of the config file, you can see a sensitivity setting, which if you increase, will make it more likely to pick up small movements. However, if you increase it too much, it's going to produce false readings. The movement detection used here is a similar method to the measuring speed with Raspberry Pi video.

Check that out, link down below. And a big thanks goes to Claude Pajot once again, as I based a lot of this on his previous work, particularly the face track demo, which I'll also link in the description.

There are a number of excellent places we can do with this. For instance, what to do when you see multiple faces, perhaps putting preferences to one face over the other based on particular parameters would be a worthy code addition. Or picture the scenario where there's people you're interested in, but they're all wearing masks. In that case, instead of running facial detection, we'd be better off running an object detection layer and searching just for people.

Also, there are many types of pan tilt methods, and I have attempted to make it so that all these can be incorporated successfully and painlessly into the code I've created. Many common pan tilt systems use slow moving brushless DC motors, and some even use stepper motors. A little tinkering with the code and you'll be off to the races running any pan tilt system successfully.

So hopefully that has got you planning ways to forge ahead in a new direction with machine learning. There is a lot of magic and an endless amount of creativity possible within this world. I'm only just getting started. So until next time, stay cozy.