RPi Cluster - SETI@Home

What is SETI@home? 

SETI@home is a scientific experiment, based at UC Berkeley, that uses Internet-connected computers in the Search for Extraterrestrial Intelligence (SETI). You can participate by running a free program that downloads and analyzes radio telescope data. You can learn more about Seti@Home on to the project!

How it works

After about 3 days of googling, learning and much swearing I got it working. Once I had got my head around mpiexec I then thought of what I could use it for. Enter seti@home. After more setup, config and swearing I had it running. I wanted to show the status of each pi via cmd line so using shell scripts and mpiexec to display it on a small screen. Then I learned about the blinkt leds strips so got one for each rpi zero on the hat and pythoned up the blinkt strips to display % complete of tasks being performed on that particular pi.

The main rpi3 GPIO pins were used by the clusterhat but it still displayed the status on the screen. Then disaster! I used the washer, dryer and kettle at the same time and the power safety switch got triggered. After reboot and restart I decided I needed a UPS. From an old motorcycle battery, a solar charge controller and a cigarette car charger I cobbled it all together. The switch from mains power to battery is triggered via the network cable on the main rpi3 that is connected to an old router. If the power goes off on the router then the ups kicks in.

Parts for the Cluster:

• 1 RPI3b+
• 4 RPI zeros
• 4 Blinkt strips
• 1 Network cable
• 1 usb to usbmicro cable
• 1 Router (you could also plug the network cable into another computer)

Parts for the UPS (Power Supply)

• 1 12v motorcycle battery 
• 1 usb car cigarette charger
• 1 solar charge controller
• 1 usb to usbmicro cable
• 1 12v power adaptor

How the UPS Works

The UPS battery is connected to the solar charge controller and is being charged by the 12v power adaptor. The USB car cigarette charger bumps the voltage from 12v to 5.1v. The usb to usbmicro is the power cable for the cluster. When the mains power goes out, the solar charge controller switches from mains and charging to battery power.

Configuring the RPi Cluster

Each rpi has a getstats.sh script that collects the info and stores it in a file. Only the main rpi3 has the UPS part of the script at the bottom

Read code comments for more information on how it all comes together

getstats.sh
========================================================
#! /bin/bash
#echo $HOSTNAME

##remove previous files
rm tmpstats*

##create new blank files
touch tmpstats_$HOSTNAME
##add the hostname and a line
hostname > tmpstats_$HOSTNAME
echo "==========" >> tmpstats_$HOSTNAME
##get the uptime of the computer and write to file
uptime -p | tr -d , >> tmpstats_$HOSTNAME
##get CPU usage and write to file
echo "CPU:" $[100-$(vmstat 1 2|tail -1|awk '{print $15}')] >> tmpstats_$HOSTNAME

##get memory stats and write to file
free -th | grep Total | awk '{print "Total Mem: "$2}' >> tmpstats_$HOSTNAME
free -th | grep Total | awk '{print "Used Mem: "$3}' >> tmpstats_$HOSTNAME
free -th | grep Total | awk '{print "Free Mem: "$4}' >> tmpstats_$HOSTNAME

##get RPI stats and write to file
vcgencmd measure_temp >> tmpstats_$HOSTNAME
vcgencmd measure_volts core >> tmpstats_$HOSTNAME
vcgencmd get_mem arm >> tmpstats_$HOSTNAME
vcgencmd get_mem gpu >> tmpstats_$HOSTNAME

##get bounc stats and write to file
boinccmd --get_simple_gui_info | grep fraction | sed -r 's/( )+//g' >> /home/pi/tmpstats_$HOSTNAME

##get ups stats and write to file
echo "UPS Stats" >> tmpstats_upsd
echo "==========" >> tmpstats_upsd
upsd -i >> tmpstats_upsd
echo
========================================================

The main rpi3 has a runscript that collects all the files and displays them on screen.
run001.sh
========================================================
##clear screen, display date/time and message
clear
date
echo "Finding swarm machines"

##run the command hostname on each node in the cluster
mpiexec -f /home/pi/machinefile -n 5 hostname

echo "Running swarm scripts"

##run getstats.sh on each node in the cluster
mpiexec -f ./machinefile -n 5 /home/pi/getstats.sh

##clear screen, display date/time
clear
date

##print to screen in columns the contents of 2 files below
pr -m -t  ~/tmpstats_RPC ~/tmpstats_upsd

##print blank line
echo

##print to screen in columns the contents of 4 files below
pr -m -t /var/lib/clusterctrl/nfs/p1/home/pi/tmpstats_p1 /var/lib/clusterctrl/nfs/p2/home/pi/tmpstats_p2 /var/lib/clusterctrl/nfs/p3/home/pi/tmpstats_p3 /var/lib/clusterctrl/nfs/p4/home/pi/tmpstats_p4
========================================================

Pi Zero Configuration

Each Pi Zero has a python script to collect and display the percentage on the blinkt strips. (The results of the boinccmd percentage done is to 6 decimal places so I had a bit of fun with the leds) This is still a work in progress as I want to see how much info I can display via the blinkt strips percent_done.py

Read code comments for more information on how it all comes together

#!/usr/bin/env python

import time
import subprocess
from sys import exit

##check if module is installed
try:
	import psutil
except ImportError:
	exit("This script requires the psutil module\nInstall with: sudo pip install psutil")

import blinkt

##when script stops turn off leds
blinkt.set_clear_on_exit()

def show_graph(v, r, g, b):

##set all the pixels to red
	v *= blinkt.NUM_PIXELS
	for x in range(blinkt.NUM_PIXELS):
    	if v < 0:
        	r, g, b = 1, 0, 0
##fade in blue pixels based on while true calculations
    	else:
        	r, g, b = [int(min(v, 1.0) * c) for c in [r, g, b]]
    	blinkt.set_pixel(x, r, g, b)
    	v -= 1

	blinkt.show()

blinkt.set_brightness(0.1)

##this bit splits the 6 decimal places number into
##percentages and assigns them to parameters
##for fading the blue pixels
while True:
	v = subprocess.check_output("boinccmd --get_simple_gui_info | grep fraction | sed -r 's/( )+//g' | tail -c 7 | head -c 2", shell=True)
	c = subprocess.check_output("boinccmd --get_simple_gui_info | grep fraction | sed -r 's/( )+//g' | tail -c 5 | head -c 2", shell=True)
	b = subprocess.check_output("boinccmd --get_simple_gui_info | grep fraction | sed -r 's/( )+//g' | tail -c 3 | head -c 2", shell=True)
	v = int(v)
	c = int(c)
	b = int(b)
	v = v / 100.0
	v = float(v)

##runs show_graph, passing in values for blue led fade
	show_graph(v, 0, 0, b)
	time.sleep(1.0)

========================================================

Attachment - Project Files