Monitoring Raspberry Pi

Published on Friday, May 20, 2016

Before commissioning the Raspberry Pi; it would be nice if we setup some monitoring; so we can correlate any issue in the field with device status. This becomes important especially for devices like Raspberry Pi that has limited resources. The simplest and easiest way is to setup SNMP; its the protocol to collect and organize information about the managed devices on IP networks. Given Raspbian is a just another Linux; we can easily setup SNMPD; a SNMP daemon; and can monitor the device remotely or even from within the device. To install SNMPD; issue the following commands; and once installed backup the /etc/snmp/snmpd.conf

$ sudo apt-get install snmpd
$ sudo cp /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.original

Next; edit the snmpd.conf; remove everything and punch in the following

agentAddress    127.0.0.1:161
rocommunity     public

Restart the snmpd using

$ sudo /etc/init.d/snmpd restart

With above arrangements in place; we are basically running the SNMPD (SNMP Agent) on localhost 161 UDP port and have configured “public” a read only community (password / key). We can now “query” SNMPD using snmp utilities. To install them issue

$ sudo apt-get install snmp

Once installed; issue the following command to query free CPU percentage via snmp and we will have an output like this

$ snmpget -v 1 -c public localhost .1.3.6.1.4.1.2021.11.11.0
iso.3.6.1.4.1.2021.11.11.0 = INTEGER: 90

.1.3.6.1.4.1… is an OID; there are well known OID (Object Identifier) for free CPU; there are other OIDs for things like this; some of which we will use later.

Next we want to “expose” CPU Temperature through SNMP; by default its not there and given device IO is done in Linux through files and SNMPD has an option to get data by running a script and expose it through additional OID. Lets make a script for CPU temperature

$ nano snmp-cpu-temp.sh

Punch in the following

#!/bin/bash
if [ "$1" = "-g" ]
then
        echo .1.3.6.1.2.1.25.1.8
        echo gauge
        cat /sys/class/thermal/thermal_zone0/temp
fi
exit 0

Make the script executable and run it with -g

$ chmod +x snmp-cpu-temp.sh
$ ./snmp-cpu-temp.sh -g
.1.3.6.1.2.1.25.1.8
gauge
49768

The temperature is 49.768 degree Celsius; lets edit snmpd.conf ($ sudo nano /etc/snmp/snmpd.conf) to add this script; make it look like this; have highlighted the new lines

agentAddress    127.0.0.1:161
rocommunity     public
pass            .1.3.6.1.2.1.25.1.8 /bin/sh /home/pi/snmp-cpu-temp.sh

Restart the snmpd and query the OID using snmpget and we will have the value

$ sudo /etc/init.d/snmpd restart
[ ok ] Restarting snmpd (via systemctl): snmpd.service.
$ snmpget -v 1 -c public localhost .1.3.6.1.2.1.25.1.8
iso.3.6.1.2.1.25.1.8 = Gauge32: 50458

As we already have the NGINX installed; we can easily setup MRTG; its light weight and widely used monitoring setup that generate graphs using data gathered over SNMP that we can host in NGINX and view remotely. To install issue this command:

$ sudo apt-get install mrtg

MRTG comes with helper utilities like CFGMAKER and INDEXMAKER; we can use cfgmaker to make mrtg.cfg (in /etc) but given we also want to include our own additional OID; lets make the mrtg.cfg ourselves; take a backup of original /etc/mrtg.cfg; remove everything and punch in the following

WorkDir: /var/www/mrtg
EnableIPv6: no
LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt

Target[CPU]: 100 - .1.3.6.1.4.1.2021.11.11.0&.1.3.6.1.4.1.2021.11.11.0:public@localhost
Options[CPU]: integer, gauge, nopercent, growright, unknaszero, noo
MaxBytes[CPU]: 100
YLegend[CPU]: CPU %
ShortLegend[CPU]: %
LegendI[CPU]: CPU
Legend1[CPU]: CPU usage
Title[CPU]: Raspberry Pi CPU load
PageTop[CPU]: <H1>Raspberry Pi - CPU load</H1>

Target[Memory]: .1.3.6.1.2.1.25.2.3.1.6.1&.1.3.6.1.2.1.25.2.3.1.6.3:public@localhost
Options[Memory]: integer, gauge, nopercent, growright, unknaszero, noo
MaxBytes[Memory]: 100524288
YLegend[Memory]: Mem - 1K pages
Factor[Memory]: 1024
ShortLegend[Memory]: B
LegendI[Memory]: Physical
LegendO[Memory]: Virtual
Legend1[Memory]: Physical
Legend2[Memory]: Virtual Memory
Title[Memory]: Raspberry Pi Memory Usage
PageTop[Memory]: <H1>Raspberry Pi - Memory Usage</H1>

Target[CPU-temp]: .1.3.6.1.2.1.25.1.7.0&.1.3.6.1.2.1.25.1.8:public@localhost
Options[CPU-temp]: integer, gauge, nopercent, growright, unknaszero, noi
Factor[CPU-temp]: 0.001
MaxBytes[CPU-temp]: 100000
Title[CPU-temp]: CPU temperature on Raspberry Pi
YLegend[CPU-temp]: Temperature °C
ShortLegend[CPU-temp]: °C
Legend2[CPU-temp]: CPU temperature in °C
LegendO[CPU-temp]: CPU temperature
PageTop[CPU-temp]: <H1>Raspberry Pi - CPU Temperature</H1>

Target[Ethernet]: 2:public@localhost
MaxBytes[Ethernet]: 12500000
Title[Ethernet]: Raspberry Pi Ethernet Traffic Usage
PageTop[Ethernet]: <H1>Raspberry Pi - Ethernet Traffic Usage</H1>

mrtg has an option to run it as daemon and it will poll the assigned snmps and make graphs periodically. Next we need script for /etc/init.d through which not only we can run the mrtg as daemon but it will also start across system reboots. We can google the interenet; people out there has already made such script; here is one such script that you can save as /etc/init.d/mrtg ($ sudo nano /etc/init.d/mrtg)

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON="/usr/bin/mrtg"
PARAM=" --user=root /etc/mrtg.cfg --logging /var/log/mrtg.log"
NAME="MRTG"
DESC="Multi Router Traffic Grapher Daemon"

test -f $DAEMON || exit 0
set -e
case "$1" in
start)
        echo -n "Starting $DESC: "
        env LANG=C $DAEMON $PARAM
        echo "$NAME."
        ;;
stop)
        echo -n "Stopping $DESC: "
        killall -9 mrtg
        echo "$NAME."
        ;;
restart|force-reload)
        echo -n "Restarting $DESC: "
        killall -9 mrtg
        sleep 1
        env LANG=C $DAEMON $PARAM
        echo "$NAME."
        ;;
*)
        N=/etc/init.d/$NAME
        echo "Usage: $N {start|stop|restart|force-reload}"
        exit 1
        ;;
esac
exit 0

Once saved; make it executable; create a directory/var/www/mrtg where it will store the graphs and start the daemon. It will give few warnings and its ok when running first time.

$ sudo chmod +x /etc/init.d/mrtg
$ sudo mkdir /var/www/mrtg
$ sudo /etc/init.d/mrtg start

All we need now is to expose the graphs through NGINX; lets edit /etc/nginx/sites-available/default and make it look like this ($ sudo nano /etc/nginx/sites-available/default), have highlighted the new lines

server {
        listen 80 default_server;
        listen [::]:80 default_server;
        index index.html index.htm index.nginx-debian.html;
        server_name _;
        location /mrtg {
                alias /var/www/mrtg;
        }
        location /node {
                proxy_pass
http://localhost:3000;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection 'upgrade';
                proxy_set_header Host $host;
                proxy_cache_bypass $http_upgrade;
        }
}

Create an index.html using mrtg’s indexmaker utility and restart the nginx. While using indexmaker; it might give permission error; use sudo –i and logout when its done

$ sudo indexmaker /etc/mrtg.cfg >> /var/www/mrtg/index.html
-bash: /var/www/mrtg/index.html: Permission denied
$ sudo -i
# indexmaker /etc/mrtg.cfg >> /var/www/mrtg/index.html
# service nginx restart
# logout

Check the graphs at http://ip/mrtg; after a while it will look something like this

image

Go ahead and click the graph to view the details!

With this in place; we can deploy our applications and send our Raspberries into the field and once they are there we can remotely check how they are performing! We can expose the SNMP to some central monitoring system on the same lines where sys-admins can keep an eye on them; if such arrangement is in place!

Happy coding and deployments!