Mike Baranski's Development Blog

RPIMON - Raspberry Pi Monitoring

In this article we will look at a simple metrics gathering system for multiple Raspberry Pi devices using Python, RabbitMQ, PostgreSQL and Metabase with Docker.

Requirements

We’ll need at last 1 Pi device. For the server device, we should install Ubuntu 64 bit and add some packages:

sudo apt-get install docker.io docker-compose python3-pip

git clone https://github.com/mikebski/rpi-mon.git

git clone https://github.com/mikebski/rpi-pymon.git

pip3 install virtualenv

The Components

Each Pi device that we will monitor will run a simple program that reports stats (load, cpu temp, and more) to a central messaging server running RabbitMQ.
We will setup a program that pulls messages off of the message queue and inserts them into a PostgreSQL database.
We will setup a serivce that reads data from the queue and writes it to a PostgreSQL table
We will setup a tool to visualize the data in the PostgreSQL table - Metabase - what a royal pain in the ass this is to get working on Docker for Pi (their default Dockerfile does not support the ARM architecture)

Here’s an ugly diagram of our system:

architecture

Setup the server

We’ll need to pick one machine to be the “server”. This machine will run Docker and several Docker containers:

RabbitMQ Docker container
PostgreSQL Docker container
RPI Queue Monitoring container (this reads the queue and writes to the database)
Visualization software (Metabase)

We will also need to have a static IP (or DNS entry) for the server machine.

Install OS

The first thing we need is an OS - we’ll use Ubuntu 64 bit in this tutorial - that supports Docker. Some Pi devices come with “noobs”, but for now we’ll just assume Ubuntu is on the Pi and it’s ready to go.

First, we update all packages:

sudo apt-get update

sudo apt-get upgrade

Install Docker & Docker Compose

To install Docker, we’ll run:

sudo apt-get install docker.io docker-compose

To test, we run Docker’s hello-world

docker run hello-world

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete 
Digest: sha256:4cf9c47f86df71d48364001ede3a4fcd85ae80ce02ebad74156906caff5378bc
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Now, using a docker-compose file, we’ll define our Docker containers. We will examine each section in turn.

Download compose file

The docker-compose file we will be using is here: https://gist.github.com/mikebski/cf37043c769967fc2c6c7c4dd4b375cd

Notably, we don’t have Metabase in this file. Rest assured we’ll get to that later. They don’t provide ARM images so we have to build our own.

    version: '3.1'
    
    services:
    
      db:
        image: postgres:alpine
        restart: always
        environment:
          POSTGRES_PASSWORD: 83riafdfbak8793qjqktlk;b;'q0[
        volumes:
          - "postgres:/var/lib/postgresql/data"
        networks:
          - monitoring
    
      adminer:
        image: adminer
        restart: always
        ports:
          - 8080:8080
        networks:
          - monitoring
    
      rmqmon:
        image: rabbitmq:3-management-alpine
        restart: always
        ports:
          - 15672:15672
          - 5671:5671
          - 5672:5672
        volumes:
          - "rabbitmq:/var/lib/rabbitmq"
        networks:
          - monitoring
    
      pymon:
        image: pymon
        restart: always
        networks:
          - monitoring
    
    volumes:
      postgres:
      rabbitmq:
    
    networks:
      monitoring:

Let’s create a directory with this file and download it. Login as the “pi” user and run:

cd && mkdir server && cd server

wget https://gist.githubusercontent.com/mikebski/cf37043c769967fc2c6c7c4dd4b375cd/raw/d47732958299b9863522690d1a5b793dcc1f9966/docker-compose.yml

ls -al

Volumes

Now, let’s look at the relevant bits of the file. There are containers, volumes and networks defined. The networks and volumes are pretty simple, one volume for postgres data (so data does not go away during reboots) and one volume for rabbitmq persistence. These are like “disks” for Docker and allows the data written by one container to stay around after it dies. The next container mounts the data (after a reboot, say) and carries on.

Networks

The network called monitoring is just a common network so all the containers can talk to each other.

Services

db

The DB service is Postgres, with a persistent volume and the username and password defined. Pretty simple.

adminer

Adminer is a UI for Postgres (sort of like PHPMyAdmin) and is we can access it on port 8080 of the pi.

rmqmon

rmqmon is our RabbitMQ server

pymon

pymon is our service that reads messages from the rmqmon queue and writes them to db (the PostgreSQL database).

It’s pymon like Python, by the way. Here’s how to build it as the pi user:

Let’s check out the project: git clone https://github.com/mikebski/rpi-pymon.git

Run cd rpi-pymon && ./build.sh - enter root password if prompted

That’s it - the container is built, you can see it by running sudo docker images | grep pymon

Running services

After buliding the pymon container we are ready to start our containers. We do this by downloading the dockder-compose.yml file and running docker-compose up -d (the -d disconnects the tty and runs our containers in the background).

Let’s try it. From the directory that contains our docker-compose.yml file, we run:

sudo docker-compose up -d

Starting mon_db_1      ... done
Starting mon_adminer_1 ... done
Starting mon_pymon_1   ... done
Starting mon_rmqmon_1  ... done

Now that we have our stack up and running we can use sudo docker ps to see what containers are up. It looks something like this:

root@alpharpi4:~/mon# docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                                                                                         NAMES
5874249ce4c6        rabbitmq:3-management-alpine   "docker-entrypoint.s…"   40 minutes ago      Up 14 minutes       4369/tcp, 0.0.0.0:5671-5672->5671-5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp   mon_rmqmon_1
61a99fa7292e        adminer                        "entrypoint.sh docke…"   40 minutes ago      Up 14 minutes       0.0.0.0:8080->8080/tcp                                                                                        mon_adminer_1
8d98ad21bf71        pymon                          "python /opt/consume…"   40 minutes ago      Up 14 minutes                                                                                                                     mon_pymon_1
b570aa791820        postgres:alpine                "docker-entrypoint.s…"   40 minutes ago      Up 14 minutes       5432/tcp                                                                                                      mon_db_1
root@alpharpi4:~/mon#

So, now we have our setup except for a couple of things:

Create the Postgres database
Create the process that sends messages from the Pi(s) to the RabbitMQ server
The Metabase server (we’ll do this last)

Setup Database

Now that Postgre is up, we need to create our database and a table. To do that, we’ll use the adminer tool and run some SQL. Go to your Pi’s IP address port 8080. In our example that’s http://192.168.3.107:8080:

Note that the Postgres user password is in the compose file

adminer login

Create a database called pimon

adminer create db

Run these SQL commands:

DROP TABLE IF EXISTS "stats";
DROP SEQUENCE IF EXISTS stats_id_seq;
CREATE SEQUENCE stats_id_seq INCREMENT 1 MINVALUE 1 MAXVALUE 2147483647 START 1 CACHE 1;

CREATE TABLE "public"."stats" (
    "id" integer DEFAULT nextval('stats_id_seq') NOT NULL,
    "stats_json" json NOT NULL,
    CONSTRAINT "stats_pkey" PRIMARY KEY ("id")
) WITH (oids = false);


DROP VIEW IF EXISTS "v_stats";
CREATE TABLE "v_stats" ("hostname" json, "temp_c" numeric, "time_epoch" numeric, "time_timestamp" timestamptz, "cpu_percentage" numeric, "load_1_min" numeric, "load_5_min" numeric, "load_15_min" numeric);


DROP TABLE IF EXISTS "v_stats";
CREATE VIEW "v_stats" AS SELECT stats.stats_json ->> 'hostname' AS hostname,
    (((stats.stats_json -> 'temp'::text))::character varying(10))::numeric AS temp_c,
    (((stats.stats_json -> 'time'::text))::character varying(10))::numeric AS time_epoch,
    to_timestamp(((((stats.stats_json -> 'time'::text))::character varying(10))::numeric)::double precision) AS time_timestamp,
    (((stats.stats_json -> 'cpu_percentage'::text))::character varying(10))::numeric AS cpu_percentage,
    ((((stats.stats_json -> 'load'::text) -> 0))::character varying(10))::numeric AS load_1_min,
    ((((stats.stats_json -> 'load'::text) -> 1))::character varying(10))::numeric AS load_5_min,
    ((((stats.stats_json -> 'load'::text) -> 2))::character varying(10))::numeric AS load_15_min
   FROM stats;

To test our database setup (and to see how many messages we have received) we can run this:

docker exec -it mon_db_1 psql -U postgres -d pimon -c 'select count(*) from stats'

It connects to the database container, runs the PostgreSQL client and selects a count of the statistics that we have received.

Now that our database is good, we can start the container that sends messages.

Send Stats to RabbitMQ

On each pi that we are monitoring we are going to run a Python script that generates some JSON messages and sends them to RabbitMQ, the messages will be a JSON string whose object looks like this:

{
   "hostname":"alpharpi4",
   "temp":37.972000,
   "time":1602154194391,
   "load":[
      0.220000,
      0.170000,
      0.110000
   ],
   "cpu_percentage":0.300000

Setup Data Generator

To configure this, clone the github repository and install the dependencies:

git clone https://github.com/mikebski/rpi-mon.git
python3 -m virtualenv rpi-mon/
cd rpi-mon
pip3 install -r requirements.txt

<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">Then, we can run;

<span style="color:#e6db74">`</span>python generate.py<span style="color:#e6db74">`</span> to start sending data - <span style="color:#66d9ef">if</span> it<span style="color:#e6db74">&#39;s working properly we should see a list of messages, one
</span><span style="color:#e6db74">per second, like this:
</span><span style="color:#e6db74">
</span><span style="color:#e6db74">&lt;div class=&#34;highlight&#34;&gt;&lt;pre style=&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;&gt;&lt;code class=&#34;language-json&#34; data-lang=&#34;json&#34;&gt; [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.433000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155312017&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.160000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;2.200000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.920000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155313021&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.160000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;0.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.433000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155314025&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.150000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;2.200000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.433000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155315029&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.150000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.000000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155316033&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.150000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155317037&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.150000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;0.800000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.433000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155318041&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.150000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.920000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155319045&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.130000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;2.200000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155320049&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.130000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.000000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.920000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155321053&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.130000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.300000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155322057&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.130000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155323061&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.130000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.100000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;0.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.459000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155324065&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.120000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;2.200000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;37.972000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155325069&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.120000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.000000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;38.946000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155326073&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.120000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1.500000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;
</span><span style="color:#e6db74"> [&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;x&lt;/span&gt;] &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;Sent&lt;/span&gt; &lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;{&lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;hostname&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#e6db74&#34;&gt;&amp;#34;alpharpi4&amp;#34;&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;temp&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;39.920000&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;time&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;1602155327076&lt;/span&gt;, &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;load&amp;#34;&lt;/span&gt;: [&lt;span style=&#34;color:#ae81ff&#34;&gt;0.120000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;, &lt;span style=&#34;color:#ae81ff&#34;&gt;0.090000&lt;/span&gt;], &lt;span style=&#34;color:#f92672&#34;&gt;&amp;#34;cpu_percentage&amp;#34;&lt;/span&gt;: &lt;span style=&#34;color:#ae81ff&#34;&gt;0.800000&lt;/span&gt;}&lt;span style=&#34;color:#960050;background-color:#1e0010&#34;&gt;&amp;#39;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt; 
</span><span style="color:#e6db74">
</span><span style="color:#e6db74">Note that it&#39;</span>s running in a terminal in the foreground, so <span style="color:#66d9ef">if</span> we logout of the pi it will stop sending data.

We will fix that in a later post.

For now, you can see that we are sending the hostname to the RabbitMQ server, so <span style="color:#66d9ef">if</span> you have multiple Pi
devices you can run this script on each to build your database.

<span style="color:#75715e">#### Validate data is going to Postgres</span>

Get your docker container id by <span style="color:#e6db74">`</span>docker ps<span style="color:#e6db74">`</span> and looking <span style="color:#66d9ef">for</span> <span style="color:#e6db74">`</span>ubuntu_db_1<span style="color:#e6db74">`</span>, assuming our hostname is ubuntu.

&lt;div class<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;highlight&#34;</span>&gt;&lt;pre style<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4&#34;</span>&gt;&lt;code class<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;language-text&#34;</span> data-lang<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;text&#34;</span>&gt;CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                                                                                                         NAMES
8d6b26905052        postgres:alpine                &amp;<span style="color:#75715e">#34;docker-entrypoint.s…&amp;#34;   About an hour ago   Up About an hour    0.0.0.0:5432-&amp;gt;5432/tcp                                                                                        ubuntu_db_1</span>
db1ffe9bfd6a        pymon                          &amp;<span style="color:#75715e">#34;python /opt/consume…&amp;#34;   About an hour ago   Up About an hour                                                                                                                  ubuntu_pymon_1</span>
73b0bff52797        metabasepi:latest              &amp;<span style="color:#75715e">#34;/app/bin/start&amp;#34;         About an hour ago   Up About an hour    0.0.0.0:3000-&amp;gt;3000/tcp                                                                                        ubuntu_metabase_1</span>
139ce66234e0        rabbitmq:3-management-alpine   &amp;<span style="color:#75715e">#34;docker-entrypoint.s…&amp;#34;   About an hour ago   Up About an hour    4369/tcp, 0.0.0.0:5671-5672-&amp;gt;5671-5672/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:15672-&amp;gt;15672/tcp   ubuntu_rmqmon_1</span>
c02b9f507c7f        adminer                        &amp;<span style="color:#75715e">#34;entrypoint.sh docke…&amp;#34;   About an hour ago   Up About an hour    0.0.0.0:8080-&amp;gt;8080/tcp                                                                                        ubuntu_adminer_1&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</span>

Then run:

<span style="color:#e6db74">`</span>docker exec -it 8d6b26905052 psql -U postgres -c <span style="color:#e6db74">&#39;select count(*) from stats&#39;</span> pimon<span style="color:#e6db74">`</span>

We should get a non-zero answer, and that means we are getting data into the database!

<span style="color:#75715e">### Visualization</span>

Now we<span style="color:#960050;background-color:#1e0010">&#39;</span>re ready <span style="color:#66d9ef">for</span> visualization - Metabase will be the tool. However, they have not 
provided a Docker image that will run on an ARM host. This is a pain in the ass, and
they <span style="color:#66d9ef">do</span> not seem interested in a pull request, but we can fix it. 

To vote and encourage them to provide one, comment/vote on the issue here: 
https://github.com/metabase/metabase/issues/13119</code></pre></div>

Raspberry Pi IOT Dashboard

Thursday Oct 1, 2020