Monitoring

Monitoring

Overview

There is a Raspberry Pi running:

home page to list services (http://sensing.shhm.uk)
Packaged Software
- InfluxDB to collect measurements (http://sensing.shhm.uk:8086)
- Prometheus to collect measurements (http://sensing.shhm.uk:9090)
- Grafana to display dashboards of measurements (http://sensing.shhm.uk:3000)
Custom-made Software
- MQTT listener which posts to InfluxDB (code)
- several ESP8266/etc sending data to MQTT (code)
- Shelly Pro 3EM Prometheus exporter (source) for power metrics

Code

generic ESP8266 C++ code for sensors to send measurements to MQTT
- https://github.com/sheffieldhackspace/hackspace-sensing/
MQTT to InfluxDB listener (on Pi)
- https://github.com/sheffieldhackspace/hackspace-sensing-mqtt-listener/

Sensors

MQTT Message Format

any environment monitoring MQTT message that looks like “SHHNoT/environment/#” will attempt to be parsed by the MQTT listener. It *should* have the format:

SHHNoT/environment/room<X>/sensor<Y>/<measurement> <value>

e.g.,

SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/temperature 23.414

Measurements

All sensors should broadcast the measurements:

a special “status” measurement which should be “on” when they connect to MQTT (see code above)
data they measure (e.g., “temperature”/“co2”/“humidity”)
RSSI WiFi strength

an example payload for a single device could be:

SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/temperature 22.218
SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/co2  587
SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/humidity 64.819
SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/rssi  -60

Sensors in use

1. SCD40 (CO2/humidity/temp)

2. SCD40 (CO2/humidity/temp) and DS18B20 (temperature)

(the red wire goes from D5/GPIO14)

3. SDS011 (particulate matter – PM2.5 and PM10)

Brother exporter

setup:

# install
git clone https://github.com/d0ugal/brother-exporter
# build
go mod download
VERSION=${VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo "dev")}
COMMIT=${COMMIT:-$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")}
BUILD_DATE=${BUILD_DATE:-$(date -u +"%Y-%m-%dT%H:%M:%SZ")}
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -ldflags="-s -w \
  -X github.com/d0ugal/brother-exporter/internal/version.Version=$VERSION \
  -X github.com/d0ugal/brother-exporter/internal/version.Commit=$COMMIT \
  -X github.com/d0ugal/brother-exporter/internal/version.BuildDate=$BUILD_DATE" \
  -o brother-exporter ./cmd/main.go

# config
cp config.example.yaml config.yaml
sed 's+host: "192.168.1.100"+host: "10.3.50.21"+'

# service file
cat << EOUFILE | sudo tee /etc/systemd/system/brother-exporter.service
[Unit]
Description=Brother Printer Exporter
Documentation=https://wiki.sheffieldhackspace.org.uk/members/projects/monitoring#brother_exporter
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
Restart=on-failure
RestartSec=10
WorkingDirectory=/usr/shhm/brother-exporter/
ExecStart=/usr/shhm/brother-exporter/brother-exporter

[Install]
WantedBy=multi-user.target
EOUFILE
sudo systemctl enable brother-exporter.service
sudo systemctl start brother-exporter.service
sudo systemctl status brother-exporter.service

# test
curl http://sensing.shhm.uk:8080/metrics

Mothership

To install InfluxDB and Grafana on the Raspberry Pi, run something like:

# add users
sudo useradd -r -s /bin/false influxdb
sudo useradd -r -s /bin/false grafana
sudo useradd -r -s /bin/false mqttlistener
sudo useradd -r -s /bin/false prometheus

# create folders
mkdir -p /usr/shhm/influxdb
mkdir -p /usr/shhm/grafana

# install influxdb
sudo apt update
sudo apt upgrade
curl https://repos.influxdata.com/influxdata-archive.key | gpg --dearmor | sudo tee /usr/share/keyrings/influxdb-archive-keyring.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/influxdb-archive-keyring.gpg] https://repos.influxdata.com/debian stable main" | sudo tee /etc/apt/sources.list.d/influxdb.list
sudo apt update
sudo apt install influxdb2
sudo systemctl status influxdb.service

# install grafana
sudo apt-get install -y apt-transport-https software-properties-common wget
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
sudo apt-get update
sudo apt-get install grafana

# install prometheus
cd /usr/shhm/
wget https://github.com/prometheus/prometheus/releases/download/v2.53.5/prometheus-2.53.5.linux-arm64.tar.gz
tar -xzf prometheus-2.53.5.linux-arm64.tar.gz
rm prometheus-2.53.5.linux-arm64.tar.gz
mv prometheus-2.53.5.linux-arm64 prometheus
sudo chown -R prometheus:prometheus prometheus
cat << EOUFILE | sudo tee /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Restart=on-failure
RestartSec=10
ExecStart=/usr/shhm/prometheus/prometheus \
  --config.file=/usr/shhm/prometheus/prometheus.yml \
  --storage.tsdb.path=/usr/shhm/prometheus/data \
  --storage.tsdb.retention.time=30d

[Install]
WantedBy=multi-user.target
EOUFILE
sudo systemctl enable prometheus.service
sudo systemctl start prometheus.service
sudo systemctl status prometheus.service

# install node_exporter
cd /usr/shhm/
wget https://github.com/prometheus/node_exporter/releases/download/v1.9.1/node_exporter-1.9.1.linux-arm64.tar.gz
tar -xzf node_exporter-1.9.1.linux-arm64.tar.gz
rm node_exporter-1.9.1.linux-arm64.tar.gz
mv node_exporter-1.9.1.linux-arm64 node_exporter
sudo chown -R prometheus:prometheus node_exporter
cat << EOUFILE | sudo tee /etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter for Prometheus
After=network.target

[Service]
Type=simple
User=prometheus
Group=users
WorkingDirectory=/usr/shhm/node_exporter
ExecStart=/usr/shhm/node_exporter/node_exporter
Restart=on-failure
RestartSec=10

[Install]
WantedBy=multi-user.target
EOUFILE
sudo systemctl enable node_exporter.service
sudo systemctl start node_exporter.service
sudo systemctl status node_exporter.service

Table of Contents