====== Monitoring ======

===== Overview =====

There is a Raspberry Pi running:

  * home page to list services (http://sensing.shhm.uk)
  * Packaged Software
    * [[https://docs.influxdata.com/influxdb/v2/|InfluxDB]] to collect measurements (http://sensing.shhm.uk:8086)
    * [[https://prometheus.io/docs/introduction/overview/|Prometheus]] to collect measurements (http://sensing.shhm.uk:9090)
    * [[https://grafana.com/docs/grafana/latest/|Grafana]] to display dashboards of measurements (http://sensing.shhm.uk:3000)
  * Custom-made Software
    * MQTT listener which posts to InfluxDB ([[https://github.com/sheffieldhackspace/hackspace-sensing-mqtt-listener/|code]])
    * several ESP8266/etc sending data to MQTT ([[https://github.com/sheffieldhackspace/hackspace-sensing|code]])
    * Shelly Pro 3EM Prometheus exporter ([[https://github.com/sheffieldhackspace/shellypro3em-mqtt-prometheus-exporter|source]]) for power metrics

{{:public:2026-06-10_environment-monitoring.webp?400|}}

===== Code =====

  * generic ESP8266 C++ code for sensors to send measurements to MQTT
    * https://github.com/sheffieldhackspace/hackspace-sensing/
  * MQTT to InfluxDB listener (on Pi)
    * https://github.com/sheffieldhackspace/hackspace-sensing-mqtt-listener/

===== Sensors =====

==== MQTT Message Format ====

any environment monitoring MQTT message that looks like "SHHNoT/environment/#" will attempt to be parsed by the MQTT listener. It *should* have the format:

  SHHNoT/environment/room<X>/sensor<Y>/<measurement> <value>

e.g.,

  SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/temperature 23.414

==== Measurements ====

All sensors should broadcast the measurements:

  * a special "status" measurement which should be "on" when they connect to MQTT (see code above)
  * data they measure (e.g., "temperature"/"co2"/"humidity")
  * RSSI WiFi strength

an example payload for a single device could be:

  SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/temperature 22.218
  SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/co2  587
  SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/humidity 64.819
  SHHNoT/environment/room-d/sensor_ESP8266-SCD40-1/rssi  -60

==== Sensors in use ====

1. SCD40 (CO2/humidity/temp)

{{public:projects:sensor_scd40.jpg?300|}}

2. SCD40 (CO2/humidity/temp) and DS18B20 (temperature)

(the red wire goes from D5/GPIO14)

{{public:projects:sensor_scd40-and-ds18b20.jpg?400|}}

3. SDS011 (particulate matter – PM2.5 and PM10)

{{public:projects:sensor_sds011.jpg?300|}}

===== Brother exporter =====

setup:

<code>
# install
git clone https://github.com/d0ugal/brother-exporter
# build
go mod download
VERSION=${VERSION:-$(git describe --tags --always --dirty 2>/dev/null || echo "dev")}
COMMIT=${COMMIT:-$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")}
BUILD_DATE=${BUILD_DATE:-$(date -u +"%Y-%m-%dT%H:%M:%SZ")}
CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -ldflags="-s -w \
  -X github.com/d0ugal/brother-exporter/internal/version.Version=$VERSION \
  -X github.com/d0ugal/brother-exporter/internal/version.Commit=$COMMIT \
  -X github.com/d0ugal/brother-exporter/internal/version.BuildDate=$BUILD_DATE" \
  -o brother-exporter ./cmd/main.go

# config
cp config.example.yaml config.yaml
sed 's+host: "192.168.1.100"+host: "10.3.50.21"+'

# service file
cat << EOUFILE | sudo tee /etc/systemd/system/brother-exporter.service
[Unit]
Description=Brother Printer Exporter
Documentation=https://wiki.sheffieldhackspace.org.uk/members/projects/monitoring#brother_exporter
After=network-online.target

[Service]
Type=simple
User=prometheus
Group=prometheus
Restart=on-failure
RestartSec=10
WorkingDirectory=/usr/shhm/brother-exporter/
ExecStart=/usr/shhm/brother-exporter/brother-exporter

[Install]
WantedBy=multi-user.target
EOUFILE
sudo systemctl enable brother-exporter.service
sudo systemctl start brother-exporter.service
sudo systemctl status brother-exporter.service

# test
curl http://sensing.shhm.uk:8080/metrics
</code>

===== Mothership =====

To install InfluxDB and Grafana on the Raspberry Pi, run something like:

  # add users
  sudo useradd -r -s /bin/false influxdb
  sudo useradd -r -s /bin/false grafana
  sudo useradd -r -s /bin/false mqttlistener
  sudo useradd -r -s /bin/false prometheus
  
  # create folders
  mkdir -p /usr/shhm/influxdb
  mkdir -p /usr/shhm/grafana

  # install influxdb
  sudo apt update
  sudo apt upgrade
  curl https://repos.influxdata.com/influxdata-archive.key | gpg --dearmor | sudo tee /usr/share/keyrings/influxdb-archive-keyring.gpg >/dev/null
  echo "deb [signed-by=/usr/share/keyrings/influxdb-archive-keyring.gpg] https://repos.influxdata.com/debian stable main" | sudo tee /etc/apt/sources.list.d/influxdb.list
  sudo apt update
  sudo apt install influxdb2
  sudo systemctl status influxdb.service
  
  # install grafana
  sudo apt-get install -y apt-transport-https software-properties-common wget
  sudo mkdir -p /etc/apt/keyrings/
  wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
  echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
  sudo apt-get update
  sudo apt-get install grafana

  # install prometheus
  cd /usr/shhm/
  wget https://github.com/prometheus/prometheus/releases/download/v2.53.5/prometheus-2.53.5.linux-arm64.tar.gz
  tar -xzf prometheus-2.53.5.linux-arm64.tar.gz
  rm prometheus-2.53.5.linux-arm64.tar.gz
  mv prometheus-2.53.5.linux-arm64 prometheus
  sudo chown -R prometheus:prometheus prometheus
  cat << EOUFILE | sudo tee /etc/systemd/system/prometheus.service
  [Unit]
  Description=Prometheus Server
  Documentation=https://prometheus.io/docs/introduction/overview/
  After=network-online.target
  
  [Service]
  User=prometheus
  Group=prometheus
  Restart=on-failure
  RestartSec=10
  ExecStart=/usr/shhm/prometheus/prometheus \
    --config.file=/usr/shhm/prometheus/prometheus.yml \
    --storage.tsdb.path=/usr/shhm/prometheus/data \
    --storage.tsdb.retention.time=30d
  
  [Install]
  WantedBy=multi-user.target
  EOUFILE
  sudo systemctl enable prometheus.service
  sudo systemctl start prometheus.service
  sudo systemctl status prometheus.service
  
  # install node_exporter
  cd /usr/shhm/
  wget https://github.com/prometheus/node_exporter/releases/download/v1.9.1/node_exporter-1.9.1.linux-arm64.tar.gz
  tar -xzf node_exporter-1.9.1.linux-arm64.tar.gz
  rm node_exporter-1.9.1.linux-arm64.tar.gz
  mv node_exporter-1.9.1.linux-arm64 node_exporter
  sudo chown -R prometheus:prometheus node_exporter
  cat << EOUFILE | sudo tee /etc/systemd/system/node_exporter.service
  [Unit]
  Description=Node Exporter for Prometheus
  After=network.target
  
  [Service]
  Type=simple
  User=prometheus
  Group=users
  WorkingDirectory=/usr/shhm/node_exporter
  ExecStart=/usr/shhm/node_exporter/node_exporter
  Restart=on-failure
  RestartSec=10
  
  [Install]
  WantedBy=multi-user.target
  EOUFILE
  sudo systemctl enable node_exporter.service
  sudo systemctl start node_exporter.service
  sudo systemctl status node_exporter.service