Add EMQX broker setup guide and bring-up watcher

Target broker is EMQX 5.x at 192.168.1.31.

Probed its current state and documented it in docs/MQTT_SETUP.md: all
listeners up, retained messages working, management API protected (401),
but the MQTT data plane has no authenticator -- anonymous connects are
accepted, and so are arbitrary made-up credentials. Until an authenticator
exists, the broker account values in the local config header are decorative
and a successful connection proves nothing about them.

The guide covers creating the password-based authenticator, separate
soil-node and homeassistant accounts, per-account ACLs (including setting
no-match to Deny, without which the rules are cosmetic), pointing the HA
MQTT integration at EMQX, and a CONNACK/PubSubClient troubleshooting table.

tools/mqtt_watch.py watches the broker directly during bring-up. It answers
what the HA UI cannot: whether the retained discovery configs are actually
on the broker. Those are what HA rebuilds entities from after a restart, so
a node can look healthy today and disappear on the next HA restart if they
are missing. It reads its broker account secret from a file or environment
variable rather than argv, which is visible process-wide.

The broker IP is deliberately kept out of the committed template header,
since the Gitea repo is public.

Co-Authored-By: Claude <noreply@anthropic.com>

README.md

@@ -23,6 +23,11 @@ pio run -e mains -t upload -t monitor
 The node appears in Home Assistant under **Settings → Devices → MQTT** as
 *Soil Node XXXXXX* within a few seconds. No YAML.
 
+Broker and Home Assistant setup — including EMQX authentication and ACLs — is
+in [docs/MQTT_SETUP.md](docs/MQTT_SETUP.md). Read it before flashing: the
+broker at 192.168.1.31 currently accepts anonymous connections, so a node that
+connects successfully proves nothing about your credentials being right.
+
 Then calibrate the probes — see [docs/CALIBRATION.md](docs/CALIBRATION.md).
 The shipped calibration constants are placeholders and will read wrong.
 
@@ -70,6 +75,8 @@ src/net.*               Wi-Fi + MQTT session, LWT
 src/ha_discovery.*      HA auto-discovery payloads
 src/node_id.*           MAC-derived unique node identity
 docs/img/*.svg          Architecture, schematics and flow diagrams
+docs/MQTT_SETUP.md      EMQX auth/ACL + Home Assistant wiring
+tools/mqtt_watch.py     Broker-side bring-up watcher
 ```
 
 ## Diagrams

docs/MQTT_SETUP.md

@@ -0,0 +1,174 @@
+# EMQX + Home Assistant setup
+
+Broker for this deployment: **EMQX 5.x at 192.168.1.31**.
+
+## 0. Starting state (probed 2026-08-14)
+
+| Check | Result |
+|---|---|
+| Listeners 1883 / 8883 / 8083 / 8084 / 18083 | all open |
+| Anonymous MQTT connect | **accepted** |
+| Connect with a made-up username/password | **accepted** |
+| Retained message publish + replay to a new client | works |
+| Existing retained topics on the broker | none — broker is empty |
+| Management API (`/api/v5/clients` etc.) | protected, HTTP 401 |
+| Public `/api/v5/status` | 200 (normal, by design) |
+
+**The MQTT data plane has no authenticator configured.** Any client on the LAN
+can connect, subscribe to `#`, and publish anywhere — including publishing to
+`homeassistant/...` discovery topics, which means anything on the network can
+create or overwrite entities in Home Assistant. The dashboard itself is fine;
+it's only the broker side that is open.
+
+Until step 1 is done, the `MQTT_USER` / `MQTT_PASSWORD` values in `secrets.h`
+are decorative — EMQX accepts them without checking, so a working node proves
+nothing about whether the credentials are right.
+
+## 1. Enable authentication in EMQX
+
+Dashboard → **http://192.168.1.31:18083**
+
+1. **Access Control → Authentication → Create**
+2. Mechanism **Password-Based**, backend **Built-in Database**
+3. Configure:
+   - User ID Type: `username`
+   - Password Hash: `bcrypt` (salt rounds 10)
+4. Create.
+
+The moment an authenticator exists, EMQX stops accepting anonymous
+connections — so create the users in step 2 before flashing anything, and
+expect Home Assistant to drop off the broker until you give it credentials too.
+
+## 2. Create one account per client
+
+Under the authenticator you just made → **Users → Add**:
+
+| Username | Used by |
+|---|---|
+| `soil-node` | every ESP32 node (they share this account; the MQTT *client id* is what stays unique, derived from each MAC) |
+| `homeassistant` | the HA MQTT integration |
+
+Separate accounts matter here: it lets you write the ACLs in step 3, and it
+means rotating the node credential doesn't knock HA off the broker.
+
+Use long random passwords. Generate them where they will not land in a
+terminal transcript, e.g. write straight to a file:
+
+```bash
+openssl rand -base64 24 > ~/soil-node.pass    # then open the file to read it
+```
+
+## 3. Restrict what each account may do (recommended)
+
+**Access Control → Authorization → Built-in Database**, then add rules.
+
+For `soil-node` — it only ever publishes, and never subscribes:
+
+| Action | Topic | Permission |
+|---|---|---|
+| Publish | `soil/#` | Allow |
+| Publish | `homeassistant/sensor/#` | Allow |
+| Subscribe | `#` | Deny |
+
+For `homeassistant`:
+
+| Action | Topic | Permission |
+|---|---|---|
+| Subscribe | `homeassistant/#` | Allow |
+| Subscribe | `soil/#` | Allow |
+| Publish | `homeassistant/status` | Allow |
+
+Then set **no-match action to `Deny`** in the authorization settings.
+Leaving it at the default `Allow` makes the rules above cosmetic — anything
+not explicitly matched is still permitted.
+
+## 4. Fill in `include/secrets.h`
+
+Already pre-filled with the broker address. Replace the four `CHANGEME` values:
+
+```c
+#define WIFI_SSID     "..."
+#define WIFI_PASSWORD "..."
+#define MQTT_HOST     "192.168.1.31"
+#define MQTT_PORT     1883
+#define MQTT_USER     "soil-node"
+#define MQTT_PASSWORD "..."
+```
+
+This file is gitignored. The broker IP is deliberately **not** in
+`secrets.h.example`, because that template is committed and the Gitea repo is
+public.
+
+### Why port 1883 and not 8883
+
+TLS is available on 8883, but PubSubClient over `WiFiClientSecure` means
+shipping a CA bundle and paying a full handshake on every deep-sleep wake —
+meaningful battery cost on the remote nodes for a sensor reading that is not
+confidential, on a LAN. Plain 1883 with per-account ACLs is the sensible trade
+here. Revisit if these ever cross a network you don't control.
+
+## 5. Point Home Assistant at EMQX
+
+Settings → Devices & Services → **MQTT** → Configure.
+
+- Broker `192.168.1.31`, port `1883`
+- Username `homeassistant`, password from step 2
+- Enable discovery, discovery prefix `homeassistant` (must match
+  `HA_DISCOVERY_PREFIX` in `include/config.h`)
+
+If MQTT is already configured against a *different* broker, repoint it —
+nodes publishing to EMQX will never appear in an HA that is listening to
+Mosquitto somewhere else. This is the single most common reason a node looks
+healthy on the wire but produces no entities.
+
+## 6. Flash and verify
+
+```bash
+pio run -e mains -t upload -t monitor
+```
+
+Expected serial output:
+
+```
+=== Soil Node A1B2C3 (aa:bb:cc:a1:b2:c3) fw 1.0.0 ===
+profile: mains | publishing every 60s
+wifi: 192.168.1.x, rssi -55
+mqtt: connecting to 192.168.1.31:1883 as soil-node-a1b2c3
+discovery: publishing entities for soil-node-a1b2c3
+state: {"node_id":"soil-node-a1b2c3","moisture_1":...}
+```
+
+Watch the broker side independently:
+
+```bash
+python3 tools/mqtt_watch.py --host 192.168.1.31 --user soil-node
+```
+
+Then confirm in HA: Settings → Devices & Services → MQTT → *Soil Node XXXXXX*
+with 6 moisture entities plus the diagnostics.
+
+## Troubleshooting
+
+`mqtt: connect failed, state N` — N is PubSubClient's code:
+
+| N | Meaning | Likely cause |
+|---|---|---|
+| 4 | Bad credentials | `MQTT_USER`/`MQTT_PASSWORD` don't match the EMQX user |
+| 5 | Not authorized | Authenticated, but ACL denied the connection |
+| 3 | Server unavailable | Broker reachable but refusing |
+| -2 | Connect failed | TCP never established — wrong IP, firewall, VLAN |
+| -4 | Connection timeout | Broker accepted TCP but never sent CONNACK |
+
+**Node publishes but no entities in HA.** Almost always HA pointed at a
+different broker (step 5), or a discovery-prefix mismatch. Check that
+`homeassistant/sensor/soil-node-xxxxxx/moisture_1/config` exists on the broker
+with the watcher script.
+
+**Entities appear then go *unavailable*.** On the mains profile that is the
+availability topic plus last-will doing its job — the node really did drop.
+On the battery profile it means `expire_after` elapsed, so the node missed
+more than ~2.5 sleep intervals.
+
+**Entities show *unknown* rather than a number.** That is the firmware
+reporting a probe outside its calibrated window as `null` on purpose, not a
+transport problem. See `CALIBRATION.md`.

tools/mqtt_watch.py

@@ -0,0 +1,131 @@
+#!/usr/bin/env python3
+"""Watch a broker for soil-node traffic and report what Home Assistant will see.
+
+Answers the two questions that actually matter during bring-up, which the HA
+UI cannot answer because it only shows you the end result:
+
+  1. Is the node publishing at all, and to the topics we expect?
+  2. Are the retained discovery configs present on the broker? HA rebuilds its
+     entities from those on every restart, so if they are missing the node
+     works today and vanishes the next time HA is restarted.
+
+Requires paho-mqtt:  pip install paho-mqtt
+
+Password is read from a file or the MQTT_PASSWORD env var, never a CLI flag --
+argv is visible to every process on the box and lands in shell history.
+"""
+
+import argparse
+import json
+import os
+import sys
+import time
+
+try:
+    import paho.mqtt.client as mqtt
+except ImportError:
+    sys.exit("paho-mqtt not installed:  pip install paho-mqtt")
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--host", default="192.168.1.31")
+    ap.add_argument("--port", type=int, default=1883)
+    ap.add_argument("--user", default=None)
+    ap.add_argument(
+        "--password-file",
+        default=None,
+        help="file containing the password; or set MQTT_PASSWORD in the environment",
+    )
+    ap.add_argument("--seconds", type=int, default=90)
+    args = ap.parse_args()
+
+    password = None
+    if args.password_file:
+        with open(args.password_file) as fh:
+            password = fh.read().strip()
+    elif os.environ.get("MQTT_PASSWORD"):
+        password = os.environ["MQTT_PASSWORD"]
+
+    discovery = {}
+    states = {}
+
+    def on_connect(client, userdata, flags, rc, properties=None):
+        if rc != 0:
+            print("connect failed, rc=%s" % rc)
+            return
+        print("connected to %s:%d\n" % (args.host, args.port))
+        client.subscribe("homeassistant/sensor/#", qos=0)
+        client.subscribe("soil/#", qos=0)
+
+    def on_message(client, userdata, msg):
+        topic = msg.topic
+        if topic.startswith("homeassistant/"):
+            # Retained discovery config. An empty payload is a deletion.
+            if not msg.payload:
+                discovery.pop(topic, None)
+                return
+            if topic not in discovery:
+                discovery[topic] = True
+                try:
+                    cfg = json.loads(msg.payload)
+                    name = cfg.get("name", "?")
+                    unit = cfg.get("unit_of_meas", "")
+                except (ValueError, TypeError):
+                    name, unit = "<unparseable>", ""
+                flag = "retained" if msg.retain else "LIVE (not retained!)"
+                print("  discovery  %-58s %s %s [%s]" % (topic, name, unit, flag))
+        elif topic.endswith("/state"):
+            states[topic] = msg.payload
+            try:
+                data = json.loads(msg.payload)
+            except ValueError:
+                print("  state      %s  <non-JSON payload>" % topic)
+                return
+            moisture = {k: v for k, v in data.items() if k.startswith("moisture_")}
+            unknown = [k for k, v in moisture.items() if v is None]
+            print(
+                "  state      %s  node=%s rssi=%s"
+                % (topic, data.get("node_id"), data.get("wifi_signal"))
+            )
+            print("             moisture: %s" % moisture)
+            if unknown:
+                print(
+                    "             NOTE %d probe(s) reporting null (out of calibrated "
+                    "range -> 'unknown' in HA): %s" % (len(unknown), ", ".join(unknown))
+                )
+        elif topic.endswith("/availability"):
+            print("  avail      %s -> %s" % (topic, msg.payload.decode(errors="replace")))
+
+    client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id="soil-watch")
+    if args.user:
+        client.username_pw_set(args.user, password)
+    client.on_connect = on_connect
+    client.on_message = on_message
+
+    client.connect(args.host, args.port, 15)
+    client.loop_start()
+    try:
+        time.sleep(args.seconds)
+    except KeyboardInterrupt:
+        pass
+    client.loop_stop()
+    client.disconnect()
+
+    print("\n--- summary after %ds ---" % args.seconds)
+    # 6 moisture + 6 raw mV + wifi_signal + uptime = 14 on the mains profile;
+    # the battery profile adds battery_level and battery_voltage.
+    print("discovery configs seen : %d" % len(discovery))
+    print("                         expect 14 per mains node, 16 per battery node")
+    print("state topics seen      : %d" % len(states))
+    if not discovery:
+        print(
+            "\nNo discovery configs. Either no node has booted since the broker last\n"
+            "started, or the node cannot reach the broker. Retained configs should\n"
+            "replay to this script the instant it subscribes, even if the node is\n"
+            "currently asleep -- if they don't, HA will not have them either."
+        )
+
+
+if __name__ == "__main__":
+    main()