# COMPOSE-SETUP.md — Carousell Monitor Docker Stack, Explained Line-by-line anatomy of the compose stack, how it connects to the rest of the homelab, and the two ways to deploy it. Complementary to `DOCUMENTATION.md` (ops) and `AGENTS.md` (agent entry); this file is the *stack reference*. --- ## 1. Topology ``` ┌────────────────────────────────────────────────┐ │ DSM host (Synology) │ │ │ Internet ─ 443 ──► │ DSM nginx reverse proxy │ │ │ (not involved for this container — │ │ │ it makes outbound calls only) │ │ ▼ │ │ Docker network bridge_hoelee (external) │ │ │ │ │ ├── carousell-monitor (this stack) │ │ │ │ │ │ │ ├─► NocoDB http://nocodb:10380 │ │ │ │ (same bridge_hoelee network) │ │ │ ├─► Carousell www.carousell.com.my │ │ │ │ (public internet, GET search)│ │ │ └─► Telegram api.telegram.org │ │ │ (pinned IPv4 via extra_hosts)│ │ │ │ │ └── nocodb container (named "nocodb") │ └────────────────────────────────────────────────┘ ``` The monitor is an **outbound-only** worker: it has no inbound port, no web UI, and is never reached through the reverse proxy. All state lives in NocoDB (Listings / Settings / IgnoredSellers / IgnoredKeywords), all alerts go out via Telegram. --- ## 2. docker-compose.yml, block by block ### Top-level services ```yaml services: carousell-monitor: build: . image: carousell-monitor:latest container_name: carousell-monitor restart: unless-stopped ``` | Key | Meaning | |---|---| | `build: .` | Image is built **locally from this directory** (Dockerfile in repo) — nothing is pulled from a registry | | `image: carousell-monitor:latest` | Local tag for the built image; docker compose will rebuild + retag on `up --build` | | `container_name` | Fixed name = stable DNS name on the network, predictable for health checks and logs | | `restart: unless-stopped` | Survive daemon restarts and DSM reboots; stop it manually to keep it down | ### Naming vs. images The image is **private**: built on DSM, never pushed to Docker Hub or any registry. `image:` is just a local convenience tag (`docker images` shows it; the build context is the repo directory). ### extra_hosts — the Telegram IPv4 pin ```yaml extra_hosts: - "api.telegram.org:149.154.166.110" ``` **Why this exists (root cause, do not delete):** - The container runs inside `bridge_hoelee`, which has **no IPv6**. - Docker's embedded DNS at `127.0.0.11` may return an **IPv6 AAAA record** for `api.telegram.org`; with no IPv6 route, the connection hangs and Telegram sends fail silently. - Pinning the correct IPv4 (as of 2026-09) in `extra_hosts` short-circuits DNS and makes `api.telegram.org` resolve straight to the IPv4. - ⚠ If Telegram calls start timing out again, **re-verify the current IP**: `nslookup api.telegram.org` / `dig +short api.telegram.org` — Telegram rotates IPs. Update the pin, then redeploy. - Note: the **live** stack pins the **same** IP as the repo (synced 2026-09-13). Treat the repo value as canonical; re-verify on any timeout. ### environment ```yaml environment: NOCODB_URL: ${NOCODB_URL:-http://nocodb:10380} NOCODB_TOKEN: ${NOCODB_TOKEN} NOCODB_BASE_ID: ${NOCODB_BASE_ID:-poqw1zjw3hnsk37} TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN} TELEGRAM_CHAT_ID: ${TELEGRAM_CHAT_ID} TICK_SECONDS: ${TICK_SECONDS:-60} FETCH_GAP_SECONDS: ${FETCH_GAP_SECONDS:-1} HEALTH_STALE_SECONDS: ${HEALTH_STALE_SECONDS:-600} ERROR_ALERT_AFTER: ${ERROR_ALERT_AFTER:-3} FAILURE_RATIO_THRESHOLD: ${FAILURE_RATIO_THRESHOLD:-1.0} TZ: Asia/Kuala_Lumpur ``` | Var | Default | Meaning | |---|---|---| | `NOCODB_URL` | `http://nocodb:10380` | NocoDB REST endpoint. `nocodb` = the NocoDB container's name on `bridge_hoelee` (container DNS). Never use a public hostname here. | | `NOCODB_TOKEN` | *(required)* | Workspace-scoped NocoDB PAT (xref `SECRETS.md`). Never hard-code; comes from the stack env/.env. | | `NOCODB_BASE_ID` | `poqw1zjw3hnsk37` | Base "Carousell" — all four tables live under it. | | `TELEGRAM_BOT_TOKEN` | *(required)* | @carousellFoundBot token (xref `SECRETS.md`). | | `TELEGRAM_CHAT_ID` | *(required)* | `5648309582` — @MrFullStackDev. | | `TICK_SECONDS` | `60` | Scheduler granularity: heartbeat + watch-list reload interval. | | `FETCH_GAP_SECONDS` | `1` | Minimum pause (s) between watch URL fetches within one tick — prevents request bursts (default 1; set `0` to disable). | | `HEALTH_STALE_SECONDS` | `600` | Docker healthcheck tolerance: if last tick older than this → unhealthy. | | `ERROR_ALERT_AFTER` | `3` | Consecutive failed ticks before a Telegram failure alert fires (debounce). Sent once on entering the failure state and once on recovery. | | `FAILURE_RATIO_THRESHOLD` | `1.0` | Fraction of watches that must fail for the tick to be reported as failed (`1.0` = all of them; `0` disables the check). | | `TZ` | `Asia/Kuala_Lumpur` | Container clock (mostly cosmetic; timestamps are written in UTC deliberately for NocoDB). | `${VAR:-default}` syntax: compose substitutes the value from the environment / `.env` file, falling back to the literal default when unset. `${NOCODB_TOKEN}` with **no** default means it's mandatory — compose errors if missing. ### volumes ```yaml volumes: - carousell-data:/data ``` Named volume `carousell-data` mounted at `/data`. Inside the container that's: - `monitor.py` → `DATA_DIR` default → `/data/health.json` (written each tick, read by the healthcheck) - Nothing else is stored there — NocoDB holds all real state. Why a volume and not a bind mount: survives container recreation, no host-path permission issues on DSM ACLs, and it's private to the stack (not exposed to the host filesystem). ### networks ```yaml networks: - bridge_hoelee networks: bridge_hoelee: external: true ``` `external: true` = the network is **pre-existing** (created by the NocoDB stack, typically). Compose does not create it, just joins it. This is what lets the monitor reach `http://nocodb:10380` by container name instead of an IP that drifts (DSM IP history: 192.168.137.2 → 192.168.1.1 → …). --- ## 3. Dockerfile ```dockerfile FROM python:3.11-alpine WORKDIR /app COPY monitor.py healthcheck.py /app/ RUN mkdir -p /data VOLUME ["/data"] HEALTHCHECK --interval=60s --timeout=15s --start-period=120s --retries=3 \ CMD python /app/healthcheck.py CMD ["python", "-u", "/app/monitor.py"] ``` - `python:3.11-alpine` — tiny, stdlib-only code needs no pip deps → fast builds, small image. - `COPY` bakes the script into the image → a rebuild is how code ships (there is no bind-mount of the repo). - `HEALTHCHECK` runs `healthcheck.py` every 60s: exits 0 iff `/data/health.json` exists, is newer than `HEALTH_STALE_SECONDS`, and `ok == true`. - `python -u` — unbuffered stdout so `docker logs` shows ticks in real time. ### healthcheck.py logic ```python age = time.time() - int(h.get("last_run_epoch", 0)) if age <= STALE and h.get("ok") is True: sys.exit(0) # healthy sys.exit(1) # unhealthy ``` One failed tick (Carousell 403/429/parse error, NocoDB down) → `ok:false` → container turns **unhealthy** until a fully successful tick. That's the tripwire: Portainer shows it red, `docker inspect` reports it, and you can alert on it. --- ## 4. Deployment paths ### A. On DSM via Portainer stack (current production) Container is owned by **Portainer stack 240** (standalone). Deploy: 1. Edit code in the repo (`D:\dev\carousell-monitor`), commit, push to Gitea (`git.hoelee.com/hoelee/carousell-monitor`). 2. Sync the build context — Portainer's project dir holds a full manual copy (not a git clone): copy changed runtime files to `/volume1/docker/portainer/compose/240/` (`monitor.py`, `healthcheck.py`, `Dockerfile`, `docker-compose.yml`). 3. If `monitor.py` / `Dockerfile` changed, rebuild the image — a standalone stack PUT does **not** run `--build`: ```bash sudo /usr/local/bin/docker build -t carousell-monitor:latest \ /volume1/docker/portainer/compose/240 ``` 4. Update the stack via the Portainer API: `PUT /api/stacks/240?endpointId=2` with the repo `docker-compose.yml` as `stackFileContent` and the **current env array** (8 entries, incl. `FETCH_GAP_SECONDS=1`; see the `portainer-api` skill — ⚠ Portainer masks secret env values, so never echo the masked `***` strings back, and keep the real Telegram token in `SECRETS.md`). Portainer recreates the container. The old `/volume1/docker/carousell-monitor` dir is **legacy — do not `compose up` there anymore**; it only kept around for reference (its `.env` is masked and stale). ### B. Local dev (Windows, against LAN NocoDB) ```bash NOCODB_URL=http://192.168.1.1:10380 \ NOCODB_TOKEN=... TELEGRAM_BOT_TOKEN=... TELEGRAM_CHAT_ID=... \ python monitor.py ``` Runs the same loop outside Docker — useful for testing code changes before shipping (uses LAN IP instead of container DNS). ⚠ `192.168.1.1` drifts; check `:10380` on the current DSM IP first. --- ## 5. What to check when something breaks | Symptom | Check | |---|---| | Container `unhealthy` | `sudo docker logs carousell-monitor --tail 100` — read the `error` line | |"No application/json state" | Carousell rate-limited/blocked; raise `check_interval_minutes` in Settings | | No Telegram | `TELEGRAM_BOT_TOKEN`/`TELEGRAM_CHAT_ID` correct? `notify` box on the watch? Telegram IP pin in `extra_hosts` still valid? | | "NOCODB_TOKEN not set" | Stack env missing the token; fix in Portainer stack env, redeploy | | NocoDB unreachable | Is `nocodb` container on `bridge_hoelee`? `sudo docker network inspect bridge_hoelee` | | Timestamps 8h off | They are **UTC by design**; NocoDB display TZ → Asia/Kuala_Lumpur | --- ## 6. Files in the repo | File | Role | |---|---| | `docker-compose.yml` | Stack definition (this doc) | | `Dockerfile` | Image build + healthcheck | | `monitor.py` | The whole monitor (stdlib only) | | `healthcheck.py` | Docker HEALTHCHECK probe | | `.env.example` | Template for local runs (secrets go in `.env`, gitignored) | | `SECRETS.md` | Credential inventory (private repo) | | `DOCUMENTATION.md` | Deployment & operations manual | | `AGENTS.md` | AI-agent entry point | | `COMPOSE-SETUP.md` | This file | --- *Last reviewed: 2026-09-13 (NocoDB 2026.08.1, Docker Compose v2.20.1 on DSM).*