carousell-monitor/COMPOSE-SETUP.md

COMPOSE-SETUP.md — Carousell Monitor Docker Stack, Explained

Line-by-line anatomy of the compose stack, how it connects to the rest of the homelab, and the two ways to deploy it. Complementary to DOCUMENTATION.md (ops) and AGENTS.md (agent entry); this file is the stack reference.

---

1. Topology

                    ┌────────────────────────────────────────────────┐
                    │ DSM host (Synology)                            │
                    │                                                │
 Internet ─ 443 ──► │ DSM nginx reverse proxy                        │
                    │      │ (not involved for this container —      │
                    │      │  it makes outbound calls only)          │
                    │      ▼                                         │
                    │  Docker network  bridge_hoelee  (external)     │
                    │      │                                         │
                    │      ├── carousell-monitor  (this stack)       │
                    │      │      │                                   │
                    │      │      ├─► NocoDB    http://nocodb:10380  │
                    │      │      │     (same bridge_hoelee network) │
                    │      │      ├─► Carousell www.carousell.com.my │
                    │      │      │     (public internet, GET search)│
                    │      │      └─► Telegram api.telegram.org      │
                    │      │            (pinned IPv4 via extra_hosts)│
                    │      │                                          │
                    │      └── nocodb container (named "nocodb")      │
                    └────────────────────────────────────────────────┘

The monitor is an outbound-only worker: it has no inbound port, no web UI, and is never reached through the reverse proxy. All state lives in NocoDB (Listings / Settings / IgnoredSellers / IgnoredKeywords), all alerts go out via Telegram.

---

2. docker-compose.yml, block by block

Top-level services

services:
  carousell-monitor:
    build: .
    image: carousell-monitor:latest
    container_name: carousell-monitor
    restart: unless-stopped

Key	Meaning
build: .	Image is built locally from this directory (Dockerfile in repo) — nothing is pulled from a registry
image: carousell-monitor:latest	Local tag for the built image; docker compose will rebuild + retag on up --build
container_name	Fixed name = stable DNS name on the network, predictable for health checks and logs
restart: unless-stopped	Survive daemon restarts and DSM reboots; stop it manually to keep it down

Naming vs. images

The image is private: built on DSM, never pushed to Docker Hub or any registry. image: is just a local convenience tag (docker images shows it; the build context is the repo directory).

extra_hosts — the Telegram IPv4 pin

    extra_hosts:
      - "api.telegram.org:149.154.166.110"

Why this exists (root cause, do not delete):

• The container runs inside bridge_hoelee, which has no IPv6.
• Docker's embedded DNS at 127.0.0.11 may return an IPv6 AAAA record for api.telegram.org; with no IPv6 route, the connection hangs and Telegram sends fail silently.
• Pinning the correct IPv4 (as of 2026-09) in extra_hosts short-circuits DNS and makes api.telegram.org resolve straight to the IPv4.
• ⚠ If Telegram calls start timing out again, re-verify the current IP: nslookup api.telegram.org / dig +short api.telegram.org — Telegram rotates IPs. Update the pin, then redeploy.
• Note: the live stack pins the same IP as the repo (synced 2026-09-13). Treat the repo value as canonical; re-verify on any timeout.

environment

    environment:
      NOCODB_URL: ${NOCODB_URL:-http://nocodb:10380}
      NOCODB_TOKEN: ${NOCODB_TOKEN}
      NOCODB_BASE_ID: ${NOCODB_BASE_ID:-poqw1zjw3hnsk37}
      TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN}
      TELEGRAM_CHAT_ID: ${TELEGRAM_CHAT_ID}
      TICK_SECONDS: ${TICK_SECONDS:-60}
      FETCH_GAP_SECONDS: ${FETCH_GAP_SECONDS:-1}
      HEALTH_STALE_SECONDS: ${HEALTH_STALE_SECONDS:-600}
      ERROR_ALERT_AFTER: ${ERROR_ALERT_AFTER:-3}
      FAILURE_RATIO_THRESHOLD: ${FAILURE_RATIO_THRESHOLD:-1.0}
      TZ: Asia/Kuala_Lumpur

Var	Default	Meaning
NOCODB_URL	http://nocodb:10380	NocoDB REST endpoint. nocodb = the NocoDB container's name on bridge_hoelee (container DNS). Never use a public hostname here.
NOCODB_TOKEN	(required)	Workspace-scoped NocoDB PAT (xref SECRETS.md). Never hard-code; comes from the stack env/.env.
NOCODB_BASE_ID	poqw1zjw3hnsk37	Base "Carousell" — all four tables live under it.
TELEGRAM_BOT_TOKEN	(required)	@carousellFoundBot token (xref SECRETS.md).
TELEGRAM_CHAT_ID	(required)	5648309582 — @MrFullStackDev.
TICK_SECONDS	60	Scheduler granularity: heartbeat + watch-list reload interval.
FETCH_GAP_SECONDS	1	Minimum pause (s) between watch URL fetches within one tick — prevents request bursts (default 1; set 0 to disable).
HEALTH_STALE_SECONDS	600	Docker healthcheck tolerance: if last tick older than this → unhealthy.
ERROR_ALERT_AFTER	3	Consecutive failed ticks before a Telegram failure alert fires (debounce). Sent once on entering the failure state and once on recovery.
FAILURE_RATIO_THRESHOLD	1.0	Fraction of watches that must fail for the tick to be reported as failed (1.0 = all of them; 0 disables the check).
TZ	Asia/Kuala_Lumpur	Container clock (mostly cosmetic; timestamps are written in UTC deliberately for NocoDB).

${VAR:-default} syntax: compose substitutes the value from the environment / .env file, falling back to the literal default when unset. ${NOCODB_TOKEN} with no default means it's mandatory — compose errors if missing.

volumes

    volumes:
      - carousell-data:/data

Named volume carousell-data mounted at /data. Inside the container that's:

• monitor.py → DATA_DIR default → /data/health.json (written each tick, read by the healthcheck)
• Nothing else is stored there — NocoDB holds all real state.

Why a volume and not a bind mount: survives container recreation, no host-path permission issues on DSM ACLs, and it's private to the stack (not exposed to the host filesystem).

networks

    networks:
      - bridge_hoelee

networks:
  bridge_hoelee:
    external: true

external: true = the network is pre-existing (created by the NocoDB stack, typically). Compose does not create it, just joins it. This is what lets the monitor reach http://nocodb:10380 by container name instead of an IP that drifts (DSM IP history: 192.168.137.2 → 192.168.1.1 → …).

---

3. Dockerfile

FROM python:3.11-alpine

WORKDIR /app
COPY monitor.py healthcheck.py /app/
RUN mkdir -p /data

VOLUME ["/data"]

HEALTHCHECK --interval=60s --timeout=15s --start-period=120s --retries=3 \
  CMD python /app/healthcheck.py

CMD ["python", "-u", "/app/monitor.py"]

• python:3.11-alpine — tiny, stdlib-only code needs no pip deps → fast builds, small image.
• COPY bakes the script into the image → a rebuild is how code ships (there is no bind-mount of the repo).
• HEALTHCHECK runs healthcheck.py every 60s: exits 0 iff /data/health.json exists, is newer than HEALTH_STALE_SECONDS, and ok == true.
• python -u — unbuffered stdout so docker logs shows ticks in real time.

healthcheck.py logic

age = time.time() - int(h.get("last_run_epoch", 0))
if age <= STALE and h.get("ok") is True:
    sys.exit(0)     # healthy
sys.exit(1)         # unhealthy

One failed tick (Carousell 403/429/parse error, NocoDB down) → ok:false → container turns unhealthy until a fully successful tick. That's the tripwire: Portainer shows it red, docker inspect reports it, and you can alert on it.

---

4. Deployment paths

A. On DSM via Portainer stack (current production)

Container is owned by Portainer stack 240 (standalone). Deploy:

1. Edit code in the repo (D:\dev\carousell-monitor), commit, push to Gitea (git.hoelee.com/hoelee/carousell-monitor).
2. Sync the build context — Portainer's project dir holds a full manual copy (not a git clone): copy changed runtime files to /volume1/docker/portainer/compose/240/ (monitor.py, healthcheck.py, Dockerfile, docker-compose.yml).
3. If monitor.py / Dockerfile changed, rebuild the image — a standalone stack PUT does not run --build:

   sudo /usr/local/bin/docker build -t carousell-monitor:latest \
     /volume1/docker/portainer/compose/240
   

4. Update the stack via the Portainer API: PUT /api/stacks/240?endpointId=2 with the repo docker-compose.yml as stackFileContent and the current env array (8 entries, incl. FETCH_GAP_SECONDS=1; see the portainer-api skill — ⚠ Portainer masks secret env values, so never echo the masked *** strings back, and keep the real Telegram token in SECRETS.md). Portainer recreates the container.

The old /volume1/docker/carousell-monitor dir is legacy — do not compose up there anymore; it only kept around for reference (its .env is masked and stale).

B. Local dev (Windows, against LAN NocoDB)

NOCODB_URL=http://192.168.1.1:10380 \
NOCODB_TOKEN=... TELEGRAM_BOT_TOKEN=... TELEGRAM_CHAT_ID=... \
python monitor.py

Runs the same loop outside Docker — useful for testing code changes before shipping (uses LAN IP instead of container DNS). ⚠ 192.168.1.1 drifts; check :10380 on the current DSM IP first.

---

5. What to check when something breaks

Symptom	Check
Container unhealthy	sudo docker logs carousell-monitor --tail 100 — read the error line
"No application/json state"	Carousell rate-limited/blocked; raise check_interval_minutes in Settings
No Telegram	TELEGRAM_BOT_TOKEN/TELEGRAM_CHAT_ID correct? notify box on the watch? Telegram IP pin in extra_hosts still valid?
"NOCODB_TOKEN not set"	Stack env missing the token; fix in Portainer stack env, redeploy
NocoDB unreachable	Is nocodb container on bridge_hoelee? sudo docker network inspect bridge_hoelee
Timestamps 8h off	They are UTC by design; NocoDB display TZ → Asia/Kuala_Lumpur

---

6. Files in the repo

File	Role
docker-compose.yml	Stack definition (this doc)
Dockerfile	Image build + healthcheck
monitor.py	The whole monitor (stdlib only)
healthcheck.py	Docker HEALTHCHECK probe
.env.example	Template for local runs (secrets go in .env, gitignored)
SECRETS.md	Credential inventory (private repo)
DOCUMENTATION.md	Deployment & operations manual
AGENTS.md	AI-agent entry point
COMPOSE-SETUP.md	This file

---

Last reviewed: 2026-09-13 (NocoDB 2026.08.1, Docker Compose v2.20.1 on DSM).