hoelee
531da933ca
Replace the hand-copied search_url (URL type) with a proper relation: - watch: LinkToAnotherRecord (belongs-to, many keywords -> one Settings row), picked from a dropdown in the UI so a wrong URL can't silently disable a keyword - search_url: Lookup column that auto-fills Settings.url through the link _ensure_keyword_watch_link() builds both columns idempotently on bootstrap and upgrades a legacy URL-typed search_url to Link+Lookup. load_ignored_keywords() now normalises Lookup values (string or array).
carousell-monitor
Watches Carousell search pages (sorted by recent) for new listings, archives every listing to a NocoDB base (with image URL + thumbnail), and alerts Telegram.
How it works
monitor.pyruns in a Docker container on DSM, self-bootstrapping its NocoDB schema (Listings+Settings+IgnoredSellers+IgnoredKeywordstables) and looping forever.- Every
TICK_SECONDSit reads the watch list from the Settings table and polls each enabled watch's URL on its owncheck_interval_minutes. - Dedupe key =
product_url(param-less listing URL). First run per watch = seed archive only (no Telegram). After that, new listings are archived and alerted as"<title>: N new listings". - The container marks itself unhealthy (Docker healthcheck) if a tick fails to extract / gets rate-limited / crashes.
Schema
Listings — product_url (unique), title, price (numeric), condition
(SingleSelect), image_url, image (Attachment → thumbnail), seller_name,
seller_url, search_title, search_url, listed_at, first_seen_at,
notified, skip_notify.
Settings — title, url, enabled, notify, check_interval_minutes,
last_checked_at. Add/remove watches here from the NocoDB UI; no redeploy needed.
IgnoredSellers — seller_name. Add/remove sellers here to suppress Telegram
alerts for their listings (still archived, marked skip_notify=true).
IgnoredKeywords — watch (Link → Settings) + keyword. Per-watch title
blocklist: pick the watch from a dropdown, add one keyword per row. A keyword only
applies to listings from the linked watch; case-insensitive substring match against
the title. search_url is a Lookup column that auto-fills from the linked watch.
Still archived.
Run
# local (against LAN NocoDB)
NOCODB_URL=http://192.168.137.2:10380 \
NOCODB_TOKEN=... TELEGRAM_BOT_TOKEN=... TELEGRAM_CHAT_ID=... \
python monitor.py
# docker
docker build -t hoelee/carousell-monitor:latest .
docker compose up -d
Deploy (DSM via Portainer)
Private build — no registry. The compose at /volume1/docker/carousell-monitor
is cloned from Gitea and built on DSM (build: .), then deployed as a Portainer
stack with the secrets passed as stack environment variables.
# on DSM
cd /volume1/docker/carousell-monitor
sudo docker compose up -d --build
Files
monitor.py— main loop, schema bootstrap, fetch/parse, NocoDB IO, Telegram.healthcheck.py— Docker HEALTHCHECK probe (/data/health.json).Dockerfile,docker-compose.yml,.env.example.AGENTS.md— AI-agent entry.SECRETS.md— credentials (private repo).