# AGENTS.md

Project: Carousell new-listing monitor (Python stdlib, Docker, NocoDB, Telegram).

## What it does

`monitor.py` polls Carousell search URLs (sort_by=3 = recent), extracts listings
from the server-rendered `<script type="application/json">` Redux state
(`SearchListing.listingCards`), dedupes by `product_url` (param-less), archives to a
NocoDB base, and alerts Telegram `"<title>: N new listings"`. Listings whose
`seller_name` is in the `IgnoredSellers` table are archived but never alerted
(`skip_notify=true`). Listings whose **title** contains a keyword listed for their
watch in the `IgnoredKeywords` table (keyed by `search_url`, case-insensitive) are
also archived but never alerted. Runs 24/7 as a Docker container on DSM (network
`bridge_hoelee`, reaches NocoDB at `http://nocodb:10380`).

## Iron rules

- Secrets NEVER in code or compose — only env vars / `SECRETS.md` (private repo).
  Operational knobs (`enabled` / `notify` / `check_interval_minutes`) live in the
  NocoDB **Settings** table, adjustable from the UI without redeploy.
- Ignored sellers live in the NocoDB **IgnoredSellers** table (one `seller_name`
  per row). When a pending listing's `seller_name` matches an ignored seller, the
  monitor sets `skip_notify=true` + `notified=true` and does NOT send Telegram.
  The list is reloaded every notification cycle, so UI add/remove takes effect
  immediately.
- Ignored keywords live in the NocoDB **IgnoredKeywords** table: `watch` is a real
  **Link column → `Settings`** (pick the watch from a dropdown) and `search_url` is a
  **Lookup column** that auto-fills `Settings.url` from that link. A listing is
  silenced when its **title** contains any keyword for its watch, case-insensitive
  substring match. Per-watch, not global. Reloaded every cycle, so UI edits take
  effect immediately. Bootstrap creates both columns and upgrades a legacy URL-typed
  `search_url` to Link+Lookup automatically.
- Dedupe key is `product_url` (`https://www.carousell.com.my/p/<id>/`), not the raw
  listing id and never the query-string URL.
- First run per watch seeds the archive with **no** Telegram alert (`last_checked_at`
  null == unseeded).
- Docker HEALTHCHECK: container is unhealthy if the last tick is >10 min old or the
  last run had a failure (failed extract / 403 / rate-limit).
- Image thumbnail: `image` Attachment field stores the remote URL (NocoDB hotlinks
  it — media.karousell.com is GCS-backed, `Access-Control-Allow-Origin: *`). The raw
  URL is also kept in `image_url`.

## Verified facts (2026-09-02)

- Carousell search page: 1.6 MB HTML, state blob ~1.27 MB, ~49 `listingCards` per
  load. Card fields: `listingID`, `title`, `price` (e.g. "RM85"), `thumbnailURL`,
  `seller.username`, `aboveFold[time_created].timestampContent.seconds.low`,
  `belowFold` paragraphs where `paragraph[1]` = condition (Like new/Brand new/...).
- `https://www.carousell.com.my/p/<id>/` 301-redirects to the canonical slug URL.
- NocoDB instance: DSM `http://192.168.137.2:10380` (IP drifts; container DNS
  `nocodb:10380` on bridge_hoelee). Base `Carousell` = `poqw1zjw3hnsk37`.
  Workspace token in SECRETS.md. v2 API: records use column *titles* as JSON keys;
  Attachment field accepts a JSON string `[{"path","mimetype","title"}]` and keeps the
  remote URL (does not re-host).

## Build / deploy

Private build on DSM — NO registry (do not push to Docker Hub):

```bash
# on DSM (repo cloned to /volume1/docker/carousell-monitor)
cd /volume1/docker/carousell-monitor
sudo docker compose up -d --build
```