Fix Telegram notifications: multipart upload + richer message format

- 通知重构:归档与通知解耦(Listings.notified 列),tick 末尾统一发,1s 间隔,失败自动重试(30s tick)
- 每商品一条图文消息:title/price/condition/seller 名/product_url + 高清图
- 图片用 multipart 上传本地字节(原先传 URL 让 Telegram 下载,遇 carousell CDN 不稳定导致间歇 400)
- 图片 URL 去 _progressive_thumbnail 后缀(高清原图)
- condition 归一化:New→Brand new、Used→Used,加第 6 档;从所有 paragraph 找 condition(修位置漂移)
- listed_at 加 active_bump fallback(被顶置商品)
- compose extra_hosts 钉 api.telegram.org 到 IPv4 149.154.166.110(容器无 IPv6 时 DNS 只返 AAAA)
- bot 换 @carousellFoundBot,chat @MrFullStackDev
This commit is contained in:
2026-09-09 05:31:55 +08:00
parent d4bfcf92f5
commit d280e5e587
3 changed files with 215 additions and 31 deletions
+21 -11
View File
@@ -15,7 +15,9 @@ Docker container (carousell-monitor, DSM, network bridge_hoelee)
│ ├─ parse embedded JSON state → SearchListing.listingCards[]
│ ├─ dedupe by product_url (param-less listing URL)
│ ├─ INSERT new rows into NocoDB Listings table
│ └─ (after first seed) send Telegram "<title>: N new listings"
│ └─ archive w/ notified flag (first seed = silent; else pending-notify)
├─ end of each tick ─ send ALL pending (notified=false) listings, 1s apart
│ └─ each notice: photo + title/price/condition/seller/url
├─ writes /data/health.json each tick → Docker HEALTHCHECK
└─ reaches: NocoDB http://nocodb:10380 (container DNS, bridge_hoelee)
Carousell www.carousell.com.my (public internet)
@@ -51,7 +53,7 @@ Credentials are documented in `SECRETS.md` there.
| `NOCODB_URL` | `http://nocodb:10380` | container DNS on `bridge_hoelee`; LAN form `http://192.168.137.2:10380` |
| `NOCODB_TOKEN` | *(secret)* | workspace-scoped NocoDB PAT |
| `NOCODB_BASE_ID` | `poqw1zjw3hnsk37` | base "Carousell" |
| `TELEGRAM_BOT_TOKEN` | *(secret)* | @HoeleeAgentBot |
| `TELEGRAM_BOT_TOKEN` | *(secret)* | @carousellFoundBot |
| `TELEGRAM_CHAT_ID` | `5648309582` | alert destination |
| `TICK_SECONDS` | `60` | scheduler granularity |
| `HEALTH_STALE_SECONDS` | `600` | healthcheck staleness window |
@@ -75,15 +77,16 @@ deleting a table is safe; it is recreated on the next start).
| `product_url` | URL | **unique dedupe key** — `https://www.carousell.com.my/p/<id>/`, no query params |
| `title` | SingleLineText | listing title |
| `price` | Decimal | numeric price, "RM" stripped (e.g. `85.00`) — filterable/sortable |
| `condition` | SingleSelect | Brand new / Like new / Lightly used / Well used / Heavily used |
| `image_url` | URL | raw Carousell thumbnail URL (media.karousell.com) |
| `image` | Attachment | thumbnail — NocoDB hotlinks the URL; renders in grid view |
| `condition` | SingleSelect | Brand new / Like new / Lightly used / Well used / Heavily used / Used (归一化自 Carousell 的 New/Used 等写法) |
| `image_url` | URL | 高清图 URL(已去 `_progressive_thumbnail` 后缀) |
| `image` | Attachment | 高清图 — NocoDB hotlinks the URL; renders in grid view |
| `seller_name` | SingleLineText | seller username |
| `seller_url` | URL | `https://www.carousell.com.my/u/<username>/` |
| `search_title` | SingleLineText | which watch found it (denormalized) |
| `search_url` | URL | the watch's search URL |
| `listed_at` | DateTime (UTC) | listing's `time_created` on Carousell |
| `listed_at` | DateTime (UTC) | 上架时间(优先 `time_created`,被顶置商品 fallback `active_bump`) |
| `first_seen_at` | DateTime (UTC) | when the monitor first captured it |
| `notified` | Checkbox | false = 待发通知;发完/静默归档后置 true(防重复通知) |
### `Settings` (the watch list — you manage this)
@@ -106,10 +109,11 @@ deleting a table is safe; it is recreated on the next start).
- **Dedupe**: `product_url` is the identity. On startup the container loads every
existing `product_url` from NocoDB into an in-memory set; a listing is "new" only
if its URL is not in that set.
- **First run per watch** (`last_checked_at` is null): seeds the current ~49 listings
as an archive with **no Telegram alert**. This prevents a 98-message flood on setup.
- **Afterwards**: only genuinely-new listings are inserted **and** alerted, as
`"<title>: N new listings"` (one message per watch that had new items, no details).
- **First run per watch** (`last_checked_at` is null): seeds the current listings as an
archive with `notified=true` (silent, no Telegram). Prevents a flood on setup.
- **Afterwards**: new listings are inserted with `notified=false` (pending). At the end
of each tick the monitor sends every `notified=false` listing **one message each**
(photo + title/price/condition/seller/url), 1 second apart, then sets `notified=true`.
- **Failure handling**: a fetch/parse error on any watch marks that tick failed; the
container becomes **unhealthy** until the next fully-successful tick. `last_checked_at`
is only advanced on success, and a hard-failing watch is rate-limited to one attempt
@@ -188,7 +192,7 @@ Health file lives at `/data/health.json` inside the container:
| Container | `carousell-monitor` (network `bridge_hoelee`) |
| NocoDB base | `Carousell` = `poqw1zjw3hnsk37` (workspace `wal4hatt`) |
| Tables | `Listings` + `Settings` (bootstrap finds by title) |
| Telegram | `@HoeleeAgentBot` → chat `5648309582` |
| Telegram | `@carousellFoundBot` → chat `5648309582` (@MrFullStackDev) |
---
@@ -198,3 +202,9 @@ Health file lives at `/data/health.json` inside the container:
never committed. Credential inventory: `SECRETS.md` in the repo.
- The image is private (built on DSM, never pushed to Docker Hub).
- NocoDB token is workspace-scoped; regenerate in NocoDB if it leaks and update `.env`.
---
## 11. Changelog
- **2026-09-08** 通知重构:每商品一条图文消息(title/price/condition/seller/url),归档与通知解耦(`notified` 列 + tick 末尾统一发 + 1s 间隔)。图片改用高清 URL(去 `_progressive_thumbnail`)。condition 归一化(New→Brand new、Used→Used,加第 6 档)。listed_at 加 `active_bump` fallback。修复 Telegram IPv6/DNS 问题(compose `extra_hosts` 钉 IPv4)。bot 换 `@carousellFoundBot`。