tg() passed its 'method' argument (e.g. 'sendMessage') straight into
urllib Request(method=...), producing the request line
sendMessage /bot<token>/sendMessage HTTP/1.1
'sendMessage' is not an HTTP verb, so Telegram's edge rejects it with 400
(bare nginx page, no JSON). Every sendMessage-based alert therefore failed
silently: the debounced failure alert, the recovery notice, and text-only
listing notifications. Listing alerts with an image survived only because
sendPhoto goes through the multipart helper which builds its own POST.
Combined with the 1000-row pagination bug, this is why no notifications
arrived while the container still reported healthy.
- tg(): always use HTTP POST for the request, keep the Telegram method in the path
- test: assert req.get_method() == 'POST' and that body is attached
(mutation-verified: restoring the bug fails the check)
- also restore the patched global urlopen between test sections
Verified live in the container: fixed tg -> 200 ok:true, message_id 493
delivered; unfixed tg -> HTTP 400.
Two problems found while verifying the pagination fix on the live stack:
1. fetch_listings() did state['SearchListing']['listingCards'] with no guard.
Carousell serves HTTP 200 with listingCards=null under soft rate limiting,
so the tick died with TypeError: 'NoneType' object is not iterable ->
ok:false -> container unhealthy, repeatedly.
2. run_tick() set ok = (no failures at all), so a single soft-blocked watch
out of 7 marked the whole monitor failed. That flaps on transient blocks
and (now that failures alert) would spam Telegram.
- fetch_listings: explicit null check -> clear retryable RuntimeError
- add FAILURE_RATIO_THRESHOLD (default 1.0 = all watches must fail); partial
failures are reported as '[partial n/N] ...' without failing the tick
- health gains failed_watches
- wire the knob into compose/.env.example/DOCUMENTATION/COMPOSE-SETUP
- 15 new checks: null vs empty cards, real card still parses, threshold edges
The previous commit renamed the record list to 'rows' but missed the loop in
load_ignored_keywords, leaving a bare 'j' -> 'tick error: name j is not
defined' every tick. The loop died, health.json went ok:false, the container
reported unhealthy, and alert_on_health fired against a live failure.
- fix the stale reference
- use real emoji instead of literal backslash escapes in alert text
- test: exercise the real loaders end-to-end against a fake NocoDB so a
missed rename is caught (mutation-verified: reintroducing the bug fails 3
checks)
send_pending_notifications() read Listings with ?limit=1000 and no paging.
Once the table passed 1000 rows the newest records (highest Id, at the tail)
fell outside page 1, so they were never sent and never marked notified ->
notifications silently dead while health.json stayed ok:true. Found live
2026-09-22: Id 1007-1041 (35 rows, ~29h of listings) never alerted.
- add nc_list_all(): offset-paged full-table read
- use it for listings pending, seen, watches, settings, both ignore lists
- add alert_on_health(): Telegram failure alert debounced over
ERROR_ALERT_AFTER consecutive failed ticks, plus a recovery notice
(container already reported unhealthy via healthcheck.py on ok:false)
- new env knob ERROR_ALERT_AFTER wired into compose/.env.example/docs
- test_pagination.py: 30 checks incl. the page-2 regression and alert edges
hoelee