- README: "Support this project" (GitHub Sponsors + Buy Me a Coffee, both verified live)
and "Need this set up - or something built?" (setup service, custom development,
maintenance) with the Hoelee Enterprise contact block
- .github/FUNDING.yml: github + buy_me_a_coffee, so GitHub renders the Sponsor button
- Gitea repo visibility flipped to public (API private:false; anonymous access verified 200)
- monitor.py: no default base id any more, and main() now reports whichever of
NOCODB_TOKEN / NOCODB_BASE_ID is missing (writing the same text to
/data/health.json) instead of dying inside bootstrap with a traceback
- docker-compose.yml: NOCODB_BASE_ID is a required variable now (production
passes it in .env, so runtime behaviour is unchanged there)
- README: the contributing note matches reality again
Verified: python test_pagination.py passes, python -m py_compile clean, and a
run without the two variables prints "NOCODB_TOKEN, NOCODB_BASE_ID not set",
writes ok:false to health.json and exits 2.
The README/NocoDB guide used a real base id copied out of the author's own
NocoDB URL. Examples now use an obviously fake id (the compose default and
monitor.py's fallback are unchanged: production always sets NOCODB_BASE_ID).
Verified on Portainer 2.45: an inline/web-editor stack with `build: .`
fails with "compose build operation failed: failed to solve: failed to read
dockerfile: open …/Dockerfile: no such file or directory".
- README + docs/QUICKSTART-PORTAINER: Route 1 = Repository deploy (Portainer
clones the repo, so build: . resolves), staged so NocoDB comes up first and
the base id/token are added afterwards; Route 2 = web editor with a
hand-built image and the build: line removed
- docker-compose.allinone.yml header: same correction
- validated with: docker compose -f docker-compose.allinone.yml config
- README rewritten around a copy-paste quick start (CLI and Portainer),
verified alert test, day-to-day NocoDB operations, troubleshooting table, FAQ
- new docker-compose.allinone.yml: NocoDB (pinned 2026.09.0, SQLite) + monitor
on a private network, with healthchecks and the Telegram IPv4 pin documented
- docs/: QUICKSTART-PORTAINER, TELEGRAM-SETUP, NOCODB-SETUP, ARCHITECTURE,
OPERATIONS, TROUBLESHOOTING (replace DOCUMENTATION.md + COMPOSE-SETUP.md)
- secrets: SECRETS.md is gitignored and untracked; tracked template is
SECRETS.example.md; real base id / chat id removed from .env.example
- LICENSE (MIT), .gitignore/.dockerignore tidied
- AGENTS.md: layout, iron rules, verification gates; host-specific deploy
details moved to the gitignored OPS-INTERNAL.md
tg() passed its 'method' argument (e.g. 'sendMessage') straight into
urllib Request(method=...), producing the request line
sendMessage /bot<token>/sendMessage HTTP/1.1
'sendMessage' is not an HTTP verb, so Telegram's edge rejects it with 400
(bare nginx page, no JSON). Every sendMessage-based alert therefore failed
silently: the debounced failure alert, the recovery notice, and text-only
listing notifications. Listing alerts with an image survived only because
sendPhoto goes through the multipart helper which builds its own POST.
Combined with the 1000-row pagination bug, this is why no notifications
arrived while the container still reported healthy.
- tg(): always use HTTP POST for the request, keep the Telegram method in the path
- test: assert req.get_method() == 'POST' and that body is attached
(mutation-verified: restoring the bug fails the check)
- also restore the patched global urlopen between test sections
Verified live in the container: fixed tg -> 200 ok:true, message_id 493
delivered; unfixed tg -> HTTP 400.
Two problems found while verifying the pagination fix on the live stack:
1. fetch_listings() did state['SearchListing']['listingCards'] with no guard.
Carousell serves HTTP 200 with listingCards=null under soft rate limiting,
so the tick died with TypeError: 'NoneType' object is not iterable ->
ok:false -> container unhealthy, repeatedly.
2. run_tick() set ok = (no failures at all), so a single soft-blocked watch
out of 7 marked the whole monitor failed. That flaps on transient blocks
and (now that failures alert) would spam Telegram.
- fetch_listings: explicit null check -> clear retryable RuntimeError
- add FAILURE_RATIO_THRESHOLD (default 1.0 = all watches must fail); partial
failures are reported as '[partial n/N] ...' without failing the tick
- health gains failed_watches
- wire the knob into compose/.env.example/DOCUMENTATION/COMPOSE-SETUP
- 15 new checks: null vs empty cards, real card still parses, threshold edges
The previous commit renamed the record list to 'rows' but missed the loop in
load_ignored_keywords, leaving a bare 'j' -> 'tick error: name j is not
defined' every tick. The loop died, health.json went ok:false, the container
reported unhealthy, and alert_on_health fired against a live failure.
- fix the stale reference
- use real emoji instead of literal backslash escapes in alert text
- test: exercise the real loaders end-to-end against a fake NocoDB so a
missed rename is caught (mutation-verified: reintroducing the bug fails 3
checks)
send_pending_notifications() read Listings with ?limit=1000 and no paging.
Once the table passed 1000 rows the newest records (highest Id, at the tail)
fell outside page 1, so they were never sent and never marked notified ->
notifications silently dead while health.json stayed ok:true. Found live
2026-09-22: Id 1007-1041 (35 rows, ~29h of listings) never alerted.
- add nc_list_all(): offset-paged full-table read
- use it for listings pending, seen, watches, settings, both ignore lists
- add alert_on_health(): Telegram failure alert debounced over
ERROR_ALERT_AFTER consecutive failed ticks, plus a recovery notice
(container already reported unhealthy via healthcheck.py on ok:false)
- new env knob ERROR_ALERT_AFTER wired into compose/.env.example/docs
- test_pagination.py: 30 checks incl. the page-2 regression and alert edges
Replace the legacy DSM-dir 'docker compose up -d --build' flow in AGENTS.md,
COMPOSE-SETUP.md, DOCUMENTATION.md, README.md with the current deploy:
sync build context to /volume1/docker/portainer/compose/240/, rebuild image
if code changed, then PUT /api/stacks/240 (compose + env array, never echo
masked *** values). Legacy dir marked as such; live tg IP note synced.
Same tick previously fired every due watch back-to-back (burst of N requests,
worst right after a restart when all watches are due at once). Now each watch
URL fetch is followed by a pause of FETCH_GAP_SECONDS (float, default 1, env
tunable; 0 disables) on both success and failure paths.
Also thread kw_fk_col into load_ignored_keywords() inside
send_pending_notifications() so the physical-FK lookup (445040a) actually takes
effect instead of silently falling back to the watch Link column.
Previously load_ignored_keywords() relied on the 'watch' Link column's
expanded shape ({'Id': N, ...}), which NocoDB does not guarantee. Now:
- _ensure_keyword_watch_link() discovers the physical FK column title
(e.g. nc_xxx___Settings_id, uidt=ForeignKey) from the table meta and
returns it from bootstrap()
- load_ignored_keywords() prefers reading the FK column (pure numeric Id
in the records API), falling back to the watch object only if absent
- kw_fk_col is threaded bootstrap -> main -> run_tick ->
send_pending_notifications -> load_ignored_keywords
Same one-Settings-fetch-per-cycle design, no N+1.
The search_url Lookup column is redundant: the watch Link -> Settings row
already defines the association. Remove handling for the column:
- _ensure_keyword_watch_link() no longer creates a Lookup; deletes any
legacy search_url (URL or Lookup type) on bootstrap
- load_ignored_keywords(kw_tid, settings_tid) resolves Settings.url via
the watch link, one Settings fetch per cycle (no N+1)
- send_pending_notifications passes settings_tid through
- Add COMPOSE-SETUP.md: detail documentation for the compose stack
(topology, compose block-by-block, Dockerfile, healthcheck, deploy paths,
troubleshooting); referenced from README + DOCUMENTATION.md
Replace the hand-copied search_url (URL type) with a proper relation:
- watch: LinkToAnotherRecord (belongs-to, many keywords -> one Settings row),
picked from a dropdown in the UI so a wrong URL can't silently disable a keyword
- search_url: Lookup column that auto-fills Settings.url through the link
_ensure_keyword_watch_link() builds both columns idempotently on bootstrap and
upgrades a legacy URL-typed search_url to Link+Lookup. load_ignored_keywords()
now normalises Lookup values (string or array).
Silence Telegram alerts for listings whose title contains a keyword
configured for that watch, matched case-insensitively as a substring.
- New IgnoredKeywords table: search_url (watch key) + keyword (one per row)
- load_ignored_keywords() returns {search_url: {lowercase keywords}}, reloaded
every notification cycle so UI edits apply immediately
- title_matches_keyword() does the case-insensitive substring check
- send_pending_notifications() now also pulls search_url on pending records;
skip_notify=true + notified=true on a hit (still archived, no Telegram)
- Docs updated: AGENTS.md, README.md, DOCUMENTATION.md
New Listings.skip_notify checkbox and IgnoredSellers table (seller_name).\nPending listings whose seller_name matches the ignore list are marked\nskip_notify=true + notified=true and never sent to Telegram. List reloads\nevery notification cycle so UI add/remove takes effect immediately.
hoelee