feat(W4): daily full reconciliation — user/role drift repair + ReconcileLogs audit

- w4-body.js: ONE-DIRECTIONAL sync (NocoDB status = SOT, never reverse);
  drift matrix: missing user/role (provision w/ username lookup to avoid
  duplicates), dangling refs (adopt/recreate), disabled<->status, canonical
  base_path '/' + permission 16384, scopes vs calculateEffectiveGrants,
  username drift detect+log_only; verify-after-repair on every change;
  password never pushed on repair (方案 C); dry-run via ?dryRun=true
- gen_w4.js: daily 03:00 cron + webhook alist-w4-reconcile; API-import-safe
  (UUID node ids, no pinData, webhookId)
- ReconcileLogs table mktsb7le6fqr2u1 (add_reconcilelogs_table.py, idempotent,
  v2 API); FK column nc_jitu___Customers_id for bt-link writes (v2 ignores
  link column itself)
- test_w4_reconcile.js: 34 assertions / 12 mock worlds, ALL PASSED
- live-verified against production: dry-run (3 drifts + 1 error found),
  real repair (3 repaired, verify all passed, 4 log rows), idempotent rerun
- NocoDB 2026.08.x pitfalls documented (v1 col endpoints gone, row delete
  via body [{Id}]); n8n API import/activation pitfalls (activeVersionId)
- docs: W4-DESIGN.md new; DATABASE_SCHEMA 8 tables; N8N_WORKFLOWS, SETUP,
  PROJECT_STATE updated
This commit is contained in:
2026-08-23 09:34:11 +08:00
parent 29ff66de88
commit 4693c9d684
12 changed files with 1257 additions and 27 deletions
+15 -2
View File
@@ -9,7 +9,8 @@
|---|---|---|---|
| **W1** Customer Provision & Status Lifecycle | `gU2ee9Eg5w1KzB5p` | NocoDB Customers webhook | provision + 整户 status lifecycle |
| **W2** CustomerProduct Sync | `D9OWAI7JUjKnRdOn` | NocoDB CustomerProducts webhook | 重算该客有效 grants 并应用 |
| **W3** Expiry Sync | schedule `*/5` | 定时 | 全量对账 + drift 修复 |
| **W3** Expiry Sync | `zlEfjj5eK5IfILtk` | schedule `*/5` | 定时 | 全量对账 + drift 修复(scopes 层) |
| **W4** Daily Full Reconciliation | `pl8j9lpg5gjrd0dB` | 每日 03:00 + webhook `alist-w4-reconcile` | 全量对账 + 用户/角色级 drift 修复 + ReconcileLogs |
三个 workflow 共享同一套算法 `calculateEffectiveGrants`(`n8n/shared/effective-grants.js`,
构建时经 `gen_w1/w2/w3.js` 内联进各 Code 节点)。
@@ -36,6 +37,17 @@
- 逐客调 `calculateEffectiveGrants` → diff → 应用
- Env 开关:`W3_DRY_RUN=true`(只打印计划)、`ALIST_AUTO_DISABLE_ON_EXPIRE=true`(scopes 变空时 disable 用户)
## W4 — Daily Full Reconciliation(每日全量对账)
- 触发:每日 03:00 cron + 手动 webhook(`/webhook/alist-w4-reconcile`,`?dryRun=true` 出计划)
- 单向同步:NocoDB status 是 SOT,W4 只修 AList,**永不反向改 NocoDB status**
- 覆盖 W3 不管的 drift:缺 user/role → 按 username 反查防重再 provision;悬空引用 → adopt/recreate;
非 active → disable + 清 scopes;base_path/permission 修回 canonical(`/` + 16384);
username drift → detect + log_only;修复后逐个 **verify 回读**,失败记 `verify_failed`
- 写 ReconcileLogs(表 `mktsb7le6fqr2u1`),只记 drift/repair/error 行;customer 链接用 FK 列
- 密码随 repair 不推送(方案 C)
- 详见 `docs/W4-DESIGN.md`
## 共用 env 变量
n8n Code 节点内用 `$env.VAR`(`process.env` 不可靠)。下表是本项目需要的 env:
@@ -43,7 +55,8 @@ n8n Code 节点内用 `$env.VAR`(`process.env` 不可靠)。下表是本项
| 变量 | 说明 |
|---|---|
| `NOCODB_URL` / `NOCODB_TOKEN` | NocoDB 地址 + `xc-token` |
| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` | 各表 model_id |
| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` / `..._RECONCILELOGS` | 各表 model_id(ReconcileLogs = `mktsb7le6fqr2u1`) |
| `NOCODB_FK_RECONCILELOG_CUSTOMER` | ReconcileLogs.customer 的 FK 列名 `nc_jitu___Customers_id`(W4 写链接必须用 FK 列) |
| `NOCODB_LINK_PROD_RESOURCES` | Product↔Resource M2M 列 id(`c4ko4x8kwi16swg`)|
| `ALIST_URL` | `http://alist-serve-resource:5244`(n8n 容器内)|
| `ALIST_ADMIN_USERNAME` / `ALIST_ADMIN_PASSWORD` | AList admin 登录 |