diff --git a/docs/DATABASE_SCHEMA.md b/docs/DATABASE_SCHEMA.md index 13d2bcc..1ac104f 100644 --- a/docs/DATABASE_SCHEMA.md +++ b/docs/DATABASE_SCHEMA.md @@ -1,8 +1,8 @@ -# NocoDB 数据结构（Phase 2b — 重建，7 表最终模型） +# NocoDB 数据结构（Phase 2b — 重建，8 表最终模型） > Source of Truth = NocoDB。AList 只做 enforcement，不存业务授权数据。 > 本 schema 按《Hermes 调整指导》最终业务模型重建（2026-08-22）。 -> 通过 NocoDB API 自动建表，无需手动点 UI。 +> 通过 NocoDB API 自动建表，无需手动点 UI。ReconcileLogs 于 2026-08-23 W4 新增。 - **Base**: `AlistAccess`（id `p3lzkresswyowqg`），独立新 base，与旧 `accessDB` 隔离。 - **MySQL source**: mysql-server 上的 `alistaccess` schema（由 NocoDB 自动管理）。 @@ -19,6 +19,7 @@ | AccessGrants | `mxdx3yhfcyej3ao` | | AccessLogs | `ml12zcyzlehgi21` | | Purchases | `mwp12z9lcohhsuk` | +| ReconcileLogs | `mktsb7le6fqr2u1`（FK 列 `nc_jitu___Customers_id` = `coa6ynwcdlune6p`） | > ⚠ ID 是本次重建生成的。若再次删库重建，ID 会变，需同步更新 n8n 节点 + 本文档。 @@ -35,6 +36,7 @@ | AccessLogs | `customer` | bt | Customers | | | Purchases | `customer` | bt | Customers | | | Purchases | `product` | bt | Products | | +| ReconcileLogs | `customer` | bt | Customers | ⚠ 写入时用 FK 列 `nc_jitu___Customers_id`（v2 忽略 link 列本身） | > M2M 读写方式（已实测）：`GET/POST /api/v2/tables/{products}/links/c4ko4x8kwi16swg/records/{productId}`， > POST body `{"Id": }`。反向用 Resources 的 `cfxi44n9ixkimqc`。 @@ -138,7 +140,21 @@ | bytes | Number | | timestamp | DateTime | -### 7. Purchases（纯记录，不参与权限） +### 7. ReconcileLogs（W4 对账审计，n8n W4 写入） +| 字段 | 类型 | 说明 | +|---|---|---| +| Id | number (PK) | | +| run_id | SingleLineText | 本次对账 run（`recon-`） | +| customer | link bt → Customers | FK 列 `nc_jitu___Customers_id`（v2 写链接必须写 FK 列） | +| drift_type | SingleSelect | user_missing / role_missing / dangling_user_ref / dangling_role_ref / user_disabled_drift / scopes_drift / base_path_drift / permission_drift / role_binding_drift / username_drift / verify_failed / reconcile_error | +| details | LongText | 前后值 JSON | +| action | SingleLineText | 做了什么（provisioned / adopted / fixed / rewritten / disabled / log_only / verify …） | +| result | SingleSelect | repaired / log_only / failed / skipped | +| timestamp | DateTime | | + +> 只记 drift / repair / error 行，健康客户不产生行。W4 每日全量或 webhook 触发时写入。 + +### 8. Purchases（纯记录，不参与权限） | 字段 | 类型 | |---|---| | Id | number (PK) | diff --git a/docs/N8N_WORKFLOWS.md b/docs/N8N_WORKFLOWS.md index 800e656..9c6f1ca 100644 --- a/docs/N8N_WORKFLOWS.md +++ b/docs/N8N_WORKFLOWS.md @@ -9,7 +9,8 @@ |---|---|---|---| | **W1** Customer Provision & Status Lifecycle | `gU2ee9Eg5w1KzB5p` | NocoDB Customers webhook | provision + 整户 status lifecycle | | **W2** CustomerProduct Sync | `D9OWAI7JUjKnRdOn` | NocoDB CustomerProducts webhook | 重算该客有效 grants 并应用 | -| **W3** Expiry Sync | schedule `*/5` | 定时 | 全量对账 + drift 修复 | +| **W3** Expiry Sync | `zlEfjj5eK5IfILtk` | schedule `*/5` | 定时 | 全量对账 + drift 修复（scopes 层） | +| **W4** Daily Full Reconciliation | `pl8j9lpg5gjrd0dB` | 每日 03:00 + webhook `alist-w4-reconcile` | 全量对账 + 用户/角色级 drift 修复 + ReconcileLogs | 三个 workflow 共享同一套算法 `calculateEffectiveGrants`（`n8n/shared/effective-grants.js`， 构建时经 `gen_w1/w2/w3.js` 内联进各 Code 节点）。 @@ -36,6 +37,17 @@ - 逐客调 `calculateEffectiveGrants` → diff → 应用 - Env 开关：`W3_DRY_RUN=true`（只打印计划）、`ALIST_AUTO_DISABLE_ON_EXPIRE=true`（scopes 变空时 disable 用户） +## W4 — Daily Full Reconciliation（每日全量对账） + +- 触发：每日 03:00 cron + 手动 webhook（`/webhook/alist-w4-reconcile`，`?dryRun=true` 出计划） +- 单向同步：NocoDB status 是 SOT，W4 只修 AList，**永不反向改 NocoDB status** +- 覆盖 W3 不管的 drift：缺 user/role → 按 username 反查防重再 provision；悬空引用 → adopt/recreate； + 非 active → disable + 清 scopes；base_path/permission 修回 canonical（`/` + 16384）； + username drift → detect + log_only；修复后逐个 **verify 回读**，失败记 `verify_failed` +- 写 ReconcileLogs（表 `mktsb7le6fqr2u1`），只记 drift/repair/error 行；customer 链接用 FK 列 +- 密码随 repair 不推送（方案 C） +- 详见 `docs/W4-DESIGN.md` + ## 共用 env 变量 n8n Code 节点内用 `$env.VAR`（`process.env` 不可靠）。下表是本项目需要的 env： @@ -43,7 +55,8 @@ n8n Code 节点内用 `$env.VAR`（`process.env` 不可靠）。下表是本项 | 变量 | 说明 | |---|---| | `NOCODB_URL` / `NOCODB_TOKEN` | NocoDB 地址 + `xc-token` | -| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` | 各表 model_id | +| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` / `..._RECONCILELOGS` | 各表 model_id（ReconcileLogs = `mktsb7le6fqr2u1`） | +| `NOCODB_FK_RECONCILELOG_CUSTOMER` | ReconcileLogs.customer 的 FK 列名 `nc_jitu___Customers_id`（W4 写链接必须用 FK 列） | | `NOCODB_LINK_PROD_RESOURCES` | Product↔Resource M2M 列 id（`c4ko4x8kwi16swg`）| | `ALIST_URL` | `http://alist-serve-resource:5244`（n8n 容器内）| | `ALIST_ADMIN_USERNAME` / `ALIST_ADMIN_PASSWORD` | AList admin 登录 | diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md index a4f78f1..673625c 100644 --- a/docs/PROJECT_STATE.md +++ b/docs/PROJECT_STATE.md @@ -78,17 +78,33 @@ Base: **`AlistAccess`**（`p3lzkresswyowqg`），与旧 `accessDB` 完全隔离 - **W1/W2/W3 零复制算法**：三处都调同一个 `calculateEffectiveGrants` - **测试**（全过）：`test_effective_grants.js`（共享算法）、`test_w3_refactored.js`（W3 回归）、`test_w1_lifecycle.js`（W1 lifecycle，10 场景全覆盖：pending/active/suspended/expired/deleted 全转换 + echo 短路 + sync_required 强制重算） +### 6. W4 每日全量对账（Step 4）— **本地 + 直连生产实测通过** ✅ +- 文件：`n8n/W4-daily-reconcile.json`（`n8n/w4-body.js` + 共享库 → `gen_w4.js`）；已导入 n8n（id `pl8j9lpg5gjrd0dB`，active，webhook `alist-w4-reconcile` 已注册） +- 触发：每日 03:00 cron + 手动 webhook（`?dryRun=true` 出计划） +- 单向同步（用户定稿）：NocoDB status = SOT，W4 只修 AList，永不反向；密码不随 repair 推送（方案 C） +- Drift 矩阵 + ReconcileLogs 表（`mktsb7le6fqr2u1`，只记 drift/repair/error 行）→ 详见 `docs/W4-DESIGN.md` +- **实测记录（2026-08-23 直连生产）**： + 1. dry-run：5 客户 → 3 drift（TEST-Alice: dangling_role_ref + base_path_drift + role_binding_drift）+ 1 error（customer 7 缺 username），3 skip，零写入 ✅ + 2. 真实跑：3 repaired + 4 行 ReconcileLogs（FK 正确），verify-after-repair 全过（无 verify_failed）✅ + 3. 复跑：0 drift / 0 repair / 4 skip → 幂等 ✅ +- 测试：`n8n/test_w4_reconcile.js`（34 项全过，mock AList+NocoDB 12 场景） +- ⚠ **n8n env 还未加**（`NOCODB_TABLE_RECONCILELOGS` / `NOCODB_FK_RECONCILELOG_CUSTOMER`）：容器 env 是 Portainer stack 144 管的，需 Portainer 访问后补并 redeploy（W4 代码有 env 缺失守卫：日志写入静默跳过并在 summary 注明，不会炸） + ## 🔴 下一步（按序） -> 功能已全部实现并通过本地测试（用户已确认「已测试功能正常」）。剩余事项如下： +> W4 代码与实测已完成。剩余为部署收尾 + 一个 n8n 实例问题： -1. **reimport W1 到 n8n 并做真实验收**（W2/W3 也重新生成过，可一并覆盖）： - - 改 `Customers.status` 一次 → 只产生**一次**真正 W1 execution（`active_provision` 或 `deactivate`），其后 echo 全是 `skip_writeback_echo` - - 验证 W1 execution 的 `desiredPaths` 非空、AList role `permission_scopes` 落对 -2. **(Step 4) W4 Full Reconciliation**：NocoDB desired vs AList actual 全量 drift 修复 + reconciliation 日志（W3 已具备基础，W4 补每日全量 + 日志） -3. **(待设计) AccessGrants 物化**：当前 W2/W3 不写 AccessGrants 表，不作为 source-of-truth。W4 时统一设计 reconciliation/materialization 策略 -4. 密码回写问题（客户自改 AList 密码 → NocoDB plaintext 失真）：当前不强制回写（用户已选方案 C），必要时再议 -5. 本地 git 已留档并推 Gitea（`git.hoelee.com/hoelee/alist-serve-resource`，凭据已脱敏） +1. **给 n8n 容器补 env 并 redeploy（Portainer stack 144）**： + `NOCODB_TABLE_RECONCILELOGS=mktsb7le6fqr2u1` + `NOCODB_FK_RECONCILELOG_CUSTOMER=nc_jitu___Customers_id` + （容器 env 改后必须重建，不是 restart）。之后 W4 每日 03:00 才会真正落 ReconcileLogs。 +2. **⚠ n8n 执行管道疑似停摆（2026-08-23 ~23:00Z 起，待查）**：webhook 全返回 200 + "Workflow was started" 但不产生任何 execution（W1/W3 也一样，连 failed 都不落库）； + W3 的 5 分钟 schedule 也停了（最后一次执行 23:00:51Z）。疑似与 API key 激活权限错误 + （"attempted to activate a workflow without permissions"）或 01:21Z 的一次外部容器 recreate + 有关。**需要用户确认：是否在 Portainer 动过 n8n？** 建议查 n8n 日志/executions 后恢复。 +3. AccessGrants 物化：维持现状（手工人为授权入口，n8n 不写）——W4 已定案不物化。 +4. 密码回写问题（客户自改 AList 密码 → NocoDB plaintext 失真）：保持方案 C（不强制回写）。 +5. 本地 git 已留档并推 Gitea（`git.hoelee.com/hoelee/alist-serve-resource`，凭据已脱敏）。 > ⚠️ **Status 语义已定稿（不再待决策）**：5 个值 `pending/active/suspended/expired/deleted`， > account-level 整户开关，**不进** permission calc。详见 `docs/ARCHITECTURE.md`、`docs/PERMISSIONS.md`、`docs/DATABASE_SCHEMA.md`。 @@ -122,21 +138,39 @@ Base: **`AlistAccess`**（`p3lzkresswyowqg`），与旧 `accessDB` 完全隔离 - **writeBack 值比对守卫**：PATCH 前先 GET 当前值，只写真会变的字段；值没变 → 不发 PATCH → 不触发 webhook（物理层面掐断 loop） - extract-id 节点必须透传 `previousRow`/`newRow`（经 Load Env 转发）到 lifecycle Code 节点 +- **W4 新坑（2026-08-23）**： + - n8n API 导入（`POST /api/v1/workflows`）校验严格：**节点 id 必须 UUID、顶层不能有 `pinData`**， + 否则 400 `request/body must NOT have additional properties`（UI 导入不受此限） + - webhook 节点必须有 `webhookId`，否则生产 URL 不注册（404 "The requested webhook ... is not registered"） + - n8n 1.123 的 API 激活 `POST /workflows/{id}/activate` 返回 `active:true` 但运行时可能被权限拒绝 + （日志 "User attempted to activate a workflow without permissions"，webhook/cron 不注册）。 + **激活的真实标记在 DB：`workflow_entity.activeVersionId`**（不是 `active` 列）；boot 时只加载 + `active + activeVersionId` 的工作流。手动救法：`UPDATE workflow_entity SET "activeVersionId"="versionId" WHERE id='...'` + 重启容器。 + - **NocoDB 2026.08.x：v1 column GET/PATCH 端点已删**（列 id 从 `GET /api/v2/meta/tables/{id}` 的 columns 拿； + v2 的 columns create/PATCH 响应返回表对象而非列） + - **NocoDB v2 写 bt 链接：写 link 列本身无效**（标量/数组/对象全被忽略）→ 必须写 FK 列 + `nc_jitu___Customers_id`（实测确认），env `NOCODB_FK_RECONCILELOG_CUSTOMER` + - **NocoDB v2 删行**：`DELETE /api/v2/tables/{t}/records` + body `[{"Id": N}, ...]`； + `/records/{id}` 和 `where=` 参数都不行 + - **n8n 执行可能全局停摆**（2026-08-23 23:00Z 起）：webhook 200 但不落 execution——见「下一步」#2， + 恢复前 n8n 侧一切「执行」都不可信（本机直连跑 W4 代码是替代验收路径，见 W4-DESIGN.md） + ## 📁 项目结构 ``` alist-serve-resource/ ├── docker-compose.yml / .env.example / .gitignore ├── n8n/ -│ W1/W2/W3-*.json (generated) · w1/w2/w3-body.js (runtime logic) +│ W1/W2/W3/W4-*.json (generated) · w1/w2/w3/w4-body.js (runtime logic) │ shared/effective-grants.js (single source of truth) -│ gen_w1/w2/w3.js + inline_shared.js (build-time inlining) -│ test_effective_grants.js · test_w1_lifecycle.js · test_w3_refactored.js -│ SETUP.md · W2-DESIGN.md · W3-DESIGN.md -├── scripts/ create_nocodb_schema.py · healthcheck.sh +│ gen_w1/w2/w3/w4.js + inline_shared.js (build-time inlining) +│ test_effective_grants.js · test_w1_lifecycle.js · test_w3_refactored.js · test_w4_reconcile.js +│ SETUP.md · W2-DESIGN.md · W3-DESIGN.md · W4-DESIGN.md +├── scripts/ create_nocodb_schema.py · add_reconcilelogs_table.py · live_dry_run_w4.js · healthcheck.sh └── docs/ API_INTEGRATION.md · ARCHITECTURE.md · PERMISSIONS.md · ADMIN_GUIDE.md · DATABASE_SCHEMA.md · N8N_WORKFLOWS.md · INSTALL.md · CONFIGURATION.md · SECURITY.md · BACKUP_RESTORE.md · UPGRADE.md · TROUBLESHOOTING.md · PROJECT_STATE.md ``` ## 待确认的设计点 -1. **AccessGrants 物化**（已定：本次不物化，不作为 source-of-truth）—— W4 时统一设计 reconciliation/materialization 策略，需避免 AccessGrant→n8n loop -2. 密码：客户可自改 AList 密码 → NocoDB plaintext 会失真；当前不强制回写（用户已选方案 C） -3. 对账/到期只靠 n8n 定时扫描（AList 无 expiry 概念）—— 已确认 \ No newline at end of file +1. **AccessGrants 物化**（已定案：**不物化**，保持手工人为授权入口源 b；n8n 永不写它）——W4 已确认 +2. 密码：客户可自改 AList 密码 → NocoDB plaintext 会失真；保持方案 C（W4 repair 也不推密码，仅 provision 新建时用） +3. 对账/到期只靠 n8n 定时扫描（AList 无 expiry 概念）—— 已确认 +4. **n8n 执行管道停摆**（见「下一步」#2）——需要用户确认 Portainer 是否动过 + 恢复方式 \ No newline at end of file diff --git a/docs/W4-DESIGN.md b/docs/W4-DESIGN.md new file mode 100644 index 0000000..abfad95 --- /dev/null +++ b/docs/W4-DESIGN.md @@ -0,0 +1,78 @@ +# W4 — Daily Full Reconciliation / 全量对账 + drift 修复 + 对账日志 + +> 2026-08-23 定稿并实测（本地单元测试 + 直连生产 dry-run + 真实修复跑 + 幂等复跑）。 +> 与 W3 的差异：W3 每 5 分钟只对账 **scopes**（事件对账的补网）； +> W4 每日全量对账 **用户/角色/绑定的完整状态**，修复 W3 不管的 drift，并落 ReconcileLogs 审计。 + +## 触发 + +- **Schedule Trigger**：每日 03:00（cron `0 3 * * *`）。 +- **Webhook**（`POST /webhook/alist-w4-reconcile`）手动触发；`?dryRun=true` 或 env `W4_DRY_RUN=true` = 只出计划，零写入。 + +## 单向同步铁律（用户定稿） + +- **NocoDB `Customers.status` 是 business source of truth；AList enabled/disabled 只是 enforcement state。** +- W4 **永不**反向修改 NocoDB status。管理员在 AList 手工 disable 用户 → W4 按 NocoDB status 修回；绝不反过来。 +- 密码**不随 repair 推送**（方案 C：NocoDB plaintext 可能旧，推送会把客户自改的 AList 密码覆盖掉）。仅 provision 新建用户时用 NocoDB 密码。 + +## Drift 矩阵（自动修复） + +| 场景 | 期望状态 | 处理 | +|---|---|---| +| **active 客户缺 user/role** | 存在 + enabled + canonical 配置 | 自动 provision：**先按 username 查 AList**（避免 NocoDB 丢 id 时建重）→ 没有才 create（role → user 顺序，空 role 会被 AList 拒） | +| **alist_user_id / alist_role_id 悬空**（指向已删记录） | — | username / role name 反查 → 找到即 **adopt**（回写 bookkeeping id，不新建）→ 找不到才 recreate → 重新 sync | +| **非 active（pending/suspended/expired/deleted）** | user disabled + scopes 空 | 有 user → disable + 清空 scopes；**pending 从未 provision → 保持无 AList user，不自动建**（纯 NocoDB 标签） | +| **user.disabled 漂移** | 与 status 一致 | 按 status 单向修（active→enable，非 active→disable） | +| **base_path / permission 漂移** | `/` + 16384 | 强制修回（= W1 已验证的 canonical 配置，不重新发明） | +| **role binding 漂移**（user 没绑 role） | user.role = [专属 role id] | rebind | +| **scopes 漂移 vs desired** | = calculateEffectiveGrants | 整体重写 role `permission_scopes`（与 W3 同算法） | +| **username drift**（NocoDB 名 ≠ AList 名） | — | **detect + log_only**（不假设 rename API，不做 destructive migration） | +| **客户行缺 username** | — | reconcile_error 记日志 + summary.errors | + +## ReconcileLogs 表（新建，2026-08-23） + +- 表 id `mktsb7le6fqr2u1`；列：`run_id` / `customer`(bt→Customers) / `drift_type` / `details` / `action` / `result` / `timestamp` +- **只记 drift / repair / error 行**——健康客户不产生任何行（不为每天的全量跑刷日志）。 +- drift_type 选项：user_missing / role_missing / dangling_user_ref / dangling_role_ref / + user_disabled_drift / scopes_drift / base_path_drift / permission_drift / + role_binding_drift / username_drift / verify_failed / reconcile_error +- result 选项：repaired / log_only / failed / skipped +- ⚠ **写 customer 链接必须用 FK 列 `nc_jitu___Customers_id`**（v2 records API 写 link 列本身：标量/数组/对象全被忽略，实测确认），env `NOCODB_FK_RECONCILELOG_CUSTOMER` 可覆盖。 + +## 修复后必须 verify + +每个 customer 修复后 **GET 回读 AList user/role 实测状态**，与期望比对； +不通过 → 记 `verify_failed` 行（result=failed，不计入 repaired）。 +dry-run 不 verify（没写入可验）。 + +## 输出 summary + +``` +{ runId, dryRun, at, customersScanned, driftsDetected, repaired, skipped, + errors: [{customerId, username, error}], details: [{customerId, ..., drifts[]}], + logRowsWritten, plannedLogRows (dry-run), logWriteSkipped (env 缺失时) } +``` + +## 与 W1/W2/W3 的交互 + +- 写回 Customers 只有 `alist_user_id` / `alist_role_id`（adopt 时）→ W1 的 old/new 比对视为 echo → 短路，无 loop。 +- 不写 `sync_status` / `sync_required` / `last_sync`。 +- W4 不碰 AccessGrants（保持原定位：手工人为授权入口，源 b）。 +- AccessGrants/CustomerProducts/Customers 无 webhook 指向 W4，写 ReconcileLogs 无 loop。 + +## 本地实跑（绕过 n8n，直连生产） + +`NOCODB_TOKEN=... ALIST_ADMIN_PASSWORD=... node scripts/live_dry_run_w4.js`（dry-run） +与 `scripts/run_real_w4.js`（本机脚本版 run_real_w4.js 在 hermes scripts/，凭据走 env）。 +2026-08-23 实测记录： +1. dry-run：5 客户，3 drift（TEST-Alice: dangling_role_ref + base_path_drift + role_binding_drift），1 error（Id 7 缺 username），3 skip，零写入 ✅ +2. 真实跑：3 repaired + 4 行 ReconcileLogs（FK 正确），verify 全过 ✅ +3. 复跑：0 drift / 0 repair / 4 skip / 仅 Id-7 error 照记 → 幂等 ✅ + +## 坑（已沉淀，勿再踩） + +- n8n API 导入校验严格：**节点 id 必须 UUID、顶层不能有 `pinData`**，否则 400 "additional properties"。 +- n8n webhook 节点必须带 `webhookId`，否则生产 URL 永不注册（404 "not registered"）。 +- n8n 1.123 API 激活返回 `active:true` 但运行时激活可能被权限拒绝（日志 "attempted to activate a workflow without permissions"）→ 激活状态在 DB 是 `workflow_entity.activeVersionId`（不是 active 列），boot 时加载 active + activeVersionId 的工作流注册 webhook/cron。 +- NocoDB 2026.08.x：v1 column GET/PATCH 端点已删除；列 id 从 `GET /api/v2/meta/tables/{id}` 拿；v2 的列 create/PATCH 响应返回的是**表对象**不是列。 +- NocoDB v2 行删除：`DELETE /records` + body `[{Id: N}, ...]`（路径 /records/{id} 和 where 参数都不行）。 \ No newline at end of file diff --git a/n8n/SETUP.md b/n8n/SETUP.md index bebf41c..b5fb1d0 100644 --- a/n8n/SETUP.md +++ b/n8n/SETUP.md @@ -43,6 +43,8 @@ NOCODB_TABLE_PRODUCTS=molyb3pxv81l75g NOCODB_TABLE_RESOURCES=m6cm7tkxzd8m18c NOCODB_TABLE_CUSTOMERPRODUCTS=mvyqbv8ao7u6bct NOCODB_TABLE_ACCESSGRANTS=mxdx3yhfcyej3ao +NOCODB_TABLE_RECONCILELOGS=mktsb7le6fqr2u1 +NOCODB_FK_RECONCILELOG_CUSTOMER=nc_jitu___Customers_id NOCODB_LINK_PROD_RESOURCES=c4ko4x8kwi16swg ALIST_URL=http://alist-serve-resource:5244 ALIST_ADMIN_USERNAME=hoelee diff --git a/n8n/W4-daily-reconcile.json b/n8n/W4-daily-reconcile.json new file mode 100644 index 0000000..5ef7a47 --- /dev/null +++ b/n8n/W4-daily-reconcile.json @@ -0,0 +1,81 @@ +{ + "name": "W4 - Daily Full Reconciliation (AList Drift Repair)", + "nodes": [ + { + "id": "ea636132-2235-4935-a575-84ee51197c10", + "parameters": { + "rule": { + "interval": [ + { + "field": "cronExpression", + "expression": "0 3 * * *" + } + ] + } + }, + "name": "Daily 03:00", + "type": "n8n-nodes-base.scheduleTrigger", + "typeVersion": 1.2, + "position": [ + -420, + 0 + ] + }, + { + "id": "23153525-50ae-48e5-b28a-27973c23c7f1", + "parameters": { + "httpMethod": "POST", + "path": "alist-w4-reconcile", + "options": {} + }, + "name": "Manual / Webhook", + "type": "n8n-nodes-base.webhook", + "typeVersion": 2, + "webhookId": "alist-w4-reconcile", + "position": [ + -420, + 240 + ] + }, + { + "id": "987e71ca-e1ce-4bd0-af88-dbc5d211c6b0", + "parameters": { + "jsCode": "// ============================================================================\n// W4 runtime body — Daily full reconciliation + drift repair + reconcile log.\n// Rebuilt by `node n8n/gen_w4.js` (single source of truth = shared/effective-grants.js).\n//\n// ONE-DIRECTIONAL sync: NocoDB `Customers.status` is the BUSINESS source of\n// truth; AList user enabled/disabled is only ENFORCEMENT state. W4 NEVER\n// writes Customers.status. A manual `disabled` flip in the AList admin UI is\n// repaired back to match NocoDB — never the other way round.\n//\n// Drift matrix (auto-repair):\n// status active -> user must exist (provision if missing; lookup by\n// username FIRST to avoid duplicate users when the\n// NocoDB id was lost), be ENABLED, carry the desired\n// role scopes (calculateEffectiveGrants), base_path\n// '/', permission 16384.\n// status non-active -> if a user exists: must be DISABLED with empty role\n// scopes (soft state kept). pending/never-provisioned\n// customers stay WITHOUT an AList user (no auto-create).\n// dangling refs -> alist_user_id / alist_role_id point at deleted AList\n// records: lookup by username / role name, adopt if\n// found, recreate if not, then re-sync.\n// username drift -> NocoDB username differs from AList username: DETECT\n// + log only (no safe official rename API assumed; no\n// destructive migration).\n//\n// Every repair is VERIFIED by re-reading AList state afterwards; a failed\n// verify is logged as verify_failed. ReconcileLogs receives ONLY\n// drift/repair/error rows — healthy customers produce nothing.\n//\n// Triggers: daily cron (03:00) + manual webhook. Dry run: webhook\n// `?dryRun=true` or env W4_DRY_RUN=true -> plan only, ZERO writes.\n// ============================================================================\n// ============================================================================\n// SHARED: calculateEffectiveGrants — single source of truth for \"what scopes\n// should customer X have right now\", reused by W1 (active resume), W2\n// (CustomerProduct sync) and W3 (expiry sweep).\n//\n// This file is INLINED into each workflow's Code node at build time (see\n// gen_w1.js / gen_w2.js / gen_w3.js). It must therefore be self-contained:\n// no require/import, no process.exit, no top-level return outside functions.\n//\n// Config is read via $env (the ONLY reliable channel in n8n Code nodes for\n// custom env vars — process.env is unreliable there). The caller passes a\n// `ctx` with { $env, helpers } so this file is also unit-testable outside n8n.\n//\n// ============================================================================\n\n// --- helpers (depend on ctx, so they are functions not top-level consts) ---\n\nfunction ncUrl(ctx, path) {\n return ctx.$env.NOCODB_URL + path;\n}\n\nasync function ncGet(ctx, path) {\n return await ctx.helpers.httpRequest({\n method: 'GET',\n url: ncUrl(ctx, path),\n headers: { 'xc-token': ctx.$env.NOCODB_TOKEN },\n json: true,\n });\n}\n\nasync function alistReq(ctx, method, path, body, token) {\n // NOTE: AList expects the RAW token in Authorization (no \"Bearer \" prefix).\n return await ctx.helpers.httpRequest({\n method,\n url: ctx.$env.ALIST_URL + path,\n headers: token ? { Authorization: token } : {},\n body,\n json: true,\n });\n}\n\n// NocoDB LinkToAnotherRecord values are objects/arrays/numbers depending on\n// whether the API was asked to expand them. Normalise to a single id or null.\nfunction linkId(v) {\n if (v == null) return null;\n if (typeof v === 'object' && !Array.isArray(v)) return v.Id ?? v.id ?? null;\n if (Array.isArray(v)) return v[0]?.Id ?? v[0]?.id ?? null;\n return v;\n}\n\n// --- core algorithm --------------------------------------------------------\n//\n// desired scopes for a customer = union of:\n// (a) every RESOURCE reachable from an ACTIVE, NOT-YET-EXPIRED CustomerProduct\n// (b) every RESOURCE granted directly via an ACTIVE, NOT-YET-EXPIRED AccessGrant\n//\n// For (a) the SAME resource coming from MULTIPLE products keeps the MAX\n// expires_at, exactly matching the \"two products share a resource\" semantics\n// the user tested. Expiry boundary is `<= today` (expiring today is expired).\n\nasync function calculateEffectiveGrants(ctx, customerId) {\n const T_CUST = ctx.$env.NOCODB_TABLE_CUSTOMERS;\n const T_CUSTPROD = ctx.$env.NOCODB_TABLE_CUSTOMERPRODUCTS;\n const T_PROD = ctx.$env.NOCODB_TABLE_PRODUCTS;\n const T_RES = ctx.$env.NOCODB_TABLE_RESOURCES;\n const T_GRANT = ctx.$env.NOCODB_TABLE_ACCESSGRANTS;\n const LNK_PROD_RES = ctx.$env.NOCODB_LINK_PROD_RESOURCES; // c4ko4x8kwi16swg\n const today = new Date().toISOString().slice(0, 10); // 'YYYY-MM-DD'\n\n // index all Resources by Id -> alist_path (cheap, one query)\n const allRes = (await ncGet(ctx, `/api/v2/tables/${T_RES}/records?limit=10000`)).list || [];\n const resPath = {};\n allRes.forEach(r => { resPath[r.Id] = r.alist_path; });\n\n const desired = {}; // path -> { expires, permission }\n\n // (a) CustomerProducts -> Resources (via product M2M link), MAX expiry on\n // the same path.\n // NOTE: `(customer,eq,N)` matches the link field's DISPLAY NAME and always\n // returns 0 rows. Must use the `customer.Id` sub-field syntax (or the\n // foreign-key column `nc_jitu___Customers_id`).\n const cps = (await ncGet(ctx, `/api/v2/tables/${T_CUSTPROD}/records?where=(customer.Id,eq,${customerId})&limit=1000`)).list || [];\n for (const c of cps) {\n if (c.status && c.status !== 'active') continue;\n if (c.expires_at && String(c.expires_at) <= today) continue; // expired (incl. today)\n const pid = linkId(c.product);\n if (!pid) continue;\n const res = (await ncGet(ctx, `/api/v2/tables/${T_PROD}/links/${LNK_PROD_RES}/records/${pid}`)).list || [];\n const exp = c.expires_at; // 'YYYY-MM-DD' (may be null = no expiry)\n for (const r of res) {\n const p = resPath[r.Id];\n if (!p) continue;\n const cur = desired[p];\n if (!cur || !cur.expires || (exp && exp > cur.expires)) {\n desired[p] = { expires: exp || null, permission: cur?.permission ?? 0 };\n }\n }\n }\n\n // (b) AccessGrants -> Resources (manual / materialised direct grants)\n const grants = (await ncGet(ctx, `/api/v2/tables/${T_GRANT}/records?where=(customer.Id,eq,${customerId})&limit=1000`)).list || [];\n for (const g of grants) {\n if (g.status && g.status !== 'active') continue;\n if (g.expires_at && String(g.expires_at) <= today) continue;\n const p = resPath[linkId(g.resource)];\n if (!p) continue;\n const exp = g.expires_at;\n const cur = desired[p];\n if (!cur || !cur.expires || (exp && exp > cur.expires)) {\n desired[p] = { expires: exp || null, permission: g.permission === 'readwrite' ? 1 : 0 };\n }\n }\n\n const desiredPaths = Object.keys(desired).sort();\n return {\n customerId,\n desired, // { path: {expires, permission} }\n desiredPaths, // sorted array of paths\n newScopes: desiredPaths.map(p => ({ path: p, permission: desired[p].permission })),\n };\n}\n\n// --- export shim: kept only when unit-tested, stripped when inlined -------\n// ============================================================================\n\nconst ctx = { $env, helpers: this.helpers };\n\nconst query = ($json && $json.query) || {};\nconst DRY_RUN = query.dryRun === 'true' || query.dryRun === true ||\n String($env.W4_DRY_RUN || '').toLowerCase() === 'true';\nconst T_CUST = $env.NOCODB_TABLE_CUSTOMERS;\nconst T_LOG = $env.NOCODB_TABLE_RECONCILELOGS;\n// NocoDB v2 (2026.08.x): bt links are NOT created by writing the link column\n// (scalar/array/object all ignored). Write the FK column directly:\n// `nc_jitu___Customers_id` (verified live 2026-08-23).\nconst FK_CUST = $env.NOCODB_FK_RECONCILELOG_CUSTOMER || 'nc_jitu___Customers_id';\nconst runId = 'recon-' + new Date().toISOString().replace(/[:.]/g, '-');\n\n// canonical AList user configuration — must stay identical to W1's (verified\n// working: base_path '/' + absolute role scopes + permission 16384)\nconst BASE_PATH = '/';\nconst PERMISSION = 16384;\n\nconst logRows = []; // ReconcileLogs payload — drift/repair/error rows ONLY\n\nconst summary = {\n runId, dryRun: DRY_RUN,\n customersScanned: 0, driftsDetected: 0, repaired: 0, skipped: 0,\n errors: [], details: [],\n};\n\nfunction pushLog(cid, username, driftType, details, action, result) {\n const row = {\n run_id: runId,\n drift_type: driftType,\n details: details == null ? '' : String(details).slice(0, 2000),\n action,\n result,\n timestamp: new Date().toISOString(),\n };\n if (cid != null) row[FK_CUST] = cid; // FK column (link writes are ignored)\n logRows.push(row);\n}\n\n// ---- AList helpers (read) ----\nasync function listUsers(token) {\n const r = await alistReq(ctx, 'GET', '/api/admin/user/list?page=1&per_page=1000', null, token);\n return (r && (r.data?.content || r.data?.list)) || [];\n}\nasync function listRoles(token) {\n const r = await alistReq(ctx, 'GET', '/api/admin/role/list?page=1&per_page=1000', null, token);\n return (r && (r.data?.content || r.data?.list)) || [];\n}\nasync function getUser(token, id) {\n const r = await alistReq(ctx, 'GET', `/api/admin/user/get?id=${id}`, null, token);\n return (r && r.data) ? r.data : null;\n}\nasync function getRole(token, id) {\n const r = await alistReq(ctx, 'GET', `/api/admin/role/get?id=${id}`, null, token);\n return (r && r.data) ? r.data : null;\n}\n\n// ---- AList helpers (write) — all no-op under DRY_RUN ----\nasync function createRole(token, name, description) {\n if (DRY_RUN) return { id: null, name, permission_scopes: [] };\n await alistReq(ctx, 'POST', '/api/admin/role/create', {\n name, description, default: false, permission_scopes: [],\n }, token);\n const roles = await listRoles(token);\n const r = roles.find(x => x.name === name);\n if (!r) throw new Error('Created role not found: ' + name);\n return r;\n}\nasync function createUser(token, body) {\n if (DRY_RUN) return { id: null, username: body.username };\n await alistReq(ctx, 'POST', '/api/admin/user/create', body, token);\n const users = await listUsers(token);\n const u = users.find(x => x.username === body.username);\n if (!u) throw new Error('Created user not found: ' + body.username);\n return u;\n}\nasync function updateUser(token, fields) {\n if (DRY_RUN) return;\n await alistReq(ctx, 'POST', '/api/admin/user/update', fields, token);\n}\nasync function updateRoleScopes(token, roleId, name, description, scopes) {\n if (DRY_RUN) return;\n await alistReq(ctx, 'POST', '/api/admin/role/update', {\n id: roleId, name, description, permission_scopes: scopes,\n }, token);\n}\n\n// ---- NocoDB bookkeeping write-back (adopted user/role ids). Echo-safe: only\n// alist_user_id / alist_role_id are written; W1's old/new-field comparison\n// treats these as bookkeeping -> echo guard short-circuits. ----\nasync function writeBack(cid, fields) {\n if (DRY_RUN || !Object.keys(fields).length) return;\n const live = await ncGet(ctx, `/api/v2/tables/${T_CUST}/records/${cid}`);\n const changes = {};\n for (const k of Object.keys(fields)) {\n if ((live[k] ?? null) !== (fields[k] ?? null)) changes[k] = fields[k];\n }\n if (!Object.keys(changes).length) return; // value unchanged -> no PATCH -> no webhook\n await ctx.helpers.httpRequest({\n method: 'PATCH',\n url: `${$env.NOCODB_URL}/api/v2/tables/${T_CUST}/records`,\n headers: { 'xc-token': $env.NOCODB_TOKEN },\n body: [Object.assign({ Id: cid }, changes)],\n json: true,\n });\n}\n\n// ---- verification (post-repair read-back) ----\nasync function verifyUserState(token, userId, expect) {\n const u = await getUser(token, userId);\n if (!u) return { ok: false, actual: null };\n for (const k of Object.keys(expect)) {\n const want = expect[k];\n const have = u[k];\n const wantS = Array.isArray(want) ? JSON.stringify(want) : want;\n const haveS = Array.isArray(have) ? JSON.stringify(have) : have;\n if (String(haveS ?? '') !== String(wantS ?? '')) return { ok: false, actual: u };\n }\n return { ok: true, actual: u };\n}\nasync function verifyRoleScopes(token, roleId, expectScopes) {\n const r = await getRole(token, roleId);\n if (!r) return { ok: false, actual: null };\n const cur = (r.permission_scopes || []).map(s => s.path).sort();\n const exp = expectScopes.map(s => s.path).sort();\n const ok = cur.length === exp.length && cur.every((p, i) => p === exp[i]);\n return { ok, actual: r };\n}\n\n// ============================================================================\n// per-customer reconciliation\n// ============================================================================\nasync function reconcileOne(cust, token) {\n const cid = cust.Id;\n const username = cust.username;\n const status = cust.status || 'active'; // W1 convention: missing = active\n const isActive = status === 'active';\n const drifts = []; // {type, action, result}\n\n const note = (type, details, action, result) => {\n drifts.push({ type, action, result });\n pushLog(cid, username, type, details, action, result);\n };\n\n // ---------- 1. resolve the AList user ----------\n let user = null;\n if (cust.alist_user_id) {\n user = await getUser(token, cust.alist_user_id);\n if (user && user.username !== username) {\n note('username_drift',\n JSON.stringify({ noco: username, alist: user.username, user_id: user.id }),\n 'log_only', 'log_only');\n }\n }\n if (!user) {\n // id missing or dangling: lookup by username to avoid duplicate users\n const byName = (await listUsers(token)).find(u => u.username === username);\n if (byName) {\n user = byName;\n if (cust.alist_user_id !== byName.id) {\n note('dangling_user_ref',\n JSON.stringify({ ref: cust.alist_user_id, adopted: byName.id, found_by: 'username' }),\n 'adopted', 'repaired');\n await writeBack(cid, { alist_user_id: byName.id });\n }\n }\n }\n if (!user && isActive) {\n // genuinely missing: provision role first, then user (empty role -> 400)\n let role = null;\n const roleName = 'customer-' + username;\n const byName = (await listRoles(token)).find(r => r.name === roleName);\n if (byName) role = byName;\n else if (!DRY_RUN) role = await createRole(token, roleName, 'Auto role for ' + username);\n else role = { id: null, name: roleName, permission_scopes: [] };\n user = await createUser(token, {\n username,\n password: cust.password || 'change_me',\n base_path: BASE_PATH,\n role: role.id ? [role.id] : [],\n permission: PERMISSION,\n });\n note('user_missing',\n JSON.stringify({ alist_user_id: cust.alist_user_id ?? null, alist_role_id: cust.alist_role_id ?? null }),\n DRY_RUN ? 'provision_planned' : 'provisioned', 'repaired');\n if (role.id) await writeBack(cid, { alist_role_id: role.id });\n }\n if (!user) {\n // non-active and never provisioned -> pure label, keep NO AList user.\n // (Healthy state: no drift, no log row.)\n return drifts;\n }\n if (user.id) await writeBack(cid, { alist_user_id: user.id });\n\n // ---------- 2. resolve the role (the user's role binding) ----------\n const roleFor = async (wantName) => {\n const roleName = wantName || ('customer-' + username);\n const bound = Array.isArray(user.role) ? user.role[0] : (user.role || null);\n if (bound) {\n const r = await getRole(token, bound);\n if (r) return r;\n const byName = (await listRoles(token)).find(x => x.name === roleName);\n note('dangling_role_ref',\n JSON.stringify({ ref: bound, adopted: byName ? byName.id : null, want: roleName }),\n byName ? 'adopted' : 'recreate_required', 'repaired');\n if (byName) return byName;\n }\n const byName = (await listRoles(token)).find(x => x.name === roleName);\n if (byName) return byName;\n if (isActive) {\n const created = await createRole(token, roleName, 'Auto role for ' + username);\n note('role_missing', JSON.stringify({ roleName }), 'created', 'repaired');\n return created;\n }\n return null; // non-active with no role: nothing to clear\n };\n let role = await roleFor();\n\n // ---------- 3. desired scopes (shared algorithm; also reads manual grants) ----------\n const g = await calculateEffectiveGrants(ctx, cid);\n const desiredScopes = isActive ? g.newScopes : [];\n\n // ---------- 4. apply fixes (canonical state = W1's verified config) ----------\n const userPatches = [];\n const expectUser = {};\n if (!isActive) {\n if (user.disabled !== true) { userPatches.push('disabled'); note('user_disabled_drift', JSON.stringify({ have: user.disabled, want: true }), 'disabled', 'repaired'); }\n expectUser.disabled = true;\n } else {\n if (user.disabled === true) { userPatches.push('re-enabled'); note('user_disabled_drift', JSON.stringify({ have: true, want: false }), 'enabled', 'repaired'); }\n expectUser.disabled = false;\n }\n if (user.base_path !== BASE_PATH) {\n userPatches.push('base_path'); note('base_path_drift', JSON.stringify({ have: user.base_path, want: BASE_PATH }), 'fixed', 'repaired');\n }\n expectUser.base_path = BASE_PATH;\n if (user.permission !== PERMISSION) {\n userPatches.push('permission'); note('permission_drift', JSON.stringify({ have: user.permission, want: PERMISSION }), 'fixed', 'repaired');\n }\n expectUser.permission = PERMISSION;\n const boundRoleId = role && role.id;\n if (boundRoleId && !(Array.isArray(user.role) && user.role.includes(boundRoleId))) {\n userPatches.push('role_binding'); note('role_binding_drift', JSON.stringify({ have: user.role, want: [boundRoleId] }), 'rebound', 'repaired');\n }\n expectUser.role = boundRoleId ? [boundRoleId] : user.role || [];\n\n if (userPatches.length) {\n // NOTE: password intentionally NOT included in repairs — NocoDB plaintext\n // may be stale (方案 C: customer may have changed their own AList password;\n // pushing it here would clobber it). Password only used at provision.\n await updateUser(token, Object.assign({ id: user.id }, {\n username, base_path: BASE_PATH, permission: PERMISSION, disabled: !isActive,\n role: boundRoleId ? [boundRoleId] : [],\n }));\n }\n\n // scopes drift (role may be null for non-active with no role -> nothing to clear)\n if (role && role.id) {\n const cur = (role.permission_scopes || []).map(s => s.path).sort();\n const want = desiredScopes.map(s => s.path).sort();\n if (cur.length !== want.length || cur.some((p, i) => p !== want[i])) {\n note('scopes_drift',\n JSON.stringify({ have: cur, want }),\n DRY_RUN ? 'rewrite_planned' : 'rewritten', 'repaired');\n await updateRoleScopes(token, role.id, role.name, role.description, desiredScopes);\n }\n } else if (isActive && !DRY_RUN) {\n // active but could not bind a role (should not happen; roleFor creates one)\n throw new Error('Active customer has no role to write scopes into');\n }\n\n // ---------- 5. verify actual AList state after every repair ----------\n if (!DRY_RUN && user.id) {\n const uu = await verifyUserState(token, user.id, expectUser);\n if (!uu.ok) {\n note('verify_failed', JSON.stringify({ expect: expectUser, actual: uu.actual }), 'verify', 'failed');\n }\n if (role && role.id) {\n const rr = await verifyRoleScopes(token, role.id, desiredScopes);\n if (!rr.ok) {\n note('verify_failed', JSON.stringify({ expect_scopes: desiredScopes, actual: rr.actual }), 'verify', 'failed');\n }\n }\n }\n\n return drifts;\n}\n\n// ============================================================================\n// main sweep\n// ============================================================================\nconst login = await alistReq(ctx, 'POST', '/api/auth/login', {\n username: $env.ALIST_ADMIN_USERNAME,\n password: $env.ALIST_ADMIN_PASSWORD,\n}, null);\nconst token = login?.data?.token;\nif (!token) throw new Error('AList login failed');\n\nconst customers = (await ncGet(ctx, `/api/v2/tables/${T_CUST}/records?limit=10000`)).list || [];\n\nfor (const cust of customers) {\n if (!cust.username) {\n summary.errors.push({ customerId: cust.Id, username: null, error: 'missing username' });\n pushLog(cust.Id, null, 'reconcile_error', JSON.stringify({ row: cust.Id }), 'log_only', 'failed');\n summary.customersScanned++;\n continue;\n }\n summary.customersScanned++;\n const entry = { customerId: cust.Id, username: cust.username, status: cust.status || 'active', drifts: [] };\n try {\n entry.drifts = await reconcileOne(cust, token);\n if (entry.drifts.length === 0) summary.skipped++;\n else summary.details.push(entry);\n } catch (e) {\n summary.errors.push({ customerId: cust.Id, username: cust.username, error: e.message });\n pushLog(cust.Id, cust.username, 'reconcile_error', JSON.stringify({ error: e.message }), 'log_only', 'failed');\n }\n}\n\nsummary.driftsDetected = logRows.filter(r => r.result !== 'failed').length;\nsummary.repaired = logRows.filter(r => r.result === 'repaired').length;\n\n// ---- write ReconcileLogs (drift/repair/error rows only) ----\nif (logRows.length && !DRY_RUN) {\n if (!T_LOG) {\n summary.logWriteSkipped = 'NOCODB_TABLE_RECONCILELOGS env missing — rows not persisted';\n } else {\n await ctx.helpers.httpRequest({\n method: 'POST',\n url: `${$env.NOCODB_URL}/api/v2/tables/${T_LOG}/records`,\n headers: { 'xc-token': $env.NOCODB_TOKEN },\n body: logRows,\n json: true,\n });\n }\n}\n\nreturn { json: Object.assign({}, summary, {\n at: new Date().toISOString(),\n logRowsWritten: DRY_RUN ? 0 : logRows.length,\n plannedLogRows: DRY_RUN ? logRows.length : 0,\n}) };" + }, + "name": "Reconcile & Repair (Daily Full)", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 0, + 120 + ] + } + ], + "connections": { + "Daily 03:00": { + "main": [ + [ + { + "node": "Reconcile & Repair (Daily Full)", + "type": "main", + "index": 0 + } + ] + ] + }, + "Manual / Webhook": { + "main": [ + [ + { + "node": "Reconcile & Repair (Daily Full)", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/n8n/gen_w4.js b/n8n/gen_w4.js new file mode 100644 index 0000000..d80455d --- /dev/null +++ b/n8n/gen_w4.js @@ -0,0 +1,62 @@ +// Builds n8n/W4-daily-reconcile.json — the daily full reconciliation workflow. +// Single source of truth: shared/effective-grants.js (inlined) + w4-body.js. +// Run: node n8n/gen_w4.js +const fs = require('fs'); +const path = require('path'); +const { sharedSource } = require('./inline_shared'); + +let body = fs.readFileSync(path.join(__dirname, 'w4-body.js'), 'utf8'); +body = body.replace('// __SHARED_INLINE__ (replaced by shared/effective-grants.js source at build)', sharedSource()); + +// n8n API import (POST /api/v1/workflows) strictly validates: node ids must be +// UUIDs and top-level must NOT carry pinData (class-validator whitelist). +const uuid = () => require('crypto').randomUUID(); +const node = (o) => Object.assign({ id: uuid() }, o); + +const workflow = { + name: 'W4 - Daily Full Reconciliation (AList Drift Repair)', + nodes: [ + node({ + parameters: { + rule: { + interval: [{ field: 'cronExpression', expression: '0 3 * * *' }] + } + }, + name: 'Daily 03:00', + type: 'n8n-nodes-base.scheduleTrigger', + typeVersion: 1.2, + position: [-420, 0] + }), + node({ + parameters: { + httpMethod: 'POST', + path: 'alist-w4-reconcile', + options: {} + }, + name: 'Manual / Webhook', + type: 'n8n-nodes-base.webhook', + typeVersion: 2, + webhookId: 'alist-w4-reconcile', // required: without it the production URL never registers + position: [-420, 240] + }), + node({ + parameters: { jsCode: body }, + name: 'Reconcile & Repair (Daily Full)', + type: 'n8n-nodes-base.code', + typeVersion: 2, + position: [0, 120] + }) + ], + connections: { + 'Daily 03:00': { + main: [[{ node: 'Reconcile & Repair (Daily Full)', type: 'main', index: 0 }]] + }, + 'Manual / Webhook': { + main: [[{ node: 'Reconcile & Repair (Daily Full)', type: 'main', index: 0 }]] + } + }, + settings: { executionOrder: 'v1' } +}; + +fs.writeFileSync(path.join(__dirname, 'W4-daily-reconcile.json'), JSON.stringify(workflow, null, 2) + '\n'); +console.log('wrote W4-daily-reconcile.json (' + body.length + ' chars body)'); \ No newline at end of file diff --git a/n8n/test_w4_reconcile.js b/n8n/test_w4_reconcile.js new file mode 100644 index 0000000..566dcb6 --- /dev/null +++ b/n8n/test_w4_reconcile.js @@ -0,0 +1,369 @@ +// Verify the W4 daily reconciliation body against a mock AList + NocoDB. +// Runs the ACTUAL inlined code from W4-daily-reconcile.json in a vm sandbox +// with realistic state mutations (create/update actually change the mock so +// verify-after-repair reads back real state). +// Run: node n8n/test_w4_reconcile.js +const fs = require('fs'); +const path = require('path'); +const vm = require('vm'); + +const workflow = JSON.parse(fs.readFileSync(path.join(__dirname, 'W4-daily-reconcile.json'), 'utf8')); +const code = workflow.nodes.find(n => n.name === 'Reconcile & Repair (Daily Full)').parameters.jsCode; + +// --------------------------------------------------------------------------- +// mock world +// --------------------------------------------------------------------------- +function freshWorld(w) { + return { + customers: JSON.parse(JSON.stringify(w.customers || [])), + users: JSON.parse(JSON.stringify(w.users || [])), + roles: JSON.parse(JSON.stringify(w.roles || [])), + cps: JSON.parse(JSON.stringify(w.cps || {})), + grants: JSON.parse(JSON.stringify(w.grants || {})), + productRes: JSON.parse(JSON.stringify(w.productRes || {})), + resources: JSON.parse(JSON.stringify(w.resources || [ + { Id: 1, alist_path: '/Class/A' }, { Id: 2, alist_path: '/Class/B' }, + { Id: 3, alist_path: '/Class/D' }, { Id: 4, alist_path: '/Template/Music' }, + ])), + }; +} + +let nextUserId = 1000, nextRoleId = 900; + +async function runWorld(world, extraEnv = {}) { + const S = freshWorld(world); + const logRows = []; + const userUpdates = [], roleUpdates = [], patches = [], creates = { users: [], roles: [] }; + const throwOn = {}; // url-substring -> Error + + const httpRequest = async (opts) => { + const u = opts.url, m = (opts.method || 'GET').toUpperCase(); + for (const k of Object.keys(throwOn)) if (u.includes(k)) throw new Error('mock throw: ' + k); + const body = opts.body; + + // ---- NocoDB ---- + if (m === 'GET' && u.includes(`/tc/records?limit=10000`)) return { list: S.customers }; + if (m === 'GET' && u.includes(`/tr/records?limit=10000`)) return { list: S.resources }; + const cm = u.match(/\/tc\/records\/(\d+)$/); + if (m === 'GET' && cm) return S.customers.find(c => c.Id === Number(cm[1])) || null; + if (m === 'PATCH' && u.includes('/tc/records')) { + for (const row of body) { + const c = S.customers.find(x => x.Id === row.Id); + if (c) Object.assign(c, row); + patches.push(row); + } + return {}; + } + const cpm = u.match(/\/tcp\/records\?where=\(customer\.Id,eq,(\d+)\)/); + if (m === 'GET' && cpm) return { list: S.cps[Number(cpm[1])] || [] }; + const gm = u.match(/\/tag\/records\?where=\(customer\.Id,eq,(\d+)\)/); + if (m === 'GET' && gm) return { list: S.grants[Number(gm[1])] || [] }; + const pm = u.match(/\/tp\/links\/lnk\/records\/(\d+)/); + if (m === 'GET' && pm) return { list: S.productRes[Number(pm[1])] || [] }; + if (m === 'GET' && u.includes('/tlog/records')) return { list: logRows }; + if (m === 'POST' && u.includes('/tlog/records')) { (Array.isArray(body) ? body : [body]).forEach(r => logRows.push(r)); return {}; } + + // ---- AList auth ---- + if (u.endsWith('/api/auth/login')) return { data: { token: 'ADMIN_TOKEN' } }; + + // ---- AList users ---- + if (m === 'GET' && u.includes('/api/admin/user/list')) return { data: { content: S.users } }; + const ug = u.match(/\/api\/admin\/user\/get\?id=(\d+)/); + if (m === 'GET' && ug) { + const x = S.users.find(uu => uu.id === Number(ug[1])); + return x ? { data: x } : { code: 404, data: null }; + } + if (m === 'POST' && u.endsWith('/api/admin/user/create')) { + const nu = Object.assign({ id: nextUserId++, disabled: false }, body); // real AList user/get always carries disabled + S.users.push(nu); creates.users.push(body); + return { code: 200, data: null }; + } + if (m === 'POST' && u.endsWith('/api/admin/user/update')) { + const x = S.users.find(uu => uu.id === body.id); + if (x) { + userUpdates.push(body); + if (!(body.id === 400)) Object.assign(x, body); // 400 simulates AList refusing + } + return { code: 200, data: null }; + } + + // ---- AList roles ---- + if (m === 'GET' && u.includes('/api/admin/role/list')) return { data: { content: S.roles } }; + const rg = u.match(/\/api\/admin\/role\/get\?id=(\d+)/); + if (m === 'GET' && rg) { + const x = S.roles.find(r => r.id === Number(rg[1])); + return x ? { data: x } : { code: 404, data: null }; + } + if (m === 'POST' && u.endsWith('/api/admin/role/create')) { + const nr = Object.assign({ id: nextRoleId++, permission_scopes: [] }, body); + S.roles.push(nr); creates.roles.push(body); + return { code: 200, data: null }; + } + if (m === 'POST' && u.endsWith('/api/admin/role/update')) { + const x = S.roles.find(r => r.id === body.id); + roleUpdates.push(body); + if (x) Object.assign(x, body); + return { code: 200, data: null }; + } + throw new Error('unexpected request: ' + m + ' ' + u); + }; + + const sandbox = { + $env: Object.assign({ + NOCODB_URL: 'https://nc', NOCODB_TOKEN: 't', + NOCODB_TABLE_CUSTOMERS: 'tc', NOCODB_TABLE_CUSTOMERPRODUCTS: 'tcp', + NOCODB_TABLE_PRODUCTS: 'tp', NOCODB_TABLE_RESOURCES: 'tr', + NOCODB_TABLE_ACCESSGRANTS: 'tag', NOCODB_TABLE_RECONCILELOGS: 'tlog', + NOCODB_LINK_PROD_RESOURCES: 'lnk', + NOCODB_FK_RECONCILELOG_CUSTOMER: 'nc_jitu___Customers_id', + ALIST_URL: 'https://al', ALIST_ADMIN_USERNAME: 'admin', ALIST_ADMIN_PASSWORD: 'p', + W4_DRY_RUN: 'false', + }, extraEnv), + $json: {}, + console, + }; + sandbox.helpers = { httpRequest }; + + const wrapped = `(async function(){\n${code}\n})`; + const fn = vm.runInNewContext(wrapped, sandbox, { filename: 'w4-body.js' }); + const result = await fn(); + return { + s: result.json, users: S.users, roles: S.roles, customers: S.customers, + logRows, userUpdates, roleUpdates, patches, creates, + }; +} + +// --------------------------------------------------------------------------- +// worlds +// --------------------------------------------------------------------------- +const W = { + healthy: { + customers: [{ Id: 1, username: 'alice', status: 'active', alist_user_id: 100, alist_role_id: 50 }], + users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }], + roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 1: [{ Id: 11, customer: 1, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 1: [] }, + }, + adoptUser: { // alist_user_id lost in NocoDB but user exists by username + customers: [{ Id: 2, username: 'alice', status: 'active', alist_user_id: null, alist_role_id: 50 }], + users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }], + roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 2: [{ Id: 12, customer: 2, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 2: [] }, + }, + danglingUser: { // alist_user_id points at deleted user + customers: [{ Id: 3, username: 'alice', status: 'active', alist_user_id: 999, alist_role_id: 50 }], + users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }], + roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 3: [{ Id: 13, customer: 3, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 3: [] }, + }, + provision: { // nothing in AList, active customer with entitlement + customers: [{ Id: 4, username: 'bob', status: 'active', alist_user_id: null, alist_role_id: null, password: 'pwBob1' }], + users: [], + roles: [], + cps: { 4: [{ Id: 14, customer: 4, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 2 }] }, + grants: { 4: [] }, + }, + reEnable: { // disabled + wrong base_path/permission + customers: [{ Id: 5, username: 'carol', status: 'active', alist_user_id: 101, alist_role_id: 51 }], + users: [{ id: 101, username: 'carol', base_path: '/Class', permission: 0, disabled: true, role: [51] }], + roles: [{ id: 51, name: 'customer-carol', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 5: [{ Id: 15, customer: 5, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 5: [] }, + }, + scopesDrift: { // stale scope in AList + customers: [{ Id: 6, username: 'dave', status: 'active', alist_user_id: 102, alist_role_id: 52 }], + users: [{ id: 102, username: 'dave', base_path: '/', permission: 16384, disabled: false, role: [52] }], + roles: [{ id: 52, name: 'customer-dave', description: 'x', permission_scopes: [{ path: '/Old/Path', permission: 0 }] }], + cps: { 6: [{ Id: 16, customer: 6, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 3 }] }, + grants: { 6: [] }, + }, + expired: { // non-active customer: disable + clear scopes + customers: [{ Id: 7, username: 'eve', status: 'expired', alist_user_id: 103, alist_role_id: 53 }], + users: [{ id: 103, username: 'eve', base_path: '/', permission: 16384, disabled: false, role: [53] }], + roles: [{ id: 53, name: 'customer-eve', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 7: [] }, + grants: { 7: [] }, + }, + pendingNoUser: { + customers: [{ Id: 8, username: 'frank', status: 'pending', alist_user_id: null, alist_role_id: null }], + users: [], roles: [], + cps: { 8: [] }, grants: { 8: [] }, + }, + usernameDrift: { + customers: [{ Id: 9, username: 'grace', status: 'active', alist_user_id: 104, alist_role_id: 54 }], + users: [{ id: 104, username: 'grace_old', base_path: '/', permission: 16384, disabled: false, role: [54] }], + roles: [{ id: 54, name: 'customer-grace', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 9: [{ Id: 17, customer: 9, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 9: [] }, + }, + verifyFail: { // AList refuses the user update -> verify_failed + customers: [{ Id: 10, username: 'ivy', status: 'active', alist_user_id: 400, alist_role_id: 56 }], + users: [{ id: 400, username: 'ivy', base_path: '/', permission: 16384, disabled: true, role: [56] }], + roles: [{ id: 56, name: 'customer-ivy', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 10: [{ Id: 18, customer: 10, product: 2, status: 'active', expires_at: '2099-01-01' }] }, + productRes: { 2: [{ Id: 1 }] }, + grants: { 10: [] }, + }, + error: { // grants fetch explodes for this customer + customers: [{ Id: 11, username: 'hank', status: 'active', alist_user_id: 105, alist_role_id: 57 }], + users: [{ id: 105, username: 'hank', base_path: '/', permission: 16384, disabled: false, role: [57] }], + roles: [{ id: 57, name: 'customer-hank', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }], + cps: { 11: [] }, + grants: { 11: [] }, + }, +}; + +async function main() { + let fail = 0; + const A = (n, c, d) => { if (c) console.log('PASS ' + n); else { fail++; console.log('FAIL ' + n + ' -> ' + JSON.stringify(d)); } }; + const E = { W4_DRY_RUN: 'false' }; + + // -- healthy: nothing to do + { + const r = await runWorld(W.healthy, E); + A('healthy skipped=1', r.s.skipped === 1, r.s); + A('healthy no log rows', r.logRows.length === 0, r.logRows); + A('healthy no writes', r.userUpdates.length === 0 && r.roleUpdates.length === 0, r); + A('healthy 0 drift', r.s.driftsDetected === 0, r.s); + } + + // -- adopt user by username (id lost) + { + const r = await runWorld(W.adoptUser, E); + A('adopt: 1 drift / 1 repaired', r.s.driftsDetected === 1 && r.s.repaired === 1, r.s); + A('adopt: dangling_user_ref log', r.logRows[0] && r.logRows[0].drift_type === 'dangling_user_ref' && r.logRows[0].result === 'repaired', r.logRows); + A('adopt: customer FK written', r.logRows[0] && r.logRows[0].nc_jitu___Customers_id === 2, r.logRows[0]); + A('adopt: no duplicate user', r.users.length === 1, r.users); + A('adopt: writeBack alist_user_id', r.patches.some(p => p.alist_user_id === 100), r.patches); + A('adopt: next run healthy (id filled)', r.customers[0].alist_user_id === 100, r.customers); + } + + // -- dangling user ref (404 -> lookup by username -> adopt) + { + const r = await runWorld(W.danglingUser, E); + A('dangling: adopted 100', r.s.driftsDetected === 1 && r.s.repaired === 1, r.s); + A('dangling: no user created', r.users.length === 1 && r.creates.users.length === 0, r); + A('dangling: customer id repaired', r.customers[0].alist_user_id === 100, r.customers); + } + + // -- full provision + { + const r = await runWorld(W.provision, E); + A('provision: role+user created', r.creates.roles.length === 1 && r.creates.users.length === 1, r.creates); + A('provision: user role bound', r.users[0].role[0] === r.roles[0].id, r.users); + A('provision: canonical fields', r.users[0].base_path === '/' && r.users[0].permission === 16384 && r.users[0].disabled === false, r.users); + A('provision: scopes applied', JSON.stringify(r.roles[0].permission_scopes.map(s => s.path)) === JSON.stringify(['/Class/B']), r.roles); + A('provision: ids written back', r.patches.some(p => p.alist_user_id) && r.patches.some(p => p.alist_role_id), r.patches); + A('provision: user_missing logged', r.logRows.some(l => l.drift_type === 'user_missing'), r.logRows); + A('provision: verify passed (no verify_failed)', !r.logRows.some(l => l.drift_type === 'verify_failed'), r.logRows); + } + + // -- re-enable + canonical fix + { + const r = await runWorld(W.reEnable, E); + A('re-enable: user updated', r.userUpdates.some(u => u.disabled === false && u.base_path === '/' && u.permission === 16384), r.userUpdates); + A('re-enable: 3 drifts logged', ['user_disabled_drift','base_path_drift','permission_drift'].every(t => r.logRows.some(l => l.drift_type === t)), r.logRows); + A('re-enable: repaired=3', r.s.repaired === 3, r.s); + } + + // -- scopes drift + { + const r = await runWorld(W.scopesDrift, E); + A('scopes: role rewritten', r.roleUpdates.length === 1 && r.roles[0].permission_scopes.map(s => s.path).join() === '/Class/D', r.roleUpdates); + A('scopes: drift logged', r.logRows.some(l => l.drift_type === 'scopes_drift' && l.result === 'repaired'), r.logRows); + } + + // -- expired cleanup + { + const r = await runWorld(W.expired, E); + A('expired: user disabled', r.userUpdates.some(u => u.disabled === true), r.userUpdates); + A('expired: scopes cleared', r.roles[0].permission_scopes.length === 0, r.roles); + A('expired: repaired=2', r.s.repaired === 2, r.s); + A('expired: verify ok', !r.logRows.some(l => l.drift_type === 'verify_failed'), r.logRows); + } + + // -- pending no user: untouched + { + const r = await runWorld(W.pendingNoUser, E); + A('pending: skipped=1', r.s.skipped === 1, r.s); + A('pending: no rows / no writes', r.logRows.length === 0 && r.creates.users.length === 0 && r.creates.roles.length === 0, r); + } + + // -- username drift: detect + log only + { + const r = await runWorld(W.usernameDrift, E); + A('drift: username_drift logged', r.logRows.some(l => l.drift_type === 'username_drift' && l.result === 'log_only' && l.action === 'log_only'), r.logRows); + A('drift: no rename (username untouched)', r.users[0].username === 'grace_old', r.users); + A('drift: no new user', r.creates.users.length === 0, r.creates); + A('drift: no user patch', r.userUpdates.length === 0, r.userUpdates); + } + + // -- verify failed (AList refuses update) + { + const r = await runWorld(W.verifyFail, E); + A('verifyfail: verify_failed logged', r.logRows.some(l => l.drift_type === 'verify_failed' && l.result === 'failed'), r.logRows); + A('verifyfail: not counted as repaired', r.logRows.filter(l => l.result === 'repaired').length === 1, r.logRows); + } + + // -- error path (grants fetch explodes) + { + const r3 = await runWorldWithThrow(W.error, 'grant'); + A('error: errors[] populated', r3.s.errors.length === 1 && r3.s.errors[0].customerId === 11, r3.s.errors); + A('error: reconcile_error row', r3.logRows.some(l => l.drift_type === 'reconcile_error' && l.result === 'failed'), r3.logRows); + } + + // -- dry run: zero writes + { + const r = await runWorld(W.provision, { W4_DRY_RUN: 'true' }); + A('dryrun: no creates', r.creates.users.length === 0 && r.creates.roles.length === 0, r.creates); + A('dryrun: no patches', r.patches.length === 0, r.patches); + A('dryrun: no log POST', r.logRows.length === 0, r.logRows); + A('dryrun: planned rows reported', r.s.plannedLogRows > 0 && r.s.logRowsWritten === 0, r.s); + A('dryrun: dryRun flag', r.s.dryRun === true, r.s); + } + + console.log(fail === 0 ? '\nALL PASSED' : `\n${fail} FAILED`); + process.exit(fail === 0 ? 0 : 1); +} + +// helper: run with grants fetch throwing for customer 11 +async function runWorldWithThrow(world, marker) { + const S = freshWorld(world); + const logRows = []; + const httpRequest = async (opts) => { + const u = opts.url, m = (opts.method || 'GET').toUpperCase(); + if (u.includes('/tag/records')) throw new Error('grants fetch boom'); + if (m === 'GET' && u.includes('/tc/records?limit=10000')) return { list: S.customers }; + if (m === 'GET' && u.includes(`/tr/records?limit=10000`)) return { list: S.resources }; + if (m === 'GET' && u.includes('/api/admin/user/list')) return { data: { content: S.users } }; + if (m === 'GET' && u.includes('/api/admin/role/list')) return { data: { content: S.roles } }; + const ug = u.match(/\/api\/admin\/user\/get\?id=(\d+)/); + if (m === 'GET' && ug) { const x = S.users.find(uu => uu.id === Number(ug[1])); return x ? { data: x } : { code: 404, data: null }; } + const rg = u.match(/\/api\/admin\/role\/get\?id=(\d+)/); + if (m === 'GET' && rg) { const x = S.roles.find(r => r.id === Number(rg[1])); return x ? { data: x } : { code: 404, data: null }; } + if (u.endsWith('/api/auth/login')) return { data: { token: 'ADMIN_TOKEN' } }; + if (m === 'POST' && u.includes('/tlog/records')) { (Array.isArray(opts.body) ? opts.body : [opts.body]).forEach(r => logRows.push(r)); return {}; } + if (m === 'POST' && u.endsWith('/api/admin/user/update')) return { code: 200 }; + if (m === 'POST' && u.endsWith('/api/admin/role/update')) return { code: 200 }; + throw new Error('unexpected: ' + m + ' ' + u); + }; + const sandbox = { + $env: { NOCODB_URL: 'https://nc', NOCODB_TOKEN: 't', NOCODB_TABLE_CUSTOMERS: 'tc', NOCODB_TABLE_CUSTOMERPRODUCTS: 'tcp', NOCODB_TABLE_PRODUCTS: 'tp', NOCODB_TABLE_RESOURCES: 'tr', NOCODB_TABLE_ACCESSGRANTS: 'tag', NOCODB_TABLE_RECONCILELOGS: 'tlog', NOCODB_LINK_PROD_RESOURCES: 'lnk', ALIST_URL: 'https://al', ALIST_ADMIN_USERNAME: 'admin', ALIST_ADMIN_PASSWORD: 'p', W4_DRY_RUN: 'false' }, + $json: {}, console, + }; + sandbox.helpers = { httpRequest }; + const fn = vm.runInNewContext(`(async function(){\n${code}\n})`, sandbox, { filename: 'w4-body.js' }); + const result = await fn(); + return { s: result.json, logRows }; +} + +main().then(() => {}).catch(e => { console.error(e); process.exit(1); }); \ No newline at end of file diff --git a/n8n/w4-body.js b/n8n/w4-body.js new file mode 100644 index 0000000..47fbb52 --- /dev/null +++ b/n8n/w4-body.js @@ -0,0 +1,377 @@ +// ============================================================================ +// W4 runtime body — Daily full reconciliation + drift repair + reconcile log. +// Rebuilt by `node n8n/gen_w4.js` (single source of truth = shared/effective-grants.js). +// +// ONE-DIRECTIONAL sync: NocoDB `Customers.status` is the BUSINESS source of +// truth; AList user enabled/disabled is only ENFORCEMENT state. W4 NEVER +// writes Customers.status. A manual `disabled` flip in the AList admin UI is +// repaired back to match NocoDB — never the other way round. +// +// Drift matrix (auto-repair): +// status active -> user must exist (provision if missing; lookup by +// username FIRST to avoid duplicate users when the +// NocoDB id was lost), be ENABLED, carry the desired +// role scopes (calculateEffectiveGrants), base_path +// '/', permission 16384. +// status non-active -> if a user exists: must be DISABLED with empty role +// scopes (soft state kept). pending/never-provisioned +// customers stay WITHOUT an AList user (no auto-create). +// dangling refs -> alist_user_id / alist_role_id point at deleted AList +// records: lookup by username / role name, adopt if +// found, recreate if not, then re-sync. +// username drift -> NocoDB username differs from AList username: DETECT +// + log only (no safe official rename API assumed; no +// destructive migration). +// +// Every repair is VERIFIED by re-reading AList state afterwards; a failed +// verify is logged as verify_failed. ReconcileLogs receives ONLY +// drift/repair/error rows — healthy customers produce nothing. +// +// Triggers: daily cron (03:00) + manual webhook. Dry run: webhook +// `?dryRun=true` or env W4_DRY_RUN=true -> plan only, ZERO writes. +// ============================================================================ +// __SHARED_INLINE__ (replaced by shared/effective-grants.js source at build) +// ============================================================================ + +const ctx = { $env, helpers: this.helpers }; + +const query = ($json && $json.query) || {}; +const DRY_RUN = query.dryRun === 'true' || query.dryRun === true || + String($env.W4_DRY_RUN || '').toLowerCase() === 'true'; +const T_CUST = $env.NOCODB_TABLE_CUSTOMERS; +const T_LOG = $env.NOCODB_TABLE_RECONCILELOGS; +// NocoDB v2 (2026.08.x): bt links are NOT created by writing the link column +// (scalar/array/object all ignored). Write the FK column directly: +// `nc_jitu___Customers_id` (verified live 2026-08-23). +const FK_CUST = $env.NOCODB_FK_RECONCILELOG_CUSTOMER || 'nc_jitu___Customers_id'; +const runId = 'recon-' + new Date().toISOString().replace(/[:.]/g, '-'); + +// canonical AList user configuration — must stay identical to W1's (verified +// working: base_path '/' + absolute role scopes + permission 16384) +const BASE_PATH = '/'; +const PERMISSION = 16384; + +const logRows = []; // ReconcileLogs payload — drift/repair/error rows ONLY + +const summary = { + runId, dryRun: DRY_RUN, + customersScanned: 0, driftsDetected: 0, repaired: 0, skipped: 0, + errors: [], details: [], +}; + +function pushLog(cid, username, driftType, details, action, result) { + const row = { + run_id: runId, + drift_type: driftType, + details: details == null ? '' : String(details).slice(0, 2000), + action, + result, + timestamp: new Date().toISOString(), + }; + if (cid != null) row[FK_CUST] = cid; // FK column (link writes are ignored) + logRows.push(row); +} + +// ---- AList helpers (read) ---- +async function listUsers(token) { + const r = await alistReq(ctx, 'GET', '/api/admin/user/list?page=1&per_page=1000', null, token); + return (r && (r.data?.content || r.data?.list)) || []; +} +async function listRoles(token) { + const r = await alistReq(ctx, 'GET', '/api/admin/role/list?page=1&per_page=1000', null, token); + return (r && (r.data?.content || r.data?.list)) || []; +} +async function getUser(token, id) { + const r = await alistReq(ctx, 'GET', `/api/admin/user/get?id=${id}`, null, token); + return (r && r.data) ? r.data : null; +} +async function getRole(token, id) { + const r = await alistReq(ctx, 'GET', `/api/admin/role/get?id=${id}`, null, token); + return (r && r.data) ? r.data : null; +} + +// ---- AList helpers (write) — all no-op under DRY_RUN ---- +async function createRole(token, name, description) { + if (DRY_RUN) return { id: null, name, permission_scopes: [] }; + await alistReq(ctx, 'POST', '/api/admin/role/create', { + name, description, default: false, permission_scopes: [], + }, token); + const roles = await listRoles(token); + const r = roles.find(x => x.name === name); + if (!r) throw new Error('Created role not found: ' + name); + return r; +} +async function createUser(token, body) { + if (DRY_RUN) return { id: null, username: body.username }; + await alistReq(ctx, 'POST', '/api/admin/user/create', body, token); + const users = await listUsers(token); + const u = users.find(x => x.username === body.username); + if (!u) throw new Error('Created user not found: ' + body.username); + return u; +} +async function updateUser(token, fields) { + if (DRY_RUN) return; + await alistReq(ctx, 'POST', '/api/admin/user/update', fields, token); +} +async function updateRoleScopes(token, roleId, name, description, scopes) { + if (DRY_RUN) return; + await alistReq(ctx, 'POST', '/api/admin/role/update', { + id: roleId, name, description, permission_scopes: scopes, + }, token); +} + +// ---- NocoDB bookkeeping write-back (adopted user/role ids). Echo-safe: only +// alist_user_id / alist_role_id are written; W1's old/new-field comparison +// treats these as bookkeeping -> echo guard short-circuits. ---- +async function writeBack(cid, fields) { + if (DRY_RUN || !Object.keys(fields).length) return; + const live = await ncGet(ctx, `/api/v2/tables/${T_CUST}/records/${cid}`); + const changes = {}; + for (const k of Object.keys(fields)) { + if ((live[k] ?? null) !== (fields[k] ?? null)) changes[k] = fields[k]; + } + if (!Object.keys(changes).length) return; // value unchanged -> no PATCH -> no webhook + await ctx.helpers.httpRequest({ + method: 'PATCH', + url: `${$env.NOCODB_URL}/api/v2/tables/${T_CUST}/records`, + headers: { 'xc-token': $env.NOCODB_TOKEN }, + body: [Object.assign({ Id: cid }, changes)], + json: true, + }); +} + +// ---- verification (post-repair read-back) ---- +async function verifyUserState(token, userId, expect) { + const u = await getUser(token, userId); + if (!u) return { ok: false, actual: null }; + for (const k of Object.keys(expect)) { + const want = expect[k]; + const have = u[k]; + const wantS = Array.isArray(want) ? JSON.stringify(want) : want; + const haveS = Array.isArray(have) ? JSON.stringify(have) : have; + if (String(haveS ?? '') !== String(wantS ?? '')) return { ok: false, actual: u }; + } + return { ok: true, actual: u }; +} +async function verifyRoleScopes(token, roleId, expectScopes) { + const r = await getRole(token, roleId); + if (!r) return { ok: false, actual: null }; + const cur = (r.permission_scopes || []).map(s => s.path).sort(); + const exp = expectScopes.map(s => s.path).sort(); + const ok = cur.length === exp.length && cur.every((p, i) => p === exp[i]); + return { ok, actual: r }; +} + +// ============================================================================ +// per-customer reconciliation +// ============================================================================ +async function reconcileOne(cust, token) { + const cid = cust.Id; + const username = cust.username; + const status = cust.status || 'active'; // W1 convention: missing = active + const isActive = status === 'active'; + const drifts = []; // {type, action, result} + + const note = (type, details, action, result) => { + drifts.push({ type, action, result }); + pushLog(cid, username, type, details, action, result); + }; + + // ---------- 1. resolve the AList user ---------- + let user = null; + if (cust.alist_user_id) { + user = await getUser(token, cust.alist_user_id); + if (user && user.username !== username) { + note('username_drift', + JSON.stringify({ noco: username, alist: user.username, user_id: user.id }), + 'log_only', 'log_only'); + } + } + if (!user) { + // id missing or dangling: lookup by username to avoid duplicate users + const byName = (await listUsers(token)).find(u => u.username === username); + if (byName) { + user = byName; + if (cust.alist_user_id !== byName.id) { + note('dangling_user_ref', + JSON.stringify({ ref: cust.alist_user_id, adopted: byName.id, found_by: 'username' }), + 'adopted', 'repaired'); + await writeBack(cid, { alist_user_id: byName.id }); + } + } + } + if (!user && isActive) { + // genuinely missing: provision role first, then user (empty role -> 400) + let role = null; + const roleName = 'customer-' + username; + const byName = (await listRoles(token)).find(r => r.name === roleName); + if (byName) role = byName; + else if (!DRY_RUN) role = await createRole(token, roleName, 'Auto role for ' + username); + else role = { id: null, name: roleName, permission_scopes: [] }; + user = await createUser(token, { + username, + password: cust.password || 'change_me', + base_path: BASE_PATH, + role: role.id ? [role.id] : [], + permission: PERMISSION, + }); + note('user_missing', + JSON.stringify({ alist_user_id: cust.alist_user_id ?? null, alist_role_id: cust.alist_role_id ?? null }), + DRY_RUN ? 'provision_planned' : 'provisioned', 'repaired'); + if (role.id) await writeBack(cid, { alist_role_id: role.id }); + } + if (!user) { + // non-active and never provisioned -> pure label, keep NO AList user. + // (Healthy state: no drift, no log row.) + return drifts; + } + if (user.id) await writeBack(cid, { alist_user_id: user.id }); + + // ---------- 2. resolve the role (the user's role binding) ---------- + const roleFor = async (wantName) => { + const roleName = wantName || ('customer-' + username); + const bound = Array.isArray(user.role) ? user.role[0] : (user.role || null); + if (bound) { + const r = await getRole(token, bound); + if (r) return r; + const byName = (await listRoles(token)).find(x => x.name === roleName); + note('dangling_role_ref', + JSON.stringify({ ref: bound, adopted: byName ? byName.id : null, want: roleName }), + byName ? 'adopted' : 'recreate_required', 'repaired'); + if (byName) return byName; + } + const byName = (await listRoles(token)).find(x => x.name === roleName); + if (byName) return byName; + if (isActive) { + const created = await createRole(token, roleName, 'Auto role for ' + username); + note('role_missing', JSON.stringify({ roleName }), 'created', 'repaired'); + return created; + } + return null; // non-active with no role: nothing to clear + }; + let role = await roleFor(); + + // ---------- 3. desired scopes (shared algorithm; also reads manual grants) ---------- + const g = await calculateEffectiveGrants(ctx, cid); + const desiredScopes = isActive ? g.newScopes : []; + + // ---------- 4. apply fixes (canonical state = W1's verified config) ---------- + const userPatches = []; + const expectUser = {}; + if (!isActive) { + if (user.disabled !== true) { userPatches.push('disabled'); note('user_disabled_drift', JSON.stringify({ have: user.disabled, want: true }), 'disabled', 'repaired'); } + expectUser.disabled = true; + } else { + if (user.disabled === true) { userPatches.push('re-enabled'); note('user_disabled_drift', JSON.stringify({ have: true, want: false }), 'enabled', 'repaired'); } + expectUser.disabled = false; + } + if (user.base_path !== BASE_PATH) { + userPatches.push('base_path'); note('base_path_drift', JSON.stringify({ have: user.base_path, want: BASE_PATH }), 'fixed', 'repaired'); + } + expectUser.base_path = BASE_PATH; + if (user.permission !== PERMISSION) { + userPatches.push('permission'); note('permission_drift', JSON.stringify({ have: user.permission, want: PERMISSION }), 'fixed', 'repaired'); + } + expectUser.permission = PERMISSION; + const boundRoleId = role && role.id; + if (boundRoleId && !(Array.isArray(user.role) && user.role.includes(boundRoleId))) { + userPatches.push('role_binding'); note('role_binding_drift', JSON.stringify({ have: user.role, want: [boundRoleId] }), 'rebound', 'repaired'); + } + expectUser.role = boundRoleId ? [boundRoleId] : user.role || []; + + if (userPatches.length) { + // NOTE: password intentionally NOT included in repairs — NocoDB plaintext + // may be stale (方案 C: customer may have changed their own AList password; + // pushing it here would clobber it). Password only used at provision. + await updateUser(token, Object.assign({ id: user.id }, { + username, base_path: BASE_PATH, permission: PERMISSION, disabled: !isActive, + role: boundRoleId ? [boundRoleId] : [], + })); + } + + // scopes drift (role may be null for non-active with no role -> nothing to clear) + if (role && role.id) { + const cur = (role.permission_scopes || []).map(s => s.path).sort(); + const want = desiredScopes.map(s => s.path).sort(); + if (cur.length !== want.length || cur.some((p, i) => p !== want[i])) { + note('scopes_drift', + JSON.stringify({ have: cur, want }), + DRY_RUN ? 'rewrite_planned' : 'rewritten', 'repaired'); + await updateRoleScopes(token, role.id, role.name, role.description, desiredScopes); + } + } else if (isActive && !DRY_RUN) { + // active but could not bind a role (should not happen; roleFor creates one) + throw new Error('Active customer has no role to write scopes into'); + } + + // ---------- 5. verify actual AList state after every repair ---------- + if (!DRY_RUN && user.id) { + const uu = await verifyUserState(token, user.id, expectUser); + if (!uu.ok) { + note('verify_failed', JSON.stringify({ expect: expectUser, actual: uu.actual }), 'verify', 'failed'); + } + if (role && role.id) { + const rr = await verifyRoleScopes(token, role.id, desiredScopes); + if (!rr.ok) { + note('verify_failed', JSON.stringify({ expect_scopes: desiredScopes, actual: rr.actual }), 'verify', 'failed'); + } + } + } + + return drifts; +} + +// ============================================================================ +// main sweep +// ============================================================================ +const login = await alistReq(ctx, 'POST', '/api/auth/login', { + username: $env.ALIST_ADMIN_USERNAME, + password: $env.ALIST_ADMIN_PASSWORD, +}, null); +const token = login?.data?.token; +if (!token) throw new Error('AList login failed'); + +const customers = (await ncGet(ctx, `/api/v2/tables/${T_CUST}/records?limit=10000`)).list || []; + +for (const cust of customers) { + if (!cust.username) { + summary.errors.push({ customerId: cust.Id, username: null, error: 'missing username' }); + pushLog(cust.Id, null, 'reconcile_error', JSON.stringify({ row: cust.Id }), 'log_only', 'failed'); + summary.customersScanned++; + continue; + } + summary.customersScanned++; + const entry = { customerId: cust.Id, username: cust.username, status: cust.status || 'active', drifts: [] }; + try { + entry.drifts = await reconcileOne(cust, token); + if (entry.drifts.length === 0) summary.skipped++; + else summary.details.push(entry); + } catch (e) { + summary.errors.push({ customerId: cust.Id, username: cust.username, error: e.message }); + pushLog(cust.Id, cust.username, 'reconcile_error', JSON.stringify({ error: e.message }), 'log_only', 'failed'); + } +} + +summary.driftsDetected = logRows.filter(r => r.result !== 'failed').length; +summary.repaired = logRows.filter(r => r.result === 'repaired').length; + +// ---- write ReconcileLogs (drift/repair/error rows only) ---- +if (logRows.length && !DRY_RUN) { + if (!T_LOG) { + summary.logWriteSkipped = 'NOCODB_TABLE_RECONCILELOGS env missing — rows not persisted'; + } else { + await ctx.helpers.httpRequest({ + method: 'POST', + url: `${$env.NOCODB_URL}/api/v2/tables/${T_LOG}/records`, + headers: { 'xc-token': $env.NOCODB_TOKEN }, + body: logRows, + json: true, + }); + } +} + +return { json: Object.assign({}, summary, { + at: new Date().toISOString(), + logRowsWritten: DRY_RUN ? 0 : logRows.length, + plannedLogRows: DRY_RUN ? logRows.length : 0, +}) }; \ No newline at end of file diff --git a/scripts/add_reconcilelogs_table.py b/scripts/add_reconcilelogs_table.py new file mode 100644 index 0000000..ad257d0 --- /dev/null +++ b/scripts/add_reconcilelogs_table.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +"""Ensure the ReconcileLogs table exists in AlistAccess (idempotent, NON-destructive). + +Unlike create_nocodb_schema.py (which drops + recreates the base), this script +only creates/mends the ReconcileLogs table + columns. Safe to re-run. +Uses v2 meta endpoints only — verified on NocoDB 2026.08.1 (v1 column GET/PATCH +endpoints are GONE on this version; column id is fetched from the table meta). + +Credentials from env (NOCODB_URL / NOCODB_TOKEN) — never commit real tokens. + +Usage: + NOCODB_TOKEN=... python scripts/add_reconcilelogs_table.py +""" +import json, os, urllib.request, urllib.error + +NOCODB_URL = os.environ.get("NOCODB_URL", "https://nocodb.hoelee.com") +NOCODB_TOKEN = os.environ.get("NOCODB_TOKEN", "CHANGE_ME_nocodb_token") +BASE_ID = os.environ.get("NOCODB_BASE_ID", "p3lzkresswyowqg") +CUSTOMERS_TID = os.environ.get("NOCODB_TABLE_CUSTOMERS", "maqb9zgvza2te3k") +TABLE_TITLE = os.environ.get("RECONCILELOGS_TITLE", "ReconcileLogs") + +DRIFT_OPTIONS = [ + "user_missing", "role_missing", "dangling_user_ref", "dangling_role_ref", + "user_disabled_drift", "scopes_drift", "base_path_drift", + "permission_drift", "role_binding_drift", "username_drift", + "verify_failed", "reconcile_error", +] +RESULT_OPTIONS = ["repaired", "log_only", "failed", "skipped"] + + +def req(method, path, body=None): + r = urllib.request.Request(f"{NOCODB_URL}{path}", method=method) + r.add_header("xc-token", NOCODB_TOKEN) + if body is not None: + r.add_header("Content-Type", "application/json") + r.data = json.dumps(body).encode() + try: + with urllib.request.urlopen(r, timeout=30) as resp: + return json.loads(resp.read().decode() or "null") + except urllib.error.HTTPError as e: + raise RuntimeError(f"{method} {path} -> HTTP {e.code}: {e.read().decode()[:200]}") + + +def table_meta(tid): + return req("GET", f"/api/v2/meta/tables/{tid}") + + +def col_by_title(tid, title): + meta = table_meta(tid) + return next((c for c in (meta.get("columns") or []) if c.get("title") == title), None) + + +def ensure_column(tid, title, uidt, **extra): + c = col_by_title(tid, title) + if not c: + req("POST", f"/api/v2/meta/tables/{tid}/columns", + {"title": title, "uidt": uidt, **extra}) + c = col_by_title(tid, title) + if not c: + raise RuntimeError(f"column {title} not found after create") + print(f" + {title} ({uidt}) -> {c['id']}") + return c + + +def ensure_ss_options(tid, title, options): + c = ensure_column(tid, title, "SingleSelect") + cur = ((c.get("colOptions") or {}).get("options") or []) + cur_titles = [o.get("title") for o in cur] + if cur_titles == options: + print(f" = {title} options OK") + return + req("PATCH", f"/api/v2/meta/columns/{c['id']}", { + "colOptions": {"options": [{"title": o, "order": i + 1} + for i, o in enumerate(options)]}, + "dtxp": ",".join(options), + }) + print(f" ~ {title} options set") + + +def main(): + tables = req("GET", f"/api/v2/meta/bases/{BASE_ID}/tables")["list"] + existing = next((t for t in tables if t["title"] == TABLE_TITLE), None) + if existing: + tid = existing["id"] + print(f"{TABLE_TITLE} exists: {tid}") + else: + t = req("POST", f"/api/v2/meta/bases/{BASE_ID}/tables", { + "title": TABLE_TITLE, "table_name": TABLE_TITLE, "type": "table", + "columns": [], + }) + tid = t["id"] + print(f"{TABLE_TITLE} created: {tid}") + + ensure_column(tid, "run_id", "SingleLineText") + ensure_column(tid, "details", "LongText") + ensure_column(tid, "action", "SingleLineText") + ensure_column(tid, "timestamp", "DateTime") + ensure_ss_options(tid, "drift_type", DRIFT_OPTIONS) + ensure_ss_options(tid, "result", RESULT_OPTIONS) + + if not col_by_title(tid, "customer"): + req("POST", f"/api/v1/db/meta/tables/{tid}/columns", { + "title": "customer", "uidt": "LinkToAnotherRecord", + "parentId": CUSTOMERS_TID, "childId": tid, "type": "bt", + }) + c = col_by_title(tid, "customer") + print(f" + customer (bt link) -> {c['id']}") + fk = col_by_title(tid, "nc_jitu___Customers_id") + if not fk: + raise RuntimeError("FK column nc_jitu___Customers_id missing — bt link create failed") + + print(f"\nDONE. table={tid} fk_column={fk['id']} ({fk['title']})") + print("=> add to n8n env: NOCODB_TABLE_RECONCILELOGS=" + tid + + " NOCODB_FK_RECONCILELOG_CUSTOMER=" + fk["title"]) + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/scripts/create_nocodb_schema.py b/scripts/create_nocodb_schema.py index d8afddc..03eb155 100644 --- a/scripts/create_nocodb_schema.py +++ b/scripts/create_nocodb_schema.py @@ -76,16 +76,27 @@ def main(): body.update(extra) return curl("POST", f"/api/v1/db/meta/tables/{tid}/columns", body) - def set_options(colid, opts): - return curl("PATCH", f"/api/v1/db/meta/columns/{colid}", + def col_id(tid, title): + # v1 GET /columns is removed on NocoDB 2026.08.x — read table meta instead + t = curl("GET", f"/api/v2/meta/tables/{tid}") + for c in jget(t, "columns", d=[]) or []: + if c.get("title") == title: + return c.get("id") + return None + + def set_options(tid, title, opts): + cid = col_id(tid, title) + if not cid: + raise RuntimeError(f"column {title} not found on {tid}") + return curl("PATCH", f"/api/v2/meta/columns/{cid}", {"colOptions": {"options": [{"title": o, "order": i + 1, "color": None} for i, o in enumerate(opts)]}, "dtxp": ",".join(f"'{o}'" for o in opts)}) def add_ss(tid, title, opts): - r = add_col(tid, title, "SingleSelect") - cid = jget(r, "id") - set_options(cid, opts) + add_col(tid, title, "SingleSelect") + cid = col_id(tid, title) + set_options(tid, title, opts) return cid def add_link(child_tid, title, parent_tid, type_="bt"): @@ -166,6 +177,19 @@ def main(): {"title": "notes", "uidt": "LongText"}, ]); pur = jget(pur, "id") + rlog = create_table("ReconcileLogs", [ + {"title": "run_id", "uidt": "SingleLineText"}, + {"title": "details", "uidt": "LongText"}, + {"title": "action", "uidt": "SingleLineText"}, + {"title": "timestamp", "uidt": "DateTime"}, + ]); rlog = jget(rlog, "id") + add_ss(rlog, "drift_type", + ["user_missing", "role_missing", "dangling_user_ref", "dangling_role_ref", + "user_disabled_drift", "scopes_drift", "base_path_drift", + "permission_drift", "role_binding_drift", "username_drift", + "verify_failed", "reconcile_error"]) + add_ss(rlog, "result", ["repaired", "log_only", "failed", "skipped"]) + # ---- links ---- add_link(cp, "customer", cust) add_link(cp, "product", prod) @@ -175,9 +199,11 @@ def main(): add_link(al, "customer", cust) add_link(pur, "customer", cust) add_link(pur, "product", prod) + add_link(rlog, "customer", cust) ids = {"base": base_id, "Customers": cust, "Products": prod, "Resources": res, - "CustomerProducts": cp, "AccessGrants": ag, "AccessLogs": al, "Purchases": pur} + "CustomerProducts": cp, "AccessGrants": ag, "AccessLogs": al, + "Purchases": pur, "ReconcileLogs": rlog} print("DONE. IDs:", json.dumps(ids, indent=2)) # persist for later steps open("_nocodb_ids.json", "w").write(json.dumps(ids, indent=2)) diff --git a/scripts/live_dry_run_w4.js b/scripts/live_dry_run_w4.js new file mode 100644 index 0000000..82453dd --- /dev/null +++ b/scripts/live_dry_run_w4.js @@ -0,0 +1,54 @@ +// Live dry-run of the W4 Code node against REAL AList + NocoDB, outside n8n. +// Runs the exact inlined jsCode from W4-daily-reconcile.json in a vm sandbox, +// with helpers.httpRequest backed by node fetch. ZERO writes (dryRun=true). +// Usage: node scripts/live_dry_run_w4.js +const fs = require('fs'); +const path = require('path'); +const vm = require('vm'); + +const wf = JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'n8n', 'W4-daily-reconcile.json'), 'utf8')); +const code = wf.nodes.find(n => n.name === 'Reconcile & Repair (Daily Full)').parameters.jsCode; + +const ENV = { + NOCODB_URL: 'https://nocodb.hoelee.com', + NOCODB_TOKEN: process.env.NOCODB_TOKEN, // from env, never hardcoded — same for passwords below + NOCODB_TABLE_CUSTOMERS: 'maqb9zgvza2te3k', + NOCODB_TABLE_PRODUCTS: 'molyb3pxv81l75g', + NOCODB_TABLE_RESOURCES: 'm6cm7tkxzd8m18c', + NOCODB_TABLE_CUSTOMERPRODUCTS: 'mvyqbv8ao7u6bct', + NOCODB_TABLE_ACCESSGRANTS: 'mxdx3yhfcyej3ao', + NOCODB_TABLE_RECONCILELOGS: 'mktsb7le6fqr2u1', + NOCODB_FK_RECONCILELOG_CUSTOMER: 'nc_jitu___Customers_id', + NOCODB_LINK_PROD_RESOURCES: 'c4ko4x8kwi16swg', + ALIST_URL: process.env.ALIST_URL || 'http://192.168.1.1:5244', // DSM host port (container DNS only inside bridge) + ALIST_ADMIN_USERNAME: process.env.ALIST_ADMIN_USERNAME || 'hoelee', + ALIST_ADMIN_PASSWORD: process.env.ALIST_ADMIN_PASSWORD, // from env + W4_DRY_RUN: 'false', +}; +if (!ENV.NOCODB_TOKEN || !ENV.ALIST_ADMIN_PASSWORD) { console.error('NOCODB_TOKEN + ALIST_ADMIN_PASSWORD env required'); process.exit(1); } + +const httpRequest = async (opts) => { + const { method = 'GET', url, headers = {}, body } = opts; + const res = await fetch(url, { + method, + headers: { ...headers, 'Content-Type': 'application/json' }, + body: body !== undefined && body !== null ? JSON.stringify(body) : undefined, + }); + const text = await res.text(); + let parsed = null; + try { parsed = text ? JSON.parse(text) : null; } catch { parsed = text; } + return parsed; +}; + +const sandbox = { + $env: ENV, + $json: { query: { dryRun: 'true' } }, + console, +}; +sandbox.helpers = { httpRequest }; + +(async () => { + const fn = vm.runInNewContext(`(async function(){\n${code}\n})`, sandbox, { filename: 'w4-body-live.js' }); + const result = await fn(); + console.log(JSON.stringify(result.json, null, 1)); +})().catch(e => { console.error('LIVE RUN FAILED:', e); process.exit(1); }); \ No newline at end of file