Compare commits
9
Commits
w3-verified
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 hoelee
|
a61cd0f984 | ||
|
hoelee
|
d7fad26d9d | ||
|
hoelee
|
414bf31ba0 | ||
|
hoelee
|
7dd14f9f25 | ||
|
hoelee
|
0c2f60e922 | ||
|
hoelee
|
773fd3f2d5 | ||
|
hoelee
|
fd62775fcd | ||
|
hoelee
|
f26164023e | ||
|
hoelee
|
4693c9d684 |
@@ -0,0 +1,34 @@
|
||||
# AGENTS.md — AI 会话入口（开工先读这个）
|
||||
|
||||
> 本文件是 AI agent 的路由表：新会话从这里开始，按需再往下读，不要一上来通读全部文档。
|
||||
|
||||
## 开工四步
|
||||
1. 读本文件（30 秒）
|
||||
2. 读 `docs/PROJECT_STATE.md`（当前进度 + 下一步，≤150 行）
|
||||
3. 凭据/地址 → **只看 `SECRETS.md`**（唯一凭据文件；外发给别人 = 删它即可）
|
||||
4. 连不上系统？DSM IP 可能漂移了——先探测内网（ping / 试 5244、10380、5678 端口），别死磕旧 IP
|
||||
|
||||
## 任务路由表
|
||||
| 要做的事 | 去哪里 |
|
||||
|---|---|
|
||||
| 改 workflow 逻辑 | `n8n/w1..w4-body.js` 或 `n8n/shared/effective-grants.js`，改完跑 `node n8n/gen_w1.js`…重新生成 JSON。**禁止手编 `W*.json`** |
|
||||
| 查表/字段/link 列 ID | `docs/DATABASE_SCHEMA.md`（ID 不是机密，留在文档里） |
|
||||
| 查 AList / NocoDB / n8n API 怪癖 | skill `alist-serve-resource` 的 Pitfalls（唯一权威，比本文档全） |
|
||||
| 理解权限算法 | `n8n/shared/effective-grants.js` 注释 + `docs/PERMISSIONS.md` |
|
||||
| 理解整体架构 | `docs/ARCHITECTURE.md` |
|
||||
| 查某次事故结论 | `docs/PROJECT_STATE.md` 结案区 |
|
||||
|
||||
## 铁律（违反必返工）
|
||||
1. **勿手编 `n8n/W*.json`** —— 它们是生成物，源头在 body/shared
|
||||
2. **验证 n8n 是否真的执行：看 AList/NocoDB 实际状态变化**，别数 executions 表（`SAVE_ON_SUCCESS=none`）
|
||||
3. **DSM IP 会漂移** —— 连不上先探测（probe_env.py），别死磕旧 IP
|
||||
4. AList 多目录授权：`base_path="/"` + role scope 绝对路径（设 `/Class` 会双重前缀全 403）
|
||||
5. expiry 边界是 `<= today`（到期当天即失效）；NocoDB link 查询必须 `(link.Id,eq,N)`
|
||||
6. 改共享代码后 grep 一遍裸 `(customer,eq,` / `(product,eq,` —— 此 bug 复发过两次
|
||||
|
||||
## 项目一句话
|
||||
NocoDB(业务 SOT) --webhook/schedule--> n8n W1-W4 --> AList(role scopes)。
|
||||
Customers → CustomerProducts → Products ↔ Resources 决定客户在 AList 能看哪些目录。
|
||||
|
||||
## 外发 SOP
|
||||
把项目给其他人：**删除 `SECRETS.md` 这一个文件即可**，其余内容均可公开。
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
# SECRETS.md — 凭据总表（唯一）
|
||||
|
||||
> ⚠️ **本文件随私有仓库入库。把项目发给任何人之前：删除本文件即可完成脱敏。**
|
||||
> 技术文档里的表 ID / workflow ID 不是机密（没有下面的 token/密码谁也访问不了），留在原处。
|
||||
|
||||
## DSM（Synology NAS）
|
||||
- SSH：`192.168.137.2:9998`（**IP 会漂移**，连不上先探测内网）user `hoelee` / `987][pOIUY/.,mnbVCXZ`
|
||||
- sudo 免密；docker 命令用 `/usr/local/bin/docker`
|
||||
- paramiko 只能用密码认证（ed25519 对其 OpenSSH 8.2 失败）；sshd 无 SFTP 子系统
|
||||
|
||||
## AList（本项目核心服务）
|
||||
- 容器 `alist-serve-resource`（xhofe/alist v3.63.0），网络 bridge_hoelee
|
||||
- 地址：Windows 直连 `http://192.168.137.2:5244` · DSM host `http://localhost:5244` · n8n 容器内 `http://alist-serve-resource:5244`
|
||||
- admin：`hoelee` / `8qiksf6D5JmGJCkfGycR`（2FA 已关）
|
||||
|
||||
## NocoDB
|
||||
- 公网 `https://nocodb.hoelee.com`（⚠ 2026-08-24 起解析/连通未恢复）· 内网 `http://192.168.137.2:10380`
|
||||
- API token：`nc_pat_mDk7f6t4ZMAYYzv9GDrQvKpSr4oFZyiStxf0wxlL`
|
||||
- base `AlistAccess` = `p3lzkresswyowqg`
|
||||
- 元数据库（含业务数据物理表 `nc_jitu___*`）：mysql-server 容器，库 `nocodb`，user `nocodb` / `A834EwsSckTHpdXiXH`
|
||||
|
||||
## n8n
|
||||
- 公网 `https://auto.hoelee.com` · 内网 `http://192.168.137.2:5678`
|
||||
- REST API key：`n8n_api_73b39ec6437d9cc2185aebb8cb1fd7bc5154cf8325a38caab39f6dd399ef9e96ed5044948d1f0bdf`（postgres `user_api_keys` 表，label「My API Key」，workflow 全权）
|
||||
- 数据库：postgres-server 容器，db `n8n`，user `n8n` / `Hoelee@7296`
|
||||
- 容器由 **Portainer stack 144** 管理（改 env 必须 rebuild，restart 无效）
|
||||
|
||||
## Git 远程
|
||||
- Gitea：`https://git.hoelee.com/hoelee/alist-serve-resource.git`（先 Gitea 后 GitHub 的惯例）
|
||||
- 推送令牌：`hoelee:6495a6318ed156fe9a7b0636eb3dc30a3155a4d6`（Basic auth 用）
|
||||
@@ -106,3 +106,29 @@
|
||||
|
||||
- 只要不是 `active`，顾客就登不进去、也看不到文件。
|
||||
- 改回 `active`，系统会自动算**现在还有效**的产品权限（已经过期的不会回来）。
|
||||
|
||||
---
|
||||
|
||||
## 附 2：到期后会发生什么（重要）
|
||||
|
||||
系统**每 5 分钟自动扫一次**所有顾客的到期日（`CustomerProducts.expires_at`），
|
||||
**到期当天**（当天就算到期）就会把这些产品对应的文件夹权限从顾客身上移除——
|
||||
**这一步是全自动的，不用管理员做任何操作**（2026-08-28 定案：维持此行为，不做自动停用/通知）。
|
||||
|
||||
| 项目 | 实际行为 |
|
||||
|---|---|
|
||||
| 文件权限 | 到期当天自动移除，顾客再也打不开任何文件夹 ✅ |
|
||||
| 账号 | **不冻结**——顾客仍能登录，但登录进去目录是空的、什么都看不到 |
|
||||
| `Customers.status` | 不会自动改，仍显示 `active`（正常现象，权限其实已移除） |
|
||||
| 通知 | 不会自动发任何通知，管理员自行留意 `expires_at` 即可 |
|
||||
|
||||
> **要连登录都不让他进**（退款 / 违规 / 明确不续约）：
|
||||
> 手动把 `Customers.status` 改成 `suspended` 或 `expired`，系统**马上**冻结账号（登不进去）。
|
||||
|
||||
> **续费 / 恢复**：把 `CustomerProducts.expires_at` 改到新日期（或新增一条记录），权限自动恢复。
|
||||
> 若之前手动冻结过（status 不是 `active`），记得先把 status 改回 `active`。
|
||||
|
||||
> **顾客反馈「能登录但里面什么都没有」**：先查他的 `CustomerProducts.expires_at`，
|
||||
> 十有八九是到期了。要么改 `expires_at` 续费，要么说明服务已结束。
|
||||
> 如果 `expires_at` 还没到但看不到东西：查该产品 `Products.status` 是否 `active`、
|
||||
> Resource 的 `alist_path` 是否填对（必须真实存在的 `/` 开头路径）。
|
||||
|
||||
+19
-3
@@ -1,8 +1,8 @@
|
||||
# NocoDB 数据结构（Phase 2b — 重建，7 表最终模型）
|
||||
# NocoDB 数据结构（Phase 2b — 重建，8 表最终模型）
|
||||
|
||||
> Source of Truth = NocoDB。AList 只做 enforcement，不存业务授权数据。
|
||||
> 本 schema 按《Hermes 调整指导》最终业务模型重建（2026-08-22）。
|
||||
> 通过 NocoDB API 自动建表，无需手动点 UI。
|
||||
> 通过 NocoDB API 自动建表，无需手动点 UI。ReconcileLogs 于 2026-08-23 W4 新增。
|
||||
|
||||
- **Base**: `AlistAccess`（id `p3lzkresswyowqg`），独立新 base，与旧 `accessDB` 隔离。
|
||||
- **MySQL source**: mysql-server 上的 `alistaccess` schema（由 NocoDB 自动管理）。
|
||||
@@ -19,6 +19,7 @@
|
||||
| AccessGrants | `mxdx3yhfcyej3ao` |
|
||||
| AccessLogs | `ml12zcyzlehgi21` |
|
||||
| Purchases | `mwp12z9lcohhsuk` |
|
||||
| ReconcileLogs | `mktsb7le6fqr2u1`（FK 列 `nc_jitu___Customers_id` = `coa6ynwcdlune6p`） |
|
||||
|
||||
> ⚠ ID 是本次重建生成的。若再次删库重建，ID 会变，需同步更新 n8n 节点 + 本文档。
|
||||
|
||||
@@ -35,6 +36,7 @@
|
||||
| AccessLogs | `customer` | bt | Customers | |
|
||||
| Purchases | `customer` | bt | Customers | |
|
||||
| Purchases | `product` | bt | Products | |
|
||||
| ReconcileLogs | `customer` | bt | Customers | ⚠ 写入时用 FK 列 `nc_jitu___Customers_id`（v2 忽略 link 列本身） |
|
||||
|
||||
> M2M 读写方式（已实测）：`GET/POST /api/v2/tables/{products}/links/c4ko4x8kwi16swg/records/{productId}`，
|
||||
> POST body `{"Id": <resourceId>}`。反向用 Resources 的 `cfxi44n9ixkimqc`。
|
||||
@@ -138,7 +140,21 @@
|
||||
| bytes | Number |
|
||||
| timestamp | DateTime |
|
||||
|
||||
### 7. Purchases（纯记录，不参与权限）
|
||||
### 7. ReconcileLogs（W4 对账审计，n8n W4 写入）
|
||||
| 字段 | 类型 | 说明 |
|
||||
|---|---|---|
|
||||
| Id | number (PK) | |
|
||||
| run_id | SingleLineText | 本次对账 run（`recon-<ISO>`） |
|
||||
| customer | link bt → Customers | FK 列 `nc_jitu___Customers_id`（v2 写链接必须写 FK 列） |
|
||||
| drift_type | SingleSelect | user_missing / role_missing / dangling_user_ref / dangling_role_ref / user_disabled_drift / scopes_drift / base_path_drift / permission_drift / role_binding_drift / username_drift / verify_failed / reconcile_error |
|
||||
| details | LongText | 前后值 JSON |
|
||||
| action | SingleLineText | 做了什么（provisioned / adopted / fixed / rewritten / disabled / log_only / verify …） |
|
||||
| result | SingleSelect | repaired / log_only / failed / skipped |
|
||||
| timestamp | DateTime | |
|
||||
|
||||
> 只记 drift / repair / error 行，健康客户不产生行。W4 每日全量或 webhook 触发时写入。
|
||||
|
||||
### 8. Purchases（纯记录，不参与权限）
|
||||
| 字段 | 类型 |
|
||||
|---|---|
|
||||
| Id | number (PK) |
|
||||
|
||||
+37
-5
@@ -3,16 +3,18 @@
|
||||
> n8n 运行在现有 auto.hoelee.com（`bridge_hoelee` 网络）。它是**同步器，不是 SOT**。
|
||||
> 更新日期 2026-08-23。
|
||||
|
||||
## 已实现的三个 Workflow
|
||||
## 已实现的 Workflow
|
||||
|
||||
| Workflow | ID | 触发 | 职责 |
|
||||
|---|---|---|---|
|
||||
| **W1** Customer Provision & Status Lifecycle | `gU2ee9Eg5w1KzB5p` | NocoDB Customers webhook | provision + 整户 status lifecycle |
|
||||
| **W2** CustomerProduct Sync | `D9OWAI7JUjKnRdOn` | NocoDB CustomerProducts webhook | 重算该客有效 grants 并应用 |
|
||||
| **W3** Expiry Sync | schedule `*/5` | 定时 | 全量对账 + drift 修复 |
|
||||
| **W3** Expiry Sync | `zlEfjj5eK5IfILtk` | schedule `*/5` | 定时 | 全量对账 + drift 修复（scopes 层） |
|
||||
| **W4** Daily Full Reconciliation | `pl8j9lpg5gjrd0dB` | 每日 03:00 + webhook `alist-w4-reconcile` | 全量对账 + 用户/角色级 drift 修复 + ReconcileLogs |
|
||||
| **W5** Footer Purchase Check | （导入后由 n8n 分配） | GET webhook `dk-purchase-check` | footer 动态行数据源（公开只读） |
|
||||
|
||||
三个 workflow 共享同一套算法 `calculateEffectiveGrants`（`n8n/shared/effective-grants.js`，
|
||||
构建时经 `gen_w1/w2/w3.js` 内联进各 Code 节点）。
|
||||
W1–W4 共享同一套算法 `calculateEffectiveGrants`（`n8n/shared/effective-grants.js`，
|
||||
构建时经 `gen_w1/w2/w3/w4.js` 内联进各 Code 节点）。W5 是独立的**公开只读** webhook（不属同步器）。
|
||||
|
||||
## W1 — Customer Provision & Status Lifecycle
|
||||
|
||||
@@ -36,6 +38,35 @@
|
||||
- 逐客调 `calculateEffectiveGrants` → diff → 应用
|
||||
- Env 开关：`W3_DRY_RUN=true`（只打印计划）、`ALIST_AUTO_DISABLE_ON_EXPIRE=true`（scopes 变空时 disable 用户）
|
||||
|
||||
## W4 — Daily Full Reconciliation（每日全量对账）
|
||||
|
||||
- 触发：每日 03:00 cron + 手动 webhook（`/webhook/alist-w4-reconcile`，`?dryRun=true` 出计划）
|
||||
- 单向同步：NocoDB status 是 SOT，W4 只修 AList，**永不反向改 NocoDB status**
|
||||
- 覆盖 W3 不管的 drift：缺 user/role → 按 username 反查防重再 provision；悬空引用 → adopt/recreate；
|
||||
非 active → disable + 清 scopes；base_path/permission 修回 canonical（`/` + 16384）；
|
||||
username drift → detect + log_only；修复后逐个 **verify 回读**，失败记 `verify_failed`
|
||||
- 写 ReconcileLogs（表 `mktsb7le6fqr2u1`），只记 drift/repair/error 行；customer 链接用 FK 列
|
||||
- 密码随 repair 不推送（方案 C）
|
||||
- 详见 `docs/W4-DESIGN.md`
|
||||
|
||||
## W5 — Footer Purchase Check（公开只读，不属同步器）
|
||||
|
||||
- 触发：`GET /webhook/dk-purchase-check?u=<username>`（公网经 `auto.hoelee.com`）
|
||||
- 职责：footer v2.6 动态行数据源——按 username 查 Customers → 拉该客 CustomerProducts →
|
||||
返回 `{ username, known, purchases:[{product, expires_at, display_state}] }`
|
||||
- 安全模型（**无 HMAC**——key 会随 footer JS 下发到客户端=空话）：
|
||||
- CORS：webhook `options.responseHeaders` 设 `Access-Control-Allow-Origin: https://drive.digikedai.com`
|
||||
→ 仅 AList 门户 origin 能读响应
|
||||
- 最小数据：只回 product 名 + 到期 + display_state，**不含内部 Id**
|
||||
- 服务器枚举靠 CF WAF 在 auto.hoelee.com 上做限流 + Origin/Referer 规则（见交付指南）
|
||||
- 零 AList 认证请求：username 由 footer 本地解码用户自己的 JWT（`localStorage['token']`）取出 →
|
||||
无「空客户端设备驱逐」风险（v2.3 事故机制完全不涉及）
|
||||
- 复用现有 env（`NOCODB_URL`/`NOCODB_TOKEN`/`NOCODB_TABLE_CUSTOMERS`/`NOCODB_TABLE_CUSTOMERPRODUCTS`），**零新增 env**
|
||||
- 生成：`node n8n/gen_w5.js` → `W5-purchase-check.json`（2 节点：webhook[GET, lastNode, CORS 头] + Code「Lookup Purchases」）
|
||||
- 导入（手动）：n8n → Import from File → `W5-purchase-check.json` → 记下 Webhook URL（应为
|
||||
`https://auto.hoelee.com/webhook/dk-purchase-check`）→ 激活。测试：
|
||||
`curl 'https://auto.hoelee.com/webhook/dk-purchase-check?u=tester'` 应回 JSON 含 `known:true` + 2 条 purchases
|
||||
|
||||
## 共用 env 变量
|
||||
|
||||
n8n Code 节点内用 `$env.VAR`（`process.env` 不可靠）。下表是本项目需要的 env：
|
||||
@@ -43,7 +74,8 @@ n8n Code 节点内用 `$env.VAR`（`process.env` 不可靠）。下表是本项
|
||||
| 变量 | 说明 |
|
||||
|---|---|
|
||||
| `NOCODB_URL` / `NOCODB_TOKEN` | NocoDB 地址 + `xc-token` |
|
||||
| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` | 各表 model_id |
|
||||
| `NOCODB_TABLE_CUSTOMERS` / `..._CUSTOMERPRODUCTS` / `..._PRODUCTS` / `..._RESOURCES` / `..._ACCESSGRANTS` / `..._RECONCILELOGS` | 各表 model_id（ReconcileLogs = `mktsb7le6fqr2u1`） |
|
||||
| `NOCODB_FK_RECONCILELOG_CUSTOMER` | ReconcileLogs.customer 的 FK 列名 `nc_jitu___Customers_id`（W4 写链接必须用 FK 列） |
|
||||
| `NOCODB_LINK_PROD_RESOURCES` | Product↔Resource M2M 列 id（`c4ko4x8kwi16swg`）|
|
||||
| `ALIST_URL` | `http://alist-serve-resource:5244`（n8n 容器内）|
|
||||
| `ALIST_ADMIN_USERNAME` / `ALIST_ADMIN_PASSWORD` | AList admin 登录 |
|
||||
|
||||
+75
-125
@@ -1,142 +1,92 @@
|
||||
# Project State (Handoff) — 2026-08-23
|
||||
# Project State (Handoff) — 2026-08-25
|
||||
|
||||
> 交接文档：压缩/新会话后，按此文档续做。细节见各 doc。
|
||||
> 交接文档：压缩/新会话后按此续做。**入口顺序：AGENTS.md → 本文件 → SECRETS.md(凭据)。**
|
||||
> 本文件只记「状态与下一步」；API 怪癖/坑的唯一权威 = Hermes skill `alist-serve-resource` + `docs/API_INTEGRATION.md`。
|
||||
|
||||
## 🔑 连接凭据速查（新会话直接取用）
|
||||
## 凭据
|
||||
全部集中在根目录 **`SECRETS.md`**（唯一凭据文件；把项目给别人 = 删它即可）。
|
||||
表 ID / workflow ID 不是机密，见 `docs/DATABASE_SCHEMA.md` 与下文。
|
||||
|
||||
> ⚠️ **安全警告：本块含生产真实凭据（密码/API token/key）。只留本地用于交接——推 Gitea 前必须整块删除或脱敏，否则凭据随 git 历史永久泄露。**
|
||||
## 系统现状（2026-08-25 实测）
|
||||
- 容器全绿：`alist-serve-resource`(healthy) · `nocodb` · `n8n` · `nginx-limit-alist` · `traefik-update-alist`（08-25 全部 redeploy）
|
||||
- **公网上线 ✅**：`https://drive.digikedai.com` 经 CF Tunnel 全链路实测通
|
||||
（/ping pong、/ 与 /login 均 200 AList SPA、/@manage 403 防护有效）
|
||||
- W1–W4 四个 workflow 激活正常（postgres 复核 `activeVersionId` 全对齐；08-25 PUT 更新后仍对齐）
|
||||
- **端到端实测通过 ✅（08-25）**：真实业务事件全链路——建 Customer→W1 自动开通+写回；
|
||||
建 CP→W2 自动算 scopes（`/Class/Forex`）；AList 真实登录/范围内 200/范围外 403；
|
||||
CP 到期→W3 sweep 收权；Customer suspended→W1 禁用（客户 token 全 API 401）；
|
||||
active 恢复→重启用重算；人为漂移→W4 dryRun 只报告不写入 / 正式跑修复且幂等。
|
||||
- **08-25 三处基础设施修复**：① AList `/user/update` 必须带 `username`（否则 400 静默不禁用）
|
||||
→ w1/w3-body.js 已修并部署；② NocoDB 两 hook 改指 `http://n8n:5678`（原 auto.hoelee.com
|
||||
走旧公网链路，VPS 卡顿时事件静默丢失），W2 hook 路径 `alist-cp-sync`→`alist-customerproduct-sync`
|
||||
（原来一直 404）；③ n8n API key 已入档 SECRETS.md。
|
||||
- **n8n「执行消失」已结案**：容器 env `EXECUTIONS_DATA_SAVE_ON_SUCCESS=none`，成功执行不落库是配置使然，管道健康（详见下方结案记录）
|
||||
- NocoDB 数据：Products 176 / Resources 171（商品目录健在）；Customers/CustomerProducts 测试数据已清空（残留 1 行上个会话的 `e2e_test_01` 待用户处置）；AList 仅 guest/hoelee(+e2e_test_01 残留)
|
||||
|
||||
| 系统 | 地址 / 凭据 |
|
||||
|---|---|
|
||||
| **SSH DSM** | `192.168.1.1:9998` user `hoelee` / `<redacted>`；sudo 免密；docker 用 `/usr/local/bin/docker` |
|
||||
| **AList** | `http://alist-serve-resource:5244`（容器内）/ `localhost:5244`（DSM host）admin `hoelee` / `<redacted>`（2FA 关） |
|
||||
| **NocoDB** | `https://nocodb.hoelee.com`；API token: `<redacted>`；base AlistAccess = `p3lzkresswyowqg` |
|
||||
| **n8n** | `https://auto.hoelee.com`；API key: `<redacted>` |
|
||||
| **NocoDB 表 ID** | Customers `maqb9zgvza2te3k` · Products `molyb3pxv81l75g` · Resources `m6cm7tkxzd8m18c` · CustomerProducts `mvyqbv8ao7u6bct` · AccessGrants `mxdx3yhfcyej3ao` · AccessLogs `ml12zcyzlehgi21` · Purchases `mwp12z9lcohhsuk`；M2M link col `c4ko4x8kwi16swg` |
|
||||
| **n8n workflow ID** | W1=Cust Provision `gU2ee9Eg5w1KzB5p` · W2=CP Sync `D9OWAI7JUjKnRdOn` |
|
||||
|
||||
## 已完成 ✅
|
||||
|
||||
### 1. AList stock 部署（Phase 1）— 运行中
|
||||
- Container `alist-serve-resource` (xhofe/alist v3.63.0)，DSM port **5244**，网络 `bridge_hoelee`
|
||||
- 挂载 `/volume1/Class` `/volume1/Book` `/volume1/Comic` `/volume1/Template`（只读 → /mnt/Class 等）
|
||||
- 旧 alist (v3.54.0) 仍运行在 5243，**暂不处理**
|
||||
- Admin: username `hoelee`，2FA 已关闭；compose 在项目根 `docker-compose.yml`
|
||||
- DSM 上从 host shell 访问内网容器要用 `localhost:5244`；n8n 容器内用 `http://alist-serve-resource:5244`
|
||||
|
||||
### 2. NocoDB schema（Phase 2 重建）— 7 表最终模型
|
||||
Base: **`AlistAccess`**（`p3lzkresswyowqg`），与旧 `accessDB` 完全隔离。表 + model_id + 关键点：
|
||||
| 表 | model_id | 备注 |
|
||||
|---|---|---|
|
||||
| Customers | maqb9zgvza2te3k | plaintext password + **password_updated_at** |
|
||||
| Products | molyb3pxv81l75g | default_duration_days；M2M→Resources |
|
||||
| Resources | m6cm7tkxzd8m18c | alist_path |
|
||||
| CustomerProducts | mvyqbv8ao7u6bct | 核心表（customer+product+expiry） |
|
||||
| AccessGrants | mxdx3yhfcyej3ao | **物化状态表，不是 trigger**（禁止 loop） |
|
||||
| AccessLogs | ml12zcyzlehgi21 | 审计 |
|
||||
| Purchases | mwp12z9lcohhsuk | 纯记录 |
|
||||
|
||||
- 建表脚本：`scripts/create_nocodb_schema.py`（幂等）
|
||||
- 表/字段/链接列 ID 全表：`docs/DATABASE_SCHEMA.md`（已更新为 7 表）
|
||||
- NocoDB API token（workspace 级，来自 nc_api_tokens 表 n8n 那行）：在 `docs/API_INTEGRATION.md` / SETUP.md 有记（或查 NocoDB UI）
|
||||
|
||||
### 2. W1 客户开通 + Status Lifecycle — **实测跑通** ✅
|
||||
- 文件：`n8n/W1-customer-provision.json`（由 `n8n/w1-body.js` + 共享函数库经 `gen_w1.js` 生成）
|
||||
- 流程：Webhook → Extract customerId（同时透传 webhook 的 old/new rows）→ Load Env → **单 Code 节点 "Provision & Apply Lifecycle"**（含 login + status 分支 + write-back）
|
||||
- **Lifecycle matrix（5 状态，2026-08-23 定稿）**：
|
||||
- `active`（无 user）→ provision（建 role→建 user）+ recompute + enable
|
||||
- `active`（有 user）→ re-enable + recompute（不恢复旧 scopes）+ apply
|
||||
- `pending`/`suspended`/`expired`/`deleted`（有 user/role）→ disable AList user + 清空 role scopes（soft，保留 user/role/CP/history）
|
||||
- `pending`/`suspended`/`expired`/`deleted`（无 user）→ **no-op**（纯 NocoDB 标签，不建 AList user）
|
||||
- 所有 `X → active` resume → recompute（从当前有效 CP + expiry），**永不恢复旧 scopes**，**不自动 mutate CustomerProducts**
|
||||
- **幂等 + loop 防护**：见 `n8n/test_w1_lifecycle.js`（10 场景全过，覆盖完整 matrix + echo 短路 + sync_required 强制重算）
|
||||
- **Loop guard（2026-08-23 重写，核心）**：不再用 `sync_status==='synced'`，改用 **old/new field comparison + `sync_required` 标志**：
|
||||
- `businessChanged()` = 业务字段（status/username/password/display_name/email/notes）在 `previous_rows`→`rows` 间变化，**或** `sync_required` 置位，**或 after-insert（有 new 无 old）**
|
||||
- echo（只动了 alist_user_id/alist_role_id/sync_*/last_sync*）→ 立即 `skip_writeback_echo`，零 AList 变更、零 PATCH
|
||||
- **write-back 值比对守卫**：writeBack() 先 GET 当前记录，只 PATCH 值真会变的字段；值没变 → 无 PATCH → 无 webhook → 从物理上掐断 loop
|
||||
- 写回时始终清 `sync_required: false`
|
||||
- base_path 硬编码 `'/'`；空 role 数组先建 role 再建 user
|
||||
- ⚠ **AccessGrants 本次不物化**：不作为 lifecycle source-of-truth，保持现状，等 W4 统一设计（用户明确指示）
|
||||
|
||||
### 3. W2 CustomerProduct 权限同步 — **实测跑通** ✅
|
||||
- 文件：`n8n/W2-customerproduct-sync.json`（`n8n/w2-body.js` + 共享库 → `gen_w2.js`）
|
||||
- 流程：Webhook → Extract customerProductId → Load Env → **Compute Effective Grants & Apply**（单 Code 节点调共享 `calculateEffectiveGrants`）→ Skip if no change → No-op
|
||||
- customer link 为空时优雅跳过（`skipWriteBack:true`，不回写，补填后再触发）
|
||||
- expiry 边界 `<= today`（到期当天视为过期）
|
||||
|
||||
### 4. W3 Expiry Sync（全量对账）— **实测跑通** ✅
|
||||
- 文件：`n8n/W3-expiry-sync.json`（`n8n/w3-body.js` + 共享库 → `gen_w3.js`）
|
||||
- Schedule `*/5` → 单 Code 节点 "Reconcile & Apply"（逐客户调共享 `calculateEffectiveGrants` → diff → 应用）
|
||||
- 修过 Bearer 前缀 bug（AList 要裸 token）+ expiry 边界 `<= today`
|
||||
- Env：`W3_DRY_RUN`、`ALIST_AUTO_DISABLE_ON_EXPIRE`（默认 off）
|
||||
|
||||
### 5. 共享函数库（Step 3 核心重构）🆕
|
||||
- **`n8n/shared/effective-grants.js`** = 单一事实源：`calculateEffectiveGrants(customerId)` + `ncGet`/`alistReq`/`linkId`
|
||||
- 算法：desired = 有效 CP 派生路径 **∪** 有效 AccessGrants 路径；同 path 多 product 取 **MAX(expires_at)**；permission 取 readwrite 覆盖
|
||||
- expiry 边界 `<= today`；status 过滤非 active
|
||||
- **构建时内联**：`n8n/inline_shared.js` 的 `sharedSource()` 剥离 `module.exports`（`/* SHARED_EXPORT_BEGIN */` 之后），三个 `gen_w*.js` 把 body（`__SHARED_INLINE__` 占位）+ 共享源拼成 JSON
|
||||
- **W1/W2/W3 零复制算法**：三处都调同一个 `calculateEffectiveGrants`
|
||||
- **测试**（全过）：`test_effective_grants.js`（共享算法）、`test_w3_refactored.js`（W3 回归）、`test_w1_lifecycle.js`（W1 lifecycle，10 场景全覆盖：pending/active/suspended/expired/deleted 全转换 + echo 短路 + sync_required 强制重算）
|
||||
## 已完成 ✅（细节见各 DESIGN 文档与 git log）
|
||||
1. **Phase 1** AList stock v3.63.0 :5244（bridge_hoelee，4 只读挂载；旧 alist 已下线，compose 内注释保留）
|
||||
2. **Phase 2** NocoDB base `AlistAccess`(`p3lzkresswyowqg`) 8 表（7 业务 + ReconcileLogs）；脚本 `scripts/create_nocodb_schema.py` / `add_reconcilelogs_table.py`
|
||||
3. **W1** 客户开通 + 5 状态 lifecycle（pending/active/suspended/expired/deleted）— 实测跑通；old/new 字段比对 loop guard + `sync_required` 强制重算 + writeBack 值比对守卫（`test_w1_lifecycle.js` 10 场景全过）
|
||||
4. **W2** CustomerProduct 权限同步（调共享算法）— 实测跑通
|
||||
5. **W3** 每 5 分钟 expiry sweep（`W3_DRY_RUN` / `ALIST_AUTO_DISABLE_ON_EXPIRE` 开关）— 实测跑通
|
||||
6. **共享库** `n8n/shared/effective-grants.js` 单一事实源，构建期内联（`gen_w*.js`），三 workflow 零复制算法；union 语义：CP 派生 ∪ AccessGrants，同 path 取 MAX(expires_at)，expiry 边界 `<= today`
|
||||
7. **W4** 每日 03:00 全量对账 + ReconcileLogs 审计 — 本地 + 直连生产实测通过（dry-run→repair→复跑幂等；34 项 mock 测试过）；单向同步 NocoDB=SOT，密码不随 repair 推送（方案 C）
|
||||
8. 文档体系：ARCHITECTURE / PERMISSIONS / ADMIN_GUIDE / DATABASE_SCHEMA / N8N_WORKFLOWS 等；git 已推 Gitea
|
||||
9. **公网链路切换（08-25）**：DSM 文件统一收进 `/volume1/docker/alist-digikedai/`（容器名全部未动——n8n `ALIST_URL` 硬依赖）；CF Tunnel `drive.digikedai.com` 回源 `http://172.16.0.1:5246`；traefik 删除 `/login`→n8n 劫持路由（曾致登录页挂起）；nginx 移除 drive 域不限速白名单（现吃默认 1.5MB/s + limit_conn 10/IP）。完整记录 = DSM `/volume1/docker/alist-digikedai/alist-serve-resource/NETWORK_ARCHITECTURE.md`
|
||||
10. **W5 + footer v2.6（08-28）**：footer 动态购买行——本地解码用户 JWT 取 username → GET webhook `dk-purchase-check?u=` → NocoDB 查购买/到期 → 渲染。W5 源（`n8n/w5-body.js` + `gen_w5.js` → `W5-purchase-check.json`）已入库，**待用户手动导入 n8n + 激活**；footer v2.6 已部署线上（详见 dsm-resource-management `desc/DELIVERY_GUIDE.md` §7.10 + skill `alist-footer`）。实测 headless 9/9 PASS（含 CDP 模拟 W5 响应验证整链路）。安全= CORS 限 drive 域 + 最小数据 + 零新增 env（复用 NOCODB_*）
|
||||
|
||||
## 🔴 下一步（按序）
|
||||
**（紧迫）手动导入 W5**：n8n → Import from File → `n8n/W5-purchase-check.json` → 激活 → 记 webhook URL
|
||||
（`https://auto.hoelee.com/webhook/dk-purchase-check`）。测：`curl 'https://auto.hoelee.com/webhook/dk-purchase-check?u=tester'`
|
||||
应回 `known:true` + 2 条 purchases。导入后 footer 动态行即自动上线（无需再改 footer）。
|
||||
1. **补 n8n env 并 rebuild（Portainer stack 144，改 env 必须 rebuild）**：`NOCODB_TABLE_RECONCILELOGS=mktsb7le6fqr2u1` + `NOCODB_FK_RECONCILELOG_CUSTOMER=nc_jitu___Customers_id`（W4 审计落库当前被跳过）；顺手加 `TZ=Asia/Kuala_Lumpur` 修 UTC 时区
|
||||
2. **决策：auto.hoelee.com 是否退役/迁移**——n8n 公网访问现在只剩 UI 用途；hook 已容器化。若保留，建议迁到 CF Tunnel 新链路；若退役，同步删 VPS DNS + DSM 反代条目
|
||||
3. **清理上个会话残留**：NocoDB Customer Id=8 `e2e_test_01` + AList user 17 / role `customer-e2e_test_01`
|
||||
4. 公网限速实测：经 CF 后确认 nginx 限速按访客真实 IP 计（若 key 变 docker 内网 IP 全员共享一桶 → 调 traefik ipStrategy depth / nginx XFF）
|
||||
5. 旧链路退役清理：旧 alist 数据目录、DSM 反代 access.hoelee.com 条目、VPS DNS、compose 注释块择机删
|
||||
6. AList site_url 设为 `https://drive.digikedai.com`（分享链接需要）
|
||||
7. 密码保持方案 C（客户可自改 AList 密码 → NocoDB plaintext 失真；不强制回写）
|
||||
8. W2 不写 CP bookkeeping（sync_status/last_sync 永不更新）——现状可用；若未来有逻辑依赖这些字段再补写回节点
|
||||
|
||||
> 功能已全部实现并通过本地测试（用户已确认「已测试功能正常」）。剩余事项如下：
|
||||
## 结案记录
|
||||
- **CF Tunnel drive.digikedai.com 打不通（08-25 发现 → 当日结案）**
|
||||
- 现象：公网 TCP/TLS 能建立但 HTTP 层零响应超时；DSM 内部链路对任意 Host 头都正常 → 不是 hostname 检查
|
||||
- 根因：cloudflared 在 bridge_hoelee 网络，**容器→宿主机自身 LAN IP (192.168.1.1:5246) 被 DSM 防火墙 hairpin 丢弃**（日志 `dial tcp ... i/o timeout`）；同隧道其他站点源是别的机器所以没事
|
||||
- 修复：CF Dashboard 回源改 `http://172.16.0.1:5246`（docker0 网关=宿主机），实测公网 pong。备选：容器 DNS 名 `nginx-limit-alist:5246`
|
||||
- 同场加映：traefik `rtr-alist-login` 把 GET /login 劫持到 n8n(:5678)，n8n 无此路由 → 登录页挂起 499。已删路由 + 孤儿 middleware `mw-webhook-user-rewrite`（备份 dynamic.yml.bak-20260825）
|
||||
- **n8n 执行管道停摆（08-23 报告 → 08-24 结案，误报）**
|
||||
- 现象：webhook 全 200 但 executions 表零记录、schedule 无痕迹
|
||||
- 根因：`EXECUTIONS_DATA_SAVE_ON_SUCCESS=none` → 成功执行从不落库，只有 error 会记（期间唯一入库记录恰是一条无关备份 workflow 的 error，完全吻合）
|
||||
- 时间线：最后 success 记录 08-22 23:00:51Z ≈ 容器重建（08-23 01:21Z Started）前后；激活错位（activeVersionId）已在重建后修复
|
||||
- 若要恢复可见性：env 改 `all` / `all-with-logs`（EXECUTIONS_DATA_PRUNE 已配，不会无限涨）
|
||||
- 另注：DSM IP 已漂移为 `192.168.137.2`；Windows 直连内网端口（AList 5244 / NocoDB 10380 / n8n 5678）均通
|
||||
|
||||
1. **reimport W1 到 n8n 并做真实验收**（W2/W3 也重新生成过，可一并覆盖）：
|
||||
- 改 `Customers.status` 一次 → 只产生**一次**真正 W1 execution（`active_provision` 或 `deactivate`），其后 echo 全是 `skip_writeback_echo`
|
||||
- 验证 W1 execution 的 `desiredPaths` 非空、AList role `permission_scopes` 落对
|
||||
2. **(Step 4) W4 Full Reconciliation**：NocoDB desired vs AList actual 全量 drift 修复 + reconciliation 日志（W3 已具备基础，W4 补每日全量 + 日志）
|
||||
3. **(待设计) AccessGrants 物化**：当前 W2/W3 不写 AccessGrants 表，不作为 source-of-truth。W4 时统一设计 reconciliation/materialization 策略
|
||||
4. 密码回写问题（客户自改 AList 密码 → NocoDB plaintext 失真）：当前不强制回写（用户已选方案 C），必要时再议
|
||||
5. 本地 git 已留档并推 Gitea（`git.hoelee.com/hoelee/alist-serve-resource`，凭据已脱敏）
|
||||
## ⚠️ 坑速览（一行版；详情看 skill `alist-serve-resource` Pitfalls + API_INTEGRATION.md）
|
||||
- AList：空 role 建用户必 400（先建 role）｜base_path 必须 `/` 否则双重前缀全 403｜delete 用 `?id=`｜裸 token 无 Bearer
|
||||
- NocoDB：link 查询必须 `(link.Id,eq,N)`｜写 bt 链接要写 FK 列 `nc_jitu___*_id`｜删行 DELETE+body｜v1 列端点已废｜SingleSelect 先建空列再 PATCH
|
||||
- n8n：Code 节点 `$env` 可用、HTTP 表达式不行（Load Env 模式）｜API 导入节点 id 必须 UUID 且顶层无 pinData｜webhook 节点缺 webhookId 则生产 URL 不注册｜激活真标志=DB 的 activeVersionId｜执行消失假象=SAVE_ON_SUCCESS
|
||||
- 环境：DSM IP 会漂移（连不上先探测，08-25 起在 192.168.137.2）｜DSM sshd 无 SFTP（POST body 用 stdin pipe 传）｜**容器→宿主 LAN IP 被 hairpin 丢包，回宿主机服务用 172.16.0.1 或容器 DNS 名**
|
||||
|
||||
> ⚠️ **Status 语义已定稿（不再待决策）**：5 个值 `pending/active/suspended/expired/deleted`，
|
||||
> account-level 整户开关，**不进** permission calc。详见 `docs/ARCHITECTURE.md`、`docs/PERMISSIONS.md`、`docs/DATABASE_SCHEMA.md`。
|
||||
> **Status 语义已定稿**：`pending/active/suspended/expired/deleted`，account-level 整户开关，
|
||||
> **不进** permission calc。详见 `docs/ARCHITECTURE.md`、`docs/PERMISSIONS.md`、`docs/DATABASE_SCHEMA.md`。
|
||||
|
||||
## 🐛 已沉淀的坑（勿再踩）
|
||||
- **AList**：
|
||||
- `user/create` 空 role 数组 → 400 "admin or guest"（源码 `req.Role==0 → 默认 guest` + `IsGuest()` 检查）；先建 role 拿 id
|
||||
- `role/create`/`update` 返回 `data:null`；create 后要 **list 按 name 定位 id**
|
||||
- delete 用 `?id=` query param，不是 body
|
||||
- role/list 返回 `data.content[]` + `data.total`（PageResp）
|
||||
- GUEST=1 / ADMIN=2 / GENERAL=0；用户级 bit14=16384 = Check path limit
|
||||
- **多文件夹授权正确做法（实测确认）**：`base_path="/"` + role scope 用**绝对路径**。base_path 设成
|
||||
`/Class` 会导致 scope 绝对路径被 JoinPath 双重前缀 → 全部 403。见 API_INTEGRATION §9.8。
|
||||
- 源码位置：`internal/op/user.go` CreateUser / `server/handles/role.go` / `internal/model/roles.go`
|
||||
- **NocoDB**（`nocodb-selfhosted` skill 已更新）：
|
||||
- SingleSelect 不能建表内联选项；要先建空列再 PATCH `{"colOptions":{options},"dtxp":"a,b,c"}`
|
||||
- Link 字段用 `type:"bt"`（belongs-to）建，`hm` 会把列名搞乱
|
||||
- **Link 字段 where 查询**：`(linkField,eq,id)` 匹配的是 display name，永远返回 0 条。必须用 `(linkField.Id,eq,id)` 子字段语法或外键列 `nc_jitu___*_id`。已验证：`(customer,eq,5)`→0 条；`(customer.Id,eq,5)`→对。
|
||||
- **Link API 返回结构**：`/links/{col}/records/{id}` 只返回 `{Id,name}`（display 列），不含业务字段。如需 alist_path 等字段，要按 Id 反查全表
|
||||
- base/table/column ID 全在 DATABASE_SCHEMA.md
|
||||
- **n8n env 访问（v1.123.72）**（memory 已记）：
|
||||
- Code 节点内 `$env` ✅（**用户实测可用**，如 `$env.NOCODB_URL`）
|
||||
- HTTP 节点表达式字段里 `$env` / `process.env` ❌ 都不行
|
||||
- 模式：step 2 放 "Load Env" Code 节点输出全部 env **并透传 payload 字段**，下游 HTTP 用 `$('Load Env').item.json.VAR`
|
||||
- `gen_w*.js` 里的 Load Env 节点已含所有表 env + `previousRow`/`newRow` 透传（W1）；直接看 `n8n/gen_w1.js` 最新版为准
|
||||
|
||||
- **Webhook write-back loop（2026-08-23 已修）**：
|
||||
- 症状：n8n 写回 NocoDB（sync_status 等）→ 触发 webhook → 又写回 → 无限循环；且 `sync_status==='synced'` 做 guard 不可靠（suspend 分支也写 synced）
|
||||
- 正确做法：**old/new field comparison**。NocoDB v2 webhook payload `data.rows`（改后）+ `data.previous_rows`（改前）。只有业务字段（status/username/password/display_name/email/notes）变化才是 admin 变更；只动了 alist_user_id/alist_role_id/sync_*/last_sync* = echo → 短路
|
||||
- 加 `sync_required`（Checkbox）作为强制重算开关：admin 需要强制 provision 时勾选；写回时始终清 false
|
||||
- **writeBack 值比对守卫**：PATCH 前先 GET 当前值，只写真会变的字段；值没变 → 不发 PATCH → 不触发 webhook（物理层面掐断 loop）
|
||||
- extract-id 节点必须透传 `previousRow`/`newRow`（经 Load Env 转发）到 lifecycle Code 节点
|
||||
|
||||
## 📁 项目结构
|
||||
## 项目结构
|
||||
```
|
||||
alist-serve-resource/
|
||||
├── AGENTS.md ← 🤖 AI 会话入口（任务路由 + 铁律，开工先读）
|
||||
├── SECRETS.md ← 🔑 唯一凭据文件（外发 = 删除此文件）
|
||||
├── docker-compose.yml / .env.example / .gitignore
|
||||
├── n8n/
|
||||
│ W1/W2/W3-*.json (generated) · w1/w2/w3-body.js (runtime logic)
|
||||
│ shared/effective-grants.js (single source of truth)
|
||||
│ gen_w1/w2/w3.js + inline_shared.js (build-time inlining)
|
||||
│ test_effective_grants.js · test_w1_lifecycle.js · test_w3_refactored.js
|
||||
│ SETUP.md · W2-DESIGN.md · W3-DESIGN.md
|
||||
├── scripts/ create_nocodb_schema.py · healthcheck.sh
|
||||
└── docs/ API_INTEGRATION.md · ARCHITECTURE.md · PERMISSIONS.md · ADMIN_GUIDE.md · DATABASE_SCHEMA.md · N8N_WORKFLOWS.md · INSTALL.md · CONFIGURATION.md · SECURITY.md · BACKUP_RESTORE.md · UPGRADE.md · TROUBLESHOOTING.md · PROJECT_STATE.md
|
||||
│ ├── W1..W4-*.json (生成物，勿手编) · w1..w4-body.js (运行时逻辑源头)
|
||||
│ ├── shared/effective-grants.js (算法单一事实源) · gen_w1..w4.js + inline_shared.js (构建)
|
||||
│ ├── test_*.js · SETUP.md · W2/W3/W4-DESIGN.md
|
||||
├── scripts/ create_nocodb_schema.py · add_reconcilelogs_table.py · live_dry_run_w4.js · healthcheck.sh
|
||||
└── docs/ API_INTEGRATION · ARCHITECTURE · PERMISSIONS · ADMIN_GUIDE · DATABASE_SCHEMA ·
|
||||
N8N_WORKFLOWS · PROJECT_STATE(本文件) · INSTALL · CONFIGURATION · SECURITY ·
|
||||
BACKUP_RESTORE · UPGRADE · TROUBLESHOOTING · W4-DESIGN
|
||||
```
|
||||
|
||||
## 待确认的设计点
|
||||
1. **AccessGrants 物化**（已定：本次不物化，不作为 source-of-truth）—— W4 时统一设计 reconciliation/materialization 策略，需避免 AccessGrant→n8n loop
|
||||
2. 密码：客户可自改 AList 密码 → NocoDB plaintext 会失真；当前不强制回写（用户已选方案 C）
|
||||
3. 对账/到期只靠 n8n 定时扫描（AList 无 expiry 概念）—— 已确认
|
||||
1. AccessGrants 物化 —— **定案：不物化**（手工人为授权入口，n8n 永不写）
|
||||
2. 密码失真 —— **定案：方案 C**（不强制回写；W4 repair 不推密码，仅 provision 新建时用）
|
||||
3. 对账/到期只靠 n8n 定时扫描（AList 无 expiry 概念）—— 已确认
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# W4 — Daily Full Reconciliation / 全量对账 + drift 修复 + 对账日志
|
||||
|
||||
> 2026-08-23 定稿并实测（本地单元测试 + 直连生产 dry-run + 真实修复跑 + 幂等复跑）。
|
||||
> 与 W3 的差异：W3 每 5 分钟只对账 **scopes**（事件对账的补网）；
|
||||
> W4 每日全量对账 **用户/角色/绑定的完整状态**，修复 W3 不管的 drift，并落 ReconcileLogs 审计。
|
||||
|
||||
## 触发
|
||||
|
||||
- **Schedule Trigger**：每日 03:00（cron `0 3 * * *`）。
|
||||
- **Webhook**（`POST /webhook/alist-w4-reconcile`）手动触发；`?dryRun=true` 或 env `W4_DRY_RUN=true` = 只出计划，零写入。
|
||||
|
||||
## 单向同步铁律（用户定稿）
|
||||
|
||||
- **NocoDB `Customers.status` 是 business source of truth；AList enabled/disabled 只是 enforcement state。**
|
||||
- W4 **永不**反向修改 NocoDB status。管理员在 AList 手工 disable 用户 → W4 按 NocoDB status 修回；绝不反过来。
|
||||
- 密码**不随 repair 推送**（方案 C：NocoDB plaintext 可能旧，推送会把客户自改的 AList 密码覆盖掉）。仅 provision 新建用户时用 NocoDB 密码。
|
||||
|
||||
## Drift 矩阵（自动修复）
|
||||
|
||||
| 场景 | 期望状态 | 处理 |
|
||||
|---|---|---|
|
||||
| **active 客户缺 user/role** | 存在 + enabled + canonical 配置 | 自动 provision：**先按 username 查 AList**（避免 NocoDB 丢 id 时建重）→ 没有才 create（role → user 顺序，空 role 会被 AList 拒） |
|
||||
| **alist_user_id / alist_role_id 悬空**（指向已删记录） | — | username / role name 反查 → 找到即 **adopt**（回写 bookkeeping id，不新建）→ 找不到才 recreate → 重新 sync |
|
||||
| **非 active（pending/suspended/expired/deleted）** | user disabled + scopes 空 | 有 user → disable + 清空 scopes；**pending 从未 provision → 保持无 AList user，不自动建**（纯 NocoDB 标签） |
|
||||
| **user.disabled 漂移** | 与 status 一致 | 按 status 单向修（active→enable，非 active→disable） |
|
||||
| **base_path / permission 漂移** | `/` + 16384 | 强制修回（= W1 已验证的 canonical 配置，不重新发明） |
|
||||
| **role binding 漂移**（user 没绑 role） | user.role = [专属 role id] | rebind |
|
||||
| **scopes 漂移 vs desired** | = calculateEffectiveGrants | 整体重写 role `permission_scopes`（与 W3 同算法） |
|
||||
| **username drift**（NocoDB 名 ≠ AList 名） | — | **detect + log_only**（不假设 rename API，不做 destructive migration） |
|
||||
| **客户行缺 username** | — | reconcile_error 记日志 + summary.errors |
|
||||
|
||||
## ReconcileLogs 表（新建，2026-08-23）
|
||||
|
||||
- 表 id `mktsb7le6fqr2u1`；列：`run_id` / `customer`(bt→Customers) / `drift_type` / `details` / `action` / `result` / `timestamp`
|
||||
- **只记 drift / repair / error 行**——健康客户不产生任何行（不为每天的全量跑刷日志）。
|
||||
- drift_type 选项：user_missing / role_missing / dangling_user_ref / dangling_role_ref /
|
||||
user_disabled_drift / scopes_drift / base_path_drift / permission_drift /
|
||||
role_binding_drift / username_drift / verify_failed / reconcile_error
|
||||
- result 选项：repaired / log_only / failed / skipped
|
||||
- ⚠ **写 customer 链接必须用 FK 列 `nc_jitu___Customers_id`**（v2 records API 写 link 列本身：标量/数组/对象全被忽略，实测确认），env `NOCODB_FK_RECONCILELOG_CUSTOMER` 可覆盖。
|
||||
|
||||
## 修复后必须 verify
|
||||
|
||||
每个 customer 修复后 **GET 回读 AList user/role 实测状态**，与期望比对；
|
||||
不通过 → 记 `verify_failed` 行（result=failed，不计入 repaired）。
|
||||
dry-run 不 verify（没写入可验）。
|
||||
|
||||
## 输出 summary
|
||||
|
||||
```
|
||||
{ runId, dryRun, at, customersScanned, driftsDetected, repaired, skipped,
|
||||
errors: [{customerId, username, error}], details: [{customerId, ..., drifts[]}],
|
||||
logRowsWritten, plannedLogRows (dry-run), logWriteSkipped (env 缺失时) }
|
||||
```
|
||||
|
||||
## 与 W1/W2/W3 的交互
|
||||
|
||||
- 写回 Customers 只有 `alist_user_id` / `alist_role_id`（adopt 时）→ W1 的 old/new 比对视为 echo → 短路，无 loop。
|
||||
- 不写 `sync_status` / `sync_required` / `last_sync`。
|
||||
- W4 不碰 AccessGrants（保持原定位：手工人为授权入口，源 b）。
|
||||
- AccessGrants/CustomerProducts/Customers 无 webhook 指向 W4，写 ReconcileLogs 无 loop。
|
||||
|
||||
## 本地实跑（绕过 n8n，直连生产）
|
||||
|
||||
`NOCODB_TOKEN=... ALIST_ADMIN_PASSWORD=... node scripts/live_dry_run_w4.js`（dry-run）
|
||||
与 `scripts/run_real_w4.js`（本机脚本版 run_real_w4.js 在 hermes scripts/，凭据走 env）。
|
||||
2026-08-23 实测记录：
|
||||
1. dry-run：5 客户，3 drift（TEST-Alice: dangling_role_ref + base_path_drift + role_binding_drift），1 error（Id 7 缺 username），3 skip，零写入 ✅
|
||||
2. 真实跑：3 repaired + 4 行 ReconcileLogs（FK 正确），verify 全过 ✅
|
||||
3. 复跑：0 drift / 0 repair / 4 skip / 仅 Id-7 error 照记 → 幂等 ✅
|
||||
|
||||
## 坑（已沉淀，勿再踩）
|
||||
|
||||
- n8n API 导入校验严格：**节点 id 必须 UUID、顶层不能有 `pinData`**，否则 400 "additional properties"。
|
||||
- n8n webhook 节点必须带 `webhookId`，否则生产 URL 永不注册（404 "not registered"）。
|
||||
- n8n 1.123 API 激活返回 `active:true` 但运行时激活可能被权限拒绝（日志 "attempted to activate a workflow without permissions"）→ 激活状态在 DB 是 `workflow_entity.activeVersionId`（不是 active 列），boot 时加载 active + activeVersionId 的工作流注册 webhook/cron。
|
||||
- NocoDB 2026.08.x：v1 column GET/PATCH 端点已删除；列 id 从 `GET /api/v2/meta/tables/{id}` 拿；v2 的列 create/PATCH 响应返回的是**表对象**不是列。
|
||||
- NocoDB v2 行删除：`DELETE /records` + body `[{Id: N}, ...]`（路径 /records/{id} 和 where 参数都不行）。
|
||||
@@ -43,6 +43,8 @@ NOCODB_TABLE_PRODUCTS=molyb3pxv81l75g
|
||||
NOCODB_TABLE_RESOURCES=m6cm7tkxzd8m18c
|
||||
NOCODB_TABLE_CUSTOMERPRODUCTS=mvyqbv8ao7u6bct
|
||||
NOCODB_TABLE_ACCESSGRANTS=mxdx3yhfcyej3ao
|
||||
NOCODB_TABLE_RECONCILELOGS=mktsb7le6fqr2u1
|
||||
NOCODB_FK_RECONCILELOG_CUSTOMER=nc_jitu___Customers_id
|
||||
NOCODB_LINK_PROD_RESOURCES=c4ko4x8kwi16swg
|
||||
ALIST_URL=http://alist-serve-resource:5244
|
||||
ALIST_ADMIN_USERNAME=hoelee
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,28 @@
|
||||
// Deploy regenerated W1..W4 workflows to live n8n via public API.
|
||||
// Usage: N8N_API_KEY=<key> N8N_BASE=http://192.168.137.2:5678 node deploy_w1_w3.mjs
|
||||
// Key lives in repo-root SECRETS.md (never hardcode here).
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
const N8N = process.env.N8N_BASE || "http://192.168.137.2:5678";
|
||||
const KEY = process.env.N8N_API_KEY;
|
||||
if (!KEY) { console.error("N8N_API_KEY env required (see SECRETS.md)"); process.exit(1); }
|
||||
const H = { "X-N8N-API-KEY": KEY, "Content-Type": "application/json" };
|
||||
|
||||
const TARGETS = [
|
||||
{ file: "W1-customer-provision.json", id: "gU2ee9Eg5w1KzB5p" },
|
||||
{ file: "W3-expiry-sync.json", id: "zlEfjj5eK5IfILtk" },
|
||||
];
|
||||
|
||||
for (const t of TARGETS) {
|
||||
const wf = JSON.parse(await readFile(t.file, "utf-8"));
|
||||
delete wf.pinData;
|
||||
delete wf.active;
|
||||
delete wf.id;
|
||||
const r = await fetch(`${N8N}/api/v1/workflows/${t.id}`, {
|
||||
method: "PUT", headers: H, body: JSON.stringify(wf),
|
||||
});
|
||||
const txt = await r.text();
|
||||
console.log(t.file, "PUT", r.status, txt.slice(0, 120));
|
||||
if (r.status !== 200) process.exit(1);
|
||||
}
|
||||
console.log("DEPLOY OK — remember to recheck workflow_entity.activeVersionId alignment");
|
||||
@@ -0,0 +1,23 @@
|
||||
// Deploy regenerated W5 (v30, adds paths) to live n8n via public API.
|
||||
// Usage: N8N_API_KEY=... node deploy_w5.mjs
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
const N8N = process.env.N8N_BASE || "http://192.168.137.2:5678";
|
||||
const KEY = process.env.N8N_API_KEY;
|
||||
if (!KEY) { console.error("N8N_API_KEY env required"); process.exit(1); }
|
||||
const H = { "X-N8N-API-KEY": KEY, "Content-Type": "application/json" };
|
||||
|
||||
const wf = JSON.parse(await readFile("W5-purchase-check.json", "utf-8"));
|
||||
delete wf.pinData;
|
||||
delete wf.active;
|
||||
delete wf.id;
|
||||
|
||||
// W5 workflow id (live) = MNC0BZc1FqCKFwIV
|
||||
const ID = "MNC0BZc1FqCKFwIV";
|
||||
const r = await fetch(`${N8N}/api/v1/workflows/${ID}`, {
|
||||
method: "PUT", headers: H, body: JSON.stringify(wf),
|
||||
});
|
||||
const txt = await r.text();
|
||||
console.log("W5 PUT", r.status, txt.slice(0, 300));
|
||||
if (r.status !== 200) process.exit(1);
|
||||
console.log("DEPLOY OK");
|
||||
@@ -0,0 +1,62 @@
|
||||
// Builds n8n/W4-daily-reconcile.json — the daily full reconciliation workflow.
|
||||
// Single source of truth: shared/effective-grants.js (inlined) + w4-body.js.
|
||||
// Run: node n8n/gen_w4.js
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { sharedSource } = require('./inline_shared');
|
||||
|
||||
let body = fs.readFileSync(path.join(__dirname, 'w4-body.js'), 'utf8');
|
||||
body = body.replace('// __SHARED_INLINE__ (replaced by shared/effective-grants.js source at build)', sharedSource());
|
||||
|
||||
// n8n API import (POST /api/v1/workflows) strictly validates: node ids must be
|
||||
// UUIDs and top-level must NOT carry pinData (class-validator whitelist).
|
||||
const uuid = () => require('crypto').randomUUID();
|
||||
const node = (o) => Object.assign({ id: uuid() }, o);
|
||||
|
||||
const workflow = {
|
||||
name: 'W4 - Daily Full Reconciliation (AList Drift Repair)',
|
||||
nodes: [
|
||||
node({
|
||||
parameters: {
|
||||
rule: {
|
||||
interval: [{ field: 'cronExpression', expression: '0 3 * * *' }]
|
||||
}
|
||||
},
|
||||
name: 'Daily 03:00',
|
||||
type: 'n8n-nodes-base.scheduleTrigger',
|
||||
typeVersion: 1.2,
|
||||
position: [-420, 0]
|
||||
}),
|
||||
node({
|
||||
parameters: {
|
||||
httpMethod: 'POST',
|
||||
path: 'alist-w4-reconcile',
|
||||
options: {}
|
||||
},
|
||||
name: 'Manual / Webhook',
|
||||
type: 'n8n-nodes-base.webhook',
|
||||
typeVersion: 2,
|
||||
webhookId: 'alist-w4-reconcile', // required: without it the production URL never registers
|
||||
position: [-420, 240]
|
||||
}),
|
||||
node({
|
||||
parameters: { jsCode: body },
|
||||
name: 'Reconcile & Repair (Daily Full)',
|
||||
type: 'n8n-nodes-base.code',
|
||||
typeVersion: 2,
|
||||
position: [0, 120]
|
||||
})
|
||||
],
|
||||
connections: {
|
||||
'Daily 03:00': {
|
||||
main: [[{ node: 'Reconcile & Repair (Daily Full)', type: 'main', index: 0 }]]
|
||||
},
|
||||
'Manual / Webhook': {
|
||||
main: [[{ node: 'Reconcile & Repair (Daily Full)', type: 'main', index: 0 }]]
|
||||
}
|
||||
},
|
||||
settings: { executionOrder: 'v1' }
|
||||
};
|
||||
|
||||
fs.writeFileSync(path.join(__dirname, 'W4-daily-reconcile.json'), JSON.stringify(workflow, null, 2) + '\n');
|
||||
console.log('wrote W4-daily-reconcile.json (' + body.length + ' chars body)');
|
||||
@@ -0,0 +1,57 @@
|
||||
// Builds n8n/W5-purchase-check.json — public footer purchase/expiry webhook.
|
||||
// Trigger: GET /webhook/dk-purchase-check?u=<username>
|
||||
// Response (JSON, CORS: https://drive.digikedai.com):
|
||||
// { username, known, purchases: [{product, expires_at, display_state}] }
|
||||
// No HMAC (key would ship client-side = theatre). Access control = CORS +
|
||||
// Cloudflare WAF rate-limit/Origin rule (documented in delivery guide).
|
||||
// Run: node n8n/gen_w5.js
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const body = fs.readFileSync(path.join(__dirname, 'w5-body.js'), 'utf8');
|
||||
|
||||
const workflow = {
|
||||
name: 'W5 - Footer Purchase Check (Public)',
|
||||
nodes: [
|
||||
{
|
||||
parameters: {
|
||||
httpMethod: 'GET',
|
||||
path: 'dk-purchase-check',
|
||||
responseMode: 'lastNode',
|
||||
options: {
|
||||
responseCode: 200,
|
||||
responseHeaders: {
|
||||
entries: [
|
||||
{ name: 'Access-Control-Allow-Origin', value: 'https://drive.digikedai.com' },
|
||||
{ name: 'Content-Type', value: 'application/json' }
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
id: 'trigger-webhook',
|
||||
name: 'Footer Purchase Check (Webhook)',
|
||||
type: 'n8n-nodes-base.webhook',
|
||||
typeVersion: 2,
|
||||
position: [0, 0],
|
||||
webhookId: 'dk-purchase-check'
|
||||
},
|
||||
{
|
||||
parameters: { jsCode: body },
|
||||
id: 'lookup',
|
||||
name: 'Lookup Purchases',
|
||||
type: 'n8n-nodes-base.code',
|
||||
typeVersion: 2,
|
||||
position: [220, 0]
|
||||
}
|
||||
],
|
||||
connections: {
|
||||
'Footer Purchase Check (Webhook)': {
|
||||
main: [[{ node: 'Lookup Purchases', type: 'main', index: 0 }]]
|
||||
}
|
||||
},
|
||||
settings: { executionOrder: 'v1' },
|
||||
pinData: {}
|
||||
};
|
||||
|
||||
fs.writeFileSync(path.join(__dirname, 'W5-purchase-check.json'), JSON.stringify(workflow, null, 2) + '\n');
|
||||
console.log('wrote W5-purchase-check.json (' + body.length + ' chars body)');
|
||||
@@ -0,0 +1,369 @@
|
||||
// Verify the W4 daily reconciliation body against a mock AList + NocoDB.
|
||||
// Runs the ACTUAL inlined code from W4-daily-reconcile.json in a vm sandbox
|
||||
// with realistic state mutations (create/update actually change the mock so
|
||||
// verify-after-repair reads back real state).
|
||||
// Run: node n8n/test_w4_reconcile.js
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const workflow = JSON.parse(fs.readFileSync(path.join(__dirname, 'W4-daily-reconcile.json'), 'utf8'));
|
||||
const code = workflow.nodes.find(n => n.name === 'Reconcile & Repair (Daily Full)').parameters.jsCode;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// mock world
|
||||
// ---------------------------------------------------------------------------
|
||||
function freshWorld(w) {
|
||||
return {
|
||||
customers: JSON.parse(JSON.stringify(w.customers || [])),
|
||||
users: JSON.parse(JSON.stringify(w.users || [])),
|
||||
roles: JSON.parse(JSON.stringify(w.roles || [])),
|
||||
cps: JSON.parse(JSON.stringify(w.cps || {})),
|
||||
grants: JSON.parse(JSON.stringify(w.grants || {})),
|
||||
productRes: JSON.parse(JSON.stringify(w.productRes || {})),
|
||||
resources: JSON.parse(JSON.stringify(w.resources || [
|
||||
{ Id: 1, alist_path: '/Class/A' }, { Id: 2, alist_path: '/Class/B' },
|
||||
{ Id: 3, alist_path: '/Class/D' }, { Id: 4, alist_path: '/Template/Music' },
|
||||
])),
|
||||
};
|
||||
}
|
||||
|
||||
let nextUserId = 1000, nextRoleId = 900;
|
||||
|
||||
async function runWorld(world, extraEnv = {}) {
|
||||
const S = freshWorld(world);
|
||||
const logRows = [];
|
||||
const userUpdates = [], roleUpdates = [], patches = [], creates = { users: [], roles: [] };
|
||||
const throwOn = {}; // url-substring -> Error
|
||||
|
||||
const httpRequest = async (opts) => {
|
||||
const u = opts.url, m = (opts.method || 'GET').toUpperCase();
|
||||
for (const k of Object.keys(throwOn)) if (u.includes(k)) throw new Error('mock throw: ' + k);
|
||||
const body = opts.body;
|
||||
|
||||
// ---- NocoDB ----
|
||||
if (m === 'GET' && u.includes(`/tc/records?limit=10000`)) return { list: S.customers };
|
||||
if (m === 'GET' && u.includes(`/tr/records?limit=10000`)) return { list: S.resources };
|
||||
const cm = u.match(/\/tc\/records\/(\d+)$/);
|
||||
if (m === 'GET' && cm) return S.customers.find(c => c.Id === Number(cm[1])) || null;
|
||||
if (m === 'PATCH' && u.includes('/tc/records')) {
|
||||
for (const row of body) {
|
||||
const c = S.customers.find(x => x.Id === row.Id);
|
||||
if (c) Object.assign(c, row);
|
||||
patches.push(row);
|
||||
}
|
||||
return {};
|
||||
}
|
||||
const cpm = u.match(/\/tcp\/records\?where=\(customer\.Id,eq,(\d+)\)/);
|
||||
if (m === 'GET' && cpm) return { list: S.cps[Number(cpm[1])] || [] };
|
||||
const gm = u.match(/\/tag\/records\?where=\(customer\.Id,eq,(\d+)\)/);
|
||||
if (m === 'GET' && gm) return { list: S.grants[Number(gm[1])] || [] };
|
||||
const pm = u.match(/\/tp\/links\/lnk\/records\/(\d+)/);
|
||||
if (m === 'GET' && pm) return { list: S.productRes[Number(pm[1])] || [] };
|
||||
if (m === 'GET' && u.includes('/tlog/records')) return { list: logRows };
|
||||
if (m === 'POST' && u.includes('/tlog/records')) { (Array.isArray(body) ? body : [body]).forEach(r => logRows.push(r)); return {}; }
|
||||
|
||||
// ---- AList auth ----
|
||||
if (u.endsWith('/api/auth/login')) return { data: { token: 'ADMIN_TOKEN' } };
|
||||
|
||||
// ---- AList users ----
|
||||
if (m === 'GET' && u.includes('/api/admin/user/list')) return { data: { content: S.users } };
|
||||
const ug = u.match(/\/api\/admin\/user\/get\?id=(\d+)/);
|
||||
if (m === 'GET' && ug) {
|
||||
const x = S.users.find(uu => uu.id === Number(ug[1]));
|
||||
return x ? { data: x } : { code: 404, data: null };
|
||||
}
|
||||
if (m === 'POST' && u.endsWith('/api/admin/user/create')) {
|
||||
const nu = Object.assign({ id: nextUserId++, disabled: false }, body); // real AList user/get always carries disabled
|
||||
S.users.push(nu); creates.users.push(body);
|
||||
return { code: 200, data: null };
|
||||
}
|
||||
if (m === 'POST' && u.endsWith('/api/admin/user/update')) {
|
||||
const x = S.users.find(uu => uu.id === body.id);
|
||||
if (x) {
|
||||
userUpdates.push(body);
|
||||
if (!(body.id === 400)) Object.assign(x, body); // 400 simulates AList refusing
|
||||
}
|
||||
return { code: 200, data: null };
|
||||
}
|
||||
|
||||
// ---- AList roles ----
|
||||
if (m === 'GET' && u.includes('/api/admin/role/list')) return { data: { content: S.roles } };
|
||||
const rg = u.match(/\/api\/admin\/role\/get\?id=(\d+)/);
|
||||
if (m === 'GET' && rg) {
|
||||
const x = S.roles.find(r => r.id === Number(rg[1]));
|
||||
return x ? { data: x } : { code: 404, data: null };
|
||||
}
|
||||
if (m === 'POST' && u.endsWith('/api/admin/role/create')) {
|
||||
const nr = Object.assign({ id: nextRoleId++, permission_scopes: [] }, body);
|
||||
S.roles.push(nr); creates.roles.push(body);
|
||||
return { code: 200, data: null };
|
||||
}
|
||||
if (m === 'POST' && u.endsWith('/api/admin/role/update')) {
|
||||
const x = S.roles.find(r => r.id === body.id);
|
||||
roleUpdates.push(body);
|
||||
if (x) Object.assign(x, body);
|
||||
return { code: 200, data: null };
|
||||
}
|
||||
throw new Error('unexpected request: ' + m + ' ' + u);
|
||||
};
|
||||
|
||||
const sandbox = {
|
||||
$env: Object.assign({
|
||||
NOCODB_URL: 'https://nc', NOCODB_TOKEN: 't',
|
||||
NOCODB_TABLE_CUSTOMERS: 'tc', NOCODB_TABLE_CUSTOMERPRODUCTS: 'tcp',
|
||||
NOCODB_TABLE_PRODUCTS: 'tp', NOCODB_TABLE_RESOURCES: 'tr',
|
||||
NOCODB_TABLE_ACCESSGRANTS: 'tag', NOCODB_TABLE_RECONCILELOGS: 'tlog',
|
||||
NOCODB_LINK_PROD_RESOURCES: 'lnk',
|
||||
NOCODB_FK_RECONCILELOG_CUSTOMER: 'nc_jitu___Customers_id',
|
||||
ALIST_URL: 'https://al', ALIST_ADMIN_USERNAME: 'admin', ALIST_ADMIN_PASSWORD: 'p',
|
||||
W4_DRY_RUN: 'false',
|
||||
}, extraEnv),
|
||||
$json: {},
|
||||
console,
|
||||
};
|
||||
sandbox.helpers = { httpRequest };
|
||||
|
||||
const wrapped = `(async function(){\n${code}\n})`;
|
||||
const fn = vm.runInNewContext(wrapped, sandbox, { filename: 'w4-body.js' });
|
||||
const result = await fn();
|
||||
return {
|
||||
s: result.json, users: S.users, roles: S.roles, customers: S.customers,
|
||||
logRows, userUpdates, roleUpdates, patches, creates,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// worlds
|
||||
// ---------------------------------------------------------------------------
|
||||
const W = {
|
||||
healthy: {
|
||||
customers: [{ Id: 1, username: 'alice', status: 'active', alist_user_id: 100, alist_role_id: 50 }],
|
||||
users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }],
|
||||
roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 1: [{ Id: 11, customer: 1, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 1: [] },
|
||||
},
|
||||
adoptUser: { // alist_user_id lost in NocoDB but user exists by username
|
||||
customers: [{ Id: 2, username: 'alice', status: 'active', alist_user_id: null, alist_role_id: 50 }],
|
||||
users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }],
|
||||
roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 2: [{ Id: 12, customer: 2, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 2: [] },
|
||||
},
|
||||
danglingUser: { // alist_user_id points at deleted user
|
||||
customers: [{ Id: 3, username: 'alice', status: 'active', alist_user_id: 999, alist_role_id: 50 }],
|
||||
users: [{ id: 100, username: 'alice', base_path: '/', permission: 16384, disabled: false, role: [50] }],
|
||||
roles: [{ id: 50, name: 'customer-alice', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 3: [{ Id: 13, customer: 3, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 3: [] },
|
||||
},
|
||||
provision: { // nothing in AList, active customer with entitlement
|
||||
customers: [{ Id: 4, username: 'bob', status: 'active', alist_user_id: null, alist_role_id: null, password: 'pwBob1' }],
|
||||
users: [],
|
||||
roles: [],
|
||||
cps: { 4: [{ Id: 14, customer: 4, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 2 }] },
|
||||
grants: { 4: [] },
|
||||
},
|
||||
reEnable: { // disabled + wrong base_path/permission
|
||||
customers: [{ Id: 5, username: 'carol', status: 'active', alist_user_id: 101, alist_role_id: 51 }],
|
||||
users: [{ id: 101, username: 'carol', base_path: '/Class', permission: 0, disabled: true, role: [51] }],
|
||||
roles: [{ id: 51, name: 'customer-carol', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 5: [{ Id: 15, customer: 5, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 5: [] },
|
||||
},
|
||||
scopesDrift: { // stale scope in AList
|
||||
customers: [{ Id: 6, username: 'dave', status: 'active', alist_user_id: 102, alist_role_id: 52 }],
|
||||
users: [{ id: 102, username: 'dave', base_path: '/', permission: 16384, disabled: false, role: [52] }],
|
||||
roles: [{ id: 52, name: 'customer-dave', description: 'x', permission_scopes: [{ path: '/Old/Path', permission: 0 }] }],
|
||||
cps: { 6: [{ Id: 16, customer: 6, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 3 }] },
|
||||
grants: { 6: [] },
|
||||
},
|
||||
expired: { // non-active customer: disable + clear scopes
|
||||
customers: [{ Id: 7, username: 'eve', status: 'expired', alist_user_id: 103, alist_role_id: 53 }],
|
||||
users: [{ id: 103, username: 'eve', base_path: '/', permission: 16384, disabled: false, role: [53] }],
|
||||
roles: [{ id: 53, name: 'customer-eve', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 7: [] },
|
||||
grants: { 7: [] },
|
||||
},
|
||||
pendingNoUser: {
|
||||
customers: [{ Id: 8, username: 'frank', status: 'pending', alist_user_id: null, alist_role_id: null }],
|
||||
users: [], roles: [],
|
||||
cps: { 8: [] }, grants: { 8: [] },
|
||||
},
|
||||
usernameDrift: {
|
||||
customers: [{ Id: 9, username: 'grace', status: 'active', alist_user_id: 104, alist_role_id: 54 }],
|
||||
users: [{ id: 104, username: 'grace_old', base_path: '/', permission: 16384, disabled: false, role: [54] }],
|
||||
roles: [{ id: 54, name: 'customer-grace', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 9: [{ Id: 17, customer: 9, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 9: [] },
|
||||
},
|
||||
verifyFail: { // AList refuses the user update -> verify_failed
|
||||
customers: [{ Id: 10, username: 'ivy', status: 'active', alist_user_id: 400, alist_role_id: 56 }],
|
||||
users: [{ id: 400, username: 'ivy', base_path: '/', permission: 16384, disabled: true, role: [56] }],
|
||||
roles: [{ id: 56, name: 'customer-ivy', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 10: [{ Id: 18, customer: 10, product: 2, status: 'active', expires_at: '2099-01-01' }] },
|
||||
productRes: { 2: [{ Id: 1 }] },
|
||||
grants: { 10: [] },
|
||||
},
|
||||
error: { // grants fetch explodes for this customer
|
||||
customers: [{ Id: 11, username: 'hank', status: 'active', alist_user_id: 105, alist_role_id: 57 }],
|
||||
users: [{ id: 105, username: 'hank', base_path: '/', permission: 16384, disabled: false, role: [57] }],
|
||||
roles: [{ id: 57, name: 'customer-hank', description: 'x', permission_scopes: [{ path: '/Class/A', permission: 0 }] }],
|
||||
cps: { 11: [] },
|
||||
grants: { 11: [] },
|
||||
},
|
||||
};
|
||||
|
||||
async function main() {
|
||||
let fail = 0;
|
||||
const A = (n, c, d) => { if (c) console.log('PASS ' + n); else { fail++; console.log('FAIL ' + n + ' -> ' + JSON.stringify(d)); } };
|
||||
const E = { W4_DRY_RUN: 'false' };
|
||||
|
||||
// -- healthy: nothing to do
|
||||
{
|
||||
const r = await runWorld(W.healthy, E);
|
||||
A('healthy skipped=1', r.s.skipped === 1, r.s);
|
||||
A('healthy no log rows', r.logRows.length === 0, r.logRows);
|
||||
A('healthy no writes', r.userUpdates.length === 0 && r.roleUpdates.length === 0, r);
|
||||
A('healthy 0 drift', r.s.driftsDetected === 0, r.s);
|
||||
}
|
||||
|
||||
// -- adopt user by username (id lost)
|
||||
{
|
||||
const r = await runWorld(W.adoptUser, E);
|
||||
A('adopt: 1 drift / 1 repaired', r.s.driftsDetected === 1 && r.s.repaired === 1, r.s);
|
||||
A('adopt: dangling_user_ref log', r.logRows[0] && r.logRows[0].drift_type === 'dangling_user_ref' && r.logRows[0].result === 'repaired', r.logRows);
|
||||
A('adopt: customer FK written', r.logRows[0] && r.logRows[0].nc_jitu___Customers_id === 2, r.logRows[0]);
|
||||
A('adopt: no duplicate user', r.users.length === 1, r.users);
|
||||
A('adopt: writeBack alist_user_id', r.patches.some(p => p.alist_user_id === 100), r.patches);
|
||||
A('adopt: next run healthy (id filled)', r.customers[0].alist_user_id === 100, r.customers);
|
||||
}
|
||||
|
||||
// -- dangling user ref (404 -> lookup by username -> adopt)
|
||||
{
|
||||
const r = await runWorld(W.danglingUser, E);
|
||||
A('dangling: adopted 100', r.s.driftsDetected === 1 && r.s.repaired === 1, r.s);
|
||||
A('dangling: no user created', r.users.length === 1 && r.creates.users.length === 0, r);
|
||||
A('dangling: customer id repaired', r.customers[0].alist_user_id === 100, r.customers);
|
||||
}
|
||||
|
||||
// -- full provision
|
||||
{
|
||||
const r = await runWorld(W.provision, E);
|
||||
A('provision: role+user created', r.creates.roles.length === 1 && r.creates.users.length === 1, r.creates);
|
||||
A('provision: user role bound', r.users[0].role[0] === r.roles[0].id, r.users);
|
||||
A('provision: canonical fields', r.users[0].base_path === '/' && r.users[0].permission === 16384 && r.users[0].disabled === false, r.users);
|
||||
A('provision: scopes applied', JSON.stringify(r.roles[0].permission_scopes.map(s => s.path)) === JSON.stringify(['/Class/B']), r.roles);
|
||||
A('provision: ids written back', r.patches.some(p => p.alist_user_id) && r.patches.some(p => p.alist_role_id), r.patches);
|
||||
A('provision: user_missing logged', r.logRows.some(l => l.drift_type === 'user_missing'), r.logRows);
|
||||
A('provision: verify passed (no verify_failed)', !r.logRows.some(l => l.drift_type === 'verify_failed'), r.logRows);
|
||||
}
|
||||
|
||||
// -- re-enable + canonical fix
|
||||
{
|
||||
const r = await runWorld(W.reEnable, E);
|
||||
A('re-enable: user updated', r.userUpdates.some(u => u.disabled === false && u.base_path === '/' && u.permission === 16384), r.userUpdates);
|
||||
A('re-enable: 3 drifts logged', ['user_disabled_drift','base_path_drift','permission_drift'].every(t => r.logRows.some(l => l.drift_type === t)), r.logRows);
|
||||
A('re-enable: repaired=3', r.s.repaired === 3, r.s);
|
||||
}
|
||||
|
||||
// -- scopes drift
|
||||
{
|
||||
const r = await runWorld(W.scopesDrift, E);
|
||||
A('scopes: role rewritten', r.roleUpdates.length === 1 && r.roles[0].permission_scopes.map(s => s.path).join() === '/Class/D', r.roleUpdates);
|
||||
A('scopes: drift logged', r.logRows.some(l => l.drift_type === 'scopes_drift' && l.result === 'repaired'), r.logRows);
|
||||
}
|
||||
|
||||
// -- expired cleanup
|
||||
{
|
||||
const r = await runWorld(W.expired, E);
|
||||
A('expired: user disabled', r.userUpdates.some(u => u.disabled === true), r.userUpdates);
|
||||
A('expired: scopes cleared', r.roles[0].permission_scopes.length === 0, r.roles);
|
||||
A('expired: repaired=2', r.s.repaired === 2, r.s);
|
||||
A('expired: verify ok', !r.logRows.some(l => l.drift_type === 'verify_failed'), r.logRows);
|
||||
}
|
||||
|
||||
// -- pending no user: untouched
|
||||
{
|
||||
const r = await runWorld(W.pendingNoUser, E);
|
||||
A('pending: skipped=1', r.s.skipped === 1, r.s);
|
||||
A('pending: no rows / no writes', r.logRows.length === 0 && r.creates.users.length === 0 && r.creates.roles.length === 0, r);
|
||||
}
|
||||
|
||||
// -- username drift: detect + log only
|
||||
{
|
||||
const r = await runWorld(W.usernameDrift, E);
|
||||
A('drift: username_drift logged', r.logRows.some(l => l.drift_type === 'username_drift' && l.result === 'log_only' && l.action === 'log_only'), r.logRows);
|
||||
A('drift: no rename (username untouched)', r.users[0].username === 'grace_old', r.users);
|
||||
A('drift: no new user', r.creates.users.length === 0, r.creates);
|
||||
A('drift: no user patch', r.userUpdates.length === 0, r.userUpdates);
|
||||
}
|
||||
|
||||
// -- verify failed (AList refuses update)
|
||||
{
|
||||
const r = await runWorld(W.verifyFail, E);
|
||||
A('verifyfail: verify_failed logged', r.logRows.some(l => l.drift_type === 'verify_failed' && l.result === 'failed'), r.logRows);
|
||||
A('verifyfail: not counted as repaired', r.logRows.filter(l => l.result === 'repaired').length === 1, r.logRows);
|
||||
}
|
||||
|
||||
// -- error path (grants fetch explodes)
|
||||
{
|
||||
const r3 = await runWorldWithThrow(W.error, 'grant');
|
||||
A('error: errors[] populated', r3.s.errors.length === 1 && r3.s.errors[0].customerId === 11, r3.s.errors);
|
||||
A('error: reconcile_error row', r3.logRows.some(l => l.drift_type === 'reconcile_error' && l.result === 'failed'), r3.logRows);
|
||||
}
|
||||
|
||||
// -- dry run: zero writes
|
||||
{
|
||||
const r = await runWorld(W.provision, { W4_DRY_RUN: 'true' });
|
||||
A('dryrun: no creates', r.creates.users.length === 0 && r.creates.roles.length === 0, r.creates);
|
||||
A('dryrun: no patches', r.patches.length === 0, r.patches);
|
||||
A('dryrun: no log POST', r.logRows.length === 0, r.logRows);
|
||||
A('dryrun: planned rows reported', r.s.plannedLogRows > 0 && r.s.logRowsWritten === 0, r.s);
|
||||
A('dryrun: dryRun flag', r.s.dryRun === true, r.s);
|
||||
}
|
||||
|
||||
console.log(fail === 0 ? '\nALL PASSED' : `\n${fail} FAILED`);
|
||||
process.exit(fail === 0 ? 0 : 1);
|
||||
}
|
||||
|
||||
// helper: run with grants fetch throwing for customer 11
|
||||
async function runWorldWithThrow(world, marker) {
|
||||
const S = freshWorld(world);
|
||||
const logRows = [];
|
||||
const httpRequest = async (opts) => {
|
||||
const u = opts.url, m = (opts.method || 'GET').toUpperCase();
|
||||
if (u.includes('/tag/records')) throw new Error('grants fetch boom');
|
||||
if (m === 'GET' && u.includes('/tc/records?limit=10000')) return { list: S.customers };
|
||||
if (m === 'GET' && u.includes(`/tr/records?limit=10000`)) return { list: S.resources };
|
||||
if (m === 'GET' && u.includes('/api/admin/user/list')) return { data: { content: S.users } };
|
||||
if (m === 'GET' && u.includes('/api/admin/role/list')) return { data: { content: S.roles } };
|
||||
const ug = u.match(/\/api\/admin\/user\/get\?id=(\d+)/);
|
||||
if (m === 'GET' && ug) { const x = S.users.find(uu => uu.id === Number(ug[1])); return x ? { data: x } : { code: 404, data: null }; }
|
||||
const rg = u.match(/\/api\/admin\/role\/get\?id=(\d+)/);
|
||||
if (m === 'GET' && rg) { const x = S.roles.find(r => r.id === Number(rg[1])); return x ? { data: x } : { code: 404, data: null }; }
|
||||
if (u.endsWith('/api/auth/login')) return { data: { token: 'ADMIN_TOKEN' } };
|
||||
if (m === 'POST' && u.includes('/tlog/records')) { (Array.isArray(opts.body) ? opts.body : [opts.body]).forEach(r => logRows.push(r)); return {}; }
|
||||
if (m === 'POST' && u.endsWith('/api/admin/user/update')) return { code: 200 };
|
||||
if (m === 'POST' && u.endsWith('/api/admin/role/update')) return { code: 200 };
|
||||
throw new Error('unexpected: ' + m + ' ' + u);
|
||||
};
|
||||
const sandbox = {
|
||||
$env: { NOCODB_URL: 'https://nc', NOCODB_TOKEN: 't', NOCODB_TABLE_CUSTOMERS: 'tc', NOCODB_TABLE_CUSTOMERPRODUCTS: 'tcp', NOCODB_TABLE_PRODUCTS: 'tp', NOCODB_TABLE_RESOURCES: 'tr', NOCODB_TABLE_ACCESSGRANTS: 'tag', NOCODB_TABLE_RECONCILELOGS: 'tlog', NOCODB_LINK_PROD_RESOURCES: 'lnk', ALIST_URL: 'https://al', ALIST_ADMIN_USERNAME: 'admin', ALIST_ADMIN_PASSWORD: 'p', W4_DRY_RUN: 'false' },
|
||||
$json: {}, console,
|
||||
};
|
||||
sandbox.helpers = { httpRequest };
|
||||
const fn = vm.runInNewContext(`(async function(){\n${code}\n})`, sandbox, { filename: 'w4-body.js' });
|
||||
const result = await fn();
|
||||
return { s: result.json, logRows };
|
||||
}
|
||||
|
||||
main().then(() => {}).catch(e => { console.error(e); process.exit(1); });
|
||||
+4
-1
@@ -153,7 +153,10 @@ if (status !== 'active') {
|
||||
}
|
||||
const actions = [];
|
||||
if (alistUserId) {
|
||||
await alistReq(ctx, 'POST', '/api/admin/user/update', { id: alistUserId, disabled: true }, token);
|
||||
// AList v3.63 /user/update REQUIRES username (validation: Username required).
|
||||
// Sending {id,disabled} alone -> 400 -> silent no-disable (found live 2026-08-25).
|
||||
const disRes = await alistReq(ctx, 'POST', '/api/admin/user/update', { id: alistUserId, username, disabled: true }, token);
|
||||
if (!disRes || disRes.code !== 200) throw new Error('AList disable failed: ' + JSON.stringify(disRes));
|
||||
actions.push('disable_user');
|
||||
}
|
||||
if (alistRoleId) {
|
||||
|
||||
+5
-2
@@ -56,8 +56,11 @@ for (const cust of customers) {
|
||||
applied.push('role_scopes');
|
||||
}
|
||||
if (shouldDisable) {
|
||||
if (!DRY_RUN && cust.alist_user_id)
|
||||
await alistReq(ctx, 'POST', '/api/admin/user/update', { id: cust.alist_user_id, disabled: true }, token);
|
||||
// AList v3.63 /user/update REQUIRES username — {id,disabled} alone = 400 silent no-op (2026-08-25).
|
||||
if (!DRY_RUN && cust.alist_user_id) {
|
||||
const disRes = await alistReq(ctx, 'POST', '/api/admin/user/update', { id: cust.alist_user_id, username: cust.username, disabled: true }, token);
|
||||
if (!disRes || disRes.code !== 200) throw new Error('AList disable failed: ' + JSON.stringify(disRes));
|
||||
}
|
||||
applied.push('disable');
|
||||
}
|
||||
if (applied.length) {
|
||||
|
||||
+377
@@ -0,0 +1,377 @@
|
||||
// ============================================================================
|
||||
// W4 runtime body — Daily full reconciliation + drift repair + reconcile log.
|
||||
// Rebuilt by `node n8n/gen_w4.js` (single source of truth = shared/effective-grants.js).
|
||||
//
|
||||
// ONE-DIRECTIONAL sync: NocoDB `Customers.status` is the BUSINESS source of
|
||||
// truth; AList user enabled/disabled is only ENFORCEMENT state. W4 NEVER
|
||||
// writes Customers.status. A manual `disabled` flip in the AList admin UI is
|
||||
// repaired back to match NocoDB — never the other way round.
|
||||
//
|
||||
// Drift matrix (auto-repair):
|
||||
// status active -> user must exist (provision if missing; lookup by
|
||||
// username FIRST to avoid duplicate users when the
|
||||
// NocoDB id was lost), be ENABLED, carry the desired
|
||||
// role scopes (calculateEffectiveGrants), base_path
|
||||
// '/', permission 16384.
|
||||
// status non-active -> if a user exists: must be DISABLED with empty role
|
||||
// scopes (soft state kept). pending/never-provisioned
|
||||
// customers stay WITHOUT an AList user (no auto-create).
|
||||
// dangling refs -> alist_user_id / alist_role_id point at deleted AList
|
||||
// records: lookup by username / role name, adopt if
|
||||
// found, recreate if not, then re-sync.
|
||||
// username drift -> NocoDB username differs from AList username: DETECT
|
||||
// + log only (no safe official rename API assumed; no
|
||||
// destructive migration).
|
||||
//
|
||||
// Every repair is VERIFIED by re-reading AList state afterwards; a failed
|
||||
// verify is logged as verify_failed. ReconcileLogs receives ONLY
|
||||
// drift/repair/error rows — healthy customers produce nothing.
|
||||
//
|
||||
// Triggers: daily cron (03:00) + manual webhook. Dry run: webhook
|
||||
// `?dryRun=true` or env W4_DRY_RUN=true -> plan only, ZERO writes.
|
||||
// ============================================================================
|
||||
// __SHARED_INLINE__ (replaced by shared/effective-grants.js source at build)
|
||||
// ============================================================================
|
||||
|
||||
const ctx = { $env, helpers: this.helpers };
|
||||
|
||||
const query = ($json && $json.query) || {};
|
||||
const DRY_RUN = query.dryRun === 'true' || query.dryRun === true ||
|
||||
String($env.W4_DRY_RUN || '').toLowerCase() === 'true';
|
||||
const T_CUST = $env.NOCODB_TABLE_CUSTOMERS;
|
||||
const T_LOG = $env.NOCODB_TABLE_RECONCILELOGS;
|
||||
// NocoDB v2 (2026.08.x): bt links are NOT created by writing the link column
|
||||
// (scalar/array/object all ignored). Write the FK column directly:
|
||||
// `nc_jitu___Customers_id` (verified live 2026-08-23).
|
||||
const FK_CUST = $env.NOCODB_FK_RECONCILELOG_CUSTOMER || 'nc_jitu___Customers_id';
|
||||
const runId = 'recon-' + new Date().toISOString().replace(/[:.]/g, '-');
|
||||
|
||||
// canonical AList user configuration — must stay identical to W1's (verified
|
||||
// working: base_path '/' + absolute role scopes + permission 16384)
|
||||
const BASE_PATH = '/';
|
||||
const PERMISSION = 16384;
|
||||
|
||||
const logRows = []; // ReconcileLogs payload — drift/repair/error rows ONLY
|
||||
|
||||
const summary = {
|
||||
runId, dryRun: DRY_RUN,
|
||||
customersScanned: 0, driftsDetected: 0, repaired: 0, skipped: 0,
|
||||
errors: [], details: [],
|
||||
};
|
||||
|
||||
function pushLog(cid, username, driftType, details, action, result) {
|
||||
const row = {
|
||||
run_id: runId,
|
||||
drift_type: driftType,
|
||||
details: details == null ? '' : String(details).slice(0, 2000),
|
||||
action,
|
||||
result,
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
if (cid != null) row[FK_CUST] = cid; // FK column (link writes are ignored)
|
||||
logRows.push(row);
|
||||
}
|
||||
|
||||
// ---- AList helpers (read) ----
|
||||
async function listUsers(token) {
|
||||
const r = await alistReq(ctx, 'GET', '/api/admin/user/list?page=1&per_page=1000', null, token);
|
||||
return (r && (r.data?.content || r.data?.list)) || [];
|
||||
}
|
||||
async function listRoles(token) {
|
||||
const r = await alistReq(ctx, 'GET', '/api/admin/role/list?page=1&per_page=1000', null, token);
|
||||
return (r && (r.data?.content || r.data?.list)) || [];
|
||||
}
|
||||
async function getUser(token, id) {
|
||||
const r = await alistReq(ctx, 'GET', `/api/admin/user/get?id=${id}`, null, token);
|
||||
return (r && r.data) ? r.data : null;
|
||||
}
|
||||
async function getRole(token, id) {
|
||||
const r = await alistReq(ctx, 'GET', `/api/admin/role/get?id=${id}`, null, token);
|
||||
return (r && r.data) ? r.data : null;
|
||||
}
|
||||
|
||||
// ---- AList helpers (write) — all no-op under DRY_RUN ----
|
||||
async function createRole(token, name, description) {
|
||||
if (DRY_RUN) return { id: null, name, permission_scopes: [] };
|
||||
await alistReq(ctx, 'POST', '/api/admin/role/create', {
|
||||
name, description, default: false, permission_scopes: [],
|
||||
}, token);
|
||||
const roles = await listRoles(token);
|
||||
const r = roles.find(x => x.name === name);
|
||||
if (!r) throw new Error('Created role not found: ' + name);
|
||||
return r;
|
||||
}
|
||||
async function createUser(token, body) {
|
||||
if (DRY_RUN) return { id: null, username: body.username };
|
||||
await alistReq(ctx, 'POST', '/api/admin/user/create', body, token);
|
||||
const users = await listUsers(token);
|
||||
const u = users.find(x => x.username === body.username);
|
||||
if (!u) throw new Error('Created user not found: ' + body.username);
|
||||
return u;
|
||||
}
|
||||
async function updateUser(token, fields) {
|
||||
if (DRY_RUN) return;
|
||||
await alistReq(ctx, 'POST', '/api/admin/user/update', fields, token);
|
||||
}
|
||||
async function updateRoleScopes(token, roleId, name, description, scopes) {
|
||||
if (DRY_RUN) return;
|
||||
await alistReq(ctx, 'POST', '/api/admin/role/update', {
|
||||
id: roleId, name, description, permission_scopes: scopes,
|
||||
}, token);
|
||||
}
|
||||
|
||||
// ---- NocoDB bookkeeping write-back (adopted user/role ids). Echo-safe: only
|
||||
// alist_user_id / alist_role_id are written; W1's old/new-field comparison
|
||||
// treats these as bookkeeping -> echo guard short-circuits. ----
|
||||
async function writeBack(cid, fields) {
|
||||
if (DRY_RUN || !Object.keys(fields).length) return;
|
||||
const live = await ncGet(ctx, `/api/v2/tables/${T_CUST}/records/${cid}`);
|
||||
const changes = {};
|
||||
for (const k of Object.keys(fields)) {
|
||||
if ((live[k] ?? null) !== (fields[k] ?? null)) changes[k] = fields[k];
|
||||
}
|
||||
if (!Object.keys(changes).length) return; // value unchanged -> no PATCH -> no webhook
|
||||
await ctx.helpers.httpRequest({
|
||||
method: 'PATCH',
|
||||
url: `${$env.NOCODB_URL}/api/v2/tables/${T_CUST}/records`,
|
||||
headers: { 'xc-token': $env.NOCODB_TOKEN },
|
||||
body: [Object.assign({ Id: cid }, changes)],
|
||||
json: true,
|
||||
});
|
||||
}
|
||||
|
||||
// ---- verification (post-repair read-back) ----
|
||||
async function verifyUserState(token, userId, expect) {
|
||||
const u = await getUser(token, userId);
|
||||
if (!u) return { ok: false, actual: null };
|
||||
for (const k of Object.keys(expect)) {
|
||||
const want = expect[k];
|
||||
const have = u[k];
|
||||
const wantS = Array.isArray(want) ? JSON.stringify(want) : want;
|
||||
const haveS = Array.isArray(have) ? JSON.stringify(have) : have;
|
||||
if (String(haveS ?? '') !== String(wantS ?? '')) return { ok: false, actual: u };
|
||||
}
|
||||
return { ok: true, actual: u };
|
||||
}
|
||||
async function verifyRoleScopes(token, roleId, expectScopes) {
|
||||
const r = await getRole(token, roleId);
|
||||
if (!r) return { ok: false, actual: null };
|
||||
const cur = (r.permission_scopes || []).map(s => s.path).sort();
|
||||
const exp = expectScopes.map(s => s.path).sort();
|
||||
const ok = cur.length === exp.length && cur.every((p, i) => p === exp[i]);
|
||||
return { ok, actual: r };
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// per-customer reconciliation
|
||||
// ============================================================================
|
||||
async function reconcileOne(cust, token) {
|
||||
const cid = cust.Id;
|
||||
const username = cust.username;
|
||||
const status = cust.status || 'active'; // W1 convention: missing = active
|
||||
const isActive = status === 'active';
|
||||
const drifts = []; // {type, action, result}
|
||||
|
||||
const note = (type, details, action, result) => {
|
||||
drifts.push({ type, action, result });
|
||||
pushLog(cid, username, type, details, action, result);
|
||||
};
|
||||
|
||||
// ---------- 1. resolve the AList user ----------
|
||||
let user = null;
|
||||
if (cust.alist_user_id) {
|
||||
user = await getUser(token, cust.alist_user_id);
|
||||
if (user && user.username !== username) {
|
||||
note('username_drift',
|
||||
JSON.stringify({ noco: username, alist: user.username, user_id: user.id }),
|
||||
'log_only', 'log_only');
|
||||
}
|
||||
}
|
||||
if (!user) {
|
||||
// id missing or dangling: lookup by username to avoid duplicate users
|
||||
const byName = (await listUsers(token)).find(u => u.username === username);
|
||||
if (byName) {
|
||||
user = byName;
|
||||
if (cust.alist_user_id !== byName.id) {
|
||||
note('dangling_user_ref',
|
||||
JSON.stringify({ ref: cust.alist_user_id, adopted: byName.id, found_by: 'username' }),
|
||||
'adopted', 'repaired');
|
||||
await writeBack(cid, { alist_user_id: byName.id });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!user && isActive) {
|
||||
// genuinely missing: provision role first, then user (empty role -> 400)
|
||||
let role = null;
|
||||
const roleName = 'customer-' + username;
|
||||
const byName = (await listRoles(token)).find(r => r.name === roleName);
|
||||
if (byName) role = byName;
|
||||
else if (!DRY_RUN) role = await createRole(token, roleName, 'Auto role for ' + username);
|
||||
else role = { id: null, name: roleName, permission_scopes: [] };
|
||||
user = await createUser(token, {
|
||||
username,
|
||||
password: cust.password || 'change_me',
|
||||
base_path: BASE_PATH,
|
||||
role: role.id ? [role.id] : [],
|
||||
permission: PERMISSION,
|
||||
});
|
||||
note('user_missing',
|
||||
JSON.stringify({ alist_user_id: cust.alist_user_id ?? null, alist_role_id: cust.alist_role_id ?? null }),
|
||||
DRY_RUN ? 'provision_planned' : 'provisioned', 'repaired');
|
||||
if (role.id) await writeBack(cid, { alist_role_id: role.id });
|
||||
}
|
||||
if (!user) {
|
||||
// non-active and never provisioned -> pure label, keep NO AList user.
|
||||
// (Healthy state: no drift, no log row.)
|
||||
return drifts;
|
||||
}
|
||||
if (user.id) await writeBack(cid, { alist_user_id: user.id });
|
||||
|
||||
// ---------- 2. resolve the role (the user's role binding) ----------
|
||||
const roleFor = async (wantName) => {
|
||||
const roleName = wantName || ('customer-' + username);
|
||||
const bound = Array.isArray(user.role) ? user.role[0] : (user.role || null);
|
||||
if (bound) {
|
||||
const r = await getRole(token, bound);
|
||||
if (r) return r;
|
||||
const byName = (await listRoles(token)).find(x => x.name === roleName);
|
||||
note('dangling_role_ref',
|
||||
JSON.stringify({ ref: bound, adopted: byName ? byName.id : null, want: roleName }),
|
||||
byName ? 'adopted' : 'recreate_required', 'repaired');
|
||||
if (byName) return byName;
|
||||
}
|
||||
const byName = (await listRoles(token)).find(x => x.name === roleName);
|
||||
if (byName) return byName;
|
||||
if (isActive) {
|
||||
const created = await createRole(token, roleName, 'Auto role for ' + username);
|
||||
note('role_missing', JSON.stringify({ roleName }), 'created', 'repaired');
|
||||
return created;
|
||||
}
|
||||
return null; // non-active with no role: nothing to clear
|
||||
};
|
||||
let role = await roleFor();
|
||||
|
||||
// ---------- 3. desired scopes (shared algorithm; also reads manual grants) ----------
|
||||
const g = await calculateEffectiveGrants(ctx, cid);
|
||||
const desiredScopes = isActive ? g.newScopes : [];
|
||||
|
||||
// ---------- 4. apply fixes (canonical state = W1's verified config) ----------
|
||||
const userPatches = [];
|
||||
const expectUser = {};
|
||||
if (!isActive) {
|
||||
if (user.disabled !== true) { userPatches.push('disabled'); note('user_disabled_drift', JSON.stringify({ have: user.disabled, want: true }), 'disabled', 'repaired'); }
|
||||
expectUser.disabled = true;
|
||||
} else {
|
||||
if (user.disabled === true) { userPatches.push('re-enabled'); note('user_disabled_drift', JSON.stringify({ have: true, want: false }), 'enabled', 'repaired'); }
|
||||
expectUser.disabled = false;
|
||||
}
|
||||
if (user.base_path !== BASE_PATH) {
|
||||
userPatches.push('base_path'); note('base_path_drift', JSON.stringify({ have: user.base_path, want: BASE_PATH }), 'fixed', 'repaired');
|
||||
}
|
||||
expectUser.base_path = BASE_PATH;
|
||||
if (user.permission !== PERMISSION) {
|
||||
userPatches.push('permission'); note('permission_drift', JSON.stringify({ have: user.permission, want: PERMISSION }), 'fixed', 'repaired');
|
||||
}
|
||||
expectUser.permission = PERMISSION;
|
||||
const boundRoleId = role && role.id;
|
||||
if (boundRoleId && !(Array.isArray(user.role) && user.role.includes(boundRoleId))) {
|
||||
userPatches.push('role_binding'); note('role_binding_drift', JSON.stringify({ have: user.role, want: [boundRoleId] }), 'rebound', 'repaired');
|
||||
}
|
||||
expectUser.role = boundRoleId ? [boundRoleId] : user.role || [];
|
||||
|
||||
if (userPatches.length) {
|
||||
// NOTE: password intentionally NOT included in repairs — NocoDB plaintext
|
||||
// may be stale (方案 C: customer may have changed their own AList password;
|
||||
// pushing it here would clobber it). Password only used at provision.
|
||||
await updateUser(token, Object.assign({ id: user.id }, {
|
||||
username, base_path: BASE_PATH, permission: PERMISSION, disabled: !isActive,
|
||||
role: boundRoleId ? [boundRoleId] : [],
|
||||
}));
|
||||
}
|
||||
|
||||
// scopes drift (role may be null for non-active with no role -> nothing to clear)
|
||||
if (role && role.id) {
|
||||
const cur = (role.permission_scopes || []).map(s => s.path).sort();
|
||||
const want = desiredScopes.map(s => s.path).sort();
|
||||
if (cur.length !== want.length || cur.some((p, i) => p !== want[i])) {
|
||||
note('scopes_drift',
|
||||
JSON.stringify({ have: cur, want }),
|
||||
DRY_RUN ? 'rewrite_planned' : 'rewritten', 'repaired');
|
||||
await updateRoleScopes(token, role.id, role.name, role.description, desiredScopes);
|
||||
}
|
||||
} else if (isActive && !DRY_RUN) {
|
||||
// active but could not bind a role (should not happen; roleFor creates one)
|
||||
throw new Error('Active customer has no role to write scopes into');
|
||||
}
|
||||
|
||||
// ---------- 5. verify actual AList state after every repair ----------
|
||||
if (!DRY_RUN && user.id) {
|
||||
const uu = await verifyUserState(token, user.id, expectUser);
|
||||
if (!uu.ok) {
|
||||
note('verify_failed', JSON.stringify({ expect: expectUser, actual: uu.actual }), 'verify', 'failed');
|
||||
}
|
||||
if (role && role.id) {
|
||||
const rr = await verifyRoleScopes(token, role.id, desiredScopes);
|
||||
if (!rr.ok) {
|
||||
note('verify_failed', JSON.stringify({ expect_scopes: desiredScopes, actual: rr.actual }), 'verify', 'failed');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return drifts;
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// main sweep
|
||||
// ============================================================================
|
||||
const login = await alistReq(ctx, 'POST', '/api/auth/login', {
|
||||
username: $env.ALIST_ADMIN_USERNAME,
|
||||
password: $env.ALIST_ADMIN_PASSWORD,
|
||||
}, null);
|
||||
const token = login?.data?.token;
|
||||
if (!token) throw new Error('AList login failed');
|
||||
|
||||
const customers = (await ncGet(ctx, `/api/v2/tables/${T_CUST}/records?limit=10000`)).list || [];
|
||||
|
||||
for (const cust of customers) {
|
||||
if (!cust.username) {
|
||||
summary.errors.push({ customerId: cust.Id, username: null, error: 'missing username' });
|
||||
pushLog(cust.Id, null, 'reconcile_error', JSON.stringify({ row: cust.Id }), 'log_only', 'failed');
|
||||
summary.customersScanned++;
|
||||
continue;
|
||||
}
|
||||
summary.customersScanned++;
|
||||
const entry = { customerId: cust.Id, username: cust.username, status: cust.status || 'active', drifts: [] };
|
||||
try {
|
||||
entry.drifts = await reconcileOne(cust, token);
|
||||
if (entry.drifts.length === 0) summary.skipped++;
|
||||
else summary.details.push(entry);
|
||||
} catch (e) {
|
||||
summary.errors.push({ customerId: cust.Id, username: cust.username, error: e.message });
|
||||
pushLog(cust.Id, cust.username, 'reconcile_error', JSON.stringify({ error: e.message }), 'log_only', 'failed');
|
||||
}
|
||||
}
|
||||
|
||||
summary.driftsDetected = logRows.filter(r => r.result !== 'failed').length;
|
||||
summary.repaired = logRows.filter(r => r.result === 'repaired').length;
|
||||
|
||||
// ---- write ReconcileLogs (drift/repair/error rows only) ----
|
||||
if (logRows.length && !DRY_RUN) {
|
||||
if (!T_LOG) {
|
||||
summary.logWriteSkipped = 'NOCODB_TABLE_RECONCILELOGS env missing — rows not persisted';
|
||||
} else {
|
||||
await ctx.helpers.httpRequest({
|
||||
method: 'POST',
|
||||
url: `${$env.NOCODB_URL}/api/v2/tables/${T_LOG}/records`,
|
||||
headers: { 'xc-token': $env.NOCODB_TOKEN },
|
||||
body: logRows,
|
||||
json: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return { json: Object.assign({}, summary, {
|
||||
at: new Date().toISOString(),
|
||||
logRowsWritten: DRY_RUN ? 0 : logRows.length,
|
||||
plannedLogRows: DRY_RUN ? logRows.length : 0,
|
||||
}) };
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
// ============================================================================
|
||||
// W5 runtime body — footer purchase/expiry lookup (PUBLIC, READ-ONLY).
|
||||
// Trigger: GET webhook /dk-purchase-check?u=<username>
|
||||
// Returns: { username, known, purchases: [{product, expires_at, display_state, paths:[...]}] }
|
||||
//
|
||||
// v30 (2026-08-28): adds `paths` per purchase — each product's resources' alist_path
|
||||
// (via Products→Resources mm link NOCODB_LINK_PROD_RESOURCES), so the footer modal
|
||||
// can show a collapsible resource-path list. Products with no linked resources
|
||||
// return paths: [].
|
||||
//
|
||||
// Security model (no HMAC — key would ship to the client, so it is theatre):
|
||||
// - CORS: response header Access-Control-Allow-Origin: https://drive.digikedai.com
|
||||
// → only the AList portal origin may READ the response from a browser.
|
||||
// - Minimal data: product name + expiry + display_state + public folder paths only
|
||||
// (no internal NocoDB Ids, no customer PII).
|
||||
// - Server-side enumeration is mitigated by a Cloudflare WAF rate-limit +
|
||||
// Origin/Referer rule (see delivery guide).
|
||||
// - No AList auth requests of any kind (username comes from the caller's own
|
||||
// JWT, decoded client-side in the footer) → no empty-client device drift.
|
||||
// ============================================================================
|
||||
|
||||
// n8n Code node context (same pattern as W1–W4).
|
||||
const ctx = { $env, helpers: this.helpers };
|
||||
|
||||
// GET webhook: n8n puts query-string params under $json.query (fallback: body).
|
||||
const item = $json ?? ($input && $input.item ? $input.item.json : {});
|
||||
const q = item.query ?? item.body ?? {};
|
||||
const username = String(q.u ?? '').trim();
|
||||
|
||||
function linkId(v) {
|
||||
if (v == null) return null;
|
||||
if (typeof v === 'object' && !Array.isArray(v)) return v.Id ?? v.id ?? null;
|
||||
if (Array.isArray(v)) return v[0]?.Id ?? v[0]?.id ?? null;
|
||||
return v;
|
||||
}
|
||||
|
||||
// 内网直连 NocoDB：n8n 与 nocodb 同在 bridge_hoelee，NocoDB 监听 :10380。
|
||||
// 原 NOCODB_URL=https://nocodb.hoelee.com 走公网 CF tunnel，慢且抖（曾致 >60s → nginx 504
|
||||
// → footer 8s 超时每 1s 重试 → 连接堆积 → 503）。内网 30ms vs 公网 300ms+，且无 tunnel 抖动。
|
||||
const NOCODB_URL = 'http://nocodb:10380';
|
||||
|
||||
async function ncGet(path) {
|
||||
return await ctx.helpers.httpRequest({
|
||||
method: 'GET',
|
||||
url: NOCODB_URL + path,
|
||||
headers: { 'xc-token': ctx.$env.NOCODB_TOKEN },
|
||||
json: true,
|
||||
});
|
||||
}
|
||||
|
||||
const empty = () => ({ username, known: false, purchases: [] });
|
||||
|
||||
if (!username) return { json: empty() };
|
||||
|
||||
try {
|
||||
const T_CUST = ctx.$env.NOCODB_TABLE_CUSTOMERS;
|
||||
const T_CP = ctx.$env.NOCODB_TABLE_CUSTOMERPRODUCTS;
|
||||
const T_PROD = ctx.$env.NOCODB_TABLE_PRODUCTS;
|
||||
const T_RES = ctx.$env.NOCODB_TABLE_RESOURCES;
|
||||
const LNK_PROD_RES = ctx.$env.NOCODB_LINK_PROD_RESOURCES; // c4ko4x8kwi16swg
|
||||
const today = new Date().toISOString().slice(0, 10);
|
||||
|
||||
// 1. resolve username -> customer (username is unique in Customers)
|
||||
const custRows = (await ncGet(
|
||||
`/api/v2/tables/${T_CUST}/records?where=(username,eq,${encodeURIComponent(username)})&limit=1`
|
||||
)).list || [];
|
||||
const customer = custRows[0];
|
||||
if (!customer) return { json: empty() };
|
||||
|
||||
// 2. this customer's products (link query by Id, per NocoDB v2 gotchas)
|
||||
const cps = (await ncGet(
|
||||
`/api/v2/tables/${T_CP}/records?where=(customer.Id,eq,${customer.Id})&limit=500`
|
||||
)).list || [];
|
||||
|
||||
// 3. resolve paths per product (Products→Resources mm link → alist_path).
|
||||
// Fetch all Resources once, index Id → alist_path (link API only returns
|
||||
// {Id, name}, not alist_path — see effective-grants.js pattern).
|
||||
const allRes = (await ncGet(
|
||||
`/api/v2/tables/${T_RES}/records?limit=10000&fields=Id,alist_path`
|
||||
)).list || [];
|
||||
const resPath = {};
|
||||
allRes.forEach((r) => { resPath[r.Id] = r.alist_path; });
|
||||
|
||||
const prodIds = [...new Set(cps.map((c) => linkId(c.product)).filter(Boolean))];
|
||||
|
||||
const pathsByProduct = {}; // pid -> [alist_path, ...]
|
||||
for (const pid of prodIds) {
|
||||
let paths = [];
|
||||
try {
|
||||
const res = (await ncGet(
|
||||
`/api/v2/tables/${T_PROD}/links/${LNK_PROD_RES}/records/${pid}`
|
||||
)).list || [];
|
||||
paths = res.map((r) => resPath[r.Id]).filter((p) => p != null && String(p).length > 0);
|
||||
} catch (e) {
|
||||
paths = []; // graceful: no paths rather than failing the whole lookup
|
||||
}
|
||||
pathsByProduct[pid] = paths.sort();
|
||||
}
|
||||
|
||||
const purchases = cps
|
||||
.map((c) => {
|
||||
const productObj = (typeof c.product === 'object' && c.product) ? c.product : null;
|
||||
const name = productObj && productObj.name ? String(productObj.name) : 'Product';
|
||||
const pid = linkId(c.product);
|
||||
const expires = c.expires_at ? String(c.expires_at).slice(0, 10) : null;
|
||||
let state = (c.status && c.status !== 'active') ? c.status : 'active';
|
||||
if (state === 'active' && expires && expires <= today) state = 'expired';
|
||||
const paths = pid ? (pathsByProduct[pid] || []) : [];
|
||||
return { product: name, expires_at: expires, display_state: state, paths };
|
||||
})
|
||||
.sort((a, b) => {
|
||||
const rank = { active: 0, suspended: 1, revoked: 2, expired: 3 };
|
||||
if (rank[a.display_state] !== rank[b.display_state]) return rank[a.display_state] - rank[b.display_state];
|
||||
return String(a.expires_at || '').localeCompare(String(b.expires_at || ''));
|
||||
});
|
||||
|
||||
return { json: { username, known: true, purchases } };
|
||||
} catch (err) {
|
||||
// graceful: footer renders static rows only on any error
|
||||
return { json: { ...empty(), error: String((err && err.message) || err) } };
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Ensure the ReconcileLogs table exists in AlistAccess (idempotent, NON-destructive).
|
||||
|
||||
Unlike create_nocodb_schema.py (which drops + recreates the base), this script
|
||||
only creates/mends the ReconcileLogs table + columns. Safe to re-run.
|
||||
Uses v2 meta endpoints only — verified on NocoDB 2026.08.1 (v1 column GET/PATCH
|
||||
endpoints are GONE on this version; column id is fetched from the table meta).
|
||||
|
||||
Credentials from env (NOCODB_URL / NOCODB_TOKEN) — never commit real tokens.
|
||||
|
||||
Usage:
|
||||
NOCODB_TOKEN=... python scripts/add_reconcilelogs_table.py
|
||||
"""
|
||||
import json, os, urllib.request, urllib.error
|
||||
|
||||
NOCODB_URL = os.environ.get("NOCODB_URL", "https://nocodb.hoelee.com")
|
||||
NOCODB_TOKEN = os.environ.get("NOCODB_TOKEN", "CHANGE_ME_nocodb_token")
|
||||
BASE_ID = os.environ.get("NOCODB_BASE_ID", "p3lzkresswyowqg")
|
||||
CUSTOMERS_TID = os.environ.get("NOCODB_TABLE_CUSTOMERS", "maqb9zgvza2te3k")
|
||||
TABLE_TITLE = os.environ.get("RECONCILELOGS_TITLE", "ReconcileLogs")
|
||||
|
||||
DRIFT_OPTIONS = [
|
||||
"user_missing", "role_missing", "dangling_user_ref", "dangling_role_ref",
|
||||
"user_disabled_drift", "scopes_drift", "base_path_drift",
|
||||
"permission_drift", "role_binding_drift", "username_drift",
|
||||
"verify_failed", "reconcile_error",
|
||||
]
|
||||
RESULT_OPTIONS = ["repaired", "log_only", "failed", "skipped"]
|
||||
|
||||
|
||||
def req(method, path, body=None):
|
||||
r = urllib.request.Request(f"{NOCODB_URL}{path}", method=method)
|
||||
r.add_header("xc-token", NOCODB_TOKEN)
|
||||
if body is not None:
|
||||
r.add_header("Content-Type", "application/json")
|
||||
r.data = json.dumps(body).encode()
|
||||
try:
|
||||
with urllib.request.urlopen(r, timeout=30) as resp:
|
||||
return json.loads(resp.read().decode() or "null")
|
||||
except urllib.error.HTTPError as e:
|
||||
raise RuntimeError(f"{method} {path} -> HTTP {e.code}: {e.read().decode()[:200]}")
|
||||
|
||||
|
||||
def table_meta(tid):
|
||||
return req("GET", f"/api/v2/meta/tables/{tid}")
|
||||
|
||||
|
||||
def col_by_title(tid, title):
|
||||
meta = table_meta(tid)
|
||||
return next((c for c in (meta.get("columns") or []) if c.get("title") == title), None)
|
||||
|
||||
|
||||
def ensure_column(tid, title, uidt, **extra):
|
||||
c = col_by_title(tid, title)
|
||||
if not c:
|
||||
req("POST", f"/api/v2/meta/tables/{tid}/columns",
|
||||
{"title": title, "uidt": uidt, **extra})
|
||||
c = col_by_title(tid, title)
|
||||
if not c:
|
||||
raise RuntimeError(f"column {title} not found after create")
|
||||
print(f" + {title} ({uidt}) -> {c['id']}")
|
||||
return c
|
||||
|
||||
|
||||
def ensure_ss_options(tid, title, options):
|
||||
c = ensure_column(tid, title, "SingleSelect")
|
||||
cur = ((c.get("colOptions") or {}).get("options") or [])
|
||||
cur_titles = [o.get("title") for o in cur]
|
||||
if cur_titles == options:
|
||||
print(f" = {title} options OK")
|
||||
return
|
||||
req("PATCH", f"/api/v2/meta/columns/{c['id']}", {
|
||||
"colOptions": {"options": [{"title": o, "order": i + 1}
|
||||
for i, o in enumerate(options)]},
|
||||
"dtxp": ",".join(options),
|
||||
})
|
||||
print(f" ~ {title} options set")
|
||||
|
||||
|
||||
def main():
|
||||
tables = req("GET", f"/api/v2/meta/bases/{BASE_ID}/tables")["list"]
|
||||
existing = next((t for t in tables if t["title"] == TABLE_TITLE), None)
|
||||
if existing:
|
||||
tid = existing["id"]
|
||||
print(f"{TABLE_TITLE} exists: {tid}")
|
||||
else:
|
||||
t = req("POST", f"/api/v2/meta/bases/{BASE_ID}/tables", {
|
||||
"title": TABLE_TITLE, "table_name": TABLE_TITLE, "type": "table",
|
||||
"columns": [],
|
||||
})
|
||||
tid = t["id"]
|
||||
print(f"{TABLE_TITLE} created: {tid}")
|
||||
|
||||
ensure_column(tid, "run_id", "SingleLineText")
|
||||
ensure_column(tid, "details", "LongText")
|
||||
ensure_column(tid, "action", "SingleLineText")
|
||||
ensure_column(tid, "timestamp", "DateTime")
|
||||
ensure_ss_options(tid, "drift_type", DRIFT_OPTIONS)
|
||||
ensure_ss_options(tid, "result", RESULT_OPTIONS)
|
||||
|
||||
if not col_by_title(tid, "customer"):
|
||||
req("POST", f"/api/v1/db/meta/tables/{tid}/columns", {
|
||||
"title": "customer", "uidt": "LinkToAnotherRecord",
|
||||
"parentId": CUSTOMERS_TID, "childId": tid, "type": "bt",
|
||||
})
|
||||
c = col_by_title(tid, "customer")
|
||||
print(f" + customer (bt link) -> {c['id']}")
|
||||
fk = col_by_title(tid, "nc_jitu___Customers_id")
|
||||
if not fk:
|
||||
raise RuntimeError("FK column nc_jitu___Customers_id missing — bt link create failed")
|
||||
|
||||
print(f"\nDONE. table={tid} fk_column={fk['id']} ({fk['title']})")
|
||||
print("=> add to n8n env: NOCODB_TABLE_RECONCILELOGS=" + tid
|
||||
+ " NOCODB_FK_RECONCILELOG_CUSTOMER=" + fk["title"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -76,16 +76,27 @@ def main():
|
||||
body.update(extra)
|
||||
return curl("POST", f"/api/v1/db/meta/tables/{tid}/columns", body)
|
||||
|
||||
def set_options(colid, opts):
|
||||
return curl("PATCH", f"/api/v1/db/meta/columns/{colid}",
|
||||
def col_id(tid, title):
|
||||
# v1 GET /columns is removed on NocoDB 2026.08.x — read table meta instead
|
||||
t = curl("GET", f"/api/v2/meta/tables/{tid}")
|
||||
for c in jget(t, "columns", d=[]) or []:
|
||||
if c.get("title") == title:
|
||||
return c.get("id")
|
||||
return None
|
||||
|
||||
def set_options(tid, title, opts):
|
||||
cid = col_id(tid, title)
|
||||
if not cid:
|
||||
raise RuntimeError(f"column {title} not found on {tid}")
|
||||
return curl("PATCH", f"/api/v2/meta/columns/{cid}",
|
||||
{"colOptions": {"options": [{"title": o, "order": i + 1, "color": None}
|
||||
for i, o in enumerate(opts)]},
|
||||
"dtxp": ",".join(f"'{o}'" for o in opts)})
|
||||
|
||||
def add_ss(tid, title, opts):
|
||||
r = add_col(tid, title, "SingleSelect")
|
||||
cid = jget(r, "id")
|
||||
set_options(cid, opts)
|
||||
add_col(tid, title, "SingleSelect")
|
||||
cid = col_id(tid, title)
|
||||
set_options(tid, title, opts)
|
||||
return cid
|
||||
|
||||
def add_link(child_tid, title, parent_tid, type_="bt"):
|
||||
@@ -166,6 +177,19 @@ def main():
|
||||
{"title": "notes", "uidt": "LongText"},
|
||||
]); pur = jget(pur, "id")
|
||||
|
||||
rlog = create_table("ReconcileLogs", [
|
||||
{"title": "run_id", "uidt": "SingleLineText"},
|
||||
{"title": "details", "uidt": "LongText"},
|
||||
{"title": "action", "uidt": "SingleLineText"},
|
||||
{"title": "timestamp", "uidt": "DateTime"},
|
||||
]); rlog = jget(rlog, "id")
|
||||
add_ss(rlog, "drift_type",
|
||||
["user_missing", "role_missing", "dangling_user_ref", "dangling_role_ref",
|
||||
"user_disabled_drift", "scopes_drift", "base_path_drift",
|
||||
"permission_drift", "role_binding_drift", "username_drift",
|
||||
"verify_failed", "reconcile_error"])
|
||||
add_ss(rlog, "result", ["repaired", "log_only", "failed", "skipped"])
|
||||
|
||||
# ---- links ----
|
||||
add_link(cp, "customer", cust)
|
||||
add_link(cp, "product", prod)
|
||||
@@ -175,9 +199,11 @@ def main():
|
||||
add_link(al, "customer", cust)
|
||||
add_link(pur, "customer", cust)
|
||||
add_link(pur, "product", prod)
|
||||
add_link(rlog, "customer", cust)
|
||||
|
||||
ids = {"base": base_id, "Customers": cust, "Products": prod, "Resources": res,
|
||||
"CustomerProducts": cp, "AccessGrants": ag, "AccessLogs": al, "Purchases": pur}
|
||||
"CustomerProducts": cp, "AccessGrants": ag, "AccessLogs": al,
|
||||
"Purchases": pur, "ReconcileLogs": rlog}
|
||||
print("DONE. IDs:", json.dumps(ids, indent=2))
|
||||
# persist for later steps
|
||||
open("_nocodb_ids.json", "w").write(json.dumps(ids, indent=2))
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
// Live dry-run of the W4 Code node against REAL AList + NocoDB, outside n8n.
|
||||
// Runs the exact inlined jsCode from W4-daily-reconcile.json in a vm sandbox,
|
||||
// with helpers.httpRequest backed by node fetch. ZERO writes (dryRun=true).
|
||||
// Usage: node scripts/live_dry_run_w4.js
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const vm = require('vm');
|
||||
|
||||
const wf = JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'n8n', 'W4-daily-reconcile.json'), 'utf8'));
|
||||
const code = wf.nodes.find(n => n.name === 'Reconcile & Repair (Daily Full)').parameters.jsCode;
|
||||
|
||||
const ENV = {
|
||||
NOCODB_URL: 'https://nocodb.hoelee.com',
|
||||
NOCODB_TOKEN: process.env.NOCODB_TOKEN, // from env, never hardcoded — same for passwords below
|
||||
NOCODB_TABLE_CUSTOMERS: 'maqb9zgvza2te3k',
|
||||
NOCODB_TABLE_PRODUCTS: 'molyb3pxv81l75g',
|
||||
NOCODB_TABLE_RESOURCES: 'm6cm7tkxzd8m18c',
|
||||
NOCODB_TABLE_CUSTOMERPRODUCTS: 'mvyqbv8ao7u6bct',
|
||||
NOCODB_TABLE_ACCESSGRANTS: 'mxdx3yhfcyej3ao',
|
||||
NOCODB_TABLE_RECONCILELOGS: 'mktsb7le6fqr2u1',
|
||||
NOCODB_FK_RECONCILELOG_CUSTOMER: 'nc_jitu___Customers_id',
|
||||
NOCODB_LINK_PROD_RESOURCES: 'c4ko4x8kwi16swg',
|
||||
ALIST_URL: process.env.ALIST_URL || 'http://192.168.1.1:5244', // DSM host port (container DNS only inside bridge)
|
||||
ALIST_ADMIN_USERNAME: process.env.ALIST_ADMIN_USERNAME || 'hoelee',
|
||||
ALIST_ADMIN_PASSWORD: process.env.ALIST_ADMIN_PASSWORD, // from env
|
||||
W4_DRY_RUN: 'false',
|
||||
};
|
||||
if (!ENV.NOCODB_TOKEN || !ENV.ALIST_ADMIN_PASSWORD) { console.error('NOCODB_TOKEN + ALIST_ADMIN_PASSWORD env required'); process.exit(1); }
|
||||
|
||||
const httpRequest = async (opts) => {
|
||||
const { method = 'GET', url, headers = {}, body } = opts;
|
||||
const res = await fetch(url, {
|
||||
method,
|
||||
headers: { ...headers, 'Content-Type': 'application/json' },
|
||||
body: body !== undefined && body !== null ? JSON.stringify(body) : undefined,
|
||||
});
|
||||
const text = await res.text();
|
||||
let parsed = null;
|
||||
try { parsed = text ? JSON.parse(text) : null; } catch { parsed = text; }
|
||||
return parsed;
|
||||
};
|
||||
|
||||
const sandbox = {
|
||||
$env: ENV,
|
||||
$json: { query: { dryRun: 'true' } },
|
||||
console,
|
||||
};
|
||||
sandbox.helpers = { httpRequest };
|
||||
|
||||
(async () => {
|
||||
const fn = vm.runInNewContext(`(async function(){\n${code}\n})`, sandbox, { filename: 'w4-body-live.js' });
|
||||
const result = await fn();
|
||||
console.log(JSON.stringify(result.json, null, 1));
|
||||
})().catch(e => { console.error('LIVE RUN FAILED:', e); process.exit(1); });
|
||||
Reference in New Issue
Block a user