post: model gateway OAuth boundary post-mortem (EN+ZH) + LinkedIn/dev.to copy
Deploy / build (push) Successful in 39s

Argon reviewed twice: blog body, then the distribution copy. Adopted: no
absolute timestamps, seats framed as workflow isolation, 403->502 error
contract fix, two-tier cost math with output pricing, task-dispatch naming.
This commit is contained in:
hoelee committed 2026-11-06 07:56:45 +08:00
1 parent f30b5b17ae
commit b75899c875
14 files changed
+1822 -1326

No files matched your search

@@ -0,0 +1 @@
🔗 Full post-mortem, the isolation probes and the dispatcher core: https://blog.hoelee.com/posts/when-a-self-hosted-model-router-tripped-trust-and-safety/
@@ -0,0 +1,13 @@
I spent ten minutes rotating credentials that were never wrong — because my AI gateway translated an upstream policy block into HTTP 401.
In the initial architecture, a self-hosted gateway unified every model tier behind a single interface: long-context reading, refactors, cheap background jobs. One key, one picker, one spend dashboard. Good engineering for a real problem, missing exactly one thing: a boundary.
An OAuth token is bound to the client that obtained it. Isolated workflow seats egressing through a single container with normalised headers strip client telemetry and read as one anomalous cluster. Simultaneous trips across independent workflows are not manual reviews — they are one deterministic cluster rule firing.
The root failure was not the upstream block, it was the error contract: turning an upstream HTTP 403 policy rejection into an OpenAI-style 401 invalid_api_key blinded every downstream consumer. That mapping is now fixed — policy blocks return 502 with an explicit upstream_policy_block signal and raise an alert, while credential failures stay 401.
What replaced the pooled design: task dispatch. The orchestrator dispatches isolated, single-task invocations to the vendor's official CLI bound to a dedicated workflow seat — no credential pooling, no session hijacking, no rotation. Explicit working directory, closed stdin, process-group kill on timeout, zero automatic retries. Context moves through repository instruction files and an MCP memory service, not transcript dumps.
The economics held because the workload splits: nine turns in ten are routine orchestration on a $10 plan carrying a $100 model allowance (about $0.0009 for a 4k/500 turn), and the last tenth is heavy engineering on an authorised seat.
For platform teams building internal AI planes: where do you draw the execution boundary when mixing metered APIs with client-bound CLI tools — strict pure-API standardisation at ten times the cost, or a hybrid dispatcher governing official binaries? Where has that boundary broken under load?