hoelee
b75899c875
Deploy / build (push) Successful in 39s
Argon reviewed twice: blog body, then the distribution copy. Adopted: no absolute timestamps, seats framed as workflow isolation, 403->502 error contract fix, two-tier cost math with output pricing, task-dispatch naming.
14 lines
2.0 KiB
Markdown
14 lines
2.0 KiB
Markdown
I spent ten minutes rotating credentials that were never wrong — because my AI gateway translated an upstream policy block into HTTP 401.
|
|
|
|
In the initial architecture, a self-hosted gateway unified every model tier behind a single interface: long-context reading, refactors, cheap background jobs. One key, one picker, one spend dashboard. Good engineering for a real problem, missing exactly one thing: a boundary.
|
|
|
|
An OAuth token is bound to the client that obtained it. Isolated workflow seats egressing through a single container with normalised headers strip client telemetry and read as one anomalous cluster. Simultaneous trips across independent workflows are not manual reviews — they are one deterministic cluster rule firing.
|
|
|
|
The root failure was not the upstream block, it was the error contract: turning an upstream HTTP 403 policy rejection into an OpenAI-style 401 invalid_api_key blinded every downstream consumer. That mapping is now fixed — policy blocks return 502 with an explicit upstream_policy_block signal and raise an alert, while credential failures stay 401.
|
|
|
|
What replaced the pooled design: task dispatch. The orchestrator dispatches isolated, single-task invocations to the vendor's official CLI bound to a dedicated workflow seat — no credential pooling, no session hijacking, no rotation. Explicit working directory, closed stdin, process-group kill on timeout, zero automatic retries. Context moves through repository instruction files and an MCP memory service, not transcript dumps.
|
|
|
|
The economics held because the workload splits: nine turns in ten are routine orchestration on a $10 plan carrying a $100 model allowance (about $0.0009 for a 4k/500 turn), and the last tenth is heavy engineering on an authorised seat.
|
|
|
|
For platform teams building internal AI planes: where do you draw the execution boundary when mixing metered APIs with client-bound CLI tools — strict pure-API standardisation at ten times the cost, or a hybrid dispatcher governing official binaries? Where has that boundary broken under load?
|