Add post: Why I Still Bought a GPU to Run AI When Claude and GPT Are Stronger (EN+ZH)

scripts/banner-gen/generate.mjs

@@ -605,6 +605,25 @@ BANNERS['patching-workbench-26-for-mariadb'] = {
   ],
 };
 
+BANNERS['why-i-still-bought-a-local-gpu'] = {
+  titlebar: 'root@unraid — local inference',
+  lines: [
+    { t: 'prompt', text: '$' }, { t: 'cmd', text: 'llama-server -m qwen3.8-27b-Q6_K.gguf --n-gpu-layers 99' },
+    { t: 'dim', text: '24GB VRAM · ~1300 tok/s prefill · 40 tok/s decode' },
+    { t: 'prompt', text: '$' }, { t: 'cmd', text: 'agent run --task build-from-spec --iterations 50' },
+    { t: 'err', text: 'frontier api: 50 calls · metered · quota can change' },
+    { t: 'ok',  text: 'local gpu:  50 calls · $0.00 · runs as long as I want' },
+    { t: 'hl',  text: 'use the frontier to DECIDE, local compute to BUILD' },
+  ],
+  flow: [
+    { n: '1', label: 'frontier plans' },
+    { n: '2', label: 'spec locked' },
+    { n: '3', label: 'local agents run' },
+    { n: '4', label: '50× iterations' },
+    { n: '5', label: 'fixed cost ✓' },
+  ],
+};
+
 // ---------- read frontmatter ----------
 const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`);
 let category = 'devops';

scripts/og-gen/generate.mjs

@@ -191,7 +191,13 @@ const DEFAULT_TERMINAL = `
   <div class="line"><span class="prompt">$</span><span class="cmd">engineering · devops · self-hosting</span></div>
   <div class="line"><span class="prompt">&nbsp;</span><span class="fix">read the full post →</span></div>`;
 
-// ---------- read frontmatter from the post ----------
+TERMINALS['why-i-still-bought-a-local-gpu'] = `
+    <div class="line"><span class="prompt">$</span><span class="cmd">agent run --task build-from-spec --iterations 50</span></div>
+    <div class="line"><span class="prompt">&nbsp;</span><span class="err">api: 50 calls · $12.40 · quota may change</span></div>
+    <div class="line"><span class="prompt">&nbsp;</span><span class="fix">local: 50 calls · $0.00 · runs on my card</span></div>
+`;
+
+// ---------- read frontmatter ----------
 const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`);
 if (!existsSync(postPath)) {
   console.error(`Post not found: ${postPath}`);

src/content/posts/why-i-still-bought-a-local-gpu.md

@@ -0,0 +1,225 @@
+---
+title: "Why I Still Bought a GPU to Run AI When Claude and GPT Are Obviously Stronger"
+description: "Claude Opus and GPT are stronger than anything I can run locally. Here is the cost, control, and uptime argument for owning a GPU anyway — with real numbers."
+pubDate: 2026-09-21
+category: ai
+tags: [local-llm, gpu, llama-cpp, lm-studio, agents, self-hosting, cost]
+ogImage: /og/why-i-still-bought-a-local-gpu.png
+banner: /banners/why-i-still-bought-a-local-gpu.png
+draft: false
+---
+
+I have a house full of self-hosted infrastructure — Docker stacks, a NAS, a
+tunnel, a small army of cron jobs. And I still pay for frontier models.
+
+So when people find out I run local models too, the question is always the same:
+*Claude and GPT are stronger. Local models are a pain to set up, slower, and
+often just worse. So why did you spend money on a GPU?*
+
+If you are only comparing raw model capability, I agree with you completely.
+When I hit a genuinely new problem, do complex reasoning, plan a product
+architecture, or kick off a long agent run, I still reach for Claude or GPT
+first. I did not deploy local models because I think they have beaten the
+closed frontier.
+
+I did it because of a different problem that only showed up *after* AI got good
+enough to depend on. Everything I built started sitting on a service I do not
+control.
+
+---
+
+## The problem is not capability — it is dependency
+
+How many tokens you get, what a plan costs, which regions can buy the commercial
+service, when the model version changes under you — none of that is the user's
+decision. Anthropic has regional restrictions of its own; some markets cannot
+buy Claude's commercial service directly. OpenAI's plans, quotas and model
+lineup keep shifting. These are commercial services, so of course they change.
+
+That was fine when ChatGPT was something I opened occasionally to ask a
+question. If it was unavailable today, I would ask tomorrow and lose nothing.
+
+It stopped being fine once I started using AI to write code, run agents,
+analyze documents, handle support, organize company data, and wire models into
+workflows that run every day. When AI moves from *chat tool* to *infrastructure*,
+the question changes. It is no longer "which model is smarter?" It becomes:
+
+> "If I depend on this model today, and its terms change tomorrow, does my work
+> still run?"
+
+That is why I think the stronger the closed models get, the more valuable a
+local model becomes. Precisely *because* they are so good, people hand them more
+and more work. And the deeper the dependency, the more of your operation someone
+else controls.
+
+There is a second-order version of this that rarely gets said plainly. Every big
+lab talks about AI safety, model risk, and restricting certain content, regions
+or use cases — while all of them keep racing to make models stronger, because
+capability *is* the competitive advantage. Nobody stops, because if you stop,
+someone else does not.
+
+So what actually gets controlled is rarely whether AI keeps developing. It is
+who may use it, how, how much, and how far the vendor is willing to go. For an
+individual, that reads like a terms-of-service page. For a business, it is
+**supplier risk**. If your core process and every agent, tool and data pipeline
+around it are pinned to one vendor, you are not just buying tokens — you are
+handing over part of the steering wheel.
+
+---
+
+## "But local models are worse" — yes, and that is the wrong comparison
+
+This is the most honest objection, and it is true. Put a model that fits on a
+24GB or 32GB consumer card against Claude Opus and it loses on complex tasks. No
+argument.
+
+But the comparison most people run is wrong. The question is not *can Qwen beat
+Claude*. It is:
+
+> **Is the capability gap large enough to justify using the most expensive model
+> for this task, forever?**
+
+Because the trade is not symmetric. A 20–30B model is meaningfully behind the
+frontier — and costs one to two orders of magnitude less per token, or runs on
+hardware you already own. Different ratio, different decision.
+
+### I treat capability as a threshold, not a ranking
+
+If a task is below some capability line, it cannot be done at all — use the
+strong model. But once a model crosses that line, the things that decide how I
+actually work stop being benchmark scores and become cost, speed, whether it can
+run for hours, and whether I control the system.
+
+Concretely, here is how it splits for me:
+
+| Task | What I reach for |
+|---|---|
+| Vague problem, no clear direction yet | Frontier model — a wrong early call invalidates everything downstream |
+| Product architecture, data model, acceptance criteria | Frontier model |
+| Scaffolding a project to a fixed spec, editing files, running tests, fixing failures | Local model |
+| Long-running agents, retries, batch jobs, bulk content | Local model |
+
+The important shift is the second block. Once direction is settled and the agent
+is *executing a spec*, a single task may call a model 20–50 times. Do all fifty
+calls need the most expensive model on the market? Usually not. The agent is not
+deciding what the product should be — it is following specifications that are
+already written. If a local model understands those specs and knows how to use
+the tools, it has crossed the working threshold.
+
+---
+
+## Agents turn a small price difference into a large one
+
+With chat, you ask once and get one answer. If Claude costs many times more than
+a local model, you barely notice.
+
+Agents are not one call. A task can invoke the model twenty times, fifty times;
+it fails and reruns. The per-token price gap gets multiplied by every call.
+
+And anyone who has actually built these pipelines knows the expensive part is
+not the final run — it is the testing before it. You rewrite the prompt, swap
+tooling, re-chunk the data, adjust the workflow, and run the same thing ten or
+twenty times. That is normal.
+
+On metered API calls, I feel it: *I am burning money right now.* On my own card,
+there is still electricity and hardware cost, but running an agent five times or
+fifty times does not add a line to an invoice. That difference changes my
+behavior — it decides whether I dare keep testing. Most stable AI workflows are
+not written correctly the first time. They get there after failing enough times
+that you find out where they break.
+
+The value of local compute is that you move that trial-and-error into a fixed
+cost. That is a different question from "how much does a million tokens cost."
+
+---
+
+## Open models crossed the useful line, and the line keeps moving
+
+Three years ago, running a model on a consumer card that could genuinely do work
+for you was a research project. Today, 20–30B models — quantized — handle real
+work on 24GB and 32GB cards. Go up in VRAM and language, image, and now portions
+of video generation become self-hostable too.
+
+The important part is not "open source finally won." It is that work which used
+to be closed-model-only keeps falling inside the range that local hardware can
+do acceptably. If three years ago two of ten jobs could run locally, and today
+four or five can, that ratio alone has changed the value of owning the hardware.
+
+And when I buy a GPU, I am not buying a specific model. Models change. If a
+better coding model ships next month, I swap weights. If a better quantization
+scheme lands in six months, the same card suddenly fits a model it could not run
+before. What I actually bought is **compute I control**; the models on top of it
+are disposable.
+
+That is the structural difference. A closed model is stronger — and also exactly
+the version one company currently chooses to offer you, at terms it currently
+chooses. Local hardware is weaker — but as long as the card holds up, I decide
+which model runs today, which version I switch to, and how long it runs.
+
+---
+
+## Fixed compute vs. renting compute
+
+If AI stays a question-and-answer tool, I do not think most people need a GPU at
+all. But that is clearly not the direction. Agents run for long stretches. Image
+and video models want bulk generation. Companies hand more fixed processes to
+AI. We are not just *using models more often* — models are becoming a continuously
+working compute unit.
+
+At that point, owning part of the capacity and renting every execution become
+**two different cost structures**. Low usage: renting is obviously more
+convenient. High, steady usage: owning lets you control and predict cost.
+
+It also buys something no API can sell you — control. When the machine is mine,
+I can point it at internal data, connect my own tools, and run agents for hours
+without checking whether a vendor's plan, quota or limits changed overnight.
+Once AI is inside real company processes, this stops being about saving tokens.
+Some data should not leave the environment it lives in. Some agent permissions
+should not be handed wholesale to an external service. If the model runs on my
+own hardware, data, tools, permissions and execution can be designed together.
+*That* is what I mean by control — not merely having the weights on disk.
+
+---
+
+## What I actually evaluate now
+
+I no longer judge a local model by "it is dumber than Claude, therefore
+worthless." Four questions, in order:
+
+1. **Does this model clear my working threshold** for this specific job?
+2. **How many times will this task execute?**
+3. **Does it need heavy trial-and-error, or long agent runs?**
+4. **Does the data or system need to stay in my own environment?**
+
+When those stack up, a local model — even one that is not the strongest — can be
+the better tool for that slice of work. And as open models improve, that slice
+keeps getting bigger.
+
+So what I am bullish on is not one model or one GPU. It is the trend: **more AI
+work becomes doable on hardware you own.** A consumer card that used to run only
+language models now runs agents, image generation, video. Better models and
+better inference frameworks keep giving the same metal new jobs.
+
+Hardware depreciates, new cards get more expensive — that is real. But the model
+is not locked in. As long as the machine has enough compute, I can keep swapping
+in newer open models.
+
+---
+
+## The short version
+
+I no longer compare "which model is smarter." I compare **how much capability
+this job needs, how many times it will run, and how much control I am willing to
+leave in someone else's hands.**
+
+Where I need frontier reasoning, I pay for the frontier. But when work becomes
+high-volume, continuous and predictable — and a local model has crossed the
+usable threshold — there is no reason to keep paying frontier prices for every
+single step.
+
+In one line:
+
+> Use the frontier model to decide what to build. Use local compute to build it
+> a thousand times.
+
+That is why I bought the GPU.

src/content/posts/zh/why-i-still-bought-a-local-gpu.md

@@ -0,0 +1,195 @@
+---
+title: "Claude 和 GPT 明明更強，我為什麼還是買了 GPU 跑本地 AI"
+description: "Claude Opus 和 GPT 確實比我能在本地跑的任何模型都強。但我還是買了 GPU——這篇講成本、控制權與可用性，附實際的判斷標準。"
+pubDate: 2026-09-21
+category: ai
+tags: [local-llm, gpu, llama-cpp, lm-studio, agents, self-hosting, cost]
+ogImage: /og/why-i-still-bought-a-local-gpu.png
+banner: /banners/why-i-still-bought-a-local-gpu.png
+draft: false
+---
+
+我家裡塞滿了自架的東西——Docker stack、NAS、tunnel、一堆 cron job。但我
+照樣在付頂級模型的錢。
+
+所以每次別人知道我也在跑本地模型，問的都是同一句話：*Claude 和 GPT 就是
+比較強，本地模型又麻煩、又慢、效果還比較差，那你到底為什麼要花錢買 GPU？*
+
+如果只比模型能力，我完全同意。今天要處理一個全新的問題、做複雜推理、規劃
+產品架構、或者跑一個很長的 Agent 任務，我大部分時間還是先用 Claude、GPT。
+我部署本地模型，從來不是因為我覺得它已經打贏閉源模型。
+
+而是因為 AI 越來越好用之後，出現了另一個問題：我做的每一件事，都開始建立
+在一個我自己完全不能控制的服務上。
+
+---
+
+## 真正的問題不是能力，是依賴
+
+今天模型可以用多少、方案怎麼收費、什麼地區能不能用、版本什麼時候換掉，
+最後都不是使用者決定的。Anthropic 本來就有自己的地區限制，有些市場不能
+直接買 Claude 的商業服務；OpenAI 的方案、額度跟模型也一直在調。這些是商業
+服務，變動本來就可以預期。
+
+以前 ChatGPT 只是偶爾拿來問問題，今天不能用，我明天再問沒什麼差別。
+
+但當我開始拿 AI 寫程式、跑 Agent、分析文件、處理客服、整理公司資料，甚至
+把它接進每天都在跑的工作流程之後，問題就變了。當 AI 從聊天工具變成工作裡
+的基礎設施，你要考慮的就不只是「哪一個模型比較強」，而是：
+
+> 如果我今天很依賴這個模型，但有一天它的使用條件改掉，我原本的工作還能不能繼續？
+
+這就是為什麼我覺得，閉源模型越強，本地模型的重要性會越來越高。因為閉源
+模型太好用，大家就會把越來越多事情交給它；依賴越深的同時，也會越被這些
+閉源模型控制。
+
+還有一件事很值得講清楚。每一家大模型公司都在談 AI 安全、模型風險，甚至
+限制某些內容、地區或使用方式，但同時每一家又都還在拼命把模型做得更強——
+因為模型能力本身就是競爭力，你不做，別人還是會做。
+
+所以最後真正被控制的，往往不是 AI 要不要繼續發展，而是誰能用、怎麼用、
+用多少，以及模型公司願意提供到什麼程度。對一般人來說，這可能只是服務
+條款；對企業來說，這就是**供應商風險**。假設一間公司的核心流程，加上後面
+所有 Agent、工具、資料處理，全部綁在同一個供應商身上，那它其實不只是在買
+Token——它是在把一部分工作的主控權交出去。
+
+---
+
+## 「可是本地模型就是比較差啊」——對，但這是錯誤的比較方式
+
+這是最誠實的反對意見，而且它是事實。拿一個能放進 24GB、32GB 消費級顯卡的
+模型，直接跟 Claude Opus 全面比較，很多複雜任務就是會輸。這沒什麼好辯的。
+
+但大家很容易用了錯誤的比較方式。真正該問的不是「Qwen 能不能打贏 Claude」，
+而是：
+
+> **這個能力差距，到底有沒有大到值得你永遠用最貴的模型來做這件事？**
+
+因為這個交換並不對稱。二三十 B 的模型確實落後頂級模型一截，但成本低一到
+兩個數量級，或者直接跑在你已經買的硬體上。比例不同，決策就不同。
+
+### 我把模型能力看成分界線，不是排行榜
+
+如果一個任務低於某個能力門檻，那就是做不了，當然用強模型。但一旦跨過
+那條線，後面真正影響我使用方式的，就不再是 benchmark 分數，而是成本、
+速度、能不能長時間運行，以及我能不能控制這套系統。
+
+以我的實際分法：
+
+| 任務 | 我用什麼 |
+|---|---|
+| 問題還很模糊、連方向都不知道 | 頂級閉源模型——前面判斷錯了，後面做再多都沒意義 |
+| 產品架構、資料格式、驗收條件 | 頂級閉源模型 |
+| 照著已定規格建專案骨架、改檔案、跑測試、看到錯誤再修 | 本地模型 |
+| 長時間 Agent、重試、批次工作、大量內容產出 | 本地模型 |
+
+關鍵是第二塊。一旦方向確定、Agent 只是在執行規格，一個任務可能會呼叫模型
+二十次、五十次。這五十次真的每一次都需要市面上最貴的模型嗎？通常不需要。
+Agent 已經不是在幫我決定產品要怎麼做，它是在按照前面已經寫好的規格執行。
+只要本地模型可以正確理解這些規格、知道工具怎麼用，它就已經跨過可以工作的
+門檻。
+
+---
+
+## Agent 會把小小的價差一直放大
+
+用聊天，是問一次、回答一次。Claude 一次比本地模型貴很多，你也不會有特別
+感受。
+
+但 Agent 不是一次。一個任務可能呼叫模型二十次、五十次，失敗之後還會重新
+跑。模型單次價格的差距，放進這種連續任務裡就會被一直放大。
+
+而真正做過這些流程的人都知道，最花資源的往往還不是最後正式執行，而是前面
+測試。你會改 Prompt、換工具、重新切資料、調整工作流程，同一個東西跑十次、
+二十次很正常。
+
+每次都是 API，我自己就會很明顯感覺到：又在燒錢。用自己那張卡，當然還是有
+電費跟硬體成本，但我今天讓一個 Agent 跑五次還是五十次，不會每跑一次就多
+一筆帳單。這對我來說很重要，因為它會直接影響我敢不敢一直連續測試。很多
+真正穩定的 AI workflow，不是第一次寫完就成功，而是讓它失敗很多次之後，
+才慢慢找到什麼地方容易出問題。
+
+本地算力其中一個很大的價值，就是你可以把這些大量試錯搬回自己的固定成本
+裡。這跟單純比較「一百萬 Token 多少錢」其實是兩件事。
+
+---
+
+## 開放模型已經跨過可用的線，而這條線還在移動
+
+三年前，要在消費級顯卡上跑一個真的可以幫你工作的模型，限制很多。現在
+二三十 B 的模型經過量化之後，24GB、32GB 這種顯卡已經可以處理不少工作；
+再往更大的記憶體走，語言、圖片，甚至部分影片模型都開始可以自己部署。
+
+真正重要的不是「開源終於打贏閉源了」，而是原本只有閉源模型做得到的工作，
+正在一個一個掉進本地可以處理、而且可以被接受的範圍。假設三年前十個工作
+只有兩個能放本地，現在已經有四五個可以做——這個比例本身就已經改變了
+本地部署的價值。
+
+而且我買 GPU 的時候，不是在買 Qwen 這一顆模型。模型可以一直換，也會一直
+出現新的：如果之後出現一個更適合 Coding 的開放模型，我直接換權重；再過
+半年有更好的量化方式，同一張卡可能又可以裝進原本跑不動的模型。我真正買
+下來的，是**一塊自己可以控制的運算資源**，後面的模型可以一直替換。
+
+這就是本地跟閉源服務最不一樣的地方。閉源模型能力通常比較強，但你用的是
+某家公司現在願意提供給你的版本；本地硬體能力可能沒那麼高，但只要硬體
+撐得住，我可以自己決定今天跑什麼模型、換什麼版本、要跑多久。
+
+---
+
+## 固定算力 vs. 每次租算力
+
+如果 AI 永遠只是拿來問問題，我完全不覺得大家都需要買 GPU。但現在發展方向
+很明顯不是這樣：Agent 會長時間執行工作，圖片跟影片模型會需要大量生成，
+公司也會把更多固定流程交給 AI。我們不是單純使用模型的次數變多，而是模型
+開始變成一個持續工作的運算單位。
+
+這時候，「固定擁有一部分算力」跟「每一次執行都向外面租」，就會開始變成
+兩種不同的成本結構。使用量少，租一定比較方便；使用量夠大、夠固定之後，
+自己有算力就能更好控制與估算成本。
+
+它還多了一個閉源服務給不了的東西：主控權。當機器是我的，我要跑內部資料、
+接自己的工具、長時間執行 Agent，都不需要每次先確認供應商的方案、額度或者
+限制有沒有改。尤其公司真的開始把 AI 接進內部流程之後，這件事就不只是省
+Token——有些資料本來就不適合離開自己的環境，有些 Agent 權限也不可能直接
+全部交給外部服務。當模型跑在自己的環境裡，資料、工具、權限跟執行方式可以
+一起設計。這才是「控制權」真正指的東西，不只是把權重下載下來而已。
+
+---
+
+## 我現在實際怎麼判斷
+
+我不會再用「它比 Claude 笨，所以沒有價值」這種方式判斷一個本地模型。我會
+依序看四件事：
+
+1. **這顆模型能不能跨過我這個工作的門檻？**
+2. **這個任務會執行多少次？**
+3. **是不是需要大量試錯，或長時間的 Agent？**
+4. **這些資料跟系統，到底要不要留在自己的環境？**
+
+這幾個條件一疊起來，本地模型即使不是最強，還是可能比頂級閉源模型更適合
+那一段工作。而且隨著開放模型越來越強，這個範圍只會一直變大。
+
+所以我真正看好的，不是某一顆模型，也不是某一張 GPU，而是這個趨勢：
+**可以在自己硬體上完成的 AI 工作正在變多。** 以前買一張消費級顯示卡，
+可能只能拿來跑一些語言模型；現在開始可以接 Agent、圖片、影片，後面模型
+跟推理框架繼續優化，同樣的硬體又可能得到新的用途。
+
+硬體當然會折舊，新卡也可能越來越貴，但至少模型不是綁死的。只要這台機器
+還有足夠的運算能力，我就可以一直換新的開放模型。
+
+---
+
+## 一句話版本
+
+我現在已經不會只看哪個模型比較聰明。我比較的是：**這件工作到底需要多強的
+模型、它要被執行多少次，以及我願意把多少控制權留在外部服務手上。**
+
+需要頂級推理的地方，我照樣花錢用最強的模型。但當工作開始變成大量、連續、
+固定，而且本地模型的能力已經跨過可使用門檻時，我就沒有理由每一個步驟都
+繼續用最貴的閉源模型。
+
+濃縮成一句：
+
+> 用頂級模型決定要做什麼，用本地算力把它做上一千次。
+
+這就是我買 GPU 的原因。