diff --git a/public/banners/five-step-ai-roadmap-what-i-actually-run.png b/public/banners/five-step-ai-roadmap-what-i-actually-run.png new file mode 100644 index 0000000..8771c67 Binary files /dev/null and b/public/banners/five-step-ai-roadmap-what-i-actually-run.png differ diff --git a/public/og/five-step-ai-roadmap-what-i-actually-run.png b/public/og/five-step-ai-roadmap-what-i-actually-run.png new file mode 100644 index 0000000..e78cb9c Binary files /dev/null and b/public/og/five-step-ai-roadmap-what-i-actually-run.png differ diff --git a/scripts/banner-gen/generate.mjs b/scripts/banner-gen/generate.mjs index 03395f4..b4e60fa 100644 --- a/scripts/banner-gen/generate.mjs +++ b/scripts/banner-gen/generate.mjs @@ -1203,6 +1203,23 @@ BANNERS['does-hevc-actually-shrink-your-files'] = { ], }; +BANNERS['five-step-ai-roadmap-what-i-actually-run'] = { + titlebar: "root@desktop — five steps, three running", + lines: [ + { t: 'cmd', text: "roadmap: 5 steps → 1 fundamentals … 5 fine-tuning" }, { t: 'dim', text: "a syllabus says what to study, not what runs" }, + { t: 'cmd', text: "mem0 + LiteLLM + pgvector on the NAS · port 20015" }, { t: 'err', text: "502 conversion failed — multipart part had no Content-Type" }, + { t: 'cmd', text: "whisper large-v3 · RTX 3060 12 GB · 57 s video → 8 s" }, { t: 'ok', text: "POST /memories · GET /search — two calls, per user_id" }, + { t: 'hl', text: "toggle idle: 4,906 MiB → 1,321 MiB VRAM" }, { t: 'err', text: "step 5 fine-tuning: never done — and that is deliberate" }, + ], + flow: [ + { n: '1', label: "read the diagrams" }, + { n: '2', label: "prompt · automate" }, + { n: '3', label: "RAG · ingest" }, + { n: '4', label: "agents · loop" }, + { n: '5', label: "fine-tune — skipped" }, + ], +}; + // ---------- read frontmatter ---------- const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`); let category = 'devops'; diff --git a/scripts/og-gen/generate.mjs b/scripts/og-gen/generate.mjs index 078c097..82c4a83 100644 --- a/scripts/og-gen/generate.mjs +++ b/scripts/og-gen/generate.mjs @@ -367,6 +367,11 @@ TERMINALS['does-hevc-actually-shrink-your-files'] = `
$libx265 -crf 26 -> 56.7%
 min SSIM 0.988 · 22% smaller than NVENC-> measured
`; +TERMINALS['five-step-ai-roadmap-what-i-actually-run'] = ` +
$roadmap: 1 fundamentals · 2 prompting · 3 RAG · 4 agents · 5 fine-tuning
+
 step 5: never done — my failures were bytes, mounts and licences
+
$3 of 5 running here · 57 s video → text in ~8 s→ transcript ✓
`; + // ---------- read frontmatter ---------- const postPath = join(ROOT, 'src', 'content', 'posts', `${slug}.md`); if (!existsSync(postPath)) { diff --git a/src/content/posts/five-step-ai-roadmap-what-i-actually-run.md b/src/content/posts/five-step-ai-roadmap-what-i-actually-run.md new file mode 100644 index 0000000..a32a14b --- /dev/null +++ b/src/content/posts/five-step-ai-roadmap-what-i-actually-run.md @@ -0,0 +1,242 @@ +--- +title: "The Five-Step AI Roadmap Everyone Shares — and What I Actually Run in Production" +description: "The five-step AI roadmap circulates on every feed. Here is the honest version: which steps I run in production, which one I skipped, and what each is for." +pubDate: 2026-10-13 +category: ai +tags: ["llm", "rag", "agents", "self-hosting", "whisper", "mem0"] +ogImage: /og/five-step-ai-roadmap-what-i-actually-run.png +banner: /banners/five-step-ai-roadmap-what-i-actually-run.png +draft: false +--- + +There is a graphic that makes the rounds every few weeks, and by now it has +reached my feed in translated form, reposted by accounts I don't follow. It +lays out the path to working with large models in five steps: learn the +fundamentals — Transformer, neural networks, attention — and stop as soon as +you can read the diagrams; learn prompt engineering, which takes days; learn +RAG so the model answers from your own data instead of making things up; +learn agents, the biggest opportunity in the field; and then learn +fine-tuning, where the advice is to skip pretraining entirely (that belongs +to the big labs) and get comfortable with multimodal fine-tuning instead. +There is a salary band attached to each step, and the band at the end is the +reason the thing spreads. + +I am not going to argue with the list. It is not wrong — it is a syllabus. +And a syllabus is the kind of advice that behaves perfectly in a classroom +and then dies on contact with a system, because it tells you what to learn +and in what order, and never tells you which of those items you will still +be doing years later when somebody is paying you for the result. I have been +running AI workloads of my own for a while now — not research, not model +training, but real services other people depend on. So here is the honest +version of that graphic, checked against what is actually running on my +hardware tonight. + +## Why this matters + +The reason to distrust the roadmap is not that the items are wrong. It is +that a roadmap tells you what to *study*, while employment asks what you can +*operate*. Those are different skill sets, and the gap between them is where +most people who "learned AI in five steps" stall: they can describe a RAG +pipeline, but they have never had one break on them at 11pm in a way that +made them understand retrieval, cost, ingestion and evaluation in a single +evening. + +The rest of this post maps each step to something I actually run, states +plainly which steps I have never done, and gives you the order I would pick +if I had to start again. Every claim below is measured on one desktop with +an RTX 3060 (12 GB) and one NAS. + +## Step 1: Fundamentals — enough to be dangerous, then stop + +The graphic's framing here is closer to right than most engineers admit: +learn the Transformer until the diagrams make sense, then move on. My +version stopped earlier than that. I can hold a real conversation about +attention, tokens, context windows, quantization schemes and the KV cache, +because those are the levers I actually turn when a model has to fit into 12 +GB of VRAM and answer fast enough to be useful in a chat window. I could not +derive backpropagation on a whiteboard, and I would not trust a number I +produced from memory. + +What that costs me: nothing measurable so far, in systems that ship. What it +would cost me: the day I train something. If your goal is "own the AI stack +for a small business", step 1 is a reading list, not a degree — and the +honest reason to buy local hardware in the first place is not that a 12 GB +card beats the frontier models, because it does not; it is what running a +model yourself teaches you that an API never will. I wrote about that trade +honestly in +[why I still bought a local GPU to run AI](https://blog.hoelee.com/posts/why-i-still-bought-a-local-gpu/). + +## Step 2: Prompt engineering — the step with the lowest ceiling + +"Days to learn" is accurate, and that is the problem with it. Prompting is +the cheapest step to enter and the first one to get automated, so it is +where enormous numbers of people stop and start describing themselves as AI +engineers. The honest skill is not writing a better instruction; it is +deciding when an instruction should not exist at all. + +If a task has one correct answer, a deterministic input and a known output +shape, then a script is faster, cheaper and cannot hallucinate. I have +replaced agent prompts with forty lines of Python more than once, and +[wrote up the case for that](https://blog.hoelee.com/posts/stop-writing-agent-prompts-for-deterministic-work/). +What survives on the prompt side is interface design: what the model is +allowed to see, what shape its answer must take, how much of your own +context you are willing to hand it, and — the part the graphic never +mentions — how you would know the output got worse. That last item is +evaluation, and in my experience it is the difference between a demo and a +service. + +## Step 3: RAG — the hard half is everything except the retrieval + +The one-line version everyone quotes is "give the model your data so it +stops lying". I run exactly that: a self-hosted memory layer built from +mem0, LiteLLM and PostgreSQL 17 with pgvector, all on my own NAS, with +memories scoped to a `user_id` so one customer's history cannot surface in +another's answer — +[the whole stack is written up here](https://blog.hoelee.com/posts/self-hosting-mem0-memory-stack/), +including the container layout and the fact that the API is two small HTTP +calls. + +Three things the tutorial version of RAG skips, and each of them cost me +real time: + +1. **Ingestion is the boring half that breaks.** Retrieval quality gets all + the attention; getting the data in reliably is where the work is. The + most recent input I pushed through my own speech-to-text stack was a + 57-second video sitting on my desktop — it came back as text in about + eight seconds on the same 12 GB card that already serves live dictation + for my phone and laptop. That speed is what makes the stack usable + instead of a demo, and the pipeline is + [documented end to end here](https://blog.hoelee.com/posts/self-hosted-speech-to-text-api/). +2. **Byte-level mistakes look like model mistakes.** My first attempt to + feed that same service through a workflow tool reported a clean 502 + "conversion failed" from the audio layer. The real cause was a multipart + upload arriving without a `Content-Type` header, which made the broker + decode raw audio as UTF-8 text and destroy it before any of my code ran. + Nothing about that bug is AI. It is exactly the kind of failure the + five-step graphic cannot prepare you for. +3. **Retrieval is not a model problem.** When an answer is wrong, the useful + question is whether the right chunk was ever retrieved, whether it + survived chunking, and whether the model was given room to use it. + Swapping to a bigger model has never once fixed a retrieval bug for me. + +Underneath that entire layer there are two calls, which is worth seeing +once: + +```text +POST http://127.0.0.1:20015/memories # store a fact or a conversation, tagged by user_id +GET http://127.0.0.1:20015/search # semantic search, scoped to the same user_id +``` + +Nothing exotic — which is the point. A memory layer you can describe in two +lines is a memory layer you can debug, and the whole thing stays local. + +## Step 4: Agents — the opportunity is real, and the agent is the smallest part + +I agree with the graphic's ranking: this is where the work is. I disagree +with what it implies by the word "agent", because an agent is a model plus a +loop plus tools — and the part that decides whether it survives contact with +reality is the checking around it, not the model at the centre. I've written +about this as +[loop engineering](https://blog.hoelee.com/posts/loop-engineering-without-a-coding-agent/), +and the finding I keep coming back to is uncomfortable: most of my scheduled +automation has no AI in it at all, and those are precisely the jobs that +have never needed babysitting. The model is the part that adds judgement. +The rest of the system is what makes the judgement safe to trust. + +Three things that are actually running here, so the claim is concrete rather +than aspirational: + +- A Telegram support bot for my own digital-goods shop that keeps long-term + memory per customer, so a returning buyer is not a stranger on their + second message. The memory layer it writes into is the same self-hosted + mem0 stack from step 3. +- A scheduled job that manages discount settings in my Shopee seller + account, which is the least glamorous useful agent I own: it reads the + current state, decides, acts, and reports — and the entire value comes + from the fact that it checks the result afterwards instead of assuming its + click landed. +- A research run that compared twenty VPS providers in parallel by fanning + the work out across subagents, which is a pattern worth learning even if + you never build a "chatbot": + [the write-up is here](https://blog.hoelee.com/posts/how-i-vetted-20-vps-providers-with-parallel-subagents/). + +## Step 5: Fine-tuning — the step I have never done, on purpose + +The graphic says skip pretraining, which is correct, and then suggests +learning multimodal fine-tuning. For anyone whose job looks like mine, I'd +push back on the second half. + +I have never fine-tuned a model. Not because I couldn't, but because in the +time I have been running these services, none of my failures have been +model-behaviour failures. They were retrieval failures, licensing surprises, +byte-level failures and operational failures — a job that reported success +while the file it claimed to replace was untouched, a mount whose metadata +lied about which file it had swapped, an API whose real model name differed +from the one it advertised. A LoRA does not fix a schema drift. + +Where I would fine-tune: one narrow, well-defined task; thousands of +labelled examples rather than dozens; an inference bill or a latency ceiling +that a smaller tuned model would genuinely fix; and the willingness to own +an evaluation set, because a fine-tune you cannot measure is a regression +you cannot see. And I would rent the GPU for it long before I bought a +second card — the training run is a project, not a workstation purchase. + +## What I'd do differently + +If I were starting from zero with that graphic in front of me, I would not +walk it in numerical order. I would walk it in the order my problems +arrived, and accept that the first real lesson will not come from a course: + +- Treat step 1 as a reading list you finish in public documentation, not a + curriculum. Enough theory to know why quantization and context limits + hurt; no more until something forces you deeper. +- Spend a week on step 2, then immediately automate the parts of it that + have one correct answer, and move the rest of your attention to evaluation + — which the graphic omits entirely. +- Live in steps 3 and 4, because that is where nearly all employed AI work + currently sits: wiring models to real data, and making the loop around + them trustworthy. +- Keep step 5 in your back pocket until a specific problem names it. + Fine-tuning is a technique with a precondition, not a rung on a ladder. + +And notice the roadmap's real omission, which is not a step but half the +job: identity, storage, deployment, cost, monitoring, and who gets woken up +when it breaks. That is the half that decides whether a team can use you, +and it is the half this blog mostly writes about. + +## The result + +Three of those five steps run in production on one desktop GPU and one NAS, +and the honest scorecard is this: + +- A 57-second video becomes text in about eight seconds, on the same 12 GB + card that serves live dictation — and the speech server drops from **4,906 + MiB to 1,321 MiB** of VRAM when I toggle it idle, because a GPU that is + always resident is a GPU I cannot use for anything else. +- The memory layer answers in two small HTTP calls, scoped per user, backed + by PostgreSQL with pgvector on the NAS — mine, not a vendor's. +- Most of the scheduled automation that keeps this stack alive contains no + model at all, and it is the most reliable part of the system. +- Step 5 stays empty, and I would rather say that plainly than pad a + roadmap. The gap is real, and it is a choice. + +If the five-step graphic is the syllabus, this is the transcript. The +syllabus tells you what to learn. The transcript is what a recruiter or a +client can actually check. + +--- + +## Want an AI stack that runs in your business? + +I run Hoelee Enterprise, a website design and development business for +Malaysian SMEs — and the AI work above runs on the same infrastructure I +build for clients. If you want a model answering from *your* data, on *your* +hardware, without a per-seat subscription attached, I'd be glad to talk: + +- 📱 **WhatsApp:** [+60 12-797 2969](https://wa.me/60127972969) +- 📧 **Email:** + [me@hoelee.com](mailto:me@hoelee.com?subject=AI%20stack%20for%20my%20business) +- 🌐 **Website:** [hoelee.com](https://hoelee.com) + +And if you are hiring for a platform, DevOps or AI-infrastructure role, the +same channels reach me. diff --git a/src/content/posts/zh/five-step-ai-roadmap-what-i-actually-run.md b/src/content/posts/zh/five-step-ai-roadmap-what-i-actually-run.md new file mode 100644 index 0000000..ad86420 --- /dev/null +++ b/src/content/posts/zh/five-step-ai-roadmap-what-i-actually-run.md @@ -0,0 +1,103 @@ +--- +title: "全网都在转的 AI 五步学习路线 —— 我真正在生产里跑的是哪几步" +description: "AI 五步学习路线（基础、提示词、RAG、Agent、微调）到处都在转。这是诚实版：哪几步真的在跑，哪一步我跳过了，每一步到底是为了什么。" +pubDate: 2026-10-13 +category: ai +tags: ["llm", "rag", "agents", "self-hosting", "whisper", "mem0"] +ogImage: /og/five-step-ai-roadmap-what-i-actually-run.png +banner: /banners/five-step-ai-roadmap-what-i-actually-run.png +draft: false +--- + +每隔几周就会有一张图在各个信息流里转一圈，到我这里的时候已经是翻译过的版本，发它的账号我也没关注。图里把「做大模型」这条路分成五步：先学基础 —— Transformer、神经网络、注意力机制，学到能看懂图就停；再学提示词工程，几天就能上手；然后学 RAG，让模型基于你自己的数据回答，而不是张口乱编；接着学 Agent，说是这个领域最大的风口；最后是微调，这里给的建议是预训练完全不用碰（那是大厂的事），把多模态微调练熟就行。每一步后面都挂着一个薪资数字，而正是最后那个数字让这张图到处流传。 + +我不是来跟这张图抬杠的。它没说错 —— 它是一份教学大纲。而大纲这种东西，在教室里完美无缺，一碰到真实系统就失效：它只告诉你学什么、按什么顺序学，从来不告诉你，几年之后你还在做的是其中哪几件事、有谁愿意为结果付钱。我自己跑这一类负载已经有一段时间了 —— 不是做研究，也不是训练模型，而是别人真的在用的服务。所以下面是一个诚实版本：每一步对着我今晚机器上实际在跑的东西逐条核对。 + +## 为什么这件事值得说 + +不信任那张图的理由，不在于条目本身错了，而在于路线图回答的是「该学什么」，而工作问的是「你能不能让东西跑起来」。这是两套不同的能力，而多数「五步学完 AI」的人就卡在这道缝里：他们能描述一条 RAG 流水线，但从来没有在晚上十一点被它搞崩过一次，也就没有在一晚上里真正搞懂检索、成本、数据接入和评估这四件事。 + +接下来我会把五个步骤逐一对应到我实际在跑的东西上，明说哪几步我从来没做过，最后给出如果重新来一次我会选的顺序。下面所有说法，都跑在一台装了 RTX 3060（12 GB）的台式机和一台 NAS 上。 + +## 第一步：基础 —— 够用就停 + +图里在这点上比多数工程师愿意承认的更对：把 Transformer 学到能看懂图，然后往前走。我的进度比这还早一点就停了。我可以认真聊注意力、token、上下文窗口、量化方式和 KV cache，因为这些正是我在一台 12 GB 显卡上真正会去拧的旋钮 —— 模型要装得下，回答还得快到在聊天窗口里能用。但你让我在白板上推反向传播，我做不到，我也不会相信自己凭记忆报出来的数字。 + +代价是什么：到目前为止，在要交付的系统里，我没为这件事付过代价。什么时候会付：等我需要训练一个模型的那天。如果你的目标是「给一家小企业把 AI 这套东西撑起来」，第一步是一份书单，不是一个学位 —— 而买本地显卡真正诚实的理由，不是 12 GB 的卡能打赢前沿模型（它打不赢），而是「自己跑一遍」能教你 API 永远教不了的东西。这个取舍我写在[为什么我还是买了一张本地显卡](https://blog.hoelee.com/posts/zh/why-i-still-bought-a-local-gpu/)里。 + +## 第二步：提示词 —— 天花板最低的一步 + +「几天就能上手」是准确的，而这正是它的问题。提示词是门槛最低的一步，也是最早被自动化掉的一步，所以大量的人停在这里，然后开始管自己叫 AI 工程师。真正的能力不是把指令写得更漂亮，而是判断什么任务根本不该存在指令。 + +如果一个任务只有一个正确答案、输入是确定的、输出形状也是已知的，那脚本更快、更便宜，而且不会编造。我不止一次把 agent 的提示词换成四十行 Python，并把理由写进了[这里](https://blog.hoelee.com/posts/zh/stop-writing-agent-prompts-for-deterministic-work/)。提示词这一侧真正留下来的工作是接口设计：模型被允许看到什么、它的回答必须长成什么形状、你愿意把自己的多少上下文交出去，以及那张图从来不提的一件事 —— 你怎么知道它变差了。最后这件事就是评估，在我看来，它就是「演示」和「服务」之间的分界线。 + +## 第三步：RAG —— 难的那一半，跟检索没关系 + +大家都在引用的一句话是「把数据喂给模型，它就不乱说了」。我确实在跑这套东西：一个自建的记忆层，由 mem0、LiteLLM 和带 pgvector 的 PostgreSQL 17 组成，全部跑在自己的 NAS 上，记忆按 `user_id` 隔离，所以一个客户的记录不会跑到另一个客户的回答里 —— [整套栈都写在这里](https://blog.hoelee.com/posts/zh/self-hosting-mem0-memory-stack/)，包括容器怎么摆、以及它对外其实只是两个小小的 HTTP 调用。 + +教学版的 RAG 会跳过三件事，而每一件都花掉过我真实的时间： + +1. **数据接入是无聊的那一半，也是最容易坏的那一半。** 检索质量拿走全部注意力，但真正的工作是把数据可靠地送进去。我最近一次推进自己语音转文字服务的东西，是桌面上的一个 57 秒视频 —— 它变成文字花了大约八秒，用的还是那张同时给手机和笔记本提供实时听写的 12 GB 显卡。正是这个速度让它从演示变成能用的服务，整条链路[写在](https://blog.hoelee.com/posts/zh/self-hosted-speech-to-text-api/)这里。 +2. **字节级的错误，看起来像模型的错误。** 我第一次把同一个服务接到工作流工具上，拿到的是一个很干净的 502「conversion failed」。真正的原因是：multipart 上传里的文件部分少了 `Content-Type`，于是中间件把原始音频当成 UTF-8 文本解码，在我的任何一行代码跑起来之前就把它毁了。这个 bug 里没有一丝 AI 成分 —— 而它恰好就是那张五步图没办法帮你准备的失败。 +3. **检索不是一个模型问题。** 回答错了，该问的是：正确的片段到底有没有被检索出来、它在切片时有没有被切坏、模型有没有拿到足够的空间去用它。对我来说，换更大的模型从来没有修好过一个检索 bug。 + +这一层下面其实只有两个调用，值得看一眼： + +```text +POST http://127.0.0.1:20015/memories # store a fact or a conversation, tagged by user_id +GET http://127.0.0.1:20015/search # semantic search, scoped to the same user_id +``` + +没什么花活 —— 这正是重点。一个能用两行描述完的记忆层，才是你能调试的记忆层，而且它整条都留在本地。 + +## 第四步：Agent —— 机会是真的，但 Agent 是最小的一块 + +图里把它排在最值钱的位置，这点我同意：活儿确实在这里。我不同意的是「agent」这个词暗示的东西。一个 agent 是模型加循环加工具，而决定它能不能活得下去的，是围着它的那套检查，不是中间那个模型。我把这套做法写成过 [loop engineering](https://blog.hoelee.com/posts/zh/loop-engineering-without-a-coding-agent/)，而里面最让我不舒服的发现是：我大部分定时任务里根本没有 AI，偏偏就是这些任务从来没有需要我半夜起来救。模型负责的是判断力，系统的其余部分负责让这份判断力可以被信任。 + +我这边真正在跑的三件事，好让这句话不只是一句口号： + +- 一个给我自己数字商品店用的 Telegram 客服 bot，按客户保存长期记忆，所以回头客在发第二条消息的时候不是陌生人。它写入的记忆层，就是第三步那套自建的 mem0。 +- 一个定时任务，负责改我 Shopee 卖家后台的优惠券设置 —— 它是我最不性感但真有用的一类 agent：读当前状态、做决定、动手，然后汇报。它全部的价值都来自一个事实：它会去检查结果，而不是假设自己那一下点中了。 +- 一次把二十家 VPS 服务商并行比一轮的调研，做法是把活拆给多个子 agent 分头跑。这个模式值得学，哪怕你根本不打算做「聊天机器人」：[写法在这里](https://blog.hoelee.com/posts/zh/how-i-vetted-20-vps-providers-with-parallel-subagents/)。 + +## 第五步：微调 —— 我故意没做的一步 + +图里说预训练不用碰，这句是对的；但它接着说要去学多模态微调。对工作内容跟我类似的人来说，后半句我有异议。 + +我从来没有微调过任何模型。不是做不到，而是我跑这些服务这段时间里，没有任何一次失败是「模型行为」的失败。我的失败是检索失败、许可授权意外、字节级失败和运维失败 —— 一个报了成功、但它声称替换掉的文件根本没动的任务；一个对「到底换掉了哪个文件」撒谎的挂载点；一个真实模型名跟它宣传的不一样的 API。LoRA 修不了 schema 漂移。 + +我什么时候会去微调：任务足够窄且定义清楚；有成千上万条标注样本而不是几十条；推理账单或者延迟上限确实能被一个更小的调优模型解决；以及愿意自己维护一套评估集 —— 因为一个你无法测量的微调，就是一次你看不见的退化。而且真要做，我会先去租显卡，而不是买第二张 —— 训练是一次项目，不是一次装机。 + +## 如果重来一次，我会怎么做 + +如果让我拿着那张图从零开始，我不会按它的编号顺序走。我会按自己遇到的问题出现的顺序走，并且接受第一个真正的教训不会来自任何一门课： + +- 把第一步当成一份能在公开文档里读完的书单，而不是一门课程。懂到足以解释「为什么量化会和上下文上限互相打架」就够了，除非有事情逼你往深处走。 +- 第二步花一周，然后立刻把其中只有一个正确答案的部分自动化掉，剩下的注意力全部转到评估上 —— 那张图完全没提这件事。 +- 把时间花在第三、第四步上，因为现在市面上几乎所有有人付钱的 AI 工作都在这里：把模型接到真实数据上，以及让围着它的循环变得可信。 +- 第五步先放口袋里，等到某个具体问题点名它为止。微调是一门有前置条件的技术，不是楼梯上的一级。 + +还要注意那张路线图真正的遗漏：那不是一个步骤，而是这个工作的一半 —— 身份、存储、部署、成本、监控，以及出事的时候谁被叫醒。决定了团队能不能用你的，正是这一半，而这个博客写的也主要是这一半。 + +## 结果 + +那五步里有三步跑在一张桌面显卡和一台 NAS 上，诚实的成绩单是这样的： + +- 一个 57 秒的视频变成文字大约八秒，用的还是那张同时提供实时听写的 12 GB 显卡 —— 而我把语音服务切成空闲时，显存从 **4,906 MiB 掉到 1,321 MiB**，因为一张常年占着的显卡，就是一张我别的什么都干不了的显卡。 +- 记忆层两个小 HTTP 调用就给出答案，按用户隔离，后端是 NAS 上带 pgvector 的 PostgreSQL —— 是我的，不是某个厂商的。 +- 维持这套栈活着的定时任务里，大部分根本不含模型，而它们恰好是整个系统里最可靠的部分。 +- 第五步仍然是空的。我宁愿直说这件事，也不愿意把一份路线图注水填满。这个缺口是真的，而且是我选的。 + +如果那张五步图是教学大纲，这篇就是成绩单。大纲告诉你该学什么，成绩单是招聘方或者客户能真正核对的东西。 + +--- + +## 想让 AI 真正在你公司里跑起来？ + +我经营 Hoelee Enterprise，一家面向马来西亚中小企业做网站设计与开发的公司 —— 上面这些 AI 的东西，跑的就是我给客户搭的同一套基础设施。如果你想要一个基于**你自己**的数据、跑在**你自己**的硬件上、不挂按人头订阅费的模型，很欢迎聊聊： + +- 📱 **WhatsApp：** [+60 12-797 2969](https://wa.me/60127972969) +- 📧 **邮箱：** [me@hoelee.com](mailto:me@hoelee.com?subject=AI%20stack%20for%20my%20business) +- 🌐 **网站：** [hoelee.com](https://hoelee.com) + +如果你是招聘平台、DevOps 或 AI 基础设施岗位，同样的渠道也能找到我。