feat(posts): add 2 local TTS posts (EN+ZH) on CosyVoice 2 stack and percent-encoding post-mortem
Deploy / build (push) Successful in 25s
Deploy / build (push) Successful in 25s
This commit is contained in:
hoelee committed
1 parent
19023d2e4e
commit
6409f751a9
18 files changed
+637
-2
No files matched your search
@@ -0,0 +1,134 @@
|
||||
---
|
||||
title: "How I Replaced Cloud TTS with a Self-Hosted Neural Voice Stack on an RTX 3060"
|
||||
description: "How I migrated our production automation workflows from cloud speech APIs to a self-hosted CosyVoice 2 pipeline on a 12GB RTX 3060, integrated with Synology N8N."
|
||||
pubDate: 2026-10-22
|
||||
category: case-studies
|
||||
tags: ["tts", "ai", "n8n", "gpu", "cosyvoice", "self-hosting"]
|
||||
ogImage: "/og/self-hosted-tts-cosyvoice2-n8n-gpu.jpg"
|
||||
banner: "/banners/self-hosted-tts-cosyvoice2-n8n-gpu.jpg"
|
||||
draft: false
|
||||
---
|
||||
|
||||
For nearly two years, our internal business workflows, Telegram bots, and automated learning summaries relied on cloud-hosted text-to-speech APIs. Initially, Microsoft Edge-TTS provided a generous, zero-cost bridge. But running production workloads over unofficial browser-endpoint scraping is inherently fragile: sudden upstream schema changes, IP rate limits, and latency spikes across international transit routes routinely caused intermittent delivery drops. Commercial APIs like Azure Speech or Google Cloud TTS offered stability, but recurring meter billing scales awkwardly when generating long-form narration.
|
||||
|
||||
With a dedicated RTX 3060 12GB GPU passed through to a Windows 11 VM on our Unraid server, we decided to bring our entire speech synthesis stack fully on-premises.
|
||||
|
||||
Here is how we architected a self-hosted neural TTS pipeline using Alibaba's open-source **CosyVoice 2**, hooked it into our Synology DSM **n8n** automation cluster, managed GPU memory alongside our other local AI models, and achieved instant sub-second audio responses across Chinese, English, and mixed-language speech.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Design Goals
|
||||
|
||||
Before touching code, our target operational envelope required four non-negotiable criteria:
|
||||
|
||||
1. **Fluid Bilingual Code-Switching**: In Malaysian and Southeast Asian business communication, text is almost never purely monolingual. Sentences naturally mix Chinese syntax with English technical terms (e.g., *"请检查这个 pull request 并确认 webhook 状态"*). The engine must articulate both languages without sudden cadence jumps or unnatural accent shifts.
|
||||
2. **Zero-Shot Voice Variety**: We needed distinct voice profiles—cheerful child voices for storybook projects, authoritative male anchors for news roundups, and clear conversational female tones for support bots—cloned from brief reference clips without days of fine-tuning.
|
||||
3. **Constrained VRAM Footprint**: The RTX 3060 has 12GB of VRAM. Our VM regularly runs local LLMs (Qwen GGUF on LM Studio) and ComfyUI image/video pipelines. The TTS engine had to either idle under ~3.5GB of VRAM or provide frictionless cold-start toggles.
|
||||
4. **Drop-in Webhook Compatibility**: Our central automation runs on Synology DSM via n8n (`192.168.1.1:5678`). Replacing cloud nodes could not require rewriting dozens of downstream workflow schemas.
|
||||
|
||||
---
|
||||
|
||||
## 2. Model Selection: Why CosyVoice 2 Won
|
||||
|
||||
We evaluated the primary open-source contenders against our bilingual requirement:
|
||||
|
||||
| Model Architecture | Strengths | Drawbacks | Verdict |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Edge-TTS / Piper** | Ultra-lightweight, CPU-only | Robotic cadence, rigid voices, cloud dependency | Deprecated |
|
||||
| **ChatTTS** | Conversational laughter, pause tags | Unpredictable halluncinations, weak English | Not suitable |
|
||||
| **GPT-SoVITS v2** | Unrivaled character cloning accuracy | Phoneme boundaries stumble on mixed Chinese-English | Secondary / On-demand |
|
||||
| **CosyVoice 2 (0.5B)** | **Superb bilingual flow, 300ms first-packet latency** | 3.2GB VRAM baseline | **Primary Production Engine** |
|
||||
|
||||
CosyVoice 2 (the 0.5B parameter variant from Alibaba's FunAudioLLM team) emerged as the definitive winner. By combining an autoregressive language model with flow-matching diffusion decoders, it treats cross-lingual text-to-speech as a unified sequence. English terms embedded inside Chinese phrases flow with native pronunciation and natural human breath pauses.
|
||||
|
||||
---
|
||||
|
||||
## 3. Infrastructure & Network Topology
|
||||
|
||||
Our home-lab runs on a tiered hardware split:
|
||||
|
||||
```
|
||||
[ Internet / Clients ]
|
||||
│
|
||||
▼
|
||||
[ Synology DSM NAS (192.168.1.1) ]
|
||||
├── Portainer / Traefik Reverse Proxy
|
||||
└── n8n Automation Engine (Stack 272)
|
||||
│ HTTP POST / GET (/webhook/ltts)
|
||||
▼ (Local 1Gbps LAN)
|
||||
[ Unraid 7.x Server (192.168.1.6) ]
|
||||
└── Windows 11 VM (GPU Passthrough)
|
||||
├── NVIDIA GeForce RTX 3060 12GB
|
||||
└── CosyVoice 2 FastAPl Service (127.0.0.1:50000)
|
||||
```
|
||||
|
||||
The Windows VM hosts the CUDA 12.x PyTorch runtime. We exposed a lightweight FastAPI wrapper inside CosyVoice’s runtime script (`webui.py`), allowing our Synology n8n instance to trigger generation over LAN via `http://192.168.1.123:50000/tts`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Managing GPU Memory: Resident vs. On-Demand
|
||||
|
||||
Leaving every AI model loaded simultaneously will exhaust 12GB of VRAM instantly. We designed a dual-tier lifecycle:
|
||||
|
||||
1. **CosyVoice 2 as the Resident Daemon**: Consuming only **3.2 GB of VRAM**, CosyVoice 2 runs as an auto-starting background service when Windows boots (`shell:startup\CosyVoice 2.lnk`). This leaves 8.8 GB free for general desktop and lightweight inference.
|
||||
2. **One-Click State Toggle Batches**: For heavy tasks (such as running 1080p video generation in ComfyUI or 14B LLMs), we created an idempotent batch script on the desktop:
|
||||
|
||||
```bat
|
||||
@echo off
|
||||
title CosyVoice 2 Toggle
|
||||
for /f "tokens=5" %%a in ('netstat -aon ^| findstr :50000') do (
|
||||
taskkill /F /PID %%a >nul 2>&1
|
||||
echo STOPPED. GPU memory freed.
|
||||
exit /b
|
||||
)
|
||||
:: If stopped, launch background worker
|
||||
start /b python webui.py --port 50000 --model_dir pretrained_models/CosyVoice2-0.5B
|
||||
echo ONLINE - CosyVoice 2 loaded to GPU.
|
||||
```
|
||||
|
||||
A single double-click halts the process and reclaims all 3.2GB within 500ms; clicking it again restores the service in under 25 seconds.
|
||||
|
||||
---
|
||||
|
||||
## 5. Integrating with Synology N8N
|
||||
|
||||
In n8n, we configured a dedicated workflow: **`Local TTS API (CosyVoice)`** listening on `/webhook/ltts`.
|
||||
|
||||
To ensure total interoperability with existing scripts, the workflow accepts both:
|
||||
- **`GET /webhook/ltts?text=Hello&voice=child`**: Ideal for browser testing and simple URL triggers.
|
||||
- **`POST /webhook/ltts`** with JSON body `{"text": "...", "voice": "..."}`: Used by our bots and backend services.
|
||||
|
||||
The n8n pipeline consists of three streamlined nodes:
|
||||
1. **Webhook Node**: Receives incoming requests.
|
||||
2. **Parse & Clean Text (Code Node)**: Normalizes character encodings and defaults voice selections.
|
||||
3. **HTTP Request Node**: Forwards the JSON payload to `http://192.168.1.123:50000/tts` and streams the binary `audio/wav` response directly back to the requester.
|
||||
|
||||
---
|
||||
|
||||
## 6. Curating Zero-Shot Reference Voices
|
||||
|
||||
CosyVoice 2 synthesizes speech using zero-shot voice cloning: feeding a 3- to 5-second clean audio sample allows the model to clone the acoustic timbre instantly.
|
||||
|
||||
We packaged a library of curated reference samples into `asset/voices/` and mapped them to intuitive alias parameters:
|
||||
|
||||
| Voice Key | Style Profile | Sample Source |
|
||||
| :--- | :--- | :--- |
|
||||
| `child` / `girl` | Cheerful female child voice | Clean 4s preschool dialogue clip |
|
||||
| `boy` | Energetic young boy (anime style) | 5s youth animation dialogue |
|
||||
| `male` / `yunxi` | Engaging, modern tech male anchor | 5s professional narration |
|
||||
| `news` | Deep, formal broadcast presenter | 5s documentary voiceover |
|
||||
| `female` | Warm, conversational support agent | 5s acoustic reading sample |
|
||||
| `default` | Original CosyVoice high-fidelity voice | Model's baseline reference |
|
||||
|
||||
Switching voices is as simple as passing `&voice=child` in the query string or `{"voice": "male"}` in the POST payload.
|
||||
|
||||
---
|
||||
|
||||
## 7. Real-World Performance
|
||||
|
||||
Running on the RTX 3060:
|
||||
- **Generation Speed**: Real-time factor (RTF) averages **0.18**. A 10-second sentence renders in under **1.8 seconds**.
|
||||
- **Latency**: First audio packet arrives at n8n in approximately **320ms**, enabling near-instant voice replies on Telegram.
|
||||
- **Cost & Privacy**: Zero monthly cloud API charges, zero audio data leaving our local subnet, and complete immunity to external provider outages.
|
||||
|
||||
Self-hosting modern neural TTS is no longer an exercise in brittle compile scripts and robotic phoneme tables. With open-source diffusion models and a modest 12GB workstation GPU, you can run a private, high-fidelity voice engine that surpasses commercial cloud offerings in customization and control.
|
||||
@@ -0,0 +1,179 @@
|
||||
---
|
||||
title: "Why Is My TTS Model Repeating \"Percent\"? The URL-Encoding Trap in Speech Pipelines"
|
||||
description: "Why our newly deployed local TTS engine synthesized endless 'percent' repetitions instead of spoken Chinese, and how character normalization and n8n sandbox limits collided."
|
||||
pubDate: 2026-10-22
|
||||
category: notes
|
||||
tags: ["tts", "debugging", "n8n", "urlencode", "python", "ai"]
|
||||
ogImage: "/og/why-is-my-tts-model-repeating-percent.jpg"
|
||||
banner: "/banners/why-is-my-tts-model-repeating-percent.jpg"
|
||||
draft: false
|
||||
---
|
||||
|
||||
We had just wired Alibaba's open-source CosyVoice 2 into our home-lab automation cluster. The direct Python test scripts worked cleanly, but when we fired our first test request through a Synology-hosted n8n Webhook, the audio response sounded like an alien countdown.
|
||||
|
||||
Instead of the Chinese phrase *"恭喜！本地模型通过 N8N 联动测试成功！"* (Congratulations! Local model test via n8n succeeded!), the speaker rattled off a robotic avalanche:
|
||||
|
||||
> *"Percent E 6, percent 81, percent 81, percent B 9, percent A 7, percent 9... percent..."*
|
||||
|
||||
For fifteen agonizing seconds, the model faithfully enunciated dozens of hexadecimal percentage codes.
|
||||
|
||||
Here is the post-mortem of why this happened, why neural speech text frontends amplify subtle encoding oversights, and the two defensive layers required to make speech webhooks robust.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Symptom: The "Percent" Loop
|
||||
|
||||
We tested the n8n endpoint via a standard HTTP GET request from a terminal:
|
||||
|
||||
```bash
|
||||
curl -G "http://192.168.1.1:5678/webhook/ltts" \
|
||||
--data-urlencode "text=恭喜！本地模型测试成功！" \
|
||||
--data-urlencode "voice=child" \
|
||||
-o test.wav
|
||||
```
|
||||
|
||||
The server returned HTTP 200 and a valid 2.4MB WAV audio file. But when passed to a speech-to-text transcriber to inspect the synthesized words, the transcription revealed the disaster:
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "percent E 6 percent 81 percent 81 percent B 9 percent A 7 percent..."
|
||||
}
|
||||
```
|
||||
|
||||
The model had not hallucinated; it had pronounced the exact string it received with meticulous phonetic clarity.
|
||||
|
||||
---
|
||||
|
||||
## 2. Root Cause 1: Double-Encoding & Missing Query Decoding
|
||||
|
||||
To understand why, you have to trace how query parameters travel through a webhook:
|
||||
|
||||
1. When `curl` (or a web browser) sends a GET request, Unicode characters are percent-encoded into ASCII octets (UTF-8 bytes formatted as `%XX`). For example, the Chinese character `恭` (Unicode `U+606D`) becomes `%E6%81%AD`.
|
||||
2. When the n8n Webhook node received the GET query parameters, it passed the raw string directly into the node context:
|
||||
```json
|
||||
{
|
||||
"query": {
|
||||
"text": "%E6%81%AD%E5%96%9C%EF%BC%81..."
|
||||
}
|
||||
}
|
||||
```
|
||||
3. In our initial n8n HTTP Request node, the outgoing JSON body was constructed with a direct template string:
|
||||
```json
|
||||
{
|
||||
"text": "={{ $json.query.text }}"
|
||||
}
|
||||
```
|
||||
4. n8n serialized this raw percent-encoded string into the JSON payload forwarded to our local FastAPI service on `127.0.0.1:50000`.
|
||||
5. The Python backend received the string `"%E6%81%AD..."` literally.
|
||||
|
||||
---
|
||||
|
||||
## 3. Why LLM-Based Speech Frontends Don't Auto-Decode
|
||||
|
||||
In traditional text processing, non-ASCII characters might trigger an encoding error or render as replacement diamonds (``).
|
||||
|
||||
However, modern neural TTS systems like CosyVoice 2 and GPT-SoVITS rely on **Text Normalization (TN)** modules and byte/token vocabularies designed to read arbitrary technical jargon, symbols, and mathematical notation.
|
||||
|
||||
When the Text Normalizer encountered `%`:
|
||||
- It did **not** treat `%E6%81` as an escape sequence.
|
||||
- It treated `%` as the literal mathematical symbol **"percent"**.
|
||||
- It treated `E`, `6`, and `8` as individual alphanumeric characters.
|
||||
|
||||
Consequently, a 12-character Chinese sentence was translated into a 36-token alphanumeric tongue-twister. The model simply did what it was trained to do: speak whatever was written.
|
||||
|
||||
---
|
||||
|
||||
## 4. Root Cause 2: The N8N Sandbox Trap
|
||||
|
||||
We attempted to fix this inside n8n by adding an inline expression:
|
||||
|
||||
```javascript
|
||||
// The trap inside n8n node expression:
|
||||
{{ decodeURIComponent($json.query.text) }}
|
||||
```
|
||||
|
||||
This immediately broke on two edge cases:
|
||||
1. **POST requests don't have `$json.query`**: Accessing `$json.query.text` on a POST JSON call threw `Cannot read property 'text' of undefined`.
|
||||
2. **Optional Chaining (`?.`) syntax limitations**: In older embedded V8 runtimes used by certain n8n releases, modern ECMAScript features like `$json.query?.text` can cause silent parse failures or throw `Unexpected token '.'`.
|
||||
3. **Double-decoding URI errors**: If the input string contains an unescaped `%` (such as *"The price dropped by 50%"*), a naked `decodeURIComponent("50%")` throws an unhandled `URIError: URI malformed`, terminating the entire workflow.
|
||||
|
||||
---
|
||||
|
||||
## 5. The Two-Layer Defense
|
||||
|
||||
To make the speech pipeline bulletproof against any client library, we implemented defensive decoding at both the orchestration layer and the model backend.
|
||||
|
||||
### Layer 1: N8N Code Node Pre-Processing
|
||||
|
||||
Between the Webhook node and the HTTP Request node, we inserted a dedicated JavaScript **Code Node** that safely extracts and sanitizes incoming text from either GET queries or POST bodies:
|
||||
|
||||
```javascript
|
||||
const item = $input.first().json;
|
||||
const query = item.query || {};
|
||||
const body = item.body || {};
|
||||
|
||||
let text = query.text || body.text || '语音测试';
|
||||
let voice = query.voice || body.voice || 'default';
|
||||
|
||||
// Safe percent-decoding with error fallback
|
||||
if (typeof text === 'string' && text.indexOf('%') !== -1) {
|
||||
try {
|
||||
text = decodeURIComponent(text.replace(/\+/g, ' '));
|
||||
} catch (err) {
|
||||
// Retain original string if it is not valid percent-encoding
|
||||
}
|
||||
}
|
||||
|
||||
return [{
|
||||
json: {
|
||||
text: text.trim(),
|
||||
voice: String(voice).trim()
|
||||
}
|
||||
}];
|
||||
```
|
||||
|
||||
### Layer 2: Backend Python Fallback
|
||||
|
||||
On the receiving FastAPI side (`webui.py`), we added an unconditional `unquote` step in the request handler:
|
||||
|
||||
```python
|
||||
import urllib.parse
|
||||
|
||||
@app.post("/tts")
|
||||
async def api_tts(request: Request):
|
||||
body = await request.json()
|
||||
raw_text = body.get("text", "").strip()
|
||||
|
||||
# Defensive unquote handles both encoded and pre-decoded strings
|
||||
text = urllib.parse.unquote(raw_text).strip()
|
||||
voice = body.get("voice", "default")
|
||||
|
||||
# Pass clean string to the audio generator...
|
||||
```
|
||||
|
||||
`urllib.parse.unquote` is idempotent: if the string is already plain Chinese or English, it returns it unchanged. If residual percent signs from an upstream proxy survive, they are decoded before hitting the tokenizer.
|
||||
|
||||
---
|
||||
|
||||
## 6. The Result
|
||||
|
||||
After deploying both layers, we re-ran the exact same `curl` query:
|
||||
|
||||
```bash
|
||||
curl -G "http://192.168.1.1:5678/webhook/ltts" \
|
||||
--data-urlencode "text=恭喜！本地模型通过 N8N 联动测试成功！" \
|
||||
--data-urlencode "voice=child" \
|
||||
-o test_clean.wav
|
||||
```
|
||||
|
||||
We verified the output through our local Whisper/SenseVoice ASR engine:
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "恭喜！本地模型通过N8N联动测试成功！"
|
||||
}
|
||||
```
|
||||
|
||||
The "percent" loop vanished, replaced by crisp, natural speech.
|
||||
|
||||
When building AI pipelines that bridge HTTP gateways with neural text frontends, never assume the string arriving at your API boundary is in its final semantic form. Defensive decoding at both ends of the bridge costs fractions of a millisecond—and prevents your AI from reading URL headers as bedtime stories.
|
||||
@@ -0,0 +1,128 @@
|
||||
---
|
||||
title: "从云端转本地：在 12GB 显卡与群晖 N8N 上搭建中英混读高保真 TTS 语音栈"
|
||||
description: "记录如何将内部自动化工作流从依赖公网 TTS API 迁移至本地 RTX 3060 12GB 显卡，基于 CosyVoice 2 与群晖 N8N 实现中英混读与多音色零样本克隆。"
|
||||
pubDate: 2026-10-22
|
||||
category: case-studies
|
||||
tags: ["tts", "ai", "n8n", "gpu", "cosyvoice", "self-hosting"]
|
||||
ogImage: "/og/zh-self-hosted-tts-cosyvoice2-n8n-gpu.jpg"
|
||||
banner: "/banners/self-hosted-tts-cosyvoice2-n8n-gpu.jpg"
|
||||
draft: false
|
||||
---
|
||||
|
||||
在过去两年中，我们的内部业务自动化管线、Telegram 智能机器人以及视频知识库转写系统，一直依赖云端 Text-to-Speech（文本转语音）服务。起初，社区封装的 Microsoft Edge-TTS 提供了一个零成本的过渡方案；但依赖公网未公开接口的爬虫调用本质上极其脆弱——微软端点频繁变更签名、国内与海外网络抖动、以及跨国链路的偶发限流，经常导致自动化链路在半夜静默失败。而转用商业级 Azure Speech 或 Google Cloud TTS，在面对大批量长文本合成时，按字符计费的月度账单又显得不合算。
|
||||
|
||||
随着我们在 Unraid 服务器上为 Windows 11 虚拟机配置了直通的 **NVIDIA RTX 3060 12GB** 独立显卡，我们决定将整套语音合成系统完全收敛到本地局域网。
|
||||
|
||||
本文记录我们如何选型阿里巴巴开源的 **CosyVoice 2**，将其接入群晖 NAS 的 **n8n** 自动化集群，实现显存按需管理与开机常驻，并构建覆盖女童、正太、青年解说与纯正中英双语的零样本语音栈。
|
||||
|
||||
---
|
||||
|
||||
## 1. 核心需求与设计指标
|
||||
|
||||
在开工搭建之前，我们确立了四个关键要求：
|
||||
|
||||
1. **丝滑的中英混读（Code-switching）能力**：在马来西亚与东南亚的业务沟通场景中，日常对话几乎都是中英双语交织（例如：“*请确认这个 webhook 已经 trigger，并检查 logs*”）。传统的 TTS 模型在遇到英文字词时往往会生硬卡顿、甚至读错音素，引擎必须能够自然过渡。
|
||||
2. **零样本（Zero-shot）多音色快速切换**：需要支持多种人设音色——讲故事需要的活泼童声、科技资讯需要的沉稳播音男声、以及客服交互需要的自然女声，且无需耗时数小时重新微调。
|
||||
3. **显存共存与无痛调度**：12GB 显存是宝贵资产，这台工作机平时还要承载 LM Studio 的 Qwen 大模型推理以及 ComfyUI 生图生视频任务。TTS 引擎要么显存占用极小，要么能够随时一键启停释放显存。
|
||||
4. **无缝替换群晖 n8n 旧节点**：自动化调度中心跑在群晖 DSM 的 n8n (`192.168.1.1:5678`) 上，本地新接口必须兼容通用 HTTP Webhook，无需重构现有工作流业务逻辑。
|
||||
|
||||
---
|
||||
|
||||
## 2. 模型选型：为什么是 CosyVoice 2？
|
||||
|
||||
针对中文、英文以及核心的**中英混读**场景，我们横向对比了目前主流的本地开源方案：
|
||||
|
||||
| 模型方案 | 优势 | 不足 | 选型结论 |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| **Edge-TTS / Piper** | 超轻量、纯 CPU 即可跑 | 音质机械感明显、无法自主克隆 | 彻底淘汰 |
|
||||
| **ChatTTS** | 拟人化笑声与叹气细节好 | 经常胡言乱语、英文发音偏弱、长句难控 | 仅供玩票 |
|
||||
| **GPT-SoVITS (v2)** | 特定人物音色克隆还原度极高 | 语种切换边界容易卡顿顿挫、中英混读较硬 | 备选 / 按需冷启 |
|
||||
| **CosyVoice 2 (0.5B)** | **中英混读极度自然、首包延迟仅 300ms** | 常驻需要约 3.2GB 显存 | **生产主力首选** |
|
||||
|
||||
阿里开源的 **CosyVoice 2**（0.5B 自回归语言模型 + 流匹配 Flow Matching 解码器）在多语种混排上展现出了目前开源界的最佳水准。英文单词直接嵌入在中文句子中读出时，发音地道、重音准确，完全没有传统音素拼接的违和感。
|
||||
|
||||
---
|
||||
|
||||
## 3. 局域网网络与硬件拓扑
|
||||
|
||||
整套系统的部署拓扑如下：
|
||||
|
||||
```
|
||||
[ 外部调用 / 终端应用 ]
|
||||
│
|
||||
▼
|
||||
[ 群晖 DSM NAS (192.168.1.1) ]
|
||||
├── 反向代理网关 (Traefik / Nginx)
|
||||
└── n8n 自动化调度集群 (Stack 272)
|
||||
│ 局域网 HTTP POST / GET (/webhook/ltts)
|
||||
▼ (千兆内网无公网延迟)
|
||||
[ Unraid 7.x 主机 (192.168.1.6) ]
|
||||
└── Windows 11 虚拟机 (GPU 直通)
|
||||
├── NVIDIA GeForce RTX 3060 12GB
|
||||
└── CosyVoice 2 FastAPI 服务 (127.0.0.1:50000)
|
||||
```
|
||||
|
||||
通过在 CosyVoice 的服务脚本中直接挂载 FastAPI 路由，我们在 Windows 本地 `50000` 端口暴露出 `/tts` 端点，内网其他服务只需向 `http://192.168.1.123:50000/tts` 发送请求即可在 2 秒内拿回二进制 WAV 音频。
|
||||
|
||||
---
|
||||
|
||||
## 4. 显存治理：常驻守护与一键开关
|
||||
|
||||
12GB 显存如果任由各个 AI 工具同时加载，瞬间就会爆显存（OOM）。我们采取了“分级常驻 + 幂等开关”策略：
|
||||
|
||||
1. **低显存常驻（日常状态）**：CosyVoice 2 推理仅占用 **3.2 GB 显存**，剩余近 9 GB 显存完全足以同时运行日常的系统桌面和轻量任务。我们将其写入 Windows 开机启动项（`shell:startup\CosyVoice 2.lnk`），开机静默常驻后台。
|
||||
2. **一键切换脚本（高负载避让）**：当需要运行 ComfyUI 渲染整首 MV 或加载 14B 大语言模型时，桌面上放有一个免人工干预的批处理快捷方式 `CosyVoice 2 (开·关).bat`：
|
||||
|
||||
```bat
|
||||
@echo off
|
||||
title CosyVoice 2 Toggle
|
||||
for /f "tokens=5" %%a in ('netstat -aon ^| findstr :50000') do (
|
||||
taskkill /F /PID %%a >nul 2>&1
|
||||
echo STOPPED. GPU 显存已完全释放。
|
||||
exit /b
|
||||
)
|
||||
:: 若未运行，拉起后台推理服务
|
||||
start /b python webui.py --port 50000 --model_dir pretrained_models/CosyVoice2-0.5B
|
||||
echo ONLINE - CosyVoice 2 已加载至显卡，服务就绪。
|
||||
```
|
||||
|
||||
双击一次即刻杀掉进程、半秒内退还 3.2GB 显存；渲染完毕后再双击一次，25 秒内重新完成模型加载。
|
||||
|
||||
---
|
||||
|
||||
## 5. 群晖 N8N 联动接入
|
||||
|
||||
我们在群晖 n8n 上配置了名为 **`Local TTS API (CosyVoice)`** 的独立工作流，绑定路径为 `/webhook/ltts`：
|
||||
|
||||
- **GET 支持**：`http://192.168.1.1:5678/webhook/ltts?text=你好Hello&voice=child`，浏览器或监控告警可直接单链调用下载。
|
||||
- **POST 支持**：接收标准 JSON 体 `{"text": "...", "voice": "..."}`，供各类聊天机器人与后台脚本程序化触发。
|
||||
|
||||
在工作流内部，我们使用 JavaScript Code 节点对传入文本做字符解码与清洗，再通过 HTTP Request 节点转发给本地 50000 端口，最终以 `firstEntryBinary` 模式直接将音频流送还客户端。
|
||||
|
||||
---
|
||||
|
||||
## 6. 音色库扩展：零样本克隆实践
|
||||
|
||||
得益于 CosyVoice 2 的 Zero-shot 能力，只需提供 3~5 秒无杂音的人声样音，模型即可实时复刻其声线特征。我们在服务内部预制了常用音色表：
|
||||
|
||||
| 音色代号 | 声音风格 | 样音来源与特色 |
|
||||
| :--- | :--- | :--- |
|
||||
| `child` / `girl` | 活泼女童声 | 纯净 4 秒学前童声对话，清脆甜美 |
|
||||
| `boy` | 清脆男童声（正太音） | 5 秒少儿动漫旁白 |
|
||||
| `male` / `yunxi` | 阳光青年男声 | 5 秒科技解说，**中英专业词汇混读极佳** |
|
||||
| `news` | 沉稳新闻男声 | 5 秒纪录片解说样本 |
|
||||
| `female` | 温暖知性女声 | 5 秒电台散文朗诵 |
|
||||
| `default` | 旗舰高保真女声 | CosyVoice 官方基准母声 |
|
||||
|
||||
调用时只需指定 `voice` 参数，无需重启后端或切换模型权重，即可实现毫秒级换人合成。
|
||||
|
||||
---
|
||||
|
||||
## 7. 运行成效
|
||||
|
||||
在本地 RTX 3060 上运行实测：
|
||||
- **生成效率**：实时率（RTF）稳定在 **0.18** 左右，合成一段 10 秒的标准语音仅耗时约 **1.8 秒**。
|
||||
- **首包响应**：n8n 收到请求至音频开始回传约 **320 毫秒**，完全满足交互式即时语音应答的要求。
|
||||
- **自主掌控**：不再受制于第三方厂商的封号、限频与网络审查，所有音频资产与 Prompt 完全留存在本地内网。
|
||||
|
||||
自托管高品质 AI 语音不再是大型服务器集群的专利。借助新一代自回归+扩散语音架构与消费级显卡，一套完全受控、零月租且音质超越商业云服务的全天候语音栈，已经完全可以在普通 Home-lab 环境中稳定运转。
|
||||
@@ -0,0 +1,179 @@
|
||||
---
|
||||
title: "为什么本地语音合成一直在念 “Percent”？语音管线里的 URL 编码与沙盒避坑记"
|
||||
description: "记录本地部署 CosyVoice 2 并通过群晖 n8n 对接时，语音合成陷入无限重复 “Percent” 的现象剖析，以及字符前端规范化与 n8n 沙盒表达式避坑。"
|
||||
pubDate: 2026-10-22
|
||||
category: notes
|
||||
tags: ["tts", "debugging", "n8n", "urlencode", "python", "ai"]
|
||||
ogImage: "/og/zh-why-is-my-tts-model-repeating-percent.jpg"
|
||||
banner: "/banners/why-is-my-tts-model-repeating-percent.jpg"
|
||||
draft: false
|
||||
---
|
||||
|
||||
刚把阿里巴巴开源的 CosyVoice 2 模型部署到局域网并挂载上群晖 NAS 的 n8n 自动化工作流，在控制台用 Python 脚本直调合成一切正常。然而，当我们第一次尝试通过 n8n 的 Webhook 端点发起 HTTP GET 测试时，音箱里传出来的却是一段令人窒息的“外星密码”。
|
||||
|
||||
原本传入的中文台词是：*“恭喜！本地 CosyVoice 2 模型通过群晖 N8N 联动测试成功！”*
|
||||
|
||||
但生成的音频却以极其严肃认真的语调，一口气连念了十几秒：
|
||||
|
||||
> *“Percent E 6, percent 81, percent 81, percent B 9, percent A 7, percent 9... percent...”*
|
||||
|
||||
模型没有崩溃，甚至声线极为清晰、咬字极其标准，但它在不断疯狂重复 “Percent”。
|
||||
|
||||
这是一起非常典型、但在语音合成自动化链路中极容易踩中的“文本前端（Text Normalization）灾难”。本文记录这起 Bug 的全链路成因与双层防御方案。
|
||||
|
||||
---
|
||||
|
||||
## 1. 现象复现：无限的 “Percent” 循环
|
||||
|
||||
我们在终端使用 `curl` 模拟普通外部请求调用 n8n 的 Webhook：
|
||||
|
||||
```bash
|
||||
curl -G "http://192.168.1.1:5678/webhook/ltts" \
|
||||
--data-urlencode "text=恭喜！本地模型测试成功！" \
|
||||
--data-urlencode "voice=child" \
|
||||
-o test.wav
|
||||
```
|
||||
|
||||
接口返回状态码 HTTP 200，并成功下载了一段 2.4MB 的标准 WAV 音频。但当我们将音频丢进本地的语音识别（STT）模型逆向转写核对内容时，转写文本原形毕露：
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "percent E 6 percent 81 percent 81 percent B 9 percent A 7 percent..."
|
||||
}
|
||||
```
|
||||
|
||||
模型并没有出现自回归幻觉，它只是在“百分之百忠实”地读出它所收到的字符串。
|
||||
|
||||
---
|
||||
|
||||
## 2. 根因剖析 1：被透传的 URL 百分号编码
|
||||
|
||||
追溯 HTTP 参数在网络与容器网关中的流动链路，整个过程如下：
|
||||
|
||||
1. **客户端转码**：当客户端通过 HTTP GET 发送带中文参数的请求时，所有非 ASCII 字符必须按照标准进行百分号编码（Percent-Encoding）。例如汉字 `恭`（UTF-8 编码为 `0xE6 0x81 0xAD`），在 URL Query 中会被编码为字符串 `"%E6%81%AD"`。
|
||||
2. **n8n Webhook 接收**：n8n 收到请求后，直接将原始 Query 参数挂载到数据上下文：
|
||||
```json
|
||||
{
|
||||
"query": {
|
||||
"text": "%E6%81%AD%E5%96%9C%EF%BC%81..."
|
||||
}
|
||||
}
|
||||
```
|
||||
3. **HTTP Request 节点转发**：在最初的简易工作流中，下游转发给本地 GPU 服务器的 JSON Body 直接使用了变量插值：
|
||||
```json
|
||||
{
|
||||
"text": "={{ $json.query.text }}"
|
||||
}
|
||||
```
|
||||
4. **模型收到原始转码文本**：本地运行在 50000 端口的 Python FastAPI 服务直接收到了一个含有几十个 `%` 符号的纯 ASCII 字符串 `"%E6%81%AD..."`。
|
||||
|
||||
---
|
||||
|
||||
## 3. 为什么现代神经 TTS 不会自动转义？
|
||||
|
||||
在早期的传统规则式 TTS 中，如果传入奇怪的编码字符，系统往往会直接报错或丢弃非中英文字符。
|
||||
|
||||
但像 CosyVoice 2 或 GPT-SoVITS 这类新一代基于大语言模型架构（LLM + Flow Matching / Diffusion）的 TTS 系统，其文本前端包含复杂的**文本规范化模块（Text Normalization）**。为了让模型能够读懂各种数学符号、技术参数与代码片段，它的分词器将 `%` 显式映射为了发音为 **“Percent”** 的有效 Token。
|
||||
|
||||
因此，对 TTS 模型来说：
|
||||
- 它根本不知道上游传入的是经过 URL 编码的汉字。
|
||||
- 它只看到文本里写着一个个真实的百分号、大写字母 E 和数字 6。
|
||||
- 于是，它严格按照规范将一段 15 个字的中文句子，当成了包含几十个数学符号和英文序列的混排公式，认认真真地读出了每一个 “Percent”。
|
||||
|
||||
---
|
||||
|
||||
## 4. 根因剖析 2：N8N 沙盒表达式的隐形暗坑
|
||||
|
||||
发现问题后，第一反应通常是在 n8n 的 HTTP 节点表达式中直接加一句 JS 解码：
|
||||
|
||||
```javascript
|
||||
// 表面合理的写法：
|
||||
{{ decodeURIComponent($json.query.text) }}
|
||||
```
|
||||
|
||||
但这会立刻引发二次故障：
|
||||
1. **POST 请求断流**：如果调用方改用 POST JSON 发送请求，`$json.query` 会变成 `undefined`，直接报错 `Cannot read property 'text' of undefined`。
|
||||
2. **ES 语法特性支持差异**：在较老或某些特殊打包的 n8n 环境中，内嵌的 JavaScript 解释器对可选链操作符（如 `$json.query?.text`）的支持并不稳定，极易出现 `Unexpected token '.'` 语法错误。
|
||||
3. **URIError 崩溃风险**：如果用户传入的文本本身包含合法的独立百分号（例如“*降价 50%*”），未经包装的原生 `decodeURIComponent("50%")` 会直接抛出 `URIError: URI malformed`，导致整个工作流执行中断。
|
||||
|
||||
---
|
||||
|
||||
## 5. 双层防御方案
|
||||
|
||||
为了让整条语音合成链路彻底免受各类编码问题的困扰，我们在编排层与模型层同时设置了兜底防线。
|
||||
|
||||
### 第一道防线：N8N Code 节点独立预清洗
|
||||
|
||||
在 Webhook 节点之后、HTTP 转发节点之前，插入一个纯 JavaScript **Code 节点**，统一承接 GET 与 POST 数据并做容错解码：
|
||||
|
||||
```javascript
|
||||
const item = $input.first().json;
|
||||
const query = item.query || {};
|
||||
const body = item.body || {};
|
||||
|
||||
let text = query.text || body.text || '你好，本地语音测试。';
|
||||
let voice = query.voice || body.voice || 'default';
|
||||
|
||||
// 仅在文本包含百分号时尝试解码，并在异常时优雅回退
|
||||
if (typeof text === 'string' && text.indexOf('%') !== -1) {
|
||||
try {
|
||||
text = decodeURIComponent(text.replace(/\+/g, ' '));
|
||||
} catch (err) {
|
||||
// 包含非法百分号组合时保持原文，避免流程崩溃
|
||||
}
|
||||
}
|
||||
|
||||
return [{
|
||||
json: {
|
||||
text: text.trim(),
|
||||
voice: String(voice).trim()
|
||||
}
|
||||
}];
|
||||
```
|
||||
|
||||
### 第二道防线：后端 Python 服务幂等兜底
|
||||
|
||||
在本地运行的 FastAPI 接口实现中（`webui.py`），对传入的 `text` 参数增加一层无条件但幂等的 `urllib.parse.unquote`：
|
||||
|
||||
```python
|
||||
import urllib.parse
|
||||
|
||||
@app.post("/tts")
|
||||
async def api_tts(request: Request):
|
||||
body = await request.json()
|
||||
raw_text = body.get("text", "").strip()
|
||||
|
||||
# 无论上游是否已经解码，在此做一次幂等安全解码
|
||||
text = urllib.parse.unquote(raw_text).strip()
|
||||
voice = body.get("voice", "default")
|
||||
|
||||
# 送入推理管线...
|
||||
```
|
||||
|
||||
`unquote` 是幂等的：如果文字已经是纯中文，处理后原样返回；如果有任何遗漏的 URL 编码，会被彻底还原为真实汉字，绝不把 `%` 留给下层的分词与发音模块。
|
||||
|
||||
---
|
||||
|
||||
## 6. 验证与总结
|
||||
|
||||
在部署好双层防护后，我们重新使用同样的带 URL 编码命令请求接口：
|
||||
|
||||
```bash
|
||||
curl -G "http://192.168.1.1:5678/webhook/ltts" \
|
||||
--data-urlencode "text=恭喜！本地模型通过 N8N 联动测试成功！" \
|
||||
--data-urlencode "voice=child" \
|
||||
-o test_clean.wav
|
||||
```
|
||||
|
||||
重新将生成的音频送入语音识别转写：
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "恭喜！本地模型通过N8N联动测试成功！"
|
||||
}
|
||||
```
|
||||
|
||||
所有的 “Percent” 杂音彻底消失，取而代之的是清脆甜美的正常女童声。
|
||||
|
||||
**工程经验**：
|
||||
在搭建涉及“网络网关 → 自动化编排器 → AI 神经推理模型”的多层系统时，绝不能假设到达 API 边界的数据格式天然符合预期。在网关层做安全清洗、在模型接口层做幂等兜底，两行轻量代码就能彻底杜绝 AI 模型在半夜把网络请求头念给用户听的尴尬事故。
|
||||
Reference in new issue
Block a user