diff --git a/src/content/posts/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md b/src/content/posts/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md new file mode 100644 index 0000000..ce87a12 --- /dev/null +++ b/src/content/posts/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md @@ -0,0 +1,963 @@ +--- +title: "Why My AI Songs Kept Losing Their Endings: Running YuE2 on a 12 GB GPU" +description: "Twelve Mandarin songs came out exactly 240.000 seconds long with the outros missing. The cause wasn't the lyrics — it was a hard cap, a VRAM ceiling, and a planner that ignores BPM." +pubDate: 2026-10-20 +category: ai +tags: ["yue2", "comfyui", "ai-music", "local-ai", "gpu", "vram", "mandarin"] +draft: false +--- +YuE2 is a 3B open-weights music model that sings Mandarin from a style prompt +and a lyric sheet. I ran it locally on a 12 GB RTX 3060 through ComfyUI — +twelve original songs in one serial queue — and the first batch came back with +**every single file exactly 240.000 seconds long**. That number is not a +coincidence, and it is not good news. + +## Every track was exactly 240.000 seconds + +Twelve files, twelve identical durations to the millisecond: + +``` +01_A1_..._糖画小仙.mp3 duration=240.000000 +02_A2_..._溪边捉云记.mp3 duration=240.000000 +03_A3_..._寒露拾桂子.mp3 duration=240.000000 +... +``` + +I had asked for `max_duration=240` because the style guide I was working from +said a 12-section structure would land around three and a half minutes. It +landed on 240 seconds exactly — which means the model never decided to stop. +It ran out of room. + +## A round number is a confession + +YuE2 plans a song with an ABC planner: it reads the lyric sheet, decides how +many bars each section gets, and emits a symbolic sketch. The music generator +then sings that sketch and **ends when the sketch ends**. A finished song +therefore comes out *shorter* than the cap you set — it stops on its own. + +Here is the tell, and it costs one shell command: + +```bash +d=$(ffprobe -v error -show_entries format=duration -of csv=p=0 take.mp3) +python -c "print(abs(float('$d') - 240.0) <= 0.005)" # True = hard cut +``` + +If the duration equals `max_duration` to within a few milliseconds, the render +was truncated. The lyric sheet wanted more time than the cap allowed, and the +last thing being sung at the cutoff point got sliced off mid-phrase. + +## Confirming the outros were actually missing + +Duration alone tells you *that* it was cut, not *what* was lost. So I pulled +the tail of each take and transcribed it with a local speech model (SenseVoice, +CPU — it stays running while the GPU renders): + +```bash +ffmpeg -v error -y -i take.mp3 -ss 232 -t 20 -ar 16000 -ac 1 -c:a pcm_s16le t.wav +curl -s -X POST http://127.0.0.1:20129/v1/audio/transcriptions \ + -F "file=@t.wav" -F "language=zh" +``` + +The results were unambiguous: + +| Song | Sung up to | What was missing | +|---|---|---| +| 糖画小仙 | final chorus, last line | the entire `[Outro]` | +| 溪边捉云记 | `[Bridge]` at 215 s | solo + final chorus + outro | +| 寒露拾桂子 | `[Bridge]` at 215 s | final chorus + outro | +| 纸鸢与旧木偶 | final chorus, line 5 | rest of chorus + outro | + +So: not a mixing artefact, not a fade-out. The songs were structurally +incomplete. + +## Three fixes that did nothing, and one that half-worked + +### 1. Raising the BPM — no effect + +The obvious reasoning: faster tempo, more syllables per second, more lyrics +fit in the same window. I tested this across the batch: the 117 BPM guofeng +tracks and the 140 BPM DJ remixes had **effectively identical throughput**. + +YuE2's duration comes from the ABC planner's bar count, not from the BPM token +in your style prompt. The BPM tag shapes the *feel*; it does not buy you room. +Stop tuning it for duration. + +### 2. Re-running with the same seed — silently does nothing + +ComfyUI was started with `--cache-classic`, so resubmitting a byte-identical +prompt — same style, same lyrics, same seed — hits the node cache. The job +"completes" in about ten seconds and hands you back the same truncated audio +with a `_00002` filename. + +It looks like a fast, successful retry. It is a cache hit on a known-bad take. + +### 3. Rolling new seeds — works about 3 times out of 4 + +The planner's bar allocation depends on the seed, so a different seed really can +produce a shorter, complete song from the same lyrics. This rescued three of my +four clipped tracks: + +| Song | Original | New seed | Result | +|---|---|---|---| +| 溪边捉云记 | clipped at 281 s | 20002 | 279.7 s, complete | +| 外婆的绿搪瓷碗 | clipped at 300 s | 20007 | 251.9 s, complete | +| 北溟有小鱼 | clipped at 300 s | 20005, 30005 | clipped again | + +## The budget is characters per second — and it isn't constant + +I tried to turn this into arithmetic: count the Chinese characters, divide by a +throughput figure, get the duration. The measurements refused to cooperate. + +| Song | Characters | Cap | Actual | Fits? | +|---|---|---|---|---| +| 糖画小仙 | 556 | 272 s | 258.7 s | ✅ | +| 纸鸢与旧木偶 | 572 | 300 s | 279.9 s | ✅ | +| 溪边捉云记 | 578 | 281 s | **281.0 s** | ❌ cut | +| 小楼檐下听雨眠 | 580 | 300 s | 256.3 s | ✅ | +| 画卷小锦鲤 | 634 | 300 s | 285.9 s | ✅ | +| 三楼窗台的蒲公英 | 693 | 329 s | 247.4 s | ✅ | +| 弄堂口的竹蜻蜓 | 692 | 300 s | 250.0 s | ✅ | + +Read the last two rows against the third. **693 characters finished in 247 +seconds, while 578 characters overflowed 281 seconds.** A lyrics-to-duration +formula simply does not exist here — the observed throughput ranges from about +2.0 to 2.8 Chinese characters per second, depending on how the planner chops the +sections. + +So characters are useful for a *budget warning*, not for setting a timer. + +## The limit that actually binds: VRAM + +Once you accept "give it enough room and let it close naturally", the cap wants +to be large. On a 12 GB card it cannot be. + +Measured peak VRAM while rendering, sampled every five seconds with +`nvidia-smi` during the run: + +| `max_duration` | Peak VRAM | +|---|---| +| 240 s | 10.3 GB | +| 272 s | 11.65 GB | +| 278 s | 11.07 GB | +| 281 s | 12.06 GB | +| 300 s | 12.03 GB | + +The card has 12.288 GB. **300 seconds is the practical ceiling** — past that +you are one desktop browser tab away from an out-of-memory abort in the middle +of a six-minute render. + +## The fix + +Three changes, all cheap: + +1. **Set the cap to 300 s and let the planner close the song itself.** Do not + chase a target length; give it headroom and it ends where the music ends. +2. **Archive clipped takes instead of mixing them in.** A take whose duration + equals the cap goes to `未自然结尾\` ("did not end naturally"). The output + folder then contains only complete songs, and re-renders never destroy a + usable take. +3. **Cap the seed roulette at three attempts.** Different seeds help, but a song + whose lyrics genuinely need more than the VRAM ceiling allows will fail + forever. Three strikes and you trim the lyric sheet — don't burn an hour of + GPU pretending the next seed will be the one. + +## The song that never fit + +One of the twelve never made it. **北溟有小鱼 — 593 characters — failed five +times:** + +| Attempt | Parameters | Result | +|---|---|---| +| 1 | seed 10005, cap 287 s | hit 287 s | +| 2 | seed 10005, cap 300 s | hit 300 s | +| 3 | seed 20005, cap 300 s | hit 300 s | +| 4 | seed 30005, cap 300 s | hit 300 s | +| 5 | seed 40005, both instrumental sections removed | hit 300 s | + +I even deleted the `[Interlude]` and `[Instrumental Solo]` markers — 12 sections +down to 10, not one sung word changed — and it still overflowed. Its planner +wants more time than a 12 GB card can give. The honest fix is to cut about 40 +characters of lyric and re-render, and I would rather tell you that than ship a +twelfth track with a bitten-off ending. + +## A trap I hit before any of this + +Worth its own warning: **YuE2 will sing the Chinese production notes in your +lyric sheet.** + +If you paste lyrics that look like this — which is how most lyric sheets are +written — + +``` +[Intro] +(重鼓点起，急促琵琶轮指，唢呐破空，合成器铺底) +嘿！哈！山海震！ +``` + +…then that parenthetical line becomes a lyric. The model does not know it is a +stage direction. Strip them before submitting: + +```python +clean = "\n".join(l for l in raw.split("\n") + if not re.match(r"^\s*[（(].*[）)]\s*$", l)) +``` + +Arrangement and dynamics belong in the style prompt. The lyric box gets section +tags and sung words, nothing else. + +## What I'd do differently + +- **Pre-flight the lyric sheet, not the render.** Count characters first. Over + roughly 600 characters on a 12 GB card, expect trouble; over 550, expect to + trim something. +- **Never set `max_duration` to a target length.** Set it generously above the + estimate and let the planner finish. The cap is a safety rail, not a dial. +- **Treat "duration == cap" as a hard failure in the pipeline**, not something + you notice three days later when a chorus sounds wrong. +- **Re-run with a new seed before touching the lyrics** — it is free and works + most of the time. Then stop at three. + +## Eleven tracks + +These are the ones that closed naturally, sung by a child voice, in Mandarin, +on a 12 GB card that cost less than a year of cloud music credits. Each player +below has synced lyrics — the timing was recovered by transcription and aligned +against the written lyrics, since the model emits no timestamps. + + + +### 1. 糖画小仙 — Sugar-Painting Fairy + +*Guofeng electronic* · 556 characters · 259 s + +
+ + +
+ +### 2. 溪边捉云记 — Catching Clouds by the Stream + +*Guofeng electronic* · 578 characters · 280 s + +
+ + +
+ +### 3. 寒露拾桂子 — Picking Osmanthus at Cold Dew + +*Guofeng electronic* · 578 characters · 290 s + +
+ + +
+ +### 4. 纸鸢与旧木偶 — The Kite and the Old Puppet + +*Guofeng ballad* · 572 characters · 280 s + +
+ + +
+ +### 5. 小楼檐下听雨眠 — Listening to Rain Under the Eaves + +*Guofeng ballad* · 580 characters · 256 s + +
+ + +
+ +### 6. 外婆的绿搪瓷碗 — Grandma's Green Enamel Bowl + +*Urban folk* · 679 characters · 252 s + +
+ + +
+ +### 7. 弄堂口的竹蜻蜓 — The Bamboo Dragonfly at the Alley Mouth + +*Urban folk* · 692 characters · 250 s + +
+ + +
+ +### 8. 三楼窗台的蒲公英 — Dandelion on the Third-Floor Windowsill + +*Urban folk* · 693 characters · 247 s + +
+ + +
+ +### 9. 小哪吒借红绫 — Little Nezha Borrows the Red Sash + +*Guofeng DJ remix* · 612 characters · 271 s + +
+ + +
+ +### 10. 画卷小锦鲤 — The Koi That Left the Painting + +*Guofeng DJ remix* · 634 characters · 286 s + +
+ + +
+ +### 11. 山海小当家 — Little Boss of the Classic of Mountains and Seas + +*Guofeng DJ remix* · 647 characters · 237 s + +
+ + +
+ + diff --git a/src/content/posts/zh/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md b/src/content/posts/zh/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md new file mode 100644 index 0000000..7930a20 --- /dev/null +++ b/src/content/posts/zh/yue2-on-a-12gb-gpu-songs-that-lost-their-endings.md @@ -0,0 +1,908 @@ +--- +title: "为什么我的 AI 歌曲总是丢结尾：在 12GB 显卡上跑 YuE2" +description: "12 首中文歌时长全都恰好 240.000 秒，尾奏不见了。原因不在歌词 —— 是一道硬上限、一堵显存天花板，和一个不看 BPM 的规划器。" +pubDate: 2026-10-20 +category: ai +tags: ["yue2", "comfyui", "ai-music", "local-ai", "gpu", "vram", "mandarin"] +draft: false +--- +YuE2 是一个 30 亿参数的开源音乐模型，能按风格提示词和歌词唱中文。我把它放在自己那台 12GB 显存的 RTX 3060 上用 ComfyUI 本地跑，一次排了 12 首原创歌 —— 结果第一批出来，**每一首的时长都恰好是 240.000 秒**。这个数字不是巧合，而且是个坏消息。 + +## 每首都恰好 240.000 秒 + +12 个文件，时长精确到毫秒完全一致： + +``` +01_A1_..._糖画小仙.mp3 duration=240.000000 +02_A2_..._溪边捉云记.mp3 duration=240.000000 +03_A3_..._寒露拾桂子.mp3 duration=240.000000 +... +``` + +我设的是 `max_duration=240`，因为参考指南说 12 段结构大约会落在三分半左右。结果它精确落在 240 秒 —— 这意味着模型从来没有"决定结束"，它是把时间用光了。 + +## 整数是一种自白 + +YuE2 用 ABC 规划器来编排：读取歌词，给每一段分配小节数，生成一份符号化的曲谱草稿。音乐生成器照着这份草稿唱，**草稿结束它就结束**。 + +所以一首真正唱完的歌，输出时长一定是**小于**你设的上限的 —— 它自己停下来了。 + +判断方法只要一条命令： + +```bash +d=$(ffprobe -v error -show_entries format=duration -of csv=p=0 take.mp3) +python -c "print(abs(float('$d') - 240.0) <= 0.005)" # True = 被硬切 +``` + +如果时长和 `max_duration` 相差不到几毫秒，这次渲染就是被截断的。歌词需要的时间超过了上限允许的范围，切点那一句被拦腰砍断。 + +## 确认尾奏真的丢了 + +光看时长只能知道"被切了"，不知道"丢了什么"。所以我把每首的尾巴切出来，用本地语音模型（SenseVoice，跑在 CPU 上，GPU 渲染时不用停）转写： + +```bash +ffmpeg -v error -y -i take.mp3 -ss 232 -t 20 -ar 16000 -ac 1 -c:a pcm_s16le t.wav +curl -s -X POST http://127.0.0.1:20129/v1/audio/transcriptions \ + -F "file=@t.wav" -F "language=zh" +``` + +结果没有歧义： + +| 歌 | 唱到哪 | 缺了什么 | +|---|---|---| +| 糖画小仙 | 末遍副歌最后一句 | **整段 `[Outro]`** | +| 溪边捉云记 | 215 秒还停在 `[Bridge]` | Solo + 末遍副歌 + Outro | +| 寒露拾桂子 | 215 秒还停在 `[Bridge]` | 末遍副歌 + Outro | +| 纸鸢与旧木偶 | 末遍副歌第 5 句 | 副歌后半 + Outro | + +所以不是混音问题、也不是淡出 —— 是结构不完整。 + +## 试过但没用的三件事，和半有用的一件 + +### 1. 提高 BPM —— 完全无效 + +直觉上：节奏更快、每秒塞进更多字、同样的窗口能装更多歌词。我在整批上验过：117 BPM 的国风组和 140 BPM 的 DJ 燃曲组，**吞吐几乎一样**。 + +YuE2 的时长来自 ABC 规划器分配的小节数，不看你风格提示词里的 BPM 标签。BPM 只影响"感觉"，买不到时间。别在它身上调时长。 + +### 2. 同 seed 重跑 —— 静悄悄地什么都没做 + +ComfyUI 是用 `--cache-classic` 启动的，所以重新提交一个逐字节相同的任务（同样的风格、同样的歌词、同样的 seed）会直接命中节点缓存。任务"完成"于 10 秒左右，还你一份一模一样的截断音频，文件名加个 `_00002`。 + +看起来像一次又快又成功的重试。实际上是命中了一个已知有问题 take 的缓存。 + +### 3. 换 seed —— 四次里能救三次 + +规划器分配小节的方式依赖 seed，所以换个 seed 确实可能给出更短、能唱完的版本。这一招救回了我四首里的三首： + +| 歌 | 原始 | 新 seed | 结果 | +|---|---|---|---| +| 溪边捉云记 | 卡在 281 秒 | 20002 | 279.7 秒，完整 | +| 外婆的绿搪瓷碗 | 卡在 300 秒 | 20007 | 251.9 秒，完整 | +| 北溟有小鱼 | 卡在 300 秒 | 20005、30005 | 再次撞顶 | + +## 预算是"每秒几个字"，而这个数还在变 + +我试过把它做成算术：数字数、除以吞吐、得出时长。实测不配合。 + +| 歌 | 汉字数 | 上限 | 实际 | 装下了吗 | +|---|---|---|---|---| +| 糖画小仙 | 556 | 272 秒 | 258.7 秒 | ✅ | +| 纸鸢与旧木偶 | 572 | 300 秒 | 279.9 秒 | ✅ | +| 溪边捉云记 | 578 | 281 秒 | **281.0 秒** | ❌ 被切 | +| 小楼檐下听雨眠 | 580 | 300 秒 | 256.3 秒 | ✅ | +| 画卷小锦鲤 | 634 | 300 秒 | 285.9 秒 | ✅ | +| 三楼窗台的蒲公英 | 693 | 329 秒 | 247.4 秒 | ✅ | +| 弄堂口的竹蜻蜓 | 692 | 300 秒 | 250.0 秒 | ✅ | + +把最后两行和第三行对比着看：**693 个字在 247 秒内唱完了，578 个字却撑爆了 281 秒。** 字数换算时长的公式在这里根本不存在 —— 实测吞吐落在每秒 2.0 到 2.8 个汉字之间，取决于规划器怎么切段落。 + +所以字数只适合用来**预警预算**，不适合用来掐时间。 + +## 真正卡住的瓶颈是显存 + +一旦接受"给它足够空间、让它自己收尾"，上限自然想设大一点。但在 12GB 卡上设不大。 + +渲染过程中每 5 秒采一次 `nvidia-smi` 得到的峰值显存： + +| `max_duration` | 峰值显存 | +|---|---| +| 240 秒 | 10.3 GB | +| 272 秒 | 11.65 GB | +| 278 秒 | 11.07 GB | +| 281 秒 | 12.06 GB | +| 300 秒 | 12.03 GB | + +卡总共 12.288 GB。**300 秒就是实际天花板** —— 再往上，你离"渲染到第六分钟时因为一个浏览器标签页而 OOM 崩掉"只差一步。 + +## 解法 + +三处改动，都很便宜： + +1. **把上限设到 300 秒，让规划器自己收尾。** 不要追目标时长；给足余量，它会在音乐结束的地方结束。 +2. **被切的 take 归档，不要混进成品。** 时长等于上限的 take 移到 `未自然结尾\` 目录。成品目录里就只有完整的歌，而且重跑永远不会覆盖掉一个可用的 take。 +3. **换 seed 最多试三次。** 换 seed 有用，但如果一首歌的歌词真的超过显存能撑的时长，它会永远失败。三次不成就动歌词 —— 别烧一小时 GPU 假装下一个 seed 会中。 + +## 那首始终装不下的歌 + +12 首里有一首没做成。**北溟有小鱼，593 字，失败了五次：** + +| 尝试 | 参数 | 结果 | +|---|---|---| +| 1 | seed 10005，上限 287 秒 | 撞 287 秒 | +| 2 | seed 10005，上限 300 秒 | 撞 300 秒 | +| 3 | seed 20005，上限 300 秒 | 撞 300 秒 | +| 4 | seed 30005，上限 300 秒 | 撞 300 秒 | +| 5 | seed 40005，删掉两段纯器乐 | 撞 300 秒 | + +我甚至把 `[Interlude]` 和 `[Instrumental Solo]` 两个标记删掉 —— 从 12 段降到 10 段，一个唱词都没动 —— 它还是溢出。它的规划器要的时间超过了这张卡能给的上限。诚实的解法是砍掉大约 40 个字再重跑；我宁可告诉你这件事，也不想交一首结尾被咬掉一块的第十二首歌。 + +## 在这之前踩到的一个坑 + +值得单独警告：**YuE2 会把你歌词里括号内的中文制作备注唱出来。** + +如果你贴的歌词长这样（大多数歌词稿就是这么写的）： + +``` +[Intro] +(重鼓点起，急促琵琶轮指，唢呐破空，合成器铺底) +嘿！哈！山海震！ +``` + +那么那行括号内容就成了一句歌词。模型不知道那是舞台说明。提交前先清掉： + +```python +clean = "\n".join(l for l in raw.split("\n") + if not re.match(r"^\s*[（(].*[）)]\s*$", l)) +``` + +配器和动态写进风格提示词。歌词框里只留段落标签和要唱的字。 + +## 我会怎么改 + +- **预警做在歌词阶段，不在渲染之后。** 先数字数。12GB 卡上超过约 600 字就要有心理准备，超过 550 字就该准备删点什么。 +- **永远不要把 `max_duration` 设成目标时长。** 设得比估算宽松，让它自己收尾。上限是安全护栏，不是旋钮。 +- **把"时长 == 上限"当成流水线里的硬失败**，而不是三天后觉得副歌怪怪的才发现。 +- **先换 seed，再动歌词** —— 免费，而且大多数时候有效。然后三次为止。 + +## 十一首歌 + +下面这些是自然收尾的成品，女童声、中文、跑在一张 12GB 显卡上，成本比一年的云端音乐额度还低。每个播放器都带同步歌词 —— 因为模型不输出时间戳，时间轴是用转写结果和歌词原文对齐反推出来的。 + + + +### 1. 糖画小仙 — Sugar-Painting Fairy + +*Guofeng electronic* · 556 characters · 259 s + +
+ + +
+ +### 2. 溪边捉云记 — Catching Clouds by the Stream + +*Guofeng electronic* · 578 characters · 280 s + +
+ + +
+ +### 3. 寒露拾桂子 — Picking Osmanthus at Cold Dew + +*Guofeng electronic* · 578 characters · 290 s + +
+ + +
+ +### 4. 纸鸢与旧木偶 — The Kite and the Old Puppet + +*Guofeng ballad* · 572 characters · 280 s + +
+ + +
+ +### 5. 小楼檐下听雨眠 — Listening to Rain Under the Eaves + +*Guofeng ballad* · 580 characters · 256 s + +
+ + +
+ +### 6. 外婆的绿搪瓷碗 — Grandma's Green Enamel Bowl + +*Urban folk* · 679 characters · 252 s + +
+ + +
+ +### 7. 弄堂口的竹蜻蜓 — The Bamboo Dragonfly at the Alley Mouth + +*Urban folk* · 692 characters · 250 s + +
+ + +
+ +### 8. 三楼窗台的蒲公英 — Dandelion on the Third-Floor Windowsill + +*Urban folk* · 693 characters · 247 s + +
+ + +
+ +### 9. 小哪吒借红绫 — Little Nezha Borrows the Red Sash + +*Guofeng DJ remix* · 612 characters · 271 s + +
+ + +
+ +### 10. 画卷小锦鲤 — The Koi That Left the Painting + +*Guofeng DJ remix* · 634 characters · 286 s + +
+ + +
+ +### 11. 山海小当家 — Little Boss of the Classic of Mountains and Seas + +*Guofeng DJ remix* · 647 characters · 237 s + +
+ + +
+ +