Open models include weights, tokenizers, and configuration—not a chat app. Follow one prompt through vLLM/Ollama: IDs become vectors, logits, text, then EOS. What makes it speak and stop?
下载开源大模型后，磁盘里躺着的不是可执行聊天程序，而是权重、分词器和配置等静态文件。我沿着一次真实推理拆开它们如何被 vLLM/Ollama 驱动：文本怎样变成 token、向量与概率，又怎样流式返回；最后究竟是 EOS、停止规则还是 EOF 让模型闭嘴？这篇文章还会解释 KV Cache 为何能让生成显著提速。
我把“把 PDF 做成 PPT”拆成主题判断、章节提炼、逐页表达和收尾复盘四步，并给出一段可直接复用的提示词。真正拉开效果差距的，不是让 AI 一键生成，而是先给它清晰的产出边界：谁来听、每页讲什么、结尾怎样收束。少了这些约束，页数再多，也很容易变成无法讲述的文字堆；把框架交代清楚，AI 才能成为可靠的整理助手。
vLLM tuning should not stop at checking whether the API returns an answer. TTFT, TPOT, throughput, KV Cache usage, queue length, and tail latency form the real feedback loop.
vLLM 调优不能只看服务是否返回答案，而要围绕 TTFT、TPOT、吞吐量、KV Cache 使用率和尾延迟建立监控闭环。本文用本地脚本串起启动、测量、压测和指标采集。
Speculative decoding does not let the model guess blindly. A cheap draft model proposes tokens, while the target model verifies them in parallel. The real question is whether acceptance rate and batch size make it worthwhile.
推测解码不是让大模型乱猜，而是用草稿模型先给候选，再由目标模型一次性验证。理解 Medusa、EAGLE 和大 Batch 的关系，才能判断它在生产高并发里是否真的划算。
In high-concurrency LLM serving, model weights are rarely the only memory problem. KV Cache grows with requests and tokens, so PagedAttention and Continuous Batching are the real keys to vLLM throughput.
高并发大模型服务真正容易爆掉的不是模型参数，而是随请求增长的 KV Cache。理解 PagedAttention 和 Continuous Batching，才能看懂 vLLM 为什么能把吞吐做上去。
The Core Logic of High-Quality Fine-Tuning Data Engineering: Why Data Quality Defines a Model's Ceiling Chinese version: 中文版 This is the first article
