Logo

菜单

无双的技术博客 记录 AI、Linux、网络架构、FreeSWITCH 与企业数字化实践

所有标签

#cross-functional collaboration 1 #organizational change 1 #business process reengineering 1 #CTO perspective 1 #enterprise digitalization 1 #digital transformation 1 #企业数字化 1 #CTO视角 1 #业务流程再造 1 #组织变革 1 #ERP迁移 1 #数据治理 1 #经营透明度 1 #跨部门协同 1 #数字化转型 1 #LLM serving 1 #GPU memory optimization 1 #显存优化 1 #高并发 AI 服务 1 #Continuous Batching 2 #PagedAttention 2 #KV Cache 1 #sft 2 #model evaluation 2 #data engineering 2 #LLM fine-tuning 2 #max_length 1 #OOM 1 #vLLM 3 #LoRA 1 #realm 1 #容器化 1 #docker 1 #Iptables 1 #VPN 1 #GRE 1 #eBPF 1 #Gre over IPSec 2 #MTU 1 #Freeswitch多注册 1 #中继网关 1 #Linux 0 #gprof 1 #perf 1 #BGP 1 #origin 1 #AS_PATH 1 #MED 1 #localpref 1 #AI Coding 1 #Claude Code 1 #Deepseek 1 #VOIP 3 #Freeswitch 4 #开源呼叫系统 1 #freeswitch模块 1 #fs_mod 1 #voip 1 #freeswitch 2 #Qdrant 1 #Milvus 1 #pgvector 1 #RAG 1 #Halo 0
KV Cache Is the Real Memory Killer in High-Concurrency AI Services
AI Engineering #GPU memory optimization#LLM serving#vLLM#Continuous Batching#PagedAttention

KV Cache Is the Real Memory Killer in High-Concurrency AI Services

In high-concurrency LLM serving, model weights are rarely the only memory problem. KV Cache grows with requests and tokens, so PagedAttention and Continuous Batching are the real keys to vLLM throughput.

Administrator Administrator 发布于 2026-07-06
KV Cache 才是高并发 AI 服务的显存杀手
AI Engineering #KV Cache#PagedAttention#Continuous Batching#vLLM#高并发 AI 服务#显存优化

KV Cache 才是高并发 AI 服务的显存杀手

高并发大模型服务真正容易爆掉的不是模型参数，而是随请求增长的 KV Cache。理解 PagedAttention 和 Continuous Batching，才能看懂 vLLM 为什么能把吞吐做上去。

Administrator Administrator 发布于 2026-07-06