Logo

菜单

无双的技术博客 记录 AI、Linux、网络架构、FreeSWITCH 与企业数字化实践

所有标签

#cross-functional collaboration 1 #organizational change 1 #business process reengineering 1 #CTO perspective 1 #enterprise digitalization 1 #digital transformation 1 #企业数字化 1 #CTO视角 1 #业务流程再造 1 #组织变革 1 #ERP迁移 1 #数据治理 1 #经营透明度 1 #跨部门协同 1 #数字化转型 1 #LLM serving 1 #GPU memory optimization 1 #显存优化 1 #高并发 AI 服务 1 #Continuous Batching 2 #PagedAttention 2 #KV Cache 1 #sft 2 #model evaluation 2 #data engineering 2 #LLM fine-tuning 2 #max_length 1 #OOM 1 #vLLM 3 #LoRA 1 #realm 1 #容器化 1 #docker 1 #Iptables 1 #VPN 1 #GRE 1 #eBPF 1 #Gre over IPSec 2 #MTU 1 #Freeswitch多注册 1 #中继网关 1 #Linux 0 #gprof 1 #perf 1 #BGP 1 #origin 1 #AS_PATH 1 #MED 1 #localpref 1 #AI Coding 1 #Claude Code 1 #Deepseek 1 #VOIP 3 #Freeswitch 4 #开源呼叫系统 1 #freeswitch模块 1 #fs_mod 1 #voip 1 #freeswitch 2 #Qdrant 1 #Milvus 1 #pgvector 1 #RAG 1 #Halo 0
KV Cache Is the Real Memory Killer in High-Concurrency AI Services
AI Engineering #GPU memory optimization#LLM serving#vLLM#Continuous Batching#PagedAttention

KV Cache Is the Real Memory Killer in High-Concurrency AI Services

In high-concurrency LLM serving, model weights are rarely the only memory problem. KV Cache grows with requests and tokens, so PagedAttention and Continuous Batching are the real keys to vLLM throughput.

Administrator Administrator 发布于 2026-07-06