A serving stack meets its median latency target at steady load. When short chats and long-context requests share the queue, P99 time-to-first-token and inter-token latency both spike.

Separate queueing, prefill compute, decode bandwidth, batching policy, and KV-cache pressure with measurements. Then propose mitigations that target the diagnosed phase instead of merely adding GPUs.