Skip to main content

Vinay Jayanna

Sizing LLM Inference for Production
Agentic Systems in Production

GitHub LinkedIn

Overview & Download
Preface
Why Inference Sizing Matters
Part I — Foundations
1 · Workload Characterization
2 · GPU Memory Sizing
3 · The Roofline Model
Part II — Optimization
4 · Quantization
5 · Parallelism Strategy
6 · Batching Strategy
7 · KV Cache Optimization
Part III — Production
8 · Latency-Throughput Curve
9 · Sizing Algorithm & Monitoring
First Principles, Last

7 · KV Cache Optimization

07-kv-cache-optimization

Content coming soon.

Last updated on Apr 28, 2026

6.7 Choosing the Right Strategy

7.1 PagedAttention

Field Guides

Sizing LLM Inference Systems at Scale
Agentic Systems in Production

Writing

LinkedIn Articles
Substack

Connect

GitHub
LinkedIn

© 2026 Vinay Jayanna. All rights reserved.