Skip to main content

Vinay Jayanna

Sizing LLM Inference for Production
Agentic Systems in Production

GitHub LinkedIn

Overview & Download
Preface
Why Inference Sizing Matters
Part I — Foundations
1 · Workload Characterization
2 · GPU Memory Sizing
3 · The Roofline Model
Part II — Optimization
4 · Quantization
5 · Parallelism Strategy
6 · Batching Strategy
7 · KV Cache Optimization
Part III — Production
8 · Latency-Throughput Curve
9 · Sizing Algorithm & Monitoring
First Principles, Last

6 · Batching Strategy

06-batching-strategy

Content coming soon.

Last updated on Apr 28, 2026

5.6 The Interconnect

6.1 Static Batching: Why It Fails

Field Guides

Sizing LLM Inference Systems at Scale
Agentic Systems in Production

Writing

LinkedIn Articles
Substack

Connect

GitHub
LinkedIn

© 2026 Vinay Jayanna. All rights reserved.