Skip to main content

Vinay Jayanna

Sizing LLM Inference for Production
Agentic Systems in Production

GitHub LinkedIn

Overview & Download
Preface
Why Inference Sizing Matters
Part I — Foundations
1 · Workload Characterization
2 · GPU Memory Sizing
3 · The Roofline Model
Part II — Optimization
4 · Quantization
5 · Parallelism Strategy
6 · Batching Strategy
7 · KV Cache Optimization
Part III — Production
8 · Latency-Throughput Curve
9 · Sizing Algorithm & Monitoring
First Principles, Last

1 · Workload Characterization

01-workload-characterization

Content coming soon.

Last updated on Apr 28, 2026

Why Inference Sizing Matters

1.1 Why P99 Input Length Matters

Field Guides

Sizing LLM Inference Systems at Scale
Agentic Systems in Production

Writing

LinkedIn Articles
Substack

Connect

GitHub
LinkedIn

© 2026 Vinay Jayanna. All rights reserved.