Skip to main content

Vinay Jayanna

Sizing LLM Inference for Production
Agentic Systems in Production

GitHub LinkedIn

Overview & Download
Preface
Why Inference Sizing Matters
Part I — Foundations
1 · Workload Characterization
2 · GPU Memory Sizing
3 · The Roofline Model
Part II — Optimization
4 · Quantization
5 · Parallelism Strategy
6 · Batching Strategy
7 · KV Cache Optimization
Part III — Production
8 · Latency-Throughput Curve
9 · Sizing Algorithm & Monitoring
First Principles, Last

2 · GPU Memory Sizing

02-gpu-memory-sizing

Content coming soon.

Last updated on Apr 28, 2026

1.5 Model Routing and Cascading

2.1 Model Weights: The Floor

Field Guides

Sizing LLM Inference Systems at Scale
Agentic Systems in Production

Writing

LinkedIn Articles
Substack

Connect

GitHub
LinkedIn

© 2026 Vinay Jayanna. All rights reserved.