Skip to main content

LLM Inference · GPU Systems · Production AI

LLM Inference &
ML Infrastructure

Rigorous field guides for designing, sizing, and operating large-scale AI systems. Each guide turns complex infrastructure decisions into practical engineering frameworks.

GitHubLinkedInAbout

About the author

Vinay Jayanna is a Staff/Principal Machine Learning Engineer specializing in LLM inference optimization, GPU capacity planning, and production AI infrastructure. He currently leads inference optimization and GenAI platform architecture for large-scale AI systems reaching hundreds of millions of users. Previously, he helped build and scale core inference infrastructure for Amazon SageMaker, founded Vipas.AI, and is the inventor on a pending U.S. patent covering model caching, hierarchical storage, and GPU optimization for LLM serving.

Field Guides

Comprehensive decision frameworks for real production systems—covering the architecture, trade-offs, calculations, and operating decisions that short tutorials leave out.

Field Guide · v1.1
Sizing LLM Inference for Production
Inference is where AI infrastructure cost compounds with usage growth. Many production LLM fleets remain substantially overprovisioned—not because of fundamental hardware limits, but because capacity decisions are made without a disciplined sizing framework. This guide covers the complete decision sequence: workload characterization, GPU memory sizing, parallelism strategy, KV cache optimization, operating point selection, and fleet sizing from the latency-throughput curve.
📄 107 pages◎ 9 chapters👤 Staff / Principal ML Engineers
Read the guide →
Field Guide · In Progress
Agentic Systems in Production
Production architecture for multi-agent LLM systems: orchestration patterns, tool reliability, memory and state management, latency budgets, failure modes, observability, and cost control. The guide that bridges research prototypes and production deployments.
📄 100+ pages◎ 10 chapters👤 Staff / Principal ML EngineersIn Progress

Writing

Technical articles on LLM inference, ML infrastructure, and production AI systems.

Multi-Tenant LLM Inference: Bridging Research and Reality
LinkedIn · 2024
The Great LLM Inference Showdown: TensorRT-LLM vs vLLM
LinkedIn · 2024
The Challenge of Production LLM Serving: A Ray Serve Perspective
LinkedIn · 2024
KV Cache: The Hidden Optimization Behind Real-Time AI
LinkedIn · 2024
The Infrastructure No One Talks About: How Vector Search Makes Gen AI Work
LinkedIn · 2024