DODaniel Okoro
Running stateful inference on Kubernetes without tears
StatefulSets, warm model caches, and graceful draining for GPU pods.
AK
Aisha Khan
Author@aisha
Observability & reliability
If you can't measure it, you can't autoscale it. Prometheus, OTel, SLOs.
2
Stories
306
Views
11
Likes
5
Followers
Contributions
JanFebMarAprMayJunJul
11 contributions in the last 6 monthsLessMore
MLMei Lin
Cutting LLM serving costs 60% with continuous batching
How vLLM's paged attention and dynamic batching transformed our inference economics.
DODaniel Okoro
Autoscaling inference with KEDA and custom metrics
Scale on queue depth, not CPU. Your wallet will thank you.
CRCarlos Ruiz
MIG vs time-slicing: partitioning GPUs for inference
When to slice an A100 into seven, and when not to.
MLMei Lin
Speculative decoding in production: a field report
Draft models, acceptance rates, and the latency wins that survived contact with reality.
CRCarlos Ruiz
Quantization without the accuracy cliff
INT8, FP8, and AWQ — measuring what you actually lose.