MLMei Lin
Cutting LLM serving costs 60% with continuous batching
How vLLM's paged attention and dynamic batching transformed our inference economics.
@aisha
Observability & reliability
If you can't measure it, you can't autoscale it. Prometheus, OTel, SLOs.
2
Stories
306
Views
11
Likes
5
Followers