CRCarlos Ruiz
MIG vs time-slicing: partitioning GPUs for inference
When to slice an A100 into seven, and when not to.
@carlos
GPU performance nerd
Squeezing every last token/sec out of accelerators. CUDA, MIG, and kernels.
2
Stories
364
Views
16
Likes
3
Followers