Predictive autoscaling raised inference SLO compliance from 47.8 to 65.3 percent on a 21-second cold start
AspireAI community · arXiv
InferScale is a Kubernetes-native autoscaler that combines inference-aware signals, short-horizon demand prediction, SLO-state control, a fast path for spikes and tenant-priority headroom without replacing the default scheduler. On Azure Kubernetes Service with a ResNet-50 ONNX workload and a 21-second cold start, repeated ramp-burst runs improved SLO compliance from 47.8 plus or minus 1.5 percent under a stronger external-metric Horizontal Pod Autoscaler to 65.3 plus or minus 9.0 percent under InferScale, and controlled multi-tenant contention experiments cut Gold-tier SLO standard deviation from 16.1 to 1.2 percentage points. Single-service ablations degraded 6 to 8 percentage points, and an exploratory priority ablation dropped 27 points under contention. The authors' own takeaway is that the value of prediction lies less in forecasting accuracy than in moving the controller early enough to offset cold-start delay; the paper appeared at IEEE CLOUD 2026.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.