GearUp hides model loading inside the slack a serverless DAG already exposes, cutting median latency up to 76 percent
AspireAI community · arXiv
Serverless platforms host machine learning workflows as directed acyclic graphs of interdependent functions, and beyond container startup each node must load libraries, model weights and CUDA kernels before it can run, which accumulates along dependent paths and lands on the end-to-end critical path. Existing systems optimize only one half of this: DAG-agnostic pre-loading prepares nodes the request may never reach, wasting host and GPU memory, while DAG-aware serving routes whole functions and leaves the loading stages inside a node untouched. GearUp profiles each node's loading stages and schedules them inside the slack the DAG exposes, hiding initialization behind upstream execution, and reuses initialized instances within and across requests to avoid redundant cold starts, with a probabilistic traffic-adaptive keep-alive policy balancing memory cost against cold-start risk. On the AMBER Alert and FINRA-ML workflows with NVIDIA L40S GPUs it cut median end-to-end latency by 64.5 and 75.9 percent against SMIless, for median speedups of 3.1x and 10.0x over cold execution.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.