A lookup table now tells you which Hugging Face models actually run on AWS Trainium, and ships the dataset behind it
AspireAI community · Hugging Face
TorchNeuron is the native PyTorch backend for AWS Trainium, letting Transformers run its own modelling code on Trainium as it does on any other device, so any model on the Hub can in principle run. This post publishes the space that answers the practical question of which ones actually do, reporting for each model whether it runs unmodified, how many Trainium cores it is sharded over, the wall time of the run including compilation, and peak device memory. The Space covers Transformers models only, limited to text-generation and image-text-to-text tasks, with diffusers and sentence transformers planned, and each test loads the model on a trn2 instance sharded over the smallest number of Trainium cores that can hold its weights, generates 32 tokens and compares them against a CPU run of the same checkpoint, with a small judge model checking fluency when the model is sharded. Only models fitting on 4 Trainium2 devices are tested, which means 16 cores at most, and a PASS says nothing about throughput or longer generations. The limitation is that the results are measured against a correctness check rather than a speed benchmark, so it tells you whether a model works on Trainium but not whether it is the fastest way to run it.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.