Databricks' open-source Vital Skills lifted Genie Code's BiomniBench average from 71.9 to 76.9 on in-scope biomedical tasks and recovered every eligible patient a baseline had silently dropped
AspireAI community · Databricks
Databricks published Vital Skills on October 27, 2026, a set of open-source Agent Skills that give coding agents healthcare and life-sciences domain discipline, packaged for the Agent Skills standard and built around a principle of asking before assuming rather than silently picking clinical thresholds. In paired evaluation on BiomniBench, run three times per task and graded blind by an LLM judge, Genie Code's average score rose from 71.9 to 76.9 out of 100 on the five tasks the skills covered, while the eighteen out-of-scope tasks held steady at 69.1 without the skills and 71.2 with them. On a cohort-building task, free-text cohort F1 rose from 0.62 to 1.00 and a baseline that silently dropped half the eligible patients recovered all of them, while on a TEDDY single-cell deployment task Genie Code completed all 27 readiness checks instead of 19. The skills are catalogued per discipline, including bulk RNA-seq differential expression, single-cell RNA-seq, pathway enrichment, a clinical cohort builder, propensity-adjusted comparative-effectiveness studies, HIPAA Safe-Harbor de-identification and a skill-evaluation harness. The caveat is that these are Databricks' own numbers from its own product on a benchmark it selected, and the overall lift is modest outside the covered tasks.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.