Medical vision-language models reward the wrong behaviour at their own capability boundary
AspireAI community · arXiv
Medical diagnostic reasoning needs verifiable links between visual evidence and clinical knowledge, but current medical VLMs are aligned on objectives that push the wrong way: uniform supervised fine tuning lets easy cases dominate updates while clinically critical boundary cases saturate prematurely, and sparse rewards based only on final-answer correctness invite reward hacking through answer shortcuts. MedCBR is a two-stage framework - boundary-aware preference optimisation plus rubric-grounded reinforcement learning. It builds counterfactual boundary preferences at the model's capability frontier using empirical pass rates and preference pairs that share reasoning prefixes but diverge at the decisive diagnostic branch, concentrates learning on that region with difficulty-weighted boundary preference optimisation, then distils reusable clinical rubrics from boundary trajectories and activates them by prediction entropy in entropy-gated grounded policy optimisation. A reference policy anchored to the boundary-aligned model couples population-level priors with instance-level uncertainty. On six medical benchmarks the authors report 61.2% average accuracy with the largest gains on differential diagnosis, and ablations showing super-additive gains between the two stages.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.