A dermatologist gave GPT-6 Astra and Claude Fable 5.1 200 biopsy-proven skin lesions, and a 2015 CNN beat both
AspireAI community · Lynch Lab (Dr Magnus Lynch)
**A dermatologist tested GPT-6 Astra and Claude Fable 5.1 on 200 biopsy-proven skin lesions, and a 2015 CNN won.** Dr Magnus Lynch gave each model a clinical close-up plus a dermoscopy image and asked for the most likely of six diagnoses, a confidence, and whether the lesion was safe to discharge. The CNN was Inception v3, first published by Google in 2015, fine-tuned for about twenty minutes on public dermatology images.
The CNN named the right diagnosis for 125 of 200 lesions. GPT-6 Astra got 84 and Claude Fable 5.1 got 85. On the 30 melanomas - the diagnosis that matters most - the CNN caught 13, Claude caught 11, and GPT-6 Astra caught 8. Each LLM had a signature failure: GPT-6 Astra called 75 lesions basal cell carcinoma when only 25 were, including 14 of the 30 melanomas. Both LLMs called one thin melanoma a harmless seborrhoeic keratosis and said the patient could be discharged.
Caveat: 200 lesions, one run per case, and a fine-tuned CNN is not a cleared medical device.
https://lynchlab.net/llm-skin-cancer
https://lynchlab.net/llm-skin-cancer
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.