A simulated benchmark for a low-cost robot arm reproduced the real robot's policy ranking exactly, but the best learned policies still only score near 61 percent

Low-cost arms like the SO-101 are the usual entry point into robot learning, and the benchmark options have been thin: mostly physical hardware, simple tasks, and few held-out objects. SO-101 Bench is a simulated benchmark for language-conditioned spatial manipulation built on an Isaac Lab digital twin of a real SO-101 setup, with 4 tasks scored as 5 families, 138 objects split across seen, unseen-instance and unseen-class, 916 evaluation episodes including subsets for held-out appearances, rotate-to-fit planning and occlusion memory, and 3,629 simulated demonstrations that every learned policy is fine-tuned on under an equal compute budget. The headline is a fidelity result. Five policies trained without real data, four VLAs and an LLM agent all keep their exact ranking when the same weights run on the physical arm in an unseen environment with unseen objects - Spearman rho of 1.00 and mean maximum rank variation of 0.000 - although the authors state plainly that simulation overestimates success. Across nine VLAs and world-action models, the best learned policies score near 61 percent, and spatial placement, unseen object classes and rotate-to-fit planning remain open. One result deserves separate attention: an LLM agent that writes code and hands grasping off to a learned policy scores 83.2 percent, and an ablation attributes that lead to the choice of harness and the choice of language model rather than to learned grasp ability. The caveat is scope - one arm platform, one simulated scene family, and physical validation of a small policy set in a single unseen environment.