Predicting the next geometric state as well as the next CAD command cut Chamfer distance 13.2 percent on DeepCAD text-to-CAD generation
AspireAI community · arXiv
Text-to-CAD has to choose operations whose effects compound: one wrong command can wreck every step that follows, and LLM-based methods learn to predict commands without being taught what geometry those commands should produce. CAD-WAM pairs geometry-conditioned action generation with next-state prediction, encoding surface point clouds of intermediate solids into compact latent states with a pretrained variational autoencoder, then having an LLM backbone predict both the next action and the next geometric state so training carries geometric supervision. On the DeepCAD test set it outperformed task-specific baselines in geometric accuracy and output validity, with larger gains on models needing more operations and better geometry than general-purpose LLMs prompted with the same CAD DSL, and the ablation shows next-state supervision reducing mean Chamfer distance by 13.2 percent. The evidence is one dataset and one DSL, and the method assumes its own CAD kernel can execute each generated action exactly.
Read the story and explore more from the AspireAI community.
Two sparse-autoencoder analyses have resisted scaling: matching semantically similar features across layers, and compressing large feature circuits into interpretable supernodes. This paper argues both are the same underlying problem, estimating semantic distances between SAE features that sit on different activation manifolds. Its representation treats each feature not as a single decoder vector but as an activation-weighted distribution over the hidden states expressing it, projects those distributions into a shared reference space, and compares them with Wasserstein distance. The representation is proved invariant to activation rescaling and stable under perturbation, and recovers true matches under finite-sample margin conditions. Empirically it outperformed decoder-vector and LLM-based baselines and captured subtle functional distinctions between related features, and it compressed large feature circuits into interpretable supernodes automatically. The limitation is that this is interpretability tooling: the paper demonstrates better feature matching and circuit compression, not that downstream interpretability claims become more accurate.