arXiv paper: Anatomy Contextualized Adaption of CT Foundation Models

arXiv ID: 2607.27154v1 Title: Anatomy Contextualized Adaption of CT Foundation Models Authors: Roshan Kenia, Stephanie L McNamara, William Lotter Primary category: cs.CV Categories: cs.CV, cs.AI Comment: Accepted to the European Conference on Computer Vision (ECCV) 2026 MedFM-Bench Workshop Published: 2026-07-29T17:32:57Z Updated: 2026-07-29T17:32:57Z Abstract: CT vision-language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained vision-language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level vision-language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level embeddings, which are refined via a transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once embeddings are cached. The attention weights learned by ACA's inter-anatomy transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded vision-language alignment while preserving and enhancing global anatomical context. PDF: https://arxiv.org/pdf/2607.27154v1