MIT's AI Matches X-Rays to 3D Scans in Seconds
The Problem With Flat Images
Clinicians navigate minimally invasive procedures such as catheter-based interventions and endoscopy using real-time X-rays.
The problem is that X-rays are two-dimensional. A clinician has to mentally reconstruct depth and orientation from a flat image while determining where an instrument sits inside the body.
That makes it difficult to connect what the X-ray shows with the patient's actual three-dimensional anatomy.
One solution is to align the live X-ray with a preoperative CT or MRI. This process, known as 2D/3D registration, helps determine where the instrument is relative to anatomical structures.
Manual registration can be slow and burdensome. Previous AI approaches have also struggled to generalize across patients with different anatomy.
One Model, One Patient
Researchers at MIT and collaborating institutions built a system called xvr, short for X-ray volume registration.
Its approach is different from trying to train one model that works equally well for every patient.
Instead, xvr creates a model tailored to the individual patient.
The system starts with that patient's own preoperative CT or MRI and uses physics-based simulation to generate thousands of synthetic X-rays from different viewing angles.
It can generate about 1,000 synthetic X-rays per second.
Because the synthetic images are generated from the patient's actual 3D scan using a physics-based simulation of the X-ray process, the researchers say the approach avoids the hallucination problem associated with generative image synthesis.
The synthetic images are then used to train a patient-specific neural network capable of matching the patient's real intraoperative X-rays with their 3D scan.
Five Minutes, Not Twelve Hours
There is a catch.
Training a highly accurate patient-specific registration model from scratch takes about 12 hours. That makes the approach impractical for emergency deployment.
The researchers addressed this with a foundation model.
They pretrained the system using whole-body 3D scans from more than 2,000 patients, covering different ages, imaging modalities, and anatomical regions.
When a new patient's scan becomes available, the pretrained model can be fine-tuned to that patient in about five minutes.
The resulting patient-specific model achieves registration accuracy comparable to a model trained from scratch.
Once adapted, xvr can perform the actual X-ray-to-3D registration in seconds, with sub-millimeter accuracy.
The Nature study reports that xvr improved on existing registration methods by an order of magnitude across a broad range of anatomical structures, imaging modalities, and hospitals.
Tested Across Five Hospitals
The researchers evaluated xvr using real clinical data from five hospitals.
The evaluation covered dozens of bones and organ systems in both adult and pediatric patients.
The system significantly outperformed existing AI-based registration approaches in accuracy and robustness while maintaining a speed suitable for time-sensitive procedures.
The underlying research was published in Nature on September 16, 2026, under the title Rapid patient-specific neural networks for X-ray to volume registration.
Why It Matters for Surgical Navigation
The immediate value of xvr is making a conventional X-ray more informative.
Instead of interpreting a flat image in isolation, clinicians can use it in relation to the patient's preoperative three-dimensional anatomy.
That could support navigation in procedures where precise localization matters, including minimally invasive interventions and image-guided surgery.
The researchers also see potential applications in surgical robotics.
Lead author Vivek Gopalakrishnan points to stroke care as one potential application.
Many Americans live more than an hour from a center equipped for time-critical stroke interventions.
Faster and more reliable image registration could make these procedures easier to perform outside major specialized centers.
That access benefit remains a potential application rather than a demonstrated clinical outcome of xvr.
What Comes Next
xvr is currently a research system, not a deployed surgical navigation product.
The team is working with surgical robotics companies and clinical groups to translate the research into navigation and deployment tools.
The researchers identify several areas for further work:
Faster performance for real-time deployment
Additional validation in different clinical situations
Extension to more complex scenarios
Registration involving moving anatomy
The Nature paper also makes the software available as an open-source implementation, making the underlying technology accessible to researchers working on 2D/3D registration.
Source:
