You can't train a robot to place a trocar by trial and error on a patient

Placing a trocar sounds simple until you've done it. Get the angle wrong, and you risk the bowel or a major vessel on entry. Get the depth wrong, and the seal fails mid-procedure.

It's a task built out of feel. Years of repetition calibrate that feel, not a diagram.

That's exactly the task NVIDIA picked to test whether a robot can learn real surgical dexterity without ever touching a real patient.

Why simulation, and not real-world data

Hospitals are chaotic. Every one is different. Layouts, workflows, equipment, and patient populations all vary.

Collecting exhaustive real-world training data across many hospitals isn't just expensive. NVIDIA's engineers call it economically infeasible.

Rare events matter most for safety. They're also the hardest to collect enough real examples of.

So Project Rheo, NVIDIA's blueprint for hospital and surgical robot training, builds the training ground in simulation instead. A digital twin of an operating room lets a robot attempt a task thousands of times, including the rare, dangerous variations that would be unsafe to practice on a real patient even once.

The task that actually proves something

Among the tasks Rheo trains for, Assemble Trocar is the one that matters clinically. It's a precision, two-handed manipulation task, broken into four stages that mirror the real sequence: lift, align, insert, place.

A model trained only on supervised learning, watching recorded demonstrations and imitating them, handled the first stage well. 83 percent success.

Performance collapsed as the stages compounded. By the fourth and final stage, the same model succeeded only 29 percent of the time.

Watching a task done correctly and reproducing every downstream step under slightly different conditions turned out to be very different problems.

Reinforcement learning post-training changed that. After RL: 100 percent on stage one, 92 percent through stage two, 85 percent through stage three, 82 percent through all four stages combined.

The hardest, most compounding part of the task went from failing more often than not to succeeding four times out of five.

What separates this from a research demo

A robot that copies a demonstration is fragile the moment conditions shift. That's precisely what happens across a real multi-step task.

A robot trained through repeated trial, correction, and reward in simulation builds something closer to resilience under variation. That gap, imitation versus reinforcement, is the actual finding here.

NVIDIA also tested what happens when a trained model meets a scene it wasn't built for. A surgical tray pick-and-place task, moved into unfamiliar rooms, saw a base model's success rate collapse from 64 percent to zero in two of three new scenes.

Adding synthetic scene variation during training brought performance back to 30 to 49 percent in those same unfamiliar scenes.

Robots trained on one narrow environment don't generalize by default. Training has to deliberately account for the variation a real hospital will throw at them.

What this doesn't prove yet

Nothing here happened near a patient. This is simulation performance on a defined task with defined success criteria, not a robot cleared for a real OR.

The jump from 82 percent success in simulation to something a hospital would trust with sterile instruments and a real airway is a much longer road than the numbers alone suggest.

But the method is the real story. Surgical dexterity used to be something only a human hand and years of repetition could build.

This is early evidence that a version of that same repetition, done safely in simulation first, can get a robot most of the way there before it ever needs a real patient to practice on.

Source: NVIDIA Technical Blog