How AI Can Estimate Force in Robotic Surgery Without a Force Sensor
Updated: Sep 20
Senhance and da Vinci 5 already gave surgeons some sense of touch back, each with a physical sensor built into the instrument.
A separate line of research is testing a different route: can a camera estimate the same force with no new hardware at all?
Sensorized Instruments Stayed Rare
Building a force sensor into a sterile surgical instrument is hard.
It has to survive sterilization, fit inside a shaft a few millimeters wide, and stay cheap enough to discard after a single case.
A 2024 Kyoto University study quantified how rare that combination is.
Outside the handful of systems that already ship with one, sensorized instruments have mostly stayed in the lab.
The Kyoto team asked a different question.
Surgeons already judge force by watching tissue deform on screen, so the researchers asked whether a model could learn to do the same thing from video alone.
How Close the Camera Got
The researchers still built a physical sensor, but only to generate ground truth.
A three-axis sensor was fitted to a surgical robot's forceps tip and used to press excised pig kidneys under a range of conditions.
Force data and endoscopic video were recorded in sync, and a neural network was trained to estimate force from the video alone.
The model was tested against two different questions.
One was whether the applied force crossed a 0.5-newton safety threshold, and the other was exactly how much force it was.
Restricting the model's attention to just the forceps tip lifted classification accuracy for the safety threshold from a median of 73.7 percent to 86.4 percent.
On the exact-force question, the model's average error still exceeded that same 0.5-newton threshold either way.
It could flag danger more reliably than it could report a precise number.
Speed wasn't the bottleneck.
Cropping the frame and estimating force took about 20 milliseconds per image, fast enough for real-time use.
The Gap That's Still There
Senhance and da Vinci 5 are already operating on patients.
The camera-based model was tested on excised kidneys in a box trainer, not in a living patient.
The appeal isn't that vision-based sensing beats hardware sensing.
It's that it wouldn't need a new sterile instrument to work at all.
That means it could, in principle, run on any robotic platform with a camera pointed at the tissue, not just the systems that shipped with a sensor built in.
Source: Scientific Reports
