AI Diagnostic Accuracy in Surgery: How to Read the Numbers
Updated: Oct 8
A new study appears with a familiar headline.
"AI achieves 94% accuracy."
The number sounds impressive.
It rarely tells the whole story.
Before trusting an AI accuracy claim, ask a few simple questions.
Accuracy isn't enough
Accuracy measures how often the AI gives the correct answer.
It is only one measure of performance.
Studies should also report sensitivity and specificity.
Sensitivity measures how often the AI correctly identifies patients with disease.
Specificity measures how often it correctly identifies patients without disease.
Together, these metrics provide a clearer picture of diagnostic performance.
What about AUC?
Many studies also report AUC, or Area Under the Curve.
A higher AUC generally indicates better overall diagnostic performance.
It does not show how many patients were missed.
AUC should always be interpreted alongside sensitivity and specificity.
Can the results be trusted?
Performance in one hospital does not guarantee the same performance elsewhere.
Patient populations, imaging equipment, and clinical workflows differ.
This is why external validation is important.
Studies involving multiple hospitals and larger patient populations usually provide stronger evidence than small, single-center studies.
There is no single AI accuracy
Diagnostic performance depends on the clinical task.
Detecting a brain hemorrhage is different from identifying liver metastases or predicting postoperative complications.
Accuracy should always be interpreted in the context of the specific application.
Four questions to ask
Before accepting an AI accuracy claim, ask:
What was the AI compared with?
Were sensitivity and specificity reported?
Was the model externally validated?
How large was the study?
These questions provide far more context than a single percentage.
