Surgical Intelligence Leaderboard

Surgical Data Science CollectiveThe University of Chicago Booth School of Business

Can a model understand surgical video?

Instruments are the nouns of surgical video; gestures are the verbs. We benchmark models on whether they can recognise the actions a surgeon performs.

Example

What surgical gesture is being performed in this frame?

Choose one label.

  • aspirate
  • clip
  • coagulate
  • cut
  • dissect
  • grasp
  • irrigate
  • no action
  • remove specimen
  • suture
  1. SurgMotion675.26
  2. LEMON769.19
  3. GPT-5.6 Sol857.99
  4. Kimi K3956.60
  5. Claude Opus 51045.76

Exact frame accuracy (%)

ModelExact frame accuracy
SurgMotion675.26%
LEMON769.19%
GPT-5.6 Sol857.99%
Kimi K3956.60%
Claude Opus 51045.76%

One laparoscopic procedure (21:42 of video, 10 gesture classes), annotated by expert surgeons at 15 FPS. The annotations cover 71.4% of the video; frames with overlapping or missing labels are not scored, leaving 13,001 frames.

Kimi K3 uses the documented normalization that maps its clip label to the benchmark's cut label. No confidence intervals are reported for this single-case comparison.

  1. 6 Wu, J., Holm, F., Chen, C., et al. SurgMotion: A Video-Native Foundation Model for Universal Understanding of Surgical Videos. arXiv preprint arXiv:2602.05638 (2026).
  2. 7 Che, C., Wang, C., Vercauteren, T., et al. LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings. Proceedings of CVPR 2026 CVPR 2026 (2026).
  3. 8 OpenAI. Previewing GPT-5.6 Sol. OpenAI release notes openai.com (2026).
  4. 9 Moonshot AI. Kimi K3. GitHub repository github.com/MoonshotAI/Kimi-K3 (2026).
  5. 10 Anthropic. Claude Opus 5 System Card. Anthropic anthropic.com/system-cards (2026).

About

If you found this website useful, please cite as:

@misc{skobelev2026comparativestudysurgicalai,
      title={A Comparative Study in Surgical AI: Potential and Limitations of Data, Compute, and Scaling},
      author={Kirill Skobelev and Eric Fithian and Yegor Baranovski and Jack Cook and Sandeep Angara and Shauna Otto and Zhuang-Fang Yi and John Zhu and Neeraj Mainkar and Margaux Masson-Forsythe and Daniel A. Donoho and X. Y. Han},
      year={2026},
      eprint={2603.27341},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2603.27341},
}