AI-Driven Surgical Tool Localization in Microsurgical Training Simulations

Precise localization of surgical instrument tips is essential for evaluating fine motor skills and enabling automation in microsurgical training. This study presents a deep learning framework based on keypoint heatmap regression to detect instrument tips in frames extracted from simulated surgical videos. A dataset of 1781 annotated frames from seven videos was used for evaluation. The framework was trained with different loss functions—root mean squared error (RMSE), weighted Kullback-Leibler divergence (WKLD), and Dice loss—and compared with direct coordinate regression and segmentation-based models. The RMSE-based model achieved the best performance (MAE = 7.54 pixels), while the WKLD-based model provided more stable predictions across thresholds for blank mask detection. Segmentation and direct regression models showed significantly higher errors. Statistical analyses confirmed the advantage of heatmap regression over baseline approaches. These results support the adoption of heatmap-based keypoint localization for robust tool tracking in simulated surgical environments and its integration into training systems for skill assessment.