Evaluating continuous curvilinear capsulorhexis proficiency via video analysis

Abstract

Purpose

Continuous curvilinear capsulorhexis (CCC) quality affects posterior capsule opacification and visual outcomes after cataract surgery. We developed a novel metric, the Capsulotomy Proficiency Index (CPI), to objectively evaluate the CCC manipulation during cataract surgery.

Methods

We developed a self-design code using MATLAB software to outline the motion trail of the capsulorhexis forceps, the surgical field motion amplitude, the CCC profile (circularity, deviation, decentration), the number of grasps and the total and effective CCC operating time, etc. Considering the weight of each parameter, an integrated CPI value was used to evaluate each CCC surgical video. A Mann-Whitney U test was used to compare the mean CPI from the two groups.

Results

This study included two surgeons with different total cataract surgery volumes (group A: a surgeon with experience of less than 500 cases, group B: a surgeon with experience of more than 1000 cases). 15 CCC videos from each surgeon during the same working period were randomly selected. Group A demonstrated a significantly lower CPI mean value than group B (0.47±0.32 and 0.89±0.23), indicating a significant difference in surgical level.

Conclusions

Based on the objective CCC surgical video evaluation, the CPI could reflect the surgical level difference in CCC manipulation.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Objective, reliable and timely assessment of continuous curvilinear capsulorhexis proficiency is crucial for ophthalmology residency training. However, traditional subjective assessment methods often exhibit significant variability.

WHAT THIS STUDY ADDS

• The Capsulotomy Proficiency Index (CPI) integrates and weighs various parameters derived from video analysis, including the trajectory of capsulorhexis forceps, surgical field motion amplitude and others. It can effectively differentiate surgical skill levels between surgeons with varying experience.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The CPI has the potential to serve as an objective and quantitative tool for assessing surgical skill in cataract surgery, which could lead to improved training strategies and enhanced surgical quality.

Introduction

Global ageing has significantly increased the incidence and demographic impact of age-related diseases, notably cataracts and macular degeneration.¹ Cataracts represent a condition where visual acuity can be improved considerably through surgical intervention, and the annual volume of such surgeries is on a consistent upward trend.² As refractive cataract surgery becomes more widely advocated, patients’ expectations for postoperative visual outcomes have grown more exacting, placing greater demands on surgeons’ technical skills and precision.

Although there are concerted global efforts to standardise ophthalmology residency training, the development and surgical competencies of cataract surgeons display significant variation. Traditionally, surgical skill assessment has relied on expert reviews, which are inherently subjective and lack true objectivity and consistency.³,⁵ Thus, the objective evaluation and assessment of cataract surgeons’ technical proficiency have become increasingly crucial.

Continuous curvilinear capsulorhexis (CCC) is a highly demanding technical aspect of cataract surgery, playing a crucial role in the overall assessment of surgical performance.⁶ Numerous studies emphasise the importance of enhancing surgical precision and reducing the learning curve by discussing relevant surgical techniques and improving surgical instruments.⁷,⁹

With the continuous emergence of interdisciplinary research between medicine and engineering, quantitative data analysis based on video streams has reached varying degrees of capability in data extraction. Integrating artificial intelligence algorithms has partially automated data extraction and analysis. However, the accuracy of intelligent analysis results and the reproducibility across different scenarios within video stream analysis still require significant enhancement. For instance, in 2019, Kim et al used deep learning algorithms to analyse and evaluate 99 CCC videos. Their study reported accuracies and area under curve for specific surgical instrument movement rates and optical flow field analyses at 0.848, 0.863, and 0.643, 0.803, respectively.⁵ These findings indicate that the precision and comprehensiveness of data analysis need further improvement before such technologies can be effectively implemented in clinical practice.

As a foundational endeavour for future deep research into automation and precision, this study aims to employ a semiautomated, precise analytical approach. Building on meticulous manual verification and accurate data annotation, this research uses custom-designed code to achieve precise and objective analysis of one of the most critical steps in cataract surgery: the CCC. The goal is to facilitate the objective evaluation and accurate analysis of CCC procedures performed by different surgeons with varying levels of surgical expertise.

Methods

In this section, we will elaborate on the selection of surgical videos, the extraction of video data, and the specific indicators and analytical approach for extracting CCC-related video data (figure 1).

Figure 1. Overview of the approach for extracting and analysing data based on video streams in this study. CD, continuous curvilinear capsulorhexis deviation; CI, circularity index.

Selection of surgical videos

30 age-related cataract surgical videos for Chinese patients from two surgeons (15 videos for each surgeon) with different surgical experiences were selected. Surgeon with less than 500 total cataract cases was in group A, and surgeon with over 1000 cataract cases was in group B. To further reduce the influence of potential factors on CCC operations, we have chosen medical records that exhibit comparable parameters for video analysis, including age (group A and group B: 64±6 years and 65±5 years, p=0.691), anterior chamber depth (group A and group B: 3.05±0.15 mm and 2.95±0.16 mm, p=0.633) and dilated pupil size (group A and group B: 6.87±0.22 mm and 6.83±0.30 mm, p=0.093). Both surgeons sit superiorly in the surgical procedure. However, surgeon A used Trypan blue for all cases, but surgeon B did not. Moreover, the same type, but not the same brand, of medical devices or eyedrops, such as blades, viscoelastic, topical anaesthetic eyedrops, eyelid speculums and capsulotomy forceps, were used in the CCC procedure.

Video data extraction process

Using bespoke software algorithms, individual frames from the surgical video are systematically extracted and processed in chronological order. Each frame undergoes manual annotation of the capsulorhexis forceps tip, which facilitates the extraction of detailed positional data throughout the CCC procedure. This process captures critical information regarding the forceps’ displacement, movement trajectory and open/close status.

Additionally, for every surgical video, CCC-related data analysis is performed using images taken on completion of the procedure. These images incorporate the intraocular lens (a perfect circle with a diameter of 6.0 mm) and the actual morphology of the capsulorhexis opening. The analysis employs a custom-developed script in MATLAB R2009a, V.7.8.0.347 (MathWorks, Natick, Massachusetts, USA), establishing a two-dimensional (2D) coordinate system. Here, the centre of the limbus functions as the origin, while the intraocular lens (IOL) diameter serves as the scale. The resulting 2D coordinates of the CCC’s centre position are then used to calculate specific quantitative metrics (figure 2).

Figure 2. Data processing workflow illustration. This workflow outlines the systematic processing of continuous curvilinear capsulorhexis video data using MATLAB. (A) Video frame extraction: the VideoReader function in MATLAB is employed to extract the video as sequential distinct images. (B) Feature point annotation: MATLAB function is used to annotate specific feature points, such as the tip of the forceps, the limbal boundary and the capsulorhexis edge, etc. The annotated data are then systematically stored in a coordinate file. (C) Data interpretation and metric calculation: a specialised MATLAB data reading function is responsible for interpreting the information contained within the coordinate file.

Video data analysis

Total duration of the CCC Procedure as θ₁

After the creation of the flap, the moment when the capsulorhexis forceps first make contact is designated as the starting point, while the point at which the CCC is completed serves as the endpoint. The interval between these two points is recorded in seconds. A threshold value of $\tau$=25 s has been established. The specific calculation and determination method for this parameter is as follows:

$${\displaystyle {\displaystyle {\theta }_1=\{\begin{array}{l}1,t\le \tau \\ e^{-\frac{t-\tau }{10\tau }},t>\tau \end{array}}}$$

Effective CCC operation duration as θ₂

This is the total CCC operation time minus the duration spent adjusting the position of the capsulorhexis forceps, measured in seconds. A threshold value of ${\tau }_1$=20 s has been established. The specific calculation and determination method for this parameter is as follows:

$${\displaystyle {\displaystyle {\theta }_2=\{\begin{array}{l}1,t\le {\tau }_1\\ e^{-\frac{t-{\tau }_1}{10\tau 1}},t>{\tau }_1\end{array}}}$$

Capsulorhexis forceps opening frequency as θ₃

Throughout the entire CCC procedure, this term refers to the frequency with which the forceps open to adjust their position, quantified by the number of occurrences. The established threshold for this frequency is set at $\rho$=5, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_3=\{\begin{array}{l}1,n\le \rho \\ e^{-\frac{n-\rho }{10}},n>\rho \end{array}}}$$

Total path length of the capsulorhexis forceps tip as θ₄

Starting from the tip point where the anterior capsule is first grasped after initial flap creation, this term refers to the total distance traversed by the tip of the capsulorhexis forceps until the CCC completion, measured in millimetres (mm). The established threshold for this frequency is set at $l_{up}$=130 mm, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_4=\{\begin{array}{l}1,l\le l_{up}\\ e^{-\frac{l-l_{up}}{10}},l>l_{up}\end{array}}}$$

CCC deviation as θ₅

This metric is determined by calculating the difference between the maximum and minimum radii of the actual CCC, followed by dividing this difference by the ideal target radius (2.25 mm). The established threshold for this parameter is set at $c_{up}$=0.1, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_5=\{\begin{array}{l}1,c\le c_{up}\\ e^{-\frac{c-c_{up}}{10}},c>c_{up}\end{array}}}$$

CCC decentration as θ₆

This metric is the distance between CCC centre and limbus centre, and it assesses the degree of eccentricity of the actual CCC, quantifying the decentration in terms of mm. The established threshold for this parameter is set at $C_{up2}$=0.3 mm, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_6=\{\begin{array}{l}1,C\le C_{up2}\\ e^{-\frac{C-C_{up2}}{10}},C>C_{up2}\end{array}}}$$

Horizontal field of view displacement as θ₇

With the lower-left corner of the initial frame of the surgical video serving as the origin, this metric measures the extent of horizontal displacement of the surgical field in each successive frame, expressed in mm. The established threshold for this parameter is set at $X_{up}$=3.0 mm, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_7=\{\begin{array}{l}1,X\le X_{up}\\ e^{-\frac{X-X_{up}}{10}},X>X_{up}\end{array}}}$$

Vertical field of view displacement as θ₈

Similarly, using the lower-left corner of the initial frame as the origin, this metric quantifies the vertical displacement of the surgical field throughout the procedure in each frame, also expressed in mm. The established threshold for this parameter is set at $Y_{up}$=3.0 mm, the methods for calculation and specific criteria for determination are as follows:

$${\displaystyle {\displaystyle {\theta }_8=\{\begin{array}{l}1,Y\le Y_{up}\\ e^{-\frac{Y-Y_{up}}{10}},Y>Y_{up}\end{array}}}$$

Capsulotomy Proficiency Index (CPI)

Perform the final performance evaluation of the CPI using the circularity index (CI) as the standard. The CI is determined by multiplying the total area of the actual CCC by 4π, and then dividing by the square of the CCC perimeter. The established threshold is set at $C{I}_{low}$=0.9, the procedure for determining the final value of the CPI is as follows:

$${\displaystyle {\displaystyle S_a=\{\begin{array}{l}\prod _{i=1}^8{\theta }_i,CI>C{I}_{low}\\ 0,CI\le C{I}_{low}\end{array}}}$$

Statistical analyses were performed using SPSS software (V.21.0). Continuous variables were expressed as mean±SD. The Shapiro-Wilk test was used to assess the normality of the distribution of the continuous variables prior to conducting significance tests. For data following a normal distribution, an independent sample t-test was employed; otherwise, the Mann-Whitney U test was used. The conventional significance level for individual comparisons was initially set at p<0.05. However, given the multiple comparisons conducted for the CPI across nine variables within this study, a Bonferroni correction was applied. Consequently, the adjusted threshold for statistical significance was established at p<0.006.

The sample size for comparing means was calculated using MedCalc software (V.20.216; Medcalc Software), with a type 1 error rate of 0.05 and a type 2 error rate (1−power) of 0.10. Based on the data from the study by Laude et al, assuming a mean difference in the proposed performance metric of 0.4432, with a SD of 0.2322 in group 1 and 0.1211 in group 2, and assuming the ratio of sample sizes between group 1 and group 2 is 1, the calculated minimum sample size required is five in each group.¹⁰

Results

A statistical analysis of the results from 15 surgical videos in each group identified significant differences between the two groups across various observational indicators, as shown in table 1. Figure 3 illustrates the trajectory of the CCC forceps, the displacement of the surgical field and the morphology of the CCC in representative cases from both groups.

Table 1. Comparison of the specific analytical parameters between the two groups.

Parameters	Group A (n=15)	Group B (n=15)	Difference (A−B)	P value
Total CCC time (s)	31±9	20±4	11±3	<0.001
Effective CCC time (s)	23±6	16±3	7±2	<0.001
Proportion of effective CCC time (%)	76±8	80±6	4±2	0.184*
Forceps open times	5±1	4±1	1.5±0.4	0.002
Forceps total path length (mm)	134±31	110±18	23±9	0.018
Forceps velocity (mm/s)	4.40±0.75	5.62±0.67	−1.21±0.26	<0.001
CCC deviation	0.15±0.04	0.10±0.03	0.04±0.01	0.002
CCC maximum radius (mm)	2.76±0.15	2.65±0.18	0.11±0.06	0.08
CCC minimum radius (mm)	2.36±0.14	2.37±0.20	−0.01±0.06	0.881
CCC decentration (mm)	1.49±0.90	0.34±0.22	1.15±0.24	<0.001
Horizontal field of view displacement (mm)	3.21±0.64	1.73±0.46	1.48±0.20	<0.001
Vertical field of view displacement (mm)	3.47±1.12	2.06±0.31	1.42±0.30	<0.001
Circularity index	0.97±0.01	0.98±0.01	−0.01±0.002	0.002*
CPI	0.47±0.32	0.89±0.23	−0.42±0.10	<0.001*

^*Mann-Whitney U test.

CCC, continuous curvilinear capsulorhexis; CPI, Capsulotomy Proficiency Index.

Figure 3. Illustration of the continuous curvilinear capsulorhexis (CCC) forceps trajectory, the surgical field displacement and the CCC morphology between the two groups. The first row illustrates a case from group A with a Capsulotomy Proficiency Index (CPI) of 0.07, while the second row shows a case from group B with a CPI of 1.00. (A,D) The CCC forceps trajectory, with total path lengths of 153 mm and 81 mm, respectively. The red circle denotes the starting position, the green asterisk marks the endpoint, and the blue circle with a cross indicates the best-fit circle and its centre. (B,E) The surgical field movement throughout the CCC procedure. (C,F) The morphology and size of the CCC.

Control of CCC forceps

Group A demonstrated longer total and effective operation times for CCC than group B (p<0.001). However, the proportion of effective operation time showed no statistically significant difference between the two groups (p=0.184). Additionally, the frequency of opening CCC forceps was significantly higher in group A than in group B (p=0.002). The path length of the CCC forceps was 23 mm longer in group A compared with group B, but no significant difference (p=0.018). Regarding the average speed of the CCC forceps, group B was significantly faster than group A (p<0.001).

Surgical field displacement

Group A exhibited significantly less control over the surgical field throughout the entire CCC procedure than group B. This was most apparent in the displacement range along both horizontal and vertical directions, where substantial differences were observed (p<0.001).

Efficiency of CCC

Although there were no significant differences between group A and group B in terms of the mean maximum and minimum radii of the CCC (both p>0.05), group A exhibited a significantly greater CCC deviation compared with group B (p=0.002). Moreover, the CCC decentration was significantly greater in group A than in group B (p<0.001). Additionally, the CI, which assesses the integrity and regularity of the completed CCC, was significantly higher in group B compared with group A (p=0.002).

Synthesising the analyses from the three areas mentioned above, the final comparison of the CPI indicates that group A scored significantly lower than group B (p<0.001).

Discussion

This study employed meticulously and manually annotated video stream data, along with custom-designed software, to develop the CPI by weighing various quantitative indicators of the CCC procedure. This approach enabled a comprehensive and objective comparison of cataract surgeons with different experience levels in CCC proficiency. The results indicate that the CPI can serve as a novel benchmark for the objective evaluation of this surgical technique. Unlike some studies that use artificial intelligence for automated analysis, this research is based on manually annotated frame-by-frame image data, potentially offering higher accuracy. Moreover, previous research on the automated evaluation of CCC still requires significant improvements in accuracy and should incorporate a broader range of influencing indicators.³,^{510 11} Consequently, the evaluation and analysis approach centred on specific CCC details in this study could provide valuable insights for future advancements in deep learning-based intelligent data extraction and analysis, which may also be applicable to other surgical steps.

The objective quantification of surgical proficiency has long been essential to resident training and evaluation. compared with subjective assessment methods, the objective evaluation of cataract surgical procedures can better eliminate subjective bias, provide standardised assessments and ensure assessment quality, thereby improving the efficiency and effectiveness of resident training.¹² By objectively analysing the main aspects and key steps of operator manoeuvres (such as controlling the size and regularity during capsulorhexis and mastering pivot operations), the learning curve can be shortened and personalised feedback can be provided, thereby enhancing the manipulation proficiency of residents. The objective assessments, analysis data and methods also help establish more comprehensive simulation training devices for medical education and training and assist in the development of future automated robotic surgery.¹³ However, achieving quantifiable assessments has proven challenging and relies mainly on experts’ subjective judgement of video quality. The study by Annadanam et al concentrated on comparing the differences and relationships between subjective evaluations using expert questionnaires—specifically, the International Council of Ophthalmology’s Ophthalmology Surgical Competency Assessment Rubric for Phacoemulsification—and objective video analysis methods for assessing residents’ CCC eccentricity, roundness, size and centration. The main finding indicated that the correlations between objective and subjective scores were low, ranging from 0.09 to 0.39. Nonetheless, the study underscored the importance of objective assessments in evaluating surgical proficiency.¹¹

In the study conducted by Laude et al, several influencing factors during the capsulorhexis step—such as total grasp time, total CCC operation time, number of grasps, CCC decentration and the circularity index—were analysed to assess the operator’s proficiency, thereby providing valuable references for the development of our research.¹⁰ In contrast, our study introduced new key weighted indicators, including CCC path length, field of view displacement and CCC deviation, to facilitate a more accurate and comprehensive evaluation of CCC proficiency.

Experienced surgeons exhibit superior precision in controlling the capsulorhexis forceps during the CCC procedure.¹⁴ This is further corroborated by the significant advantage demonstrated by the group B surgeon in controlling the capsulorhexis forceps within this study. This indicates that cataract surgeons can, through refined control of surgical instrument force, effect a reduction in capsulorhexis duration, minimise forceps opening frequency, optimise effective procedure time, shorten the forceps’ movement paths and enhance operational speed.^{6 15} The significant difference in surgical field displacement between the two groups in this study indicates that less experienced cataract surgeons may face ‘tunnel vision’ challenges during the CCC procedure. They often exhibit considerable shifts in the surgical field position due to inadequate hand control, which can potentially lead to unnecessary surgical complications. This highlights the importance of surgeons focusing on specific manipulations and comprehensive awareness of the entire surgical field. By adopting a holistic perspective, surgeons can better ensure that the procedure adheres to established standards and maintains safety.

In the absence of capsulorhexis assistance devices or digital navigation systems, beginners or those with limited surgical experience often rely solely on personal judgement or anatomical landmarks, such as the pupillary margin and limbus, to subjectively control the position and size of the CCC. This method is significantly reliant on the surgeon’s level of experience. Our study results demonstrate that experienced surgeons surpass beginners in CCC quality, particularly in terms of CCC deviation, CCC decentration and CI. This highlights the vital role of capsulorhexis assistance devices and digital navigation systems in resident training and enhancing surgical precision.^{16 17}

In this study, CPI offers a more objective assessment of a surgeon’s CCC performance. It achieves this by thoroughly considering the weight and significance of various CCC-related indicators.⁴ Our study reveals that the CPI effectively differentiates the surgeon’s proficiency and mastery of this technique. The CPI surgical evaluation algorithm can provide surgeons with an objective benchmark for CCC efficiency and quantify the impact of technique modifications, thereby optimising surgical performance and improving patient outcomes in the near future. However, it is essential to acknowledge that the threshold values for each quantitative indicator in this study were set based on subjective experience. The choice of these threshold values can significantly influence the final calculation of the CPI, highlighting the necessity for further research to establish objective validation in this domain.

This study presents several limitations: first, our analysis relies on 2D video images, lacking essential 3-dimensional (3D) spatial data. This limitation constrains the depth and spatial interpretation of surgical procedures. Future research aims to incorporate 3D spatial data to improve the study’s quality and efficiency. In addition, this study employs the number of surgical cases solely as the criterion for group distinction, which may not fully encapsulate the complexity and diversity of the surgeon’s CCC techniques. This limitation arises from the differing learning curves among physicians. Furthermore, even when comparing surgical videos with similar ocular parameters, it remains challenging to eliminate potential influences from variables such as the surgical environment, the surgeon’s condition and the patient’s level of cooperation during the procedure. Despite these limitations, our study provides valuable insights into the objective assessment of capsulotomy surgical proficiency.

In conclusion, this research demonstrates that the CPI, derived from quantitative data in CCC surgical videos, is a robust and quantifiable indicator for objectively evaluating CCC proficiency.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.