Abstract
Background:
Near infrared autofluorescence (NIRAF) detection has previously demonstrated significant potential for real-time parathyroid gland identification. However, the performance of a NIRAF detection device - PTeye® - remains to be evaluated relative to a surgeon’s own ability to identify parathyroid glands.
Methods:
Patients eligible for thyroidectomy and/or parathyroidectomy were enrolled under 6 endocrine surgeons at 3 high-volume institutions. Participating surgeons were categorized based on years of experience. All surgeons were blinded to output of PTeye® when identifying tissues. The surgeon’s performance for parathyroid discrimination was then compared with PTeye®. Histology served as gold standard for excised specimens, while expert surgeon’s opinion was used to validate in-situ tissues.
Results:
PTeye® achieved 92.7% accuracy across 167 patients recruited. Junior surgeons (<5 years of experience) were found to have lower confidence in parathyroid identification and higher tissue misclassification rate per specimen when compared to PTeye® and senior surgeons (>10 years of experience).
Conclusions:
NIRAF detection with PTeye® can be a valuable intraoperative adjunct technology to aid in parathyroid identification for surgeons.
Keywords: parathyroid glands, surgical guidance, thyroidectomy, parathyroidectomy, near infrared autofluorescence
INTRODUCTION
Post-operative hypocalcemia is a major complication that may occur following thyroidectomy due to accidentally excising healthy parathyroid glands or inadvertent damage to their blood supply. (1, 2) In addition, failure to localize all diseased parathyroid glands during parathyroidectomies leads to persistent hyperparathyroidism and costly repeat surgeries. (3, 4) Most surgeons tend to visually identify parathyroid glands by relying on their accrued surgical experience, which may be highly subjective and not always accurate, as parathyroid glands could resemble thyroid nodules, fat, thymus, and lymph nodes. While frozen section analysis or tissue aspirate parathyroid hormone analysis can aid in intraoperative parathyroid identification, these techniques are invasive, labor-intensive and require 20–30 minutes per sample. (5)
When parathyroid glands were found to have strong near-infrared auto-fluorescence (NIRAF) than other soft tissues in the neck (6), this discovery had profound implications. NIRAF detection is a simple, real-time, non-invasive and label-free approach that could identify both healthy and diseased parathyroids with ~97% accuracy. (7) With several studies validating NIRAF detection for identifying parathyroid glands (8), FDA-clearance was granted for Fluobeam® (a camera-based system), and PTeye® (a probe-based system) to adjunctively aid intraoperative parathyroid identification. (9) Nonetheless, various groups have evaluated the impact of NIRAF detection and obtained conflicting results for (i) minimizing post-operative hypocalcemia and (ii) improving parathyroid localization and minimizing frozen section analysis. (10–17) The lack of human factors testing, i.e. variability in the surgeon’s experience or technique, may account for these contrasting results. Given that the effectiveness of NIRAF detection may be highly contingent on the experience of the surgeons using these devices, human factors variability should be considered to determine whether NIRAF-based parathyroid identification could be truly impactful. Our current study aimed to assess human factors variability through a surgeon-blinded multi-centric study with PTeye® (AiBiomed, Santa Barbara, CA). The study sought to compare the accuracy of multiple surgeons in identifying parathyroid glands intraoperatively, based on their own surgical experience, while evaluating their performance relative to PTeye®.
MATERIAL AND METHODS
Study Design.
Patients aged ≥18 years, undergoing thyroidectomy and/or parathyroidectomy, provided informed written consent at 3 high-volume centers. The blinded study was granted approval by the Institutional Review Boards at these sites. Data obtained from patients with secondary hyperparathyroidism were excluded from further analysis, due to irregular NIRAF observed in parathyroid glands of these patients. (7, 18) Participating endocrine surgeons were designated as Surgeons A and B for Site 1, Surgeons C and D for Site 2, Surgeons E and F for Site 3. The surgeons were further grouped based on their years of surgeon experience, i.e. independent surgical practice, at the time of study. Based on this descriptor, surgeons were eventually categorized as (i) senior (Surgeons A, C and E): > 10 years of experience and (ii) junior (Surgeons B, D and F): < 5 years of experience. The number of procedures performed per annum were additionally noted for each surgeon.
Instrumentation of PTeye®
PTeye® (see Figure 1A), mainly consists of (i) a console that houses a 785 nm laser, a photodiode detector and relevant internal circuitry, (ii) a display interface to provide real-time NIRAF information to the surgeon, (iii) a foot-pedal to activate the laser for NIRAF detection and (iv) a sterile detachable fiber-optic probe. The internal circuitry of PTeye® enables NIRAF detection without interference from ambient operating room lights. Tissue NIRAF measured is displayed as (i) ‘Detection Level’ – absolute NIRAF intensity measured from tissue and (ii) ‘Detection Ratio’ – absolute NIRAF intensity of tissue normalized to the baseline NIRAF intensity background established for each patient. PTeye® is designed such that when ‘Detection Ratio’ exceeds 1.2, the device identifies/classifies the tissue as ‘parathyroid’. (18)
Figure 1:
(A) Instrumentation of a fiber probe-based NIRAF detection modality – PTeye® – that consists of (1) the console that encloses the near infrared laser and the detector,(2) the display interface for guiding the surgeon, (3) a foot-pedal that is used by the surgeon to activate the laser for tissue NIRAF measurements, and (4) a detachable fiber-optic probe that illuminates the tissue and collects the resultant tissue NIRAF back to the detector in the console. (B) For each patient, a baseline NIRAF is established by taking 5 measurements on random sites on the thyroid gland. (C) NIRAF measurements are obtained by placing the fiber-optic probe on tissue of interest and pressing the pedal. NIRAF measurements with PTeye® are made with ambient operation room lights remaining on. The display monitor of PTeye® will then inform if the suspect tissue is parathyroid (D) or not (E)
NIRAF detection with PTeye®.
For each patient, the surgeon first establishes a baseline NIRAF by obtaining measurements on five random sites on the patient’s thyroid (or neck muscle, if patient had prior thyroidectomy) (Figure 1B). To obtain a NIRAF measurement, the surgeon places the sterile fiber-optic probe on the tissue (Figure 1C), and the study coordinator then presses the foot-pedal. After establishing the baseline, whenever a prospective PG candidate was visualized, the surgeon’s confidence in identifying the tissue as PG was first recorded as either ‘high’, ‘moderate’ or ‘low’. After noting the surgeon’s confidence, NIRAF parameters – Detection Level and Detection Ratio – as displayed on PTeye® console were recorded from these tissues of interest. A ‘positive’ and ‘negative’ PTeye measurement for PG tissues are depicted in Figure 1D and 1E, respectively. All surgeons were blinded to the auditory-visual output of PTeye® to obtain the surgeon’s unbiased assessment of the visualized ‘tissue’ and not alter the procedure course. An example of the surgeon-blinded methodology followed with the device has been demonstrated in Video 1.
While healthy parathyroids were mostly left in-situ, diseased parathyroids that were excised were sent for frozen section analysis and/or routine histopathology. If needed, healthy parathyoids were also histologically validated before auto-transplantation. Performance in parathyroid identification were noted for each surgeon via: (i) the number of ‘low’, ‘moderate’ and ‘high’ confidence identification of parathyroid glands per case, (ii) number of frozen section analysis per case and (iii) number of tissue specimens misclassified when validated with histology. Tissues were considered misclassified when: (i) surgeon had ‘high’ confidence that a tissue was parathyroid, but histology was negative, (ii) surgeon had ‘low’ confidence that a tissue was parathyroid, but histology indicated that the tissue was parathyroid and (iii) surgeon had a ‘high’ confidence that the tissue was not a parathyroid, but histology indicated that it was.
Data Analysis
Detection Ratios recorded for parathyroid glands, thyroid and other non-parathyroid tissues were reported as median with inter-quartile range (IQR) at each study site. Detection Ratios across different tissues were analyzed using the 2-tailed t-test for unequal variance. One-way analysis of variance test was applied to evaluate if quantitative parameters such as age, body mass index, number of parathyroids confirmed and number of frozen section analysis differed significantly across study sites and surgeons. A Chi-squared test was utilized to quantify significance in the distribution of categorical variables such as sex, ethnicity and disease types in the 3 study sites. Chi-squared tests were further used to determine if there was a significant difference between senior and junior surgeons for – (i) incidence of low confidence in parathyroid identification and (ii) misclassification rate of excised tissues validated with histology. Chi-squared analysis was utilized to determine if there was a difference between tissue misclassification rates of the surgeons versus PTeye®. Analysis of variables like ‘low confidence incidence’, ‘number of frozen section analysis’ or ‘tissue misclassification rate’, were performed for surgeons as a group (senior vs junior), instead of individual surgeons. For all quantitative or categorical data analyzed, a p-value (p) ≤ 0.05 was considered statistically significant. Performance of PTeye® was validated using (i) histology for excised/biopsied tissues and (ii) the surgeon’s visual confirmation – with high/moderate confidence – for in-situ tissues. Tissues identified with low confidence with no corresponding histology were excluded from performance analysis of PTeye®.
The effect of study design variables on accuracy of PTeye® and the participant surgeons were analyzed using generalized linear mixed-effect model with logistic regression, separately for in situ and/or excised tissues. Fixed-effect factors considered include study center, surgeon experience, number of thyroid surgeries per year, and tissue type. Study center and patient, as well as surgeon and patient, were modeled as nested random effects. The multi-level regression analysis yielded adjusted odds ratio and 95% confidence levels for each study design parameter considered, where a variable with p <0.05 was considered to have significant effect on accuracy of parathyroid identification. In vivo accuracy of surgeons was not assessed, as the surgeon’s opinion itself was considered the gold standard for in-situ tissues. While individual surgeon performance, disease types and procedure types were initially considered as study covariates, they could not be eventually included in the multivariable models due to lack of degrees of freedom and small sizes of certain categories.
RESULTS
Tissue NIRAF detection and performance accuracy with PTeye®
Data were collected from 167 patients across 3 study sites with the demographic and clinico-pathologic variables being described in Table 1, with no statistically significant variation seen across all study sites. Tissue NIRAF assessment with PTeye® was conducted either in-situ or ex vivo on 386 parathyroid glands (255 healthy and 131 diseased) and 1362 non-parathyroid sites comprising of thyroid, fat, muscle, trachea, lymph nodes, thymus and fibro-adipose tissues.
Table 1:
Distribution of demographic and clinico-pathological variables for enrolled patients across all the 3 study sites.
| Patient Demographic Variables | Site 1 | Site 2 | Site 3 | Total | p-value |
|---|---|---|---|---|---|
| Total number of patients | 39 | 50 | 78 | 167 | |
| Age (in years) | 0.89 | ||||
| Median | 55 | 52 | 51 | 52 | |
| (Range) | (20 – 77) | (19 – 80) | (20 – 88) | (19 – 88) | |
| Gender | 0.38 | ||||
| Male | 8 (20.5%) | 10 (20.0%) | 23 (29.5%) | 41 (24.6%) | |
| Female | 31 (79.5%) | 40 (80.0%) | 55 (70.5%) | 126 (75.2%) | |
| Ethnicity | 0.54 | ||||
| Caucasian | 33 (84.6%) | 46 (92.0%) | 68 (87.2%) | 147 (87.9%) | |
| Non-Caucasian | 6 (15.4%) | 4 (8.0%) | 10 (12.8%) | 20 (12.1%) | |
| Body Mass Index (in kg/m 2 ) | 0.18 | ||||
| Median | 28 | 30.3 | 28.5 | 28.9 | |
| (Range) | (19.0 – 47.0) | (19.9 – 53.8) | (20.5 – 53.4) | (19.0 – 53.8) | |
| Disease Type | 0.23 | ||||
| Benign Conditions | |||||
| Nontoxic Multinodular Goiter | 10 (25.6%) | 13 (26.0%) | 13 (16.6%) | 36 (21.5%) | |
| Nontoxic Solitary Nodule | 0 (0.0%) | 2 (4.0%) | 6 (7.7%) | 8 (4.8%) | |
| Toxic Multinodular Goiter | 0 (0.0%) | 1 (2.0%) | 1 (1.3%) | 2 (1.2%) | |
| Toxic Solitary Nodule | 0 (0.0%) | 1 (2.0%) | 0 (0.0%) | 1 (0.6%) | |
| Graves' Disease | 0 (0.0%) | 4 (8.0%) | 9 (11.5%) | 13 (7.8%) | |
| Thyroiditis (including Hashimoto's) | 0 (0.0%) | 1 (2.0%) | 3 (3.9%) | 4 (2.4%) | |
| Primary Hyperparathyroidism | 21 (53.9%) | 13 (26.0%) | 32 (41.0%) | 66 (39.5%) | |
| Tertiary Hyperparathyroidism | 0 (0.0%) | 0 (0.0%) | 1 (1.3%) | 1 (0.6%) | |
| Concurrent Thyroid and Parathyroid Disease | 1 (2.6%) | 4 (8.0%) | 2 (2.6%) | 7 (4.2%) | |
| Prophylactic Thyroidectomies | 0 (0.0%) | 0 (0.0%) | 1 (1.3%) | 1 (0.6%) | |
| Malignant Conditions | |||||
| Papillary Carcinoma of Thyroid | 7 (17.9%) | 11 (22.0%) | 9 (11.5%) | 27 (16.2%) | |
| Medullary Carcinoma of Thyroid | 0 (0.0%) | 0 (0.0%) | 1 (1.3%) | 1 (0.6%) |
Median ‘Detection Ratio’ for parathyroid glands was 4.11 (IQR: 2.48–6.76), which was significantly higher than that of thyroid at 0.88 (IQR: 0.71–1.04; p=9.9×10−92) and other soft tissues at 0.43 (IQR: 0.24–0.76; p=4.1×10−103) as seen in Figure 2A. There was no significant difference in Detection Ratios for healthy versus diseased parathyroids at Site 1 (Median: 2.81 vs 2.52; p=0.22) and Site 2 (Median: 4.32 vs 2.78; p=0.34), while diseased parathyroids had lower Detection Ratios than healthy ones at Site 3 (Median: 4.24 vs 2.25; p=2.6×10−5) as depicted in Figure 2B. Since diseased parathyroids have heterogenous distribution of low and high NIRAF regions (19, 20), it is plausible that surgeons at site 3 – blinded to PTeye® and unknown to themselves – may have placed the probe more frequently on the low NIRAF regions of diseased PGs, compared to surgeons at the other sites. Based on data from the 3 sites (Table 2), PTeye® obtained 95.6% sensitivity, 91.9% specificity and 92.7% accuracy (kappa=0.80), with a positive predictive value of 76.9% and negative predictive value of 98.7%. It was however observed that Site 1 yielded relatively lower sensitivity (90.0%) in PG identification with PTeye® than Site 2 (96.5%) and Site 3 (97.6%). Logistic regression analysis further indicated that PTeye® was ~2.5 times more likely to be accurate for in-situ tissues at Site 2 and 3, compared to Site 1, albeit with marginal significance (p=0.07 for site 2 vs 1; p=0.06 for site 3 vs 1), while no such difference was observed for excised tissues.
Figure 2:
(A) A box and whisker plot representing the PTeye® Detection Ratios for parathyroid glands, thyroid glands and other soft tissues in the neck across all 3 study sites. Across all three study sites, the Detection Ratios was consistently elevated for parathyroid glands compared to the thyroid gland and other soft tissues in the neck. (B) A box and whisker plot representing the PTeye® Detection Ratio for healthy versus diseased parathyroid glands across all 3 study sites. While there is no significant difference in Detection Ratios for healthy versus diseased glands at Site 1 and 2, diseased glands had significantly lower detection ratio than healthy glands at site 3. ** - statistically significant difference where p-value ≤ 0.5
Table 2:
Performance accuracy of PTeye® in label-free intraoperative parathyroid gland identification in the blinded multi-centric study.
| Number of parathyroid glands tested | Number of non-parathyroid sites* tested | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value | False Negative Rate | False Positive Rate | Accuracy | kappa (k) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Site 1 | 90 | 405 | 90.0% | 87.2% | 60.9% | 97.5% | 10.0% | 12.8% | 87.7% | 0.65 |
| Site 2 | 85 | 404 | 96.5% | 93.8% | 77.4% | 99.2% | 3.5% | 6.2% | 94.3% | 0.82 |
| Site 3 | 211 | 553 | 97.6% | 93.9% | 86.5% | 99.0% | 2.4% | 6.1% | 94.9% | 0.88 |
| Overall | 386 | 1362 | 95.6% | 91.9% | 76.9% | 98.7% | 4.4% | 8.1% | 92.7% | 0.8 |
- Non-parathyroid sites included thyroid glands, fat, muscle, trachea, thymic tissue, lymph nodes and fibro-adipose tissues.
Variability in parathyroid identification by surgeon based on study site and surgeon experience
As seen in Table 3, number of parathyroid glands identified per patient was considerably lower for Site 2 compared to Site 1 and Site 3 (p=2.08×10−6). Significantly less parathyroids were seen or confirmed by Surgeon C, compared to the other senior surgeons (p=6.48×10−7), while no such significant difference was observed among the junior surgeons (p=0.32). Furthermore, Surgeon C explored and identified fewer parathyroids per patient for parathyroidectomy/combined procedures than for thyroidectomy (p=0.003). The data suggest that Surgeon C often employed a more focused approach for parathyroid exploration, especially during parathyroidectomy, compared to the senior surgeons at Site 1 and 3. Overall, there was no marked variation in the number of parathyroid glands seen by senior and junior surgeons (p=0.71).
Table 3:
Distribution of patients (cases) enrolled, number of parathyroid glands (PGs) visualized per patient and number of frozen section analysis (FSA) performed per case at the 3 study sites. All participant surgeons visually confirmed PGs solely based on their surgical experience and remained blinded to output from PTeye®.
| Site | Surgeon | Surgeon experience at time of study | Thyroid procedures per annum | Parathyroid + combined procedures per annum | All procedures | Thyroid procedures | Parathyroid + combined procedures | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Cases | Number of PGs* | PGs per case* | FSA per case | Cases | PGs per case* | FSA per case | Cases | PGs per case* | FSA per case | |||||
| Site 1 | Surgeon A | > 10 years | 93 | 97 | 28 | 62 | 2.21 | 0.57 | 11 | 2.09 | 0.00 | 17 | 2.29 | 0.94 |
| Surgeon B | < 5 years | 28 | 23 | 11 | 28 | 2.55 | 0.73 | 6 | 2.50 | 0.00 | 5 | 2.60 | 1.60 | |
| Total | 39 | 90 | 2.31 | 0.62 | 17 | 2.24 | 0.00 | 22 | 2.36 | 1.09 | ||||
| Site 2 | Surgeon C | > 10 years | 121 | 82 | 38 | 62 | 1.63 | 0.39 | 23 | 1.91 | 0.09 | 15 | 1.20 | 0.93 |
| Surgeon D | < 5 years | 113 | 44 | 12 | 23 | 1.92 | 0.00 | 10 | 2.10 | 0.00 | 2 | 1.00 | 0.00 | |
| Total | 50 | 85 | 1.70 | 0.30 | 33 | 1.97 | 0.06 | 17 | 1.18 | 0.81 | ||||
| Site 3 | Surgeon E | > 10 years | 140 | 130 | 50 | 141 | 2.82 | 0.32 | 30 | 2.63 | 0.10 | 20 | 3.10 | 0.65 |
| Surgeon F | < 5 years | 80 | 54 | 28 | 70 | 2.50 | 0.39 | 13 | 2.23 | 0.08 | 15 | 2.73 | 0.67 | |
| Total | 78 | 211 | 2.71 | 0.35 | 43 | 2.51 | 0.09 | 35 | 2.94 | 0.66 | ||||
| Overall | All surgeons | > 10 years | 354 | 309 | 116 | 265 | 2.28 | 0.41 | 64 | 2.28 | 0.08 | 52 | 2.29 | 0.81 |
| All surgeons | < 5 years | 221 | 121 | 51 | 121 | 2.37 | 0.37 | 29 | 2.24 | 0.03 | 22 | 2.55 | 0.82 | |
| Total | 167 | 386 | 2.31 | 0.40 | 93 | 2.27 | 0.06 | 74 | 2.36 | 0.81 | ||||
Total number of confirmed PGs: in-situ PGs identified by surgeons with moderate-to-high confidence + excised PGs validated with histology.
Upon analyzing each surgeon’s performance, it can be seen that although Surgeon B was more confident in parathyroid identification than Surgeon A at Site 1 (Figure 3A), the tissue misclassification rate was higher for Surgeon B (Figure 3B). In contrast, while the tissue misclassification rate for Surgeon D was lower than Surgeon C at Site 2, the former was clearly less confident at identifying parathyroid glands. For Site 3, Surgeon F was less confident in parathyroid identification and also had a higher tissue misclassification rate than Surgeon E. Overall, the junior surgeons were moderately or highly confident only 77.6% of the time on whether a tissue was parathyroid or not, while the senior surgeons were moderately or highly confident 93.7% of the times (p<0.00001). The findings suggest that the junior surgeons were less confident than their senior counterparts at identifying parathyroid glands with visual inspection. Furthermore, the cumulative data from all 6 surgeons indicate that junior surgeons had more than twice the tissue misclassification rate per specimen than the senior surgeons (0.25 vs 0.10, p=0.011).
Figure 3:
(A) Distribution of a surgeon’s confidence level in identifying parathyroid glands across all 3 study sites. t = number of times a potential parathyroid candidate was seen by the surgeon. (B) Distribution of tissue misclassification rates of surgeons across all 3 study sites, when excised specimens were validated with histology. n = number of tissues validated with histology for each surgeon. The overall findings imply that the lesser experienced surgeons are likely to have lower confidence and accuracy in identifying parathyroid glands. Please note that the surgeon’s skill in identifying parathyroid glands was solely based on their surgical experience, as all surgeons were blinded to the output of PTeye® in this study. * - only 2 specimens were validated histologically for Surgeon D.
Accuracy of surgeon opinion versus PTeye® in PG identification
Using histology as the gold standard, Surgeon B (Junior, Site 1), Surgeon C (Senior, Site 2) and Surgeon F (Junior, Site 3) – had a higher tissue misclassification rate per specimen when compared with PTeye® (Table 4). The performance of Surgeon A (Senior, Site 1), Surgeon D (Junior, Site 2) and Surgeon E (Senior, Site 3) denoted lower tissue misclassification rate than PTeye®. However, only 2 specimens were histologically validated for Surgeon D, compared to others who had ≥12 specimens histologically confirmed. The overall data indicate that there was no difference in the tissue misclassification rate of the PTeye® versus that of the senior surgeons (0.13 vs 0.10, p=0.51). In contrast, PTeye® demonstrated a higher performance accuracy than the junior surgeons by yielding a significantly lower tissue misclassification rate per specimen (0.06 vs 0.25, p=0.007).
Table 4:
Distribution of tissue misclassification rate for participant surgeons versus PTeye® for excised specimens confirmed with histology. All participant surgeons visually confirmed PGs solely based on their surgical experience and remained blinded to output from PTeye®. Specimens such as diseased entire thyroid lobe/thyroid gland where the tissue identity was obvious to surgeons were not included for analysis.
| Site | Surgeon | Surgeon experience at time of study | Number of specimens confirmed with histology* | Total misclassification versus histology* | Misclassification per histology* specimen | ||
|---|---|---|---|---|---|---|---|
| Surgeon | PTeye® | Surgeon | PTeye® | ||||
| Site 1 | Surgeon A | > 10 years | 28 | 2 | 5 | 0.07 | 0.18 |
| Surgeon B | < 5 years | 12 | 3 | 2 | 0.25 | 0.17 | |
| Site 2 | Surgeon C | > 10 years | 19 | 3 | 1 | 0.16 | 0.05 |
| Surgeon D | < 5 years | 2 | 0 | 1 | 0.00 | 0.50 | |
| Site 3 | Surgeon E | > 10 years | 55 | 5 | 7 | 0.09 | 0.13 |
| Surgeon F | < 5 years | 38 | 10 | 0 | 0.26 | 0.00 | |
| Overall | All surgeons | > 10 years | 103 | 10 | 13 | 0.10 | 0.13 |
| All surgeons | < 5 years | 52 | 13 | 3 | 0.25** | 0.06** | |
| Total | 155 | 23 | 16 | 0.15 | 0.10 | ||
Histology = frozen section analysis + permanent histology
p-value (Chi-square test) < 0.05.
As seen in Table 5, the results of generalized linear mixed-effect model of with logistic regression indicate that neither the study site nor the experience of surgeons significantly affected the performance accuracy of PTeye® for in-situ tissues (surgeon serving as gold standard) or excised tissues (histology serving as gold standard). When the same model was fitted to assess the performance accuracy of the surgeons themselves, it was found that compared to junior surgeons, senior surgeons (>10-year experience) are three times more likely to accurately identify parathyroid tissues (Odds Ratio: 3.14, 95% Confidence Interval: 1.13–8.70; p=0.03).
Table 5:
Effect of study site and surgeon experience variability on (i) in vivo accuracy of PTeye® with surgeon as the gold standard, (ii) ex vivo accuracy of PTeye® with histology as gold standard and (iii) ex vivo accuracy of surgeons with histology as gold standard. The effects of these study design variables were analyzed using generalized linear mixed-effect model with logistic regression for both in vivo and ex vivo analysis.
| Effect of study design variables on in vivo accuracy of PTeye®. Gold Standard: Surgeon opinion | |||
| Study Design Variable | Odds Ratio | 95% Confidence Interval | p-value |
| Site 2 vs Site 1 | 2.53 | 0.91 – 7.00 | 0.07 |
| Site 3 vs Site 1 | 2.43 | 0.97 – 6.06 | 0.06 |
| Senior vs Junior Surgeons | 1.89 | 0.84 – 4.26 | 0.13 |
| Effect of study design variables on ex vivo accuracy of PTeye®. Gold Standard: Histology | |||
| Study Design Variable | Odds Ratio | 95% Confidence Interval | p-value |
| Site 2 vs Site 1 | 2.37 | 0.44 – 12.71 | 0.31 |
| Site 3 vs Site 1 | 2.35 | 0.76 – 7.28 | 0.14 |
| Senior vs Junior Surgeons | 0.44 | 0.12 – 1.68 | 0.23 |
| Effect of study design variables on ex vivo accuracy of surgeons. Gold Standard: Histology | |||
| Study Design Variable | Odds Ratio | 95% Confidence Interval | p-value |
| Site 2 vs Site 1 | 0.64 | 0.12 – 3.38 | 0.60 |
| Site 3 vs Site 1 | 0.80 | 0.24 – 2.64 | 0.71 |
| Senior vs Junior Surgeons | 3.14 | 1.13 – 8.70 | 0.03** |
p-value < 0.05 is suggestive of that particular variable having a significant effect on the accuracy in tissue classification accuracy.
DISCUSSION
The current study reports the first blinded multi-centric study where NIRAF detection with PTeye® was applied for intraoperative parathyroid identification. Earlier studies that utilized camera-based systems were not blinded (8), as the NIRAF images seen could introduce bias in a surgeon’s own judgement in identifying parathyroid glands. The surgeon-blinded methodology in this study allowed to independently compare a surgeon’s own capability in parathyroid identification versus NIRAF detection, while evaluating if a surgeon’s own accuracy varied significantly with increasing years of practice or patient volume.
Overall, this study indicates that PTeye® has 92.7% accuracy in parathyroid identification, which concurs with earlier studies. (18, 21, 22) The observed differences between the various centers, especially for in-situ accuracy at Site 1, may be due to the variability in parathyroid exposure/dissection and probe pressure between surgeons across the 3 sites that may have led to fluctuating NIRAF levels during in-situ measurements, which is further compounded by the ‘blinded’ nature of this study leading to surgeons not receiving any real-time feedback from the device. In comparison, specificity with PTeye® ranged from 87.2% - 93.9% across the study sites, which is also consistent with prior results. (18, 21, 22) With false positives that could arise from brown fat, thyroid nodules, lymph nodes or fibrofatty tissues, appropriate mitigation strategies should therefore be implemented to counter NIRAF-related false positives and prevent surgeon error. (18)
Recent studies had investigated the impact of NIRAF-based PG detection for improving patient outcomes and enhancing surgeon’s efficiency with mixed results. (10–17) Disparity in the aforementioned findings with NIRAF detection may be due to inconsistencies in the definitive criteria adopted for primary/secondary outcomes in these studies. More importantly, the results in these studies with multiple surgeons have always been reported as a cumulative aggregate of the entire data-set, without stratification based on the site of study or surgeon’s experience. Consequently, there is limited insight on how the rate of parathyroid identification and related outcomes may itself fluctuate with varying experience/technique among multiple surgeons at different surgical centers. It thus becomes crucial to evaluate whether a surgeon’s own capability may or may not be augmented by NIRAF detection.
Since the participating surgeons were intentionally blinded from the results of PTeye®, this study determined the inherent variability across multiple surgeons in accurately identifying parathyroid glands, when relying on their own knowledge/experience. When histology was used as the gold standard, individual surgeon’s performance in PG identification could be objectively compared with that of PTeye®. However, a limitation of this study that must be noted is that histology could serve as validation for only excised specimens in this study. There was no alternative gold standard to validate healthy PGs that are often left in-situ. Of the 255 healthy parathyroids, only 2.4% (n=6) were biopsied during thyroid procedures. Future studies with these technologies could consider performing biopsies of healthy PGs, if feasible, for validation with a larger sample size.
The results of this study seem to suggest that surgeons who have been in practice longer tend to surpass their junior counterparts in their ability to identify parathyroids with higher confidence (93.6% vs 77.6%) and have lower tissue misclassification rate per specimen (0.10 vs 0.25). While all six participant surgeons are high-volume surgeons (with >25 thyroid and/or >15 parathyroid procedures per annum) (18), it must be noted that the senior surgeons accrued greater patient volume per annum than their juniors (Table 3). In terms of performance accuracy, there were no statistical difference observed between senior surgeons versus that of PTeye® (0.10 vs 0.13, p=0.51). In contrast, since junior surgeons are still traversing their learning curve for intraoperative PG identification, their tissue misclassification rate was evidently higher when compared to PTeye® (0.25 vs 0.06, p=0.007). While Surgeon D’s performance accuracy was higher than PTeye®, it needs to be pointed out that only 2 specimens were validated by histology for this junior surgeon, as compared to others who had at least 12 specimen validations. This might therefore not be reflective of Surgeon D’s true performance accuracy, as 10 out of 12 procedures performed by this particular surgeon were thyroid operations, where PGs were left in-situ and had no histological validation.
While this study compared 3 junior surgeons versus 3 senior surgeons, future studies should investigate if these findings/trends are observed in a larger data-set of participant surgeons. Furthermore, it remains to be explored if NIRAF detection devices such as PTeye® can indeed accelerate the learning curve of less-experienced surgeons so that they could become proficient in PG identification more quickly. Based on its system design, PTeye® has the potential to become a teaching tool to educate trainees/residents on how to identify parathyroid glands, in a manner similar to nerve monitoring devices. In addition, these modalities may also aid experienced surgeons in parathyroid identification, especially during cases that require extensive neck dissection or have distorted anatomy from previous surgeries.
While prospective clinical trials with NIRAF-based devices tend to mostly initiate with endocrine surgeons at high-volume centers, it would become imperative to further assess the merits of PTeye® for low-volume surgeons or non-endocrine surgeons who perform neck operations. Thus, it becomes necessary to evaluate the benefits of NIRAF-based parathyroid identification with respect to variability in surgeon volume, specialization and technique. More importantly since this study was limited to just 6 high-volume endocrine surgeons and could not be utilized to evaluate patient outcomes (e.g. postsurgical hypocalcemia) due to its blinded nature, more extensive trials are underway to fully comprehend the benefits associated with NIRAF-based parathyroid identification. (23, 24)
CONCLUSION
PTeye®, with an overall 92.7% accuracy, can be a valuable intraoperative adjunct tool that could aid surgeons for improving confidence in parathyroid identification and potentially reducing misidentification of parathyroid glands, to minimize postoperative morbidity associated with neck endocrine surgeries.
Supplementary Material
Video 1: The video demonstrates on how the surgeon first obtains baseline near infrared autofluorescence measurements on five spots of the thyroid gland. Subsequent tissue NIRAF measurements are normalized to the baseline NIRAF to generate Detection Ratio(s). As seen in the video, the surgeon identifies a potential parathyroid tissue based on surgical experience. Subsequently, the ‘blinded’ surgeon places the fiber probe on the suspect tissue. When the pedal is pressed, tissue NIRAF measurements from the suspect tissue is displayed as Detection Level and Detection Ratios in real-time on the device console. PTeye® is designed such that the device classifies/identifies the tissue as parathyroid if the Detection Ratio exceeds 1.2.
ACKNOWLEDGEMENTS
We thank the OR staff for assisting in data collection. We are also grateful to Dr. John (Jay) Wellons and Dr. Colleen M. Kiernan for their help with the manuscript. E. A. Mannoh, G. Thomas, C. Solόrzano and A. Mahadevan-Jansen were supported by Grant No. R01CA212147 (National Institute of Health).
Footnotes
DISCLOSURES
A. Mahadevan-Jansen, J. E. Phay and Vanderbilt University have a licensing agreement with AiBiomed (Santa Barbara, CA) for PTeye®. A. Mahadevan-Jansen is also an equity holder at AiBiomed. Other authors have no conflicts of interest to declare.
REFERENCES
- 1.Antakia R, Edafe O, Uttley L, Balasubramanian SP. Effectiveness of Preventative and Other Surgical Measures on Hypocalcemia Following Bilateral Thyroid Surgery: A Systematic Review and Meta-Analysis. Thyroid. 2014;25:95–106. [DOI] [PubMed] [Google Scholar]
- 2.Bergenfelz A, Nordenström E, Almquist M. Morbidity in patients with permanent hypoparathyroidism after total thyroidectomy. Surgery. 2020;167(1):124–8. [DOI] [PubMed] [Google Scholar]
- 3.Simental A, Ferris RL. Reoperative Parathyroidectomy. Otolaryngologic Clinics of North America. 2008. 2008/12/01/;41(6):1269–74. [DOI] [PubMed] [Google Scholar]
- 4.Cron DC, Kapeles SR, Andraska EA, et al. Predictors of operative failure in parathyroidectomy for primary hyperparathyroidism. The American Journal of Surgery. 2017;214:509–14. [DOI] [PubMed] [Google Scholar]
- 5.Novis DA, Zarbo RJ. Interinstitutional comparison of frozen section turnaround time. A College of American Pathologists Q-Probes study of 32868 frozen sections in 700 hospitals. Arch Pathol Lab Med. 1997. June;121(6):559–67. Epub 1997/06/01. eng. [PubMed] [Google Scholar]
- 6.Paras C, Keller M, Mahadevan-Jansen A, et al. Near-infrared autofluorescence for the detection of parathyroid glands. Journal of Biomedical Optics. 2011;16(6):067012. [DOI] [PubMed] [Google Scholar]
- 7.McWade MA, Sanders ME, Broome JT, et al. Establishing the clinical utility of autofluorescence spectroscopy for parathyroid detection. Surgery. 2016. 1//;159(1):193–203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Solórzano CC, Thomas G, Baregamian N, Mahadevan-Jansen A. Detecting the Near Infrared Autofluorescence of the Human Parathyroid: Hype or Opportunity? Annals of Surgery. 2020;272(6):973–85. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.The United States Food and Drug Administration. FDA permits marketing of two devices that detect parathyroid tissue in real-time during surgery. 2018. [November 3, 2018]; Available from: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm624982.htm.
- 10.Benmiloud F, Godiris-Petit G, Gras R, et al. Association of Autofluorescence-Based Detection of the Parathyroid Glands During Total Thyroidectomy With Postoperative Hypocalcemia Risk: Results of the PARAFLUO Multicenter Randomized Clinical Trial. JAMA Surgery. 2020;155(2):106–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Squires MH, Jarvis R, Shirley LA, Phay JE. Intraoperative Parathyroid Autofluorescence Detection in Patients with Primary Hyperparathyroidism. Annals of Surgical Oncology. 2019;26:1142–8. [DOI] [PubMed] [Google Scholar]
- 12.Dip F, Falco J, Verna S, et al. Randomized Controlled Trial Comparing White Light with Near-Infrared Autofluorescence for Parathyroid Gland Identification During Total Thyroidectomy. Journal of the American College of Surgeons. 2019;228:744–51. [DOI] [PubMed] [Google Scholar]
- 13.DiMarco A, Chotalia R, Bloxham R, et al. Autofluorescence in Parathyroidectomy: Signal Intensity Correlates with Serum Calcium and Parathyroid Hormone but Routine Clinical Use is Not Justified. World Journal of Surgery. 2019;43:1532–7. [DOI] [PubMed] [Google Scholar]
- 14.Kahramangil B, Dip F, Benmiloud F, et al. Detection of Parathyroid Autofluorescence Using Near-Infrared Imaging: A Multicenter Analysis of Concordance Between Different Surgeons. Annals of Surgical Oncology. 2018;25:957–62. [DOI] [PubMed] [Google Scholar]
- 15.Benmiloud F, Rebaudet S, Varoquaux A, et al. Impact of autofluorescence-based identification of parathyroids during total thyroidectomy on postoperative hypocalcemia: a before and after controlled study. Surgery. 2018;163(1):23–30. [DOI] [PubMed] [Google Scholar]
- 16.DiMarco A, Chotalia R, Bloxham R, et al. Does fluoroscopy prevent inadvertent parathyroidectomy in thyroid surgery? Ann R Coll Surg Engl. 2019. September;101(7):508–13. Epub 2019/07/16. eng. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kim YS, Erten O, Kahramangil B, et al. The impact of near infrared fluorescence imaging on parathyroid function after total thyroidectomy. Journal of Surgical Oncology. 2020. [DOI] [PubMed] [Google Scholar]
- 18.Thomas G, McWade MA, Paras C, et al. Developing a clinical prototype to guide surgeons for intraoperative label-free identification of parathyroid glands in real time. Thyroid. 2018;28(11):1517–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kose E, Kahramangil B, Aydin H, et al. Heterogeneous and low-intensity parathyroid autofluorescence: Patterns suggesting hyperfunction at parathyroid exploration. Surgery. 2019;165:431–7. [DOI] [PubMed] [Google Scholar]
- 20.Demarchi MS, Karenovics W, Bédat B, Triponez F. Intraoperative autofluorescence and indocyanine green angiography for the detection and preservation of parathyroid glands. Journal of Clinical Medicine. 2020;9(3):830. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Thomas G, Squires MH, Metcalf T, et al. Imaging or Fiber Probe-Based Approach? Assessing Different Methods to Detect Near Infrared Autofluorescence for Intraoperative Parathyroid Identification. Journal of the American College of Surgeons. 2019. 2019/12/01/;229(6):596–608.e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Thomas G, McWade MA, Nguyen JQ, et al. Innovative surgical guidance for label-free real-time parathyroid identification. Surgery. 2019;165(1):114–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.National Library of Medicine (clinicaltrials.gov). Evaluating Impact of NIRAF Detection for Identifying Parathyroid Glands During Parathyroidectomy. National Library of Medicine; [cited 2020 June 2 2020]; Available from: https://clinicaltrials.gov/ct2/show/NCT04299425. [Google Scholar]
- 24.National Library of Medicine (clinicaltrials.gov). Assessing Benefits of NIRAF Detection for Identifying Parathyroid Glands During Total Thyroidectomy. National Library of Medicine; [cited 2020 June 2 2020]; Available from: https://clinicaltrials.gov/ct2/show/NCT04281875. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Video 1: The video demonstrates on how the surgeon first obtains baseline near infrared autofluorescence measurements on five spots of the thyroid gland. Subsequent tissue NIRAF measurements are normalized to the baseline NIRAF to generate Detection Ratio(s). As seen in the video, the surgeon identifies a potential parathyroid tissue based on surgical experience. Subsequently, the ‘blinded’ surgeon places the fiber probe on the suspect tissue. When the pedal is pressed, tissue NIRAF measurements from the suspect tissue is displayed as Detection Level and Detection Ratios in real-time on the device console. PTeye® is designed such that the device classifies/identifies the tissue as parathyroid if the Detection Ratio exceeds 1.2.