Diagnostic accuracy of optical coherence tomography for assessing surgical margins of canine soft tissue sarcomas in observers of different specialties

Abstract

Objective:

To determine the diagnostic accuracy of optical coherence tomography (OCT) to assess surgical margins of canine soft tissue sarcoma (STS) and determine the influence of observer specialty and training.

Study design:

Blinded clinical prospective study.

Animals:

Twenty-five dogs undergoing surgical excision of STS.

Methods:

In vivo and ex vivo surgical margins were imaged with OCT after tumor resection. Representative images and videos were used to generate a training presentation and data sets. These were completed by 16 observers of four specialties (surgery, radiology, pathology, and OCT researchers). Images and videos from data sets were classified as cancerous or noncancerous.

Results:

The overall sensitivity and specificity were 88.2% and 92.8%, respectively, for in vivo tissues and 82.5% and 93.3%, respectively, for ex vivo specimens. The overall accurate classification for all specimens was 91.4% in vivo and 89.5% ex vivo. There was no difference in accuracy of interpretation of OCT imaging by observers of different specialties or experience levels.

Conclusion:

Use of OCT to accurately assess surgical margins after STS excision was associated with a high sensitivity and specificity among various specialties. Personnel of all specialties and experience levels could effectively be trained to interpret OCT imaging.

Clinical significance:

Optical coherence tomography can be used by personnel of different specialty experience levels and from various specialties to accurately identify canine STS in vivo and ex vivo after a short training session. These encouraging results provide evidence to justify further research to assess the ability of OCT to provide real-time assessments of surgical margins and its applicability to other neoplasms.

1 ∣. INTRODUCTION

Complete surgical resection is the mainstay of treatment for local control of canine (soft tissue sarcoma) STS.^1-3 Surgical margins are routinely assessed with histopathological methods after surgical resection, with results often delayed up to several days after surgery. These methods rely on trimming the tissues to assess a limited number of tissue sections microscopically. Radial trimming methodology is commonly used for small to medium-sized tumors, which involves tumor diagnosis and surgical margin assessment from as few as four sections representing the short axis and long axis through the tumor specimen.⁴ These sections represent assessment of <1% of the entire surgical margin.^5,6 This limited sampling, combined with asymmetric growth of the tumor, can result in overlooking areas in which cancer cells extend to the surgical margins, resulting in a misdiagnosis of a complete resection and a subsequent missed opportunity for adjuvant treatment to help with local tumor control.^4,5

Very few reports have focused on intraoperative real-time techniques for assessment of surgical margins in dogs. Such techniques include intraoperative cytology, shave margins, near-infrared fluorescent imaging, and optical coherence tomography (OCT).^7-13 These investigations have largely involved feasibility and have been pilot studies in nature because no large-scale clinical trials have been performed. An imaging modality that can be used to evaluate intraoperative surgical margin status is critical to improving the accuracy and efficacy of cancer surgery in companion animal species.

Optical coherence tomography is a high-resolution, depth-resolved, cross-sectional, microscopic imaging technique that uses light waves and scattering interactions with tissue to generate images at micron-scale resolution.^14-17 These images are comparable to a low-magnification histology image but with up to 2 mm of tissue imaging penetration. The use of OCT in vivo was first reported in 1993 by Swanson et al¹⁸ (followed by Fujimoto and Swanson¹⁹) to describe the microstructural features of the human retina. While it was originally used in ophthalmology, OCT has been proven efficacious in identifying residual tumor from healthy tissues in humans with breast cancer, and the appearance of tissues at the surgical margins have been described in dogs with STS.^9,10,14-17 Optical coherence tomography has the ability to help identify residual tumor cells (human breast cancer and canine STS) intraoperatively, providing real-time feedback to surgeons.^9,10,14-17 Improving accuracy at obtaining adequate surgical margins in dogs with STS may decrease the incidence of local tumor recurrence, decrease the frequency of additional surgeries and adjuvant therapy, decrease associated morbidity and mortality for dogs, and reduce costs for pet owners.^2,9,10,20

The objectives of this study were to evaluate the diagnostic accuracy of OCT image interpretation for surgical margin assessment of canine STS by various personnel after a short training session and to determine whether there was any effect of specialty or specialty experience level on accuracy through the evaluation of images by individuals of four different specialties. The authors hypothesized that accurate differentiation of canine STS from normal tissues with OCT imaging is a readily learnable skill in a wide range of personnel.

2 ∣. MATERIALS AND METHODS

Informed written owner consent was obtained prior to enrollment of dogs and acquisition of OCT images. This part of the study was performed at the University of Illinois Veterinary Teaching Hospital between January 1, 2016, and August 31, 2017. The study protocol was approved by the University of Illinois at Urbana-Champaign Institutional Animal Care and Use Committee. Observer evaluation of OCT images was performed at the Ohio State University Veterinary Medical Center.

2.1 ∣. Study population

Dogs were eligible for study enrollment if they had been diagnosed with a cutaneous or subcutaneous STS via fine-needle aspirate cytology or biopsy and if the dogs were undergoing surgical excision. Tumors could have been excised for the first time or could have been recurrent. Information collected at presentation included dog age, breed, sex, and neuter status. Surgical excision of the STS was performed by either an American College of Veterinary Surgeons (ACVS) board-certified surgeon or a trainee under the direct supervision of an ACVS board-certified surgeon. The tumors were resected with planned surgical margins determined by the primary surgeon. The planned surgical margins and the procedure performed were recorded.

2.2 ∣. Optical coherence tomography imaging

Surgical wound beds were imaged with OCT intraoperatively with a laboratory-built spectral-domain OCT system (University of Illinois Urbana-Champaign) and a custom-built handheld OCT probe (Diagnostic Photonics, Chicago, Illinois). The OCT system acquired images at a central wavelength of 1310 nm and an incident illumination power of approximately 5 mW. Image resolution was approximately 8 μm axially and 10 μm laterally with an imaging depth up to 1 to 2 mm, depending on the optical properties of the tissue. Optical coherence tomography images were acquired in proprietary software (InVivoVue 1.7; Bioptigen, Durham, North Carolina). After tumor excision, the tissue specimen was wrapped in a sterile saline-soaked gauze and set aside. Sterile ultrasound transmission gel (Aquasonic; Parker Laboratories, Fairfield, New Jersey) was applied to the OCT handheld probe. A sterile transparent probe cover (ultrasound probe drape; Spectrum Labs, Irving, Texas) was placed over the OCT system probe for in vivo intraoperative imaging. The entire surgical wound bed (in vivo) was methodically scanned in B-mode (starting circumferentially at the lateral margins and ending with the deep margins) by one investigator (L.E.S.). Two 1-cm diameter areas were identified by the surgeon as suspicious for STS or normal at the closest surgical margin (in vivo). Static OCT images were acquired from each of these areas. These tissue areas were excised to a depth of 2 to 3 mm by the surgeon. After in vivo imaging, the handheld probe was used to scan the excised tissue (ex vivo) by a trained investigator (L.E.S.) in a similar methodical manner to the in vivo tissues. Optical coherence tomography imaging was evaluated in real-time by a single investigator (L.E.S.) to ensure that the images were of diagnostic quality. The surgical specimen and two excised in vivo areas were all placed in 10% neutral buffered formalin for fixation. An American College of Veterinary Pathology board-certified pathologist (J.S.) examined the excised STS and two in vivo areas. The pathologist trimmed standard radial sections and performed tangential trimming of the surgical margins of the entire excised tissue specimen. Sections were taken through the two in vivo area samples. All sections were evaluated by a pathologist blinded to the OCT imaging results for determination of tumor grade and surgical margin status. Optical coherence tomography images of the areas of interest were compared with the corresponding histopathology results.

Optical coherence tomography images were processed by exporting the raw data to MATLAB (MathWorks, Natick, Massachusetts) to generate 600 sequential frames representing the OCT images of each 1 × 1-cm area. Videos were created in ImageJ (National Institutes of Health Bethesda, Maryland).

2.3 ∣. Optical coherence tomography-image analysis

A training presentation was created with reference OCT images that were directly compared to histopathology results and obtained from a preliminary study in surgical margin imaging of dogs after STS excision.⁹ This presentation contained an explanation of OCT technology, examples of common artifacts, and OCT images of tissues arising at surgical margins including STS, fat, muscle, and fascia (Figure 1). The training presentation consisted of 24 static images and four videos in PowerPoint (Office 365, Microsoft, Redmond, Washington). Ten static images and two videos were selected from each of the in vivo and ex vivo tissue image sets to create two practice tests (one ex vivo and one in vivo) for observers to complete independently after the training to receive feedback on their answers. No images or videos that were shown in the training or practice tests were used in the data set. Two data sets (ex vivo and in vivo) were created, and each data set included a total of 25 static images and 25 mirror image copies of these static images. Mirrored and repeated images were included to allow for assessment of intraobserver variability. Images were then placed in random order with a random number sequence generated from an open source website (https://www.random.org; Randomness and Integrity Services, Dublin, Ireland). Among the 50 images provided in each slideshow, 34 did not exhibit evidence cancer (NC) and 16 did exhibit evidence of cancer (C). In addition, eight videos were available to review for ex vivo tissues and five videos were available to review for in vivo tissues, with four NC and C each for the ex vivo videos and one C in the in vivo videos. Approximately 33% proportion of C to NC images was chosen to reflect the proportion of cases in which incomplete excision has been reported in previous canine studies.^21,22 In both the test set and the data set, observers were asked to indicate which imaging grade best fit their confidence that cancer was present or not present in the OCT surgical margin image or video (grade 1 = cancer was not present, grade 2 = cancer was most likely not present, grade 3 = cancer was most likely present, and grade 4 = cancer was present).

FIGURE 1.

Representative optical coherence tomography images of noncancerous (A-C) and cancerous (D-F) tissues. A, Adipose tissue is characterized by a low-scattering image with a honeycomb density. B, Arrow indicates where adipose tissue (left) abuts muscle (right). Notice how the muscle generates a higher scattering image with clear linear striations consistent with muscle fibers. C, Another example of normal muscle tissue with less obvious but still visible linear texture. D-F, Representations of various images obtained from STS are illustrated. Note the highly scattering images with no clear organization or texture. E, Note the low-scatter (black areas) with white outlines. These represent small irregular blood vessels (neovascularization) at the periphery of the STS. STS, soft tissue sarcoma. Scale bars = 1 mm

Sixteen observers were chosen to participate in the study. Participants were from one of four specialties including veterinary surgery, veterinary radiology, veterinary pathology, and OCT researchers. Within each specialty, two observers were experienced in their specialty (having completed specialty residency training or a graduate degree), and two observers were trainees within their specialty (residents, interns, or junior graduate students).

The training presentation was delivered in person or by video conference to groups of four observers of the same specialty by a single investigator (L.E.S.) with a standardized script during a 1-hour period. After training, observers completed the two practice tests and received direct standardized feedback regarding the responses. Observers were blinded to the dogs' identities, histopathology results, and the number of C and NC images in the data set. Observers had up to 14 days to complete the practice test and then the data set, and were permitted to refer to the training presentation or practice test results when completing the data set.

2.4 ∣. Statistical analysis

Sample size was calculated with a method described by Flahault et al²³ and Zhou et al²⁴ to estimate sample size required for description of the sensitivity and specificity of OCT imaging for STS. The confidence interval half-width (L) of 0.3 was chosen, with a power of 80% and α of .05. The number of cases of incomplete margins (n_ic) was assessed as

$$n_{ic}=\frac{{\left[\left(z_{\propto ∕2}+z_{\beta }\right)\sqrt{V\left(\widehat{\theta }\right)}\right]}^2}{(L{)}^2}$$ (1)

In this equation, Z_α/2 refers to the upper α/2 percentile of a standard normal distribution where 1-β is teh desired power. V(θ) refers to the variance function, which for sensitivity is V(θ)= Se x (1-Se), where Se is the predicted sensivitiy of the test. An estimated sensitivity and specificity of 90% was used because of the high sensitivity and specificity of incomplete margin assessment derived in a human breast cancer clinical trial.¹⁵ The estimated number of dogs required with incomplete margins was eight for each tumor type. Because of the prospective nature of the proposed clinical trial, the number of dogs with complete margins (n_c) required was calculated, with an estimate of the prevalence (Prev) of incomplete margins:

$$n_c=n_{ic}\left[\frac{1-Prev}{Prev}\right].$$ (2)

An estimated prevalence of incomplete margins of 0.33 was used because this was the reported prevalence of incomplete margins after surgical resection of malignant cutaneous tumors (including STSs and carcinomas) in a recent study.²⁵ With eight cases of dogs with incomplete margins and an estimated prevalence of 0.33 (proportion of dogs with incomplete margins of 33%), 17 dogs with complete margins would be required, so a total of 25 dogs with STS would be required.

Diagnostic accuracy statistics were calculated to assess performance in scoring images correctly as cancerous or noncancerous. Statistics included the sensitivity (true positive rate), specificity (true negative rate), and the overall correct classification rate (CCR). These statistics and the corresponding confidence intervals were calculated from marginal logistic regression models accounting for within-subject effects of each scorer. Models included no covariates for the overall statistics and either experience level or department for the group comparisons. Statistics were calculated separately for ex vivo and in vivo images. All statistical analyses were completed in SAS version 9.4 (SAS Institute, Cary, North Carolina). P < .05 was considered statistically significant.

3 ∣. RESULTS

Twenty-six dogs were initially included; however, two dogs were excluded after postoperative histopathology results revealed a lipoma and granulomatous panniculitis rather than STS, so 24 dogs remained in the final study population that had 25 STS excised. One dog was enrolled twice after developing a second STS at a separate site 7 months after the first STS was excised (Table 1). The median age of the dogs was 9.7 years (range, 5-13.7), and dogs of the after breeds were represented: mixed breed dog (9), golden retriever (3), Labrador retriever (3), boxer (2), and one each of vizsla, Scottish terrier, borzoi, border collie, Shetland sheepdog, German shepherd dog, and beagle.

TABLE 1.

Clinical data collected on dogs undergoing OCT imaging after STS excision

Dog, n	Breed	Age, y	Tumor location	Surgical dose (procedure)	Excision status	Grade
1	Mixed breed dog	7.7	Left antebrachium	Marginal resection (mass removal)	Incomplete	1
2	Golden retriever	10	Right stifle	Wide margins (mass removal)	Complete	1
3	Scottish terrier	8.5	Manubrium	Wide margins (mass removal)	Complete	3
4	Mixed breed dog	9.7	Left elbow	Marginal resection (mass removal)	Incomplete	2
5	Mixed breed dog	10.3	Left manus	Wide margins (partial limb amputation)	Complete	2
6	Golden retriever	9.1	Left elbow	Wide margins (limb amputation)	Complete	3
7	Mixed breed dog	12.4	Left fore 3rd digit	Marginal resection (digit amputation)	Incomplete	1
8	Golden retriever	12.1	Left antebrachium	Marginal resection	Incomplete	1
9	Labrador retriever	12.3	Right hip	Marginal resection	Incomplete	3
10	*Vizsla	9.6	Right crus	Marginal resection	Incomplete	1
11	Mixed breed dog	5	Left perianal region	Marginal resection	Incomplete	3
12	Mixed breed dog	11	Left hock	Marginal resection	Incomplete	1
13	Borzoi	9.7	Left tail base	Marginal resection	Incomplete	3
14	Border collie	10.6	Right periorbit	Wide margins	Incomplete	3
15	Shetland sheepdog	9.1	Right axilla	Wide margins (limb amputation)	Complete	3
16	Mixed breed dog	8.9	Right ventral thorax	Wide margins	Complete	1
17	Labrador retriever	11.2	Right body wall	Wide margins	Complete	1
18	Mixed breed dog	11.6	Right forelimb	Marginal resection	Incomplete	2
19	Boxer	8.1	Right medial hindlimb	Marginal resection	Incomplete	1
20	Vizslaa	10.2	Left shoulder	Wide margins	Incomplete	1
21	Mixed breed dog	13.7	Right hock	Marginal resection	Incomplete	2
22	Labrador retriever	7.8	Right elbow	Marginal resection (scar revision & recurrence)	Incomplete	1
23	German shepherd	9.1	Right carpus	Marginal resection	Incomplete	1
24	Boxer	9.5	Left antebrachium	Marginal resection	Incomplete	1
25	Beagle	8.1	Right antebrachium	Marginal resection	Incomplete	1

Note: The breed, age, tumor location, surgical dose, histopathologic assessment of tumor excision (complete or incomplete), and histopathological grade.^26,27 An incomplete excision was defined as tumor cells being identified <2 mm of a surgical margin (since this is what can be visualized with the OCT device) unless fascia was present between the surgical margin edge and tumor cells. Tumor grade (range 1-3) was determined from histopathology of the excised tumor.²⁷

Abbreviations: OCT, optical coherence tomography; STS, soft tissue sarcoma.

^aDog that was enrolled twice.

Sixteen observers were enrolled to participate in this study. Two observers were excluded from this study after both did not complete the data testing set within 14 days of the training (one surgery trainee, and one experienced researcher in OCT). Two additional observers were enrolled to replace those lost, and were trained identically to the others but individually.

The 16 observers completed the practice test in a median of 8 days (range, 1-13) after the training and completed the data set in a median of 13.5 days (range, 9-14) days after the training.

3.1 ∣. Diagnostic accuracy—in vivo

For the in vivo static images, 90% (360/400) of the time the observers gave the original and mirrored images the same classification and correctly graded them as cancer or no cancer, 5% (21/400) of the time the observers gave both the same classification but were incorrect, and 5% (19/400) of the time the observers graded one correctly and one incorrectly. The overall sensitivity of in vivo OCT imaging was 88.2%, and the specificity was 92.8% (Table 2). There was no difference in sensitivity or specificity between experience levels of observers (sensitivity, P > .99; specificity, P = .40; CCR, P = .52). No differences were identified across specialties (sensitivity, P = .47; specificity, P = .64; CCR, P = .21).

TABLE 2.

Diagnostic accuracy of in vivo OCT imaging of tissue at surgical margins of canine STS resections

Expertise	Observers, n	Sensitivity (95% CI)	Specificity (95% CI)	Correct classification rate (95% CI)
All	16	88.2 (82.3-92.4)	92.8 (90.1-94.8)	91.4 (89.1-93.2)
Experience level
Specialist	8	88.2 (77.8-94.1)	91.8 (87.2-94.8)	90.7 (87.1-93.4)
Trainee	8	88.2 (80.6-93.1)	93.8 (90.6-95.9)	92.0 (89.0-94.3)
Specialty
OCT researcher	4	86.8 (79.1-91.9)	92.1 (88.3-94.7)	90.4 (86.4-93.4)
Pathology	4	89.7 (77.6-95.6)	94.1 (91.5-95.9)	92.7 (88.6-95.4)
Radiology	4	94.1 (81.7-98.3)	94.1 (84.7-97.9)	94.1 (87.4-97.3)
Surgery	4	82.4 (65.8-91.9)	90.8 (83.9-94.9)	88.2 (86.0-90.0)

Note: The sensitivity, specificity, and correct classification rate results across the various comparison groups for the in vivo OCT images.

Abbreviations: OCT, optical coherence tomography; STS, soft tissue sarcoma.

3.2 ∣. Diagnostic accuracy—ex vivo

For the ex vivo static images, 87% (348/400) of the time the observers gave the original and mirrored images the same classification and correctly graded them as cancer or no cancer, 8% (33/400) of the time the observers gave both the same classification but were incorrect, and 5% (19/400) of the time the observers graded one correctly and one incorrectly. The overall sensitivity of ex vivo OCT imaging was 82.5% and the specificity was 93.3% (Table 3). No statistical differences were identified between specialty experience level (sensitivity, P = .066; specificity, P = .94; CCR P = .15) or specialty (sensitivity, P = .57; specificity, P = .98; CCR, P = .95) of observers.

TABLE 3.

Diagnostic accuracy of ex vivo OCT imaging of tissue at surgical margins of canine STS resections

Expertise	Observers, n	Sensitivity (95% CI)	Specificity (95% CI)	Correct classification rate (95% CI)
All	16	82.5 (77.3-86.7)	93.3 (87.8-96.4)	89.5 (87.1-91.6)
Experience level
Specialist	8	78.1 (71.6-83.5)	93.1 (86.0-96.7)	87.9 (84.8-90.5)
Trainee	8	86.9 (80.0-91.6)	93.4 (83.1-97.6)	91.2 (87.7-93.7)
Specialty
OCT researcher	4	86.3 (76.1-92.5)	92.1 (69.3-98.4)	90.1 (83.5-94.2)
Pathology	4	85.0 (81.2-88.1)	92.8 (78.5-97.8)	90.1 (82.2-94.7)
Radiology	4	78.8 (68.7-86.2)	94.1 (84.7-97.9)	88.8 (86.8-90.5)
Surgery	4	80.0 (64.7-89.7)	94.1 (87.4-97.3)	89.2 (85.5-92.1)

Note: The sensitivity, specificity, and correct classification rate results across the various comparison groups for the ex vivo OCT images.

Abbreviations: OCT, optical coherence tomography; STS, soft tissue sarcoma.

4 ∣. DISCUSSION

The main finding of our study is that individuals of various specialty training and expertise accurately distinguished canine STS from normal tissues in vivo and ex vivo after completion of a short, standardized training module. This diagnostic accuracy was not influenced by the individual specialty or training level of the 16 participants in this study, prompting us to propose that accurate differentiation between canine STS and surrounding tissues with OCT imaging is a skill that can be acquired with a short training period. The apparent ease associated with learning the technique justifies further evaluation of the clinical applicability of this imaging technology.

Optical coherence tomography has been described previously in the human surgical oncology field for breast cancer patients.^14-17,29 A sensitivity ranging from 80% to 100% (specificity 56%–92.1%) for accurate identification of residual disease within the surgical margins of in vivo or ex vivo specimens in humans with breast cancer is reported in previous OCT studies.^14-17,29 A similar sensitivity was identified in this study as well as a comparable to improved specificity for accurate identification of cancerous from noncancerous tissues in dogs with STS. These results are especially encouraging because of the inherent challenges involved in differentiating canine STS from surrounding muscle vs discerning human breast cancer from surrounding normal parenchyma, which is predominantly composed of adipose tissue.¹⁵ In a previous study by Selmic et al,⁹ OCT imaging of canine STS and muscle tissues appeared as dense, highly scattering images that were discernable from each other on OCT imaging only because of inherent architectural differences, whereas adipose tissues generated low-scattering images with a clearly discernable honeycomb architecture that are easily distinguishable from muscle and STS.

Ha et al¹⁷ evaluated the diagnostic accuracy of different specialists at discerning human breast cancer from surrounding parenchyma and identified a relatively high degree of accuracy among all personnel (radiology, surgery, pathology, and a nonclinical researcher), with the radiologists having the highest overall accuracy. Our findings are in agreement with those reported by Ha et al,¹⁷ who documented that personnel of various specialties can achieve a high degree of accuracy at identifying neoplastic and nonneoplastic tissues on OCT imaging with 3 hours of training or less. From the results of these studies, we conclude that this imaging modality is not constrained solely to experts in the technique and that a diverse spectrum of people can interpret OCT to aid in image acquisition and interpretation for dogs in the clinical setting, such as real-time surgical margin assessment intraoperatively by the surgeon or any available trained personnel.

A potential limitation of this study is that participants completed a training module that lasted less than 1 hour. It is possible that, with additional training, accurate interpretation of OCT imaging may be improved. In addition, similarly to ultrasonography, interpreters may have had a greater challenge in accurately evaluating still images compared to moving real-time imaging. Most of the training module, practice test, and data set used in this study comprised static images, which may have altered the results in this report. Two investigators performed the OCT imaging for all dogs. While using the imaging device was straightforward and comparable to ultrasound, there was a small learning curve associated with accomplishing ideal tissue contact to acquire high quality images. Image interpretation may vary on the basis of image quality. Another potential limitation to the results of this study was the blinding of observers. In the clinical setting, viewers have additional knowledge regarding the dog's tumor location, size, and attempted surgical margins and knowledge of the gross appearance of tissues being imaged, all of which could increase sensitivity of assessment. More research is required on this imaging modality to fully understand its potential clinical applications.

In conclusion, observers with no previous experience successfully learned to differentiate canine STS from normal tissues on OCT images with a high degree of accuracy after a short training module. Personnel at different levels of specialty experience within various specialties can be taught to accurately interpret OCT imaging, providing a wider population of personnel who can be trained to use this imaging modality in the clinical setting. On the basis of these promising results, further research should focus on assessing this technology with other tumor types to understand the potential utility for real-time surgical margin assessment in dogs.