Doctor reported outcomes: Real-world data from a tertiary eye cancer center

Abstract

Purpose:

To provide real-world data on the world-wide-web for patient and doctor awareness.

Methods:

From December 2017 to January 2020, consecutive patients with choroidal melanoma (CM), iris ciliary body melanoma (ICM), and ocular surface squamous carcinoma (OSSC) had specific outcomes recorded at each return visit. Each result was anonymized, entered in an online portal, and sent to a unique software program where it was used to create real-world data of number of patients, mean vision, local tumor control, eye salvage, systemic metastases, and length of follow-up for our eye cancer center.

Results:

A HIPAA compliant, internet-based software program was developed and linked to public access web page to collect and analyze near-real-time data pertaining to the treatment, vision, life, and follow-up time of patients. During this period, CM radiation plaque tumor control was 99.7%, median vision 20/25 (mean 20/50) and eye salvage 95.8%. ICM tumor control was 99.1% and the median vision 20/20 (mean 20/20). OSSC tumor control was 100% and the most common vision was 20/20 (mean 20/25). Rates of primary enucleation as treatment were 4.2% for CM, 2.8% for ICM, and 0% for OSSC. All patient results were updated by the ophthalmic oncology fellow at each patient visit as to reflect near-real-time outcomes at our center.

Conclusion:

Prospective data collection of returning patients was found to be a simple method to reflect patient care outcomes. This method of reporting doctor outcomes offers a measure of transparency for patients and an opportunity to compare results with other clinical practices.

Patients should have access to their doctor's results prior to and after being treated. Real-world results are outcomes that reflect how pursuing treatment with a physician will affect their sight and lives.[1,2,3,4,5] In response to this need for real-world results, we developed a doctor reported outcome (DRO) software to collect and present near-real-time public access results on our website.

Consider that without access to real-world data, patients must search on the internet, read highly complex published literature, or depend on word-of-mouth testimonials. Searching the internet relies on patients entering “key words” which generate pages of results. In turn, each result was ranked according to search engine optimization algorithms, economic influences (e.g., paid advertisement), and the sheer bulk of raw content on each resultant web site URL. Clearly, website rankings do not accurately reflect the accuracy or authoritativeness of medical information.

Furthermore, medical information on the internet is often complex and thus requires a high level of reading comprehension and familiarity with medical vocabulary (e.g., https://eyewiki.aao.org/Uveal_Melanoma). Physicians must consider their patients online health information-seeking behavior and facilitate their search for high-quality, accurate, and easy to understand real-world results information. There exists a need for transparency about potential treatments, their side-effects, and outcomes (particularly in the field of ophthalmic oncology). Certainly, making such information available offers a method to improve informed consent.

Attempts at doctor reported results include the Physician Quality Reporting System (PQRS); the doctor outcome reporting system initiated by Centers for Medicare and Medicaid Services (CMS) in the United States in 2006. PQRS gives participating physicians the opportunity to assess the quality of care they are providing to their patients and quantify how frequently they are meeting quality metrics. However, PQRS information was not readily available to patients, has been limited in scope and primarily used to affect payments to providers. It does not collect subspecialty outcomes.

In 2016, American Academy of Ophthalmology developed the IRIS registry. IRIS enables physicians to monitor and receive a constant feedback about their performance on a variety of measures quarterly.[6] Under the PQRS, the IRIS dashboard was the way for practices to get feedback on their performance to compare with a standard performance, public-access CMS-based data of other physicians. However, the Academy does not publish or make available to the public any performance data for individual physicians or practices.

Herein, we present our method of collecting and reporting real-world data to provide information for patients with iris, ciliary body, and CM as well as ocular surface squamous carcinoma (OSSC). These data have been continuously posted, open access on the internet in the form of easily understandable language that patients could readily access and understand.

Methods

This study adhered to the tenets of the Declaration of Helsinki and the Health Insurance Portability and Accountability Act (HIPAA) of 1996. We obtained approval from respective Institutional Review Board to perform a prospective chart review of patients with iris, ciliary body, and CM as well as those diagnosed with OSSC between December 2017 and January 2020. Each participant was informed that each patient's outcome results were anonymously averaged in with the others who returned for follow-up to be reported on the world-wide-web at our website.

Specific inclusion criteria

Patients with at least 1-year follow-up from initial treatment. In addition, if a patient was lost to follow-up for more than 12 months, their data were no longer included. Patients who had to have their eye removed as primary treatment were not included in our post-plaque radiation therapy tumor control and visual acuity outcomes. All clinical data were collected, entered, and thus posted online as near-real-time by the ophthalmic oncology research fellows at our center. Like the patients, the primary ophthalmic oncologist (PTF) was only aware of the current statistics when looking at the web page.

Treatment overview

Treatment for uveal melanoma comprised tumor localization and radioactive plaque insertion as to cover the entire tumor plus a 2- to 3-mm tumor-free safety margin; followed by continuous radiation over 5 to 7 days. Periodic intravitreal anti-vascular endothelial growth factor therapy was used to suppress radiation maculopathy and optic neuropathy. Treatment of ocular surface malignant squamous carcinoma was even more complex. It involved a diagnostic biopsy, followed by either monotherapy or polypharmacy with topical chemotherapy. In those, uncommon recalcitrant cases (without tumor resolution), excision with adjuvant “Finger-tip” cryotherapy was performed. Should these approaches fail, external beam radiation therapy was applied. The DRO represented all patients who have completed treatment to the point of no evidence of disease. Then, how long treatment effected persistent “local tumor destruction” over time.

HIPAA and privacy

Data collection and anonymization were accomplished by assigning a unique identification number for each patient. Then the data were collected from the electronic medical record and inserted into a password-protected interface [Fig. 1]. Passwords were made secure by using SHA-256 encryption. This level of encryption renders the passwords impossible to decipher either using brute force attack or dictionary guessing methods. User authentication was done by encrypting the password plaintext that the user provides as an input and then matching it with the password hash (encrypted string) stored on the database. The data in transit were encrypted via secure socket layer (SSL) and transport layer security (TLS) protocols, which protect the information from eavesdropping and being read by any unauthorized attackers while it was being transmitted between the server and the visitor, by using encryption keys known only to the server and the recipient's computer. The green padlock with the HTTPS sign on the website confirms that the data was secure. These data were then extrapolated on the actual website results page as an open access view of patient outcomes.

Figure 1.

Image of the data collection window with a password protected interface. Original from the DRO software created for this study

Data collection

For example, for CM the DRO software window opens to an option to input a new or follow-up patient. New patient parameters were name of condition, unique patient identification number, date of birth, vision at presentation, date of initial treatment, treatment given (e.g., enucleation/plaque brachytherapy), stage of tumor, and height of tumor. In case of a follow-up, the patient ID assigned at first visit was entered and “Look Up” was pressed on the screen. This opens the window for this patient. The parameters to be added at follow-up are date of follow-up, vision at this visit, treatment (plaque brachytherapy/enucleation), recurrence (Y/N), enucleation (Y/N), and metastasis (Y/N) [Fig. 2]. Once the patient details for an entire out-patient day have been uploaded in the system, there was an option to analyze and view analysis. The user then logs out to prevent others from viewing inputted data on that computer. The total number of data entry sessions has been 2341 over a period of 25 months. A number of data entries for CM were 1912, for iris ciliary body melanoma (ICM) were 317, and for OSSC were 112 so far.

Figure 2.

Original software image showing sample of patient data entry page

Data processing

The entire data from each follow-up were first validated against required parameters and consistency checks within the software system. If any of the data format does not match against expected data format for that value, then the program stopped the user from proceeding without correcting the discrepancy [Fig. 2].

On each successful save, each data record was assigned a unique identifier in the database, along with a timestamp so it can be traced back for audit. The data were stored in a relational format, which can seamlessly connect multiple records to a master record (patient). This enables us to collect incremental data, without overwriting the old values, thereby, measuring the impact and changes to the patient condition across the entire follow-up duration, and allows the use of multiple time periods to analyze the data.

The data for each patient were traversed and aggregated and conditional logic considered only the relevant records for summarization. This process was performed over the complete set of data, which was then summarized into key headlines to be reported. The summarized data points are stored for quick access and to avoid re-calculation until more data were added or updated, when the entire process repeats. When this happened on the server end; the data were available for consumption to the users via the internet. Thus, the most recently entered patient data was used to calculate presented DRO results.

Real-world data was defined as outcome data derived from a heterogeneous patient population in a real-world setting. This may include patient surveys, clinical trials, and observational cohort studies. Real-world data refer to observational data as opposed to data gathered in an experimental setting and typically derived from electronic health records.[1,2]

Results

For this study, we created DRO software to collect and process patient data. These include numbers of patients, treatments, visual acuity, metastasis, survival, and follow-up time of eye cancer patients. In near-real-time, we updated and presented these data as easily accessible and understandable for patient access on the internet. Since its inception, no one at our specialty center was aware of our published results prior to the public. Since December 2017, the results have been continually made public on our web site.

On the main DRO web page, one can choose the type of cancer you are looking for by clicking the tab [Fig. 3]. Then a second page displays with a picture of the tumor with an overlay figure legend instructing the viewer to “click to view results.” Clicking the image reveals patient-oriented parameters, such as patients entered, average vision, most common vision, local tumor destruction, percentage of eyes preserved, percentage of patients undergoing initial eye removal as treatment, average follow-up in years [Fig. 4]. These statistical presentations were developed to be an understandable real-world DROs.

Figure 3.

Original software snapshot of the combined result parameters published on 3 February 2020 for choroidal melanoma, ICM, and squamous conjunctival malignancies

Figure 4.

A graph showing the number of DRO web page visits to https://eyecancer.com/our-approach/doctor-reported-outcomes/ between April 2018 and January 2020

As of January 2020, 311 CM patients with mean follow-up of 7.3 years showed 99.7% local tumor control, 95.8% eye salvage, and most common vision was 20/25 (mean 20/50). Of 107 ICM patients with mean follow-up of 6.5 years, local tumor control was 99.1%, eye salvage was 97.2% and most common vision was 20/20 (mean 20/20). Of the 29 OSSC patients with mean follow-up of 2.8 years, local tumor control and eye salvage was 100% and most common vision was 20/20 (mean 20/20). OSSC patients are typically discharged if disease free for 1-2 years depending on their initial American Joint Committee on Cancer T-stage.[7] Rates of primary enucleation as treatment for CM were 4.2% and 2.8% for ICM, respectively. No patient required eye removal for invasive OSSC.

Patients visit our DRO webpage prior to and after meeting with our eye cancer specialist. Thus, they viewed a list of select treatments and the outcomes other patients in our care have experienced. Transparency strengthens the patients' understanding of the most common treatment options, expected results, and the operating physician.

Discussion

This study shows that it was possible to present near-real-time outcomes from a tertiary ophthalmic oncology center. Our PubMed.gov and Medline literature search using the key words: “internet,” “outcomes,” “reporting,” “database,” and “website” found no prior similar attempts at presenting near-real-time outcomes of treatments by physicians for patient education and awareness. A few researchers have reported methods of medical science that utilize the medium of the world-wide-web for improvement of treatment outcomes.[8,9,10,11,12] However, these studies are not directly comparable to what we have termed, the “DRO System.” These studies showed the alternative methods to improve treatment by involving the patients in healthcare through the medium of internet.

Richards and Caldwell in an interactive website and virtual specialist based model for patients, provided tailored treatment advice with a conversational agent to allow discussion of the suggested treatment while they were awaiting their specialist appointment.[13] A 6-month trial with 74 children with urinary incontinence showed that 38% of those who used the program reporting a resolution of their bed wetting without needing a specialist appointment. This model was very similar to our DRO, but it catered to non-life-threatening chronic conditions as opposed to DRO that caters to eye and life-threatening ophthalmic conditions.

Earlier attempts in the literature have also been made to study patient reported outcomes (PRO).[14,15,16,17,18] In 2002, Detmar et al. evaluated the efficacy of standardized health-related quality of life (HRQL) assessments. Physicians in the intervention group identified a greater percentage of patients with moderate-to-severe health problems in several HRQL domains as compared to control group.[14] In 2013, an observational study by Dougados et al. showed that rheumatoid arthritis treatment intensification was predominantly based on PRO as compared to DROs.[15] In 2013, Weldring et al. evaluated the PRO reflecting the ongoing health service commitment of involving patients with development and evaluation of health care service delivery and quality improvement.[16]

In comparison to our DRO, scientific journals are typically used to present the results in the form of a scientific study. In addition, journal articles require pre-publication time taken for data collection, analysis, manuscript-writing, peer-review, subsequent revision, waiting for acceptance or rejection, and resubmission formatting. If the manuscript is accepted there are processing delays such as corrections to galley proofs prior to publication. Then finally, once in press, there exist limitations to public access. Of these delays, peer-review can be particularly unpredictable. In the hands of a competitor or reviewer unfamiliar with a new approach, an objective peer-review may not occur. This is particularly true for small subspecialties such as ophthalmic oncology. Last, most publications are accepted due to the novelty of the information with a bias against negative results.

Clearly, not a substitute for peer-review publication, internet reporting of real-world data in the form of outcomes (DRO) can offer an assessment of a doctor or clinic's mean results. Though like peer-review publication, DRO results can be affected by lack of follow-up (when patients do not return), co-management by outside “non-specialty trained” physicians, and by the efficacy of one's data collection personnel [Table 1].

Table 1.

Comparison between peer-reviewed publication and doctor reported outcome available on the world-wide-web

Parameter	Peer-review publication	DRO
Real-time up to date	No	Yes
Influence decision of doctors	Yes	Yes
Influence decision of patients	No	Yes
Allow multiple centers to compare efficacy	No	Yes
Peer review criticism	Yes	No
Time consuming	Yes	No
Delay in publishing results	Yes	No
Data reviewed	Retrospective or prospective	Prospective

The main advantage of our DRO was its real-time nature that allowed users to see current results. This information can be used to compare outcomes at different centers (should they post them). Multicenter use of the DRO system would allow for validation of each center's methods and improved patient care. In addition, a near-real-time DRO can serve as an early detection system, as new therapies can be observed to improve or diminish outcomes.

Conclusion

Current electronic medical systems can be improved by revealing near-real time DROs in a language and medium that are accessible to the average patient. This initiative paves way for other specialists to post their clinical results online for patient and physician reference. Monitoring one's own waiting room results offers greater transparency and an opportunity to improve clinical practice, by giving each center a near-real-time evaluation of the effectiveness of their care. This will help both the patient and their physicians by improving the health care standards of each doctor's clinical practice. This study was a demonstration of the past quality of single clinical practice. This information was offered to help shape patient expectations, build patient–physician rapport and improve informed consent. Multi-center implementation of this DRO system offers the potential to improve patient care.

Financial support and sponsorship

Drs. Maheshwari, Jain, Tomar and Garg each received fellowship grant support from The Eye Cancer Foundation. This software was created utilizing a grant from The Eye Cancer Foundation. (http://eyecancercure.com). The Eye Cancer Foundation reserves the rights as the sole owner of this program.

Conflicts of interest

There are no conflicts of interest.