Assessing Patient Satisfaction: Using the Radiation Oncology Patient Satisfaction [ROPS] Questionnaire in a Private Practice Setting

Abstract

A new patient satisfaction measure called the Radiation Oncology Patient Satisfaction (ROPS) questionnaire is used in this study to measure satisfaction data for radiation oncology in private practice. Limitations of existing literature on patient satisfaction demonstrated a need to develop a questionnaire that was more tailored to analyze patient satisfaction among those utilizing private oncology centers within the U.S. healthcare system. This need was met by this study’s development of the ROPS questionnaire, which was a variation of 2 existing constructs that are presented in the text. The questionnaire was fielded among patients at a private cancer radiation treatment clinic in Florida (n = 950). Data analysis tested the psychometric properties of our revised construct and its predictive validity for 2 patient satisfaction outcomes: (1) likelihood of recommending this treatment center to family members/close friends and (2) overall satisfaction with treatment. Each of the ROPS variables were found to make a legitimate contribution to evaluating patients’ overall satisfaction with radiation treatment. Findings indicated organizational setting was of vital importance when conducting patient- centered research on satisfaction. Treatment factors in ROPS can be adjusted to measure satisfaction with chemotherapy or other modalities in addition to radiation treatment. Authors recommend radiation oncology clinics regularly monitor patient satisfaction, especially if/when they experience organizational changes, such as when a new physician joins the practice or if/when the national or local landscape undergoes significant shifts in norms and expectations like we have seen happen with COVID-19.

Introduction

Patient satisfaction is and will continue to be a necessary unit of analysis when assessing quality medical care in the 21st century. Existing research argues there is greater value in an assessment of patient satisfaction that is focused on specific specialties, such as radiation oncology, ¹ which this study does. The purpose of this study is to introduce a new measure of patient satisfaction, the Radiation Oncology Patient Satisfaction (ROPS) construct. Most of the work on patient satisfaction is conducted in hospital-based centers often associated with academic medical centers as well as public hospitals or clinics.^2,3 In contrast, this study aimed to examine patient satisfaction in private practice. The ROPS measure was tested at Suncoast Cancer Radiation Center (SCRC), a private radiation oncology facility in West Central Florida with the intent of producing an accessible and efficient method for assessing patient satisfaction across various dimensions of clinical care in radiation oncology that is valid, reliable, and easy to administer and analyze.

Review of the Literature

Patient satisfaction has been the subject of extensive and sometimes disparate research and conceptualization.^4,5 Linder-Pelz ⁶ defines patient satisfaction as “positive evaluations of distinct dimensions of health care” and further categorizes patient satisfaction as individuals’ attitudes toward specific care they have received. While the notion of patient satisfaction has been of interest since the 1960s, the first long-term comprehensive assessment of the concept was undertaken by Ware and Snyder ⁷ in the 1970s.

Thorough reviews of the early history of patient satisfaction in the U.S. and the British National Healthcare System were followed by a development of the Patient Satisfaction Questionnaire [PSQ] in the 1980s.^8-10 The 55-item questionnaire was developed and tested at family medicine clinics in California and Illinois in addition to a 43-item short form (Form II). The PSQ presented 8 patient satisfaction sub-constructs: (1) Interpersonal manner; (2) Technical quality/competence of provider; (3) Accessibility and convenience; (4) Finances; (5) Efficacy and outcomes; (6) Continuity of care; (7) Physical environment; (8) Availability. While these 8 sub-constructs are a useful beginning, there are several limitations to the applicability of the PSQ in the context of radiation oncology care. For example, cancer patients were not included in this body of research. The severity of illness for many participants taking the PSQ and their prognoses likely differed substantively from cancer patients.

Since many cancer patients represent an older demographic by a wide margin, ¹¹ the PSQ does not optimally capture the majority of cancer patients’ experiences. Also, the PSQ was developed in family medicine clinics from samples of predominantly lower income patients. ¹² Several other studies^13-18 have examined decidedly different healthcare settings. For example, the PSQ’s category of “efficacy/outcomes” may work well in family practice clinics where much of the focus of treatment is on short term, relatively minor medical afflictions. “Efficacy/outcomes” in oncology settings are more problematic, suggesting they should be determined over a prolonged period. In addition, since the PSQ was developed and validated in large, government-run family practice clinics, it may have limited validity on private practices with more personalized care and smaller patient-doctor ratios. ¹⁹

Analyzing the Healthcare Setting

Setting is one of the more neglected elements of healthcare research. ²⁰ Patients with severe maladies and more everyday complications can have appreciably different experiences depending on where their healthcare is delivered. This issue is particularly salient given that patient satisfaction research has been almost exclusively conducted at public health or university-affiliated medical clinics referred to above. The organizational setting examined in this study is important because it provides a specific and overlooked context to help better understand specific aspects of oncology care and patient satisfaction during radiation treatment.

At the time of this analysis, SCRC was a new facility, built to provide concierge-type^21,22 cancer radiation treatment. The practice consisted of 6 physicians and 4 RNs. The decision to invest in the latest health radiation technology allowed SCRC to treat an array of cancers, from a cancerous lung to a malignant spot the size of a pencil head on the brain. Technology available for radiation included a Tomotherapy machine HT & IGRT (Helical Tomotherapy and Image-Guided Intensity Modulated Tomography), a Varian IMRT system (Intensity-modulated radiation therapy) with a High Dose Radiation (HDR) attachment, and a Cyberknife system (LINAC Stereotactic Radiosurgery).

The waiting room featured modern amenities geared toward patient comfort, such as a high-end snack bar and computer workstations. An outdoor Zen Garden and indoor water wall were examples of SCRC’s effort to incorporate nature into the waiting process. In place of television news or sitcoms, the clinic played nature DVDs on a UHD television. Because the organization was private, they were able to allocate funds toward their waiting room that perhaps other public institutions would need to regulate. The snack bar, for example, is a cost that public clinics would have difficulty justifying to their stakeholders.

Research Questions

Based on the preceding literature review, we posed the following 4 research questions: RQ1: Is ROPS a valid and reliable instrument for measuring patient satisfaction in radiation oncology practice?

• RQ2: Do factors associated with patient care (accessibility, coordination, competence, communication and relationships, education, emotional support, personalization) adequately explain (RQ2a) overall satisfaction and (RQ2b) word-of-mouth recommendation of care?

• RQ3: Is the 8-factor structure instrument developed by Dy et al ²³ confirmed using the ROPS construct?

• RQ4: What is the construct validity of the ROPS dependent variables (overall satisfaction and word-of-mouth recommendation) when compared with qualitative patient feedback on what could be improved about patient care?

Research Methods

A survey was used to collect data on patient satisfaction at a private cancer radiation treatment clinic in a city in West Central Florida. The protocol for this research was approved by SCRC’s Institutional Review Board. Data collection spanned 3 years, yielding a dataset of 950 patients out of 1033, representing a 92% response rate of available patients. Patients were recruited to participate in the survey during their follow-up visit after treatment, 7 to 10 days after radiation was completed. Upon completion of the survey, participants deposited them into a locked box in the waiting room monitored by the SCRC’s patient advocate.

Patient Demographics

Patients ranged in age from 32 to 92 years. Fifty-six percent were male and 44% were female. Approximately 50% of patients reported having Medicare insurance with individual insurance supplements and 5% of patients had Medicaid. 15% had V.A. or TriCare insurance, 30% had private insurance, with most being insured by their employers. Personal income ranged from recipients of welfare to those whose earnings were in the top 0.05%. The majority of patients qualified as middle class, though some were homeless, and others were imprisoned in Pinellas County’s prison facility.

Questionnaire design

Two issues guided our search process of the literature to inform how we designed the survey. First, the variables in the instrument had to be well researched and presented in a peer-reviewed journal(s). Second, the instrument needed to possess face validity and had to be applicable to a primarily older patient population undergoing cancer treatment. Several instruments were considered, including the PSQ ¹⁰ long and short forms. These measures were rejected because they did not accurately reflect the circumstances of cancer radiation treatment. For example, the short form of the measure consisted of 43 questions, taking elderly and infirmed patients as long as 30 minutes to complete. Other measures of patient satisfaction were identified as limited in similar ways. For example, the McCrusker ²⁴ . instrument was also rejected because the study relied on the overly lengthy PSQ and the population included patients, home health caretakers, and surviving relatives of patients who had died. The setting did not include cancer patients who were receiving treatment at a radiation oncology center.

More audience-specific and consequently useful constructs for measuring patient satisfaction with radiation oncology care are offered by Wiggers et al ²⁵ and Dy et al ²³ but also have notable limitations. The former research used a 60-item questionnaire. Among the multitude of variables measured were: communication skills, continuity of care, nonmedical care, finances, and accessibility of care, for example. Patient satisfaction, however, was only measured indirectly as the percentage of patients who answered each question as either “very important” or “moderately important.” Dy et al ²³ performed a comprehensive literature review of elderly patients in palliative care and end-of-life situations. Their review identified 7 dimensions of patient satisfaction: (1) accessibility to care; (2) coordination of care; (3) competence of caregivers; (4) communication and relationships; (5) education about disease; (6) emotional support; (7) personalization of patient.

The Patient Satisfaction Questionnaire (PSQ) construct contained 8 subconstructs: (1) interpersonal manner; (2) technical quality; (3) accessibility/convenience; (4) finances; (5) efficacy/outcomes; (6) continuity; (7) physical environment; and (8) availability.^10,26 Sixty-eight items were included but they were not treatment specific. In later research ³ adapted the PSQ to specific medical encounters (VSQ E5 & S6), the E5 version used a 5-point “excellent to poor scale” and the S6 used a 6-point satisfaction scale. We rejected the PSQ because of length and its non-specific nature and the VSQ E5 because of the midpoint scale and the E6 scale because independent satisfaction outcome measures weren’t included.

To incorporate content validity into the study, our analysis integrated the Dy et al ²³ and the Ware & Hays ³ constructs into a 21-item questionnaire, 8 subconstruct Radiation Oncology Patient Satisfaction [ROPS] construct (see Table 1). In addition, the fourth author is one of the lead oncologists at the site where the questionnaire was administered, and he worked with the first author to ensure the content was tailored to the specific situation of radiation cancer treatment. Twenty of the 21 items were measured on a 4-point Likert-type scale ranging from “Strongly agree” to “Strongly disagree.” The 4-point scale was deemed more effective than using a midpoint scale because unlike public opinion polling, ²⁷ a neutral answer about cancer treatment is unlikely and difficult to interpret. ²⁸ Researchers cognitively pre-tested the ROPS on a random sample of 47 patients from SCRC to assess the measure’s clarity, relevance to their treatment, and ease of administration. Throughout the data collection period, none indicated they had complications understanding the questions or completing the questionnaire. This cognitive pre-test supported ROPS’s understandability, relevance, clarity, and ease of administration to patients who were under the stress of cancer radiation.

Table 1.

Mean score and standard deviation for each ROPS item.

Item	Mean	SD
1. Someone was always available to discuss my treatment and condition.	3.79	0.41
2. My doctor always took as much time as needed to help me during my treatment.	3.77	0.44
3. My treatment was given in a timely manner.	3.73	0.46
4. The people at SCRC helped me navigate through the health care system.	3.73	0.46
5. The members of my health care team worked seamlessly together to care for me.	3.81	0.39
6. The SCRC health care team has the knowledge and skill to effectively manage my treatment.	3.82	0.39
7. The SCRC health care team communicated effectively throughout my treatment.	3.80	0.36
8. Everyone at SCRC treated me with a caring attitude.	3.86	0.36
9. The SCRC health care team educated me in what to expect during my treatment and afterward.	3.74	0.45
10. The SCRC health care team educated me about my cancer in a way I could understand.	3.69	0.49
11. The SCRC team was responsive to my emotional needs during my treatment.	3.75	0.46
12. I would say the people at SCRC are compassionate people.	3.85	0.36
13. I felt the SCRC health care team treated me as a whole person and not just my disease.	3.80	0.41
14. I felt my family/significant others were included in my treatment planning and progress.	3.70	0.48
15. The SCRC health care team helped give me a sense of control during my treatment.	3.69	0.48
16. The SCRC health care team gave me the support I needed throughout my treatment.	3.77	0.42
17. Overall, I am very satisfied with my treatment at SCRC.	3.85	0.37
18. If a good friend or family member needed cancer treatment I would recommend she/he go to SCRC.	3.83	0.41
19. I never felt like I had to wait an excessive amount of time to see my doctor.	3.64	0.56
20. I never felt like I had to wait an excessive amount of time to get my radiation treatment.	3.71	0.50
21. What one thing would you change or improve upon about your experience?	(open-ended)

In addition to the 8 dimensions of patient care during cancer radiation, 2 questions (dependent variables) were posed to assess patient satisfaction directly. A final open-ended question was included for patients to describe changes SCRC should make to improve their experience. It was hypothesized that if a significant percentage of responses to the open-ended question offered substantive suggestions for improvement, this would indicate less than optimal levels of patient satisfaction no matter what the quantitative numbers were.

Data Analysis

The analysis of the ROPS construct proceeds in several steps (detailed further in the Results section). First, descriptive statistics are reported for the survey items. This is followed by an initial examination of instrument reliability and internal structure. Next, predictive validity is assessed through discriminant analysis and open-ended comments on the survey. Finally, the factor structure is analyzed via confirmatory factor analytic methods.

Results

Table 1 presents each quantitative item along with its mean score and standard deviation. The higher the mean score, the more positively participants rated that aspect of treatment at SCRC. It is clear these items closely reflect previous patient satisfaction research findings where scores are very positive with little variability.^10,13,29,30

The highest mean score was 3.86 and the lowest was 3.69. As Hall et al ³¹ conclude, this is a demand characteristic of medical care in the United States where most patients believe they pick the best medical providers because it was their personal choice. Similarly, Ware et al ¹⁰ suggest patients distort their evaluations upwardly to remove cognitive dissonance they might experience with their medical care provider. Ware ²⁶ discussed acquiescent response set (ARS) bias where patients answer favorably no matter what question is asked and suggested alternating positively and negatively worded response scales. We considered doing this and tested this method but discovered the older cancer patient population (>55) had difficulty with the back-and-forth phrasing and told us the questions were confusing. As a result, we kept all questions in the positive form, allowing patients to either agree or disagree with each statement.

Instrument Reliability

The reliability of the ROPS scale was assessed using 2 methods. Omnibus reliability of the 20-item scale was assessed using Cronbach’s alpha, ³² a multiple correlation measure of internal consistency. This omnibus reliability coefficient was α = .96. We then assessed the internal consistency of each of the sub-constructs of ROPS using their respective items. Note the sub-construct, “Communication” had only 1 item and was eliminated from the analysis. Although there are drawbacks to Cronbach’s alpha as an estimate of internal consistency in some instances, ³² it is used here as an estimate for 3 reasons. First, error comes from the sampling of content for the construct under investigation. ³⁴ Second, Cronbach’s alpha is increased by larger numbers of items in each sub-construct and since most had only 2 to 3 items, all possess good reliability. These data are presented in Table 2 below. Finally, these coefficient alphas inform analyses of the data structure of the ROPS construct in this project.

Table 2.

Reliability coefficients for each ROPS construct.

Sub-constructs	Cronbach’s alpha
Accessibility (items 1, 2, 3, 19, & 20)	.85
Coordination (items 4 & 5)	.75
Communication & Relationships (items 7 & 8)	.75
Education (items 9 & 10)	.82
Emotional Support (items 11 & 12)	.67
Personalization (items 13 & 14)	.74
Support (items 15 & 16)	.84
Outcomes (items 17 & 18)	.71

Data Structure

The relationship among the items in ROPS was assessed. To begin this assessment, a McQuitty’s ³⁵ Elementary Linkage Analysis (ELA) was performed using Pearson product moment correlations among the 20 items to reveal clusters and commonalities. This is the first and most parsimonious process for understanding data structure. It this case, there was a highly correlated set of items; the low was r = .38 and the high was r = .74. The lowest interitem correlation had 14% shared variance (r2), and the highest had 55% shared variance. Psychometrically, this is an ideal range of interrelationships because the items have a reasonable amount in common with one another without being redundant, ³⁴ and all items have a moderate to strong relationship with the outcome variables (items 17 & 18).

Figure 1 below presents the results of the ELA for all the ROPS components including the outcome variables. Seven sub-constructs had multiple items, and they were transformed into a single, composite variable. The 2 outcome variables and communication were single items and kept in that form. The ELA demonstrates close relationships among the ROPS variables. The ELA also shows that 3 clusters emerge around 3 central variables: (1) Support of patient’s decision-making; (2) Accessibility; (3) Communication & Relationships. Overall satisfaction is more closely related to the other variables than word-of-mouth-recommendation, but since the correlations among all ROPS variables are quite high, this difference is slight.

Figure 1.

Elementary linkage analysis of ROPS patient satisfaction variables.

Predictive Validity

In addition to reliability, a second part of the measure’s overall validity is predictive validity. ³⁵ A measure must have the ability to predict important outcomes. If the ROPS is useful in assessing actual patient satisfaction in connection with the radiation oncology care patients receive, the items should predict these important outcomes. In this study, this includes item 17 and item 18 (overall satisfaction and word-of-mouth recommendation of care). To test predictive validity, 2 discriminant analyses were performed. Each outcome variable was recoded into a dichotomous variable. Patients who ranked each outcome variable with a score of 1 (strongly disagree), 2 (disagree), or 3 (agree) were recoded to a value of 1 (less satisfied); patients who ranked each outcome variable as a 4 (strongly agree) were recoded as a 2 (very satisfied).

The transformations gave each outcome variable a dichotomous value of either being extremely satisfied with their care or having some reservation. The discriminant analysis forms K-1 linear combinations of discriminating variables on which the test groups will be discriminated and assigns standardized discriminant coefficients to each group of variables, such that the combination (discriminant function) provides a maximum differentiation between the test groups.^36,37 Discriminant analysis also provides several useful statistics for determining predictive validity. Chi-square is assigned to each function (in this case only one) to determine its statistical significance. In our analyses the first outcome variable, V17, yielded a canonical correlation of 648.1 which is significant at the P < .0000 level while the second outcome variable, V18, yielded a canonical correlation of 649.3, significant at the P < .0000 level. Clearly the discriminant functions support the predictive validity of the ROPS.

A canonical correlation, equivalent to a point biserial correlation (which is a correlation between interval variables and a nominal variable) was determined. In this study, that included group membership of highly satisfied and less satisfied patients. The canonical correlation for outcome variable V17 was .758, and .757 for variable V18. Both results show the discriminant functions differentiate significantly between the highly and less satisfied patient groups. In addition, by examining the valence of the variables in each discriminant analysis, we can understand which best characterizes the highly satisfied and less satisfied patient groups for variables of overall satisfaction with care and likelihood of recommending SCRC to friends or family members who might need treatment. The analysis also produced group centroids coefficients for the “greatly satisfied” and “less satisfied” groups. If the function effectively discriminates between the 2 groups, we expect the centroids to be far apart. Upon analyzing the data, the “highly satisfied” group’s coefficient was less than 0.47 and the “less satisfied” group was −2.70, reflecting a great deal of separation.

Each group was characterized by looking at the positive or negative valences. As Table 3 demonstrates, the “less satisfied” patient group is most characterized by reporting: (1) feeling their family/companions were not included in their treatment planning; (2) their physician not taking as much time as the patient perceived he/she needed; (3) not treating the patient as a “whole person and not just a disease.” Highly satisfied patients were most characterized by reporting: (1) the team of physicians and staff provided the support needed throughout their treatment; (2) the team of physicians and staff demonstrated compassion; (3) the team of physicians and staff had the knowledge and skill to manage their treatment; (4) patients believed their team of physicians and staff treated them with a caring attitude, (5) patients had a sense of control and were educated on what to expect during treatment (6) patients did not wait an excessive amount of time for radiation treatment; (7) the team educated them about their care in a way they could understand. Tables 3 and 4 present these data.

Table 3.

Significant variables in patient overall satisfaction.

Discriminant function	Standardized weights
My team gave me the support I needed throughout my treatment	0.392
I would say that the people here are compassionate	0.362
My team has the knowledge and skill to manage my treatment	0.227
Everyone treated me with a caring attitude	0.221
My team helped give me a sense of control during my treatment	0.218
My team educated me about my cancer in a way I could understand	0.202
My team educated me on what to expect during and after treatment	0.185
I never had to wait an excessive amount of time for radiation treatment	0.122
I felt my family/companions were included in my treatment planning	−0.195
My doctor always took as much time as needed	−0.144
The people here treated me like a whole person and not just a disease	−0.138

Table 4.

Significant variables in patients’ word-of-mouth recommendations.

Discriminant function	Standardized weights
Everyone treated me with a caring attitude	0.363
I would say that the people here are compassionate	0.308
My team educated me on what to expect during and after treatment	0.301
The people here treated me like a whole person and not just a disease	0.285
My doctor always took as much time as needed	0.193
I never had to wait an excessive amount of time for radiation treatment	0.189
My team educated me about my cancer in a way I could understand	−0.173

As shown in Table 4, only 7 of the 18 predictor variables were needed to maximally discriminate between the likelihood of providing word-of-mouth recommendations to friends/family regarding the treatment received at SCRC. The centroids were farther apart, − 2.70 and 0.50. Patients who were “less likely” to provide word-of-mouth recommendations to their friends/family are characterized by lower scores on feeling they had been educated about their cancer in a way they could understand. Patients “very likely” to recommend treatment at this facility to friends/family were characterized as: (1) believing physicians and staff treated them with a caring attitude; (2) believing physicians and staff were compassionate people who educated them on what to expect during and after treatment; (3) patients feeling they were treated like a whole person and not just a disease; (4) patients believing their physicians took as much time as needed during their visit; (5) patients not having to wait for an excessive amount of time for radiation treatment.

Standardized weights in discriminant analysis also offer a form of prediction on nominally scaled dependent variables. ³⁷ Discriminant functions that significantly discriminate between the categories of the dependent variable, variables with the highest standardized scores indicate they are the best predictors of that respective patient satisfaction dependent variable. Returning to Tables 3 and 4, one can see that the best predictors of high levels of patient overall satisfaction by a sizable amount are: (1) giving patients the support they needed through treatment (0.392) and (2) believing the people involved in their treatment are compassionate (0.362). The next group of weights are much lower and consequentially of less predictive power. The second dependent variable, patient word-of-mouth recommendation, is best predicted by (1) patients believing everyone treated them with a caring attitude (0.363); (2) believing the people involved in their treatment are compassionate (0.308); (3) being educated on what to expect during treatment (0.301); and (4) treating patients like they were “a whole person not just a disease” (2.85). The remaining predictor variables were much lower and of less consequential predictive importance.

Patient comments

A final test of validity of the ROPS is the open-ended question offered at the conclusion of the questionnaire. Since the mean scores of the dependent variables (overall satisfaction and willingness to provide word-of-mouth recommendations of SCRC to family and friends) presented in Table 1 are high (3.85 and 3.83, respectively), we should expect minimal “serious” recommendations for change if patients are truly satisfied with their care. In summary, there should be few substantive change recommendations. Table 5 presents these recommendations across the 950 patients surveyed.

Table 5.

Patients’ verbatim recommendations for change at SCRC.

Recommendation	%
I wouldn’t change a thing	64
This place is a wonderful place to have cancer treatment	28
Change what is served in the snack bar	4
Advise patients about copayments at the beginning	1
Have Drs. & staff wait for your answers to their questions	1
To be able to get into the treatment room earlier & spend less wait time	1
Radiation department needs to slow down and not be so pushy	1

Table 5 is consistent with the high scores on the dependent variables; only 4% of the patients had substantive recommendations for treatment improvements. This coincides closely with the percentage of patients who scored their care on the negative side of the 4-point scale. These included: (1) more/better information about their co-payments for treatment; (2) doctors and staff listening to the patient; (3) being able to get through treatment faster; (4) treatment team not being so pushy during treatment. The latter 2 suggestions are contradictory if one connects the concept of “pushiness” with an increase in “speediness.” 92% responded positively, while 4% mentioned incidental “preferences” related to the waiting room (eg, stronger coffee and pastries in the morning).

The last research question focused on how well ROPS matches the 8-factor structure hypothesized. We conducted a Confirmatory Factor Analysis (CFA) of ROPS. CFA tests the data structure from the hypothesized model fit and produces 3 indices of this fit: (1) Model chi-square; (2) Confirmatory factor index (CFI); Root mean square error of approximation (RMSEA). Furr ³⁸ recommends taking 3 steps CFA. First, the number of factors or latent variables hypothesized to underlie the scale’s items should be determined. Second, the items linked to each factor should be specified, with at least one item related to each factor and with each item linked to only one latent variable. Third, when a hypothesized model includes multiple factors, researchers should specify possible associations between factors. Based on the literature review and the initial McQuitty’s ³⁵ ELA, we posited 8 underlying factors of patient satisfaction: accessibility, competence, coordination, education, personalization, emotional support, communication & relationships, personalization, and outcomes along with the patient satisfaction dependent variable.

The results, after 3 iterations of the CFA, are provided below. The CFA revealed there were 2 unnecessary redundancies: (1) the 5-item accessibility factor was pared down to 3 items to improve the model fit (items 19 and 20 removed); (2) two other factors, communication & relationships, and emotional support also revealed better model fit when combined into one factor we now call “communication and emotional involvement” (CEI). Their proximity in the McQuitty’s ³⁵ ELA suggests greater than ideal redundancy between the 2 constructs.

As the CFA model in Figure 2 reveals, we pared the 9 patient satisfaction factors into 7 factors that include outcomes. CFA sets several standards for goodness of fit. First, correlations between the latent factors should not exceed 1.0. This standard was met with the highest correlation between latent factors coordination and communication & emotional support being 0.972. The standardized regression weights in CFA also provide an indication of the robustness of the model. In this case, the lowest standardized regression weigh was 0.72 and the highest was 0.87 showing an appreciable model robustness. We also followed Kline’s ³⁹ recommendation by reporting both the CFA Chi-square and RMSEA. The Chi-Square for the original factor model was 1136.05 and 840.15 for the revised model. Kline ³⁹ also proscribes analyzing the models’ CFIs. The first model CFI was 0.919 and the third model was 0.935. The standard is a value above 0.90 and the third model improvement is appreciably better.

Figure 2.

Confirmatory factor analysis revised factor structure.

Finally, we examined RMSEA as recommended by Brown. ⁴⁰ RMSEA should ideally be less than 0.08, but Brown ⁴⁰ suggests less than 0.10 is acceptable). RMSEA for Model 1 was 0.093 and 0.089 for Model 3 which is an acceptable improvement.

Discussion

Data analysis supports the utility and validity of using the ROPS questionnaire when assessing patient satisfaction in radiation oncology treatment centers. ROPS was designed for a patient population undergoing cancer radiation treatment and it should also be considered for use by researchers and practitioners working in other types of cancer treatment organizations since they share similar patient populations, goal structures and organizational operations. The instrument was easy to administer, taking less than 6 minutes for patients to complete. There were also no missing data from patients, suggesting participants believed the issues tested in ROPS were important and relevant to their experience.

The degree to which variables were closely linked demonstrates the importance of understanding how each of these satisfaction-related variables function. From an operational management perspective, when even 1 or 2 of the 7 constructs associated with patients being “highly satisfied” with their cancer radiation treatment are scored lower by patients, this evaluation of their satisfaction and/or willingness to recommend this treatment center to family/close friends appears to be in jeopardy.

The best predictors of a word-of-mouth recommendation as well as overall patient satisfaction are considered interpersonal issues that are “soft” in nature, such as having a compassionate treatment team, doctors/staff treating patients with a caring attitude, educating patients on what to expect during and after treatment, and treating patients as a whole person and “not just a disease.”

The “efficiency” model espoused by many largescale treatment institutions without prescribing these softer factors appears counter to achieving real patient satisfaction. ⁴¹ In fact, efficiency factors that had the lowest standardized weights in the analysis were having to wait for an excessive amount of time to receive radiation, and the physician spending as much time with the patient as needed. When assessing patient satisfaction, each of the ROPS variables makes a legitimate contribution to evaluating patients’ overall experience with radiation treatment.

Policy and practice implications

Authors recommend radiation oncology clinics regularly monitor patient satisfaction, especially if/when they experience organizational changes. Examples of these changes could be when a new physician joins the care team, or if a practice relocates or renovates its office space, or if/when the national or local political and social landscape undergoes significant shifts in norms and expectations like we have seen happen with COVID-19. The ROPS questionnaire could be adjusted to include variables measuring perceptions related to COVID-19 policies specific to a certain clinic or mask wearing. Other examples related to the context of COVID-19 to consider are the perceptions of cleanliness or safety. In the future, if/when other unexpected health crises arise on local or global proportions, such would be the case with another pandemic, telehealth and variables related to perceived needs as well as implementation and adoption of telehealth could be included in ROPS to measure reluctance, effectiveness, and satisfaction.

The authors suggest radiation oncology practices monitor patient satisfaction continuously if they do not already. This may help ensure word-of-mouth recommendation and overall satisfaction are continuously assessed in addition to each of the 18 treatment variables identified in this analysis. Treatment factors in ROPS can also be adjusted to reflect oncology practice that focuses on chemotherapy or other modalities as well as radiation therapy. Disseminating ROPS within a practice that has multiple branches in varying locations may identify issues of concern that are unique to one location or consistent across the branches. Future research across both private radiation oncology treatment centers and large public institutions like the V.A. and members of the National Cancer Care Network of not-for-profit cancer care hospitals are warranted to test the findings in this study. In addition, other specializations, such as gynecologic oncology or surgical oncology, could administer ROPS and adjust some of the language/phrasing to fit their patient population, especially if their patients are receiving radiation treatment. Each variable in ROPS receives its own score, so comparisons may be made to better understand the specifics of patients’ perspectives as they relate to treatment.

The ELA found 3 clusters around 3 central variables: (1) Support of patient’s decision-making; (2) Accessibility; (3) Communication & Relationships. Therefore, future users of the ROPS questionnaire might consider focusing on one of these 3 variables if the practice under analysis has already identified issues within one of these areas where the organization aims to improve or further examine. If, for example, the practice’s existing patients made suggestions or voiced complaints about support for decision-making, ROPS could be adjusted to measure the needs of that practice and its patients in the context of decision-making.

Study Limitations

The study is generalizable to a private radiation oncology center in the Southeastern U.S. Such centers are less traditional in the context of cancer care and as such, the results are not necessarily representative of radiation oncology centers at large. SCRC also had state-of-the-art radiation treatment equipment available at the time of the study. Environmental and technological variables should also be considered when replicating this study, especially in more modest treatment facilities. In addition to replicating instrument reliability and predictive validity, the issue of how ROPS will function in a different treatment center where a sizable number of suggestions for improvement are substantive, is a worthwhile issue.

Conclusion

The data suggest the ROPS questionnaire is a reliable measure of patient satisfaction in a private practice setting. For this study, the specialty was radiation oncology, and the questionnaire needs to be applied to other areas. The model, revised after confirmatory factor analysis, is shorter and more parsimonious. The authors recommend ROPS be tested across cancer care settings, including NCCN hospitals and outpatient cancer centers.