Abstract
Background:
Patient satisfaction surveys are increasingly utilized to measure the patient experience and as a tool to assess the quality of care delivered by medical providers. Press Ganey (PG) is the largest provider of tools for patient satisfaction measurement and analysis. The purpose of this study was to determine if patient satisfaction surveys were subject to selection and/ or nonresponse bias.
Methods:
Patients seen in an outpatient academic orthopedic clinic were included in this retrospective cohort study. Demographic data included age, race, gender, marital status, primary payer, and native language. All surveys were administered by PG Associates per internal protocols adhering to exclusion criteria within the institutional contract with PG Associates.
Results:
3.5% of outpatient encounters generated PG survey data, which were generated by 9.1% of all patients evaluated. The population of patients who were administered as well as patients who responded to the patient satisfaction survey represented a unique population with regards to age, race, gender, marital status, insurance status, and native language.
Conclusions:
Demographically, patients who were administered and patients who responded to PG surveys differed from the overall population of patients seen in an outpatient orthopedic setting, evidencing both selection and non-response bias. Because of these differences, and considering the small number of survey returned, caution should be exercised when interpreting and applying these data.
Level of Evidence: III
Keywords: survey science, non-response bias, selection bias, patient satisfaction
Introduction
Recent legislation requires evaluation of the quality of care provided by healthcare professionals, and patient-reported outcome measures (PROMs) are one metric used to measure quality.1,2 Patient satisfaction surveys are a widely recognized PROM. Press Ganey Associates is the largest provider of tools for patient satisfaction measurement, and provides the only survey approved by the Centers for Medicare and Medicaid Services (CMS). Data generated from Press Ganey (PG) surveys are being used to grade, rank, and reimburse hospitals and physicians.2,3,4 Despite widespread use, the application of PG data is controversial since PG survey scores have not been shown to correlate with other outcome metrics.5 Moreover, patient reported outcome scores have been shown to correlate with patient-specific characteristics such as age, sex, race, insurance, employment, psychological distress, and distance traveled to the healthcare delivery site.6,7,8
While PG survey results (i.e. PG scores) receive considerable attention, the methodology employed to generate patient survey data has not been broadly discussed. Survey-derived data is subject to multiple sources of bias that potentially threaten its validity. Because each institution contracting with PG may customize exclusion criteria, selection bias may inadvertently and significantly alter the surveyed population. Furthermore, non-response bias occurs when survey analysis is limited to data from respondents who differ from non-respondents.9,10 Since survey validity is undermined when the survey data is derived from a sample not representative of the whole, a thorough understanding of the surveyed and respondent population is essential to meaningfully interpret survey data.11
PG scores are regularly discussed at orthopaedic professional meetings, faculty meetings, and amongst administrators. While potential methodological shortcomings are often implied, they are typically not quantified or analyzed in depth. The purpose of this study was to analyze the methodology applied by our institution in collecting PG data, specifically 1) the response rate of the survey, 2) the exclusion criteria applied by the institution and its potential effect on survey response, and 3) the demographic characteristics of respondents, non-respondents, and the overall population.
Materials and Methods
After approval from the Institutional Review Board, all outpatient encounters from the Department of Orthopedics and Rehabilitation at our academic institution from 1/1/2015 to 6/30/2016 were identified. Demographic data were extracted from patient medical records that included age, gender, race, native language, marital status, and primary payer. All patients and those who responded to the PG survey within this population were identified and compared.
At our institution, in accordance with institutional policy, the following patients were excluded from receiving a PG survey: 1) a patient surveyed within 90 days by the orthopedics outpatient clinical service, 2) deceased patient, 3) newborn patient, 4) patient admitted or referred to another service which also surveys patients, 5) patient declined publicity (for example, patients who do not wish to be contacted for survey data), 6) patient with a primary psychiatric diagnosis, 7) prisoner or patient otherwise in custody, 8) involvement of child protective services, and 9) patient restricted due to state regulations. Beyond 90 days, PG does not retain a record of patients who were administered surveys or how many surveys were sent. Therefore, the response rate (number of surveys returned/ number of surveys administered) for this 18 month patient cohort could not be calculated retrospectively. Consequently, we prospectively collected the number and demographics of patients sent a survey from a 90 day period outside the prior 18 month study period, from 11/13/2017 to 2/16/2018.
All surveys were administered by PG according to internal protocols. Patients received survey by United States Post Service (USPS) mail or by electronic mail. Survey responses were recorded for up to 1 month for electronic mail and up to 1 year for USPS mail.
The demographic data provided for these patient populations were compared using the t-test for continuous variables and the chi-square of exact test, as appropriate, for categorical variables. A p-value of p<0.05 considered statistically significant. An analysis of maximum likelihood estimates was performed for significant variables. Analyses were completed using SAS statistical software version 9.4 (SAS Institute, Inc., Carey, NC).
Results
Comparison of respondents to all other outpatients from 18 month cohort
In this retrospective cohort study, 107,656 patient encounters were generated by 36,918 unique patients from 1/1/2015 to 6/30/2016. 3720 PG surveys were generated by 3363 of these patients. The average age of patients seen in our outpatient clinics during this time period was 40.2 ±22.5 years (standard deviation 22.5 years); 51.7% identified as female, 97.5% spoke English as a native language, 87.8% identified as white by race, 39.9% were married, 49.4% were single, 3.3% were widowed, and 1.1% were separated. By contrast, patients who returned at least one survey were on average 52.9 ±21.0 years old, 56.1% identified as female (p<0.0001), 99.0% spoke English as a native language (p<0.0001), 95.4% were white (p<0.0001), 55.2% were married (P<0.0001), 31.3% were single (p<0.0001), 3.3% were widowed (p<0.0001), 0.6% were separated (p<0.0001). Each of these categories were statistically significant between patients who had returned a PG survey and patients for whom survey data was not available (all other patients) for the encounter (Table 1).
Table 1.
Comparison of Respondents to all Other Patients Seen as Outpatients During an 18-Month Time Period
| Respondents (N=3,363) | All Other Patients (N=33,231) | p-value | |
|---|---|---|---|
| Age (years) | 52.9 (+/-21.8) | 40.2 (+/- 22.5) | <0.0001 |
| Gender (F) | 56.11% | 51.68% | <0.0001 |
| English as Native Language | 98.99% | 97.48% | <0.0001 |
| Race | |||
| White | 95.43% | 87.79% | <0.0001 |
| Non-white | 4.57% | 12.21% | <0.0001 |
| Marital status | |||
| Married | 55.19% | 39.86% | <0.0001 |
| Divorced | 6.43% | 6.17% | 0.5808 |
| Separated | 0.63% | 1.06% | 0.0273 |
| Single | 31.33% | 49.36% | <0.0001 |
| Widowed | 6.02% | 3.32% | <0.0001 |
| Life Partner | 0.38% | 0.24% | 0.1612 |
| Insurance status | |||
| Medicare | 34.14% | 17.93% | <0.0001 |
| Medicaid | 8.18% | 17.79% | <0.0001 |
| Veteran’s Affairs | 0.65% | 0.48% | 0.1663 |
| Tricare | 0.68% | 0.53% | 0.2463 |
| Commercial | 11.63% | 13.57% | 0.0016 |
| Workman’s Compensation | 6.39% | 7.19% | 0.0882 |
| Blue Cross Blue Shield | 38.15% | 41.61% | 0.0001 |
| Self-Pay | 0.12% | 0.81% | <0.0001 |
Multivariate analysis was performed using the Wald chi-square test. 3 survey Age, gender, race, marital 4 survey status, and payor status were responses found to be independent responses variables (p<0.01).
Several parameters 1% regarding the 0% returned surveys were also analyzed. 2 survey Over the course of the study period, 34.3% of patients responses generated one encounter in the outpatient clinic, 19.69% % generated two encounters, 46.1% generated three or more encounters (range: 1 encounter to 40 encounters). 3041 patients, regardless of number of encounters (1-38) returned only 1 survey (90.4% of surveys); however, 2217 patients (72.9%) generated more than a single encounter (Figure 1A and 1B).
Figure 1.
A) Number of encounters per patient over 18 month period. Over one third of patients generated one encounter, however 25% of patients generate more than five encounters. B) Number of Surveys Returned Per Patient over 18 Month Period. The majority of patients (90%) return only one survey, independent of number of outpatient visits the patient attended.
In the 18 month cohort, 3.5% of outpatient encounters generated PG survey data. Because patients frequently attend multiple visits, these data represent 9.1% of all patients receiving medical care in the outpatient setting. Per specialty, arthroplasty and spine patients were more likely to respond to a survey, whereas pediatric patients were less likely to generate survey data (Figure 2). Across specialties, respondents were statistically different with regards to age, in that respondents were about 10 years older than the overall population. Respondents generally differed with regard to payor status, with Medicaid tending to be under-represented and Medicare tending to the over-represented. On the other hand, Worker’s Compensation respondents were statistically similar to non-respondents in gender, race, language, or marital status (Table 2).
Figure 2.
Percentage of patients who responded to survey by specialty.
Table 2.
Comparison of Respondents to all Other Patients by Specialty During an 18-Month Time Period
| Age | Gender | Medicare | Medicaid | Race (white) | Primary Language (English) | Marital status (Married) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| All Other Patients | Respondents | All Other Patients | Respondents | All Other Patients | Respondents | All Other Patients | Respondents | All Other Patients | Respondents | All Other Patients | Respondents | All Other Patients | Respondents | |
| PMR | 48.4 | 53.2 (p-0.01) | 56.70% | 61.6% (p=0.4) | 23.60% | 31.5% (p=0.12) | 27.00% | 17.8% (p=0.08) | 88.10% | 91.8% (p=0.02) | 9620% | 100% (p-0.11) | 49.70% | 56.1% (p=0.32) |
| Tumor | 44.6 | 55.5 (pcO.Ol) | 48.80% | 51.6% (p=0.52) | 24.90% | 42.8% (pcO.Ol) | 15.00% | 6.92% (P=0.01) | 89.60% | 96.7% (pcO.Ol) | 97.40% | 99.4% (P-0.17) | 46.50% | 56.2% (p=0.04) |
| Foot and Ankle | 50.7 | 58.0 (pcO.Ol) | 58.70% | 58.5% (p=0.97) | 24.10% | 32.5% (pcO.Ol) | 16.30% | 8.9% (pcO.Ol) | 87.50% | 95.5% (pcO.Ol) | 98.10% | 99.2% (p=0.31) | 51.80% | 61.0% (p-0.01) |
| Trauma | 40.8 | 53.8 (p<0.01) | 46.20% | 51.8% (p=0.09) | 22.20% | 34.0% (pcO.Ol) | 23.10% | 11.1% (pcO.Ol) | 8520% | 94.1% (pcO.Ol) | 96.80% | 98.0% (p=0.29) | 33.30% | 48.2% (pcO.Ol) |
| Shoulder | 59.2 | 65.5 (p<0.01) | 43.70% | 48.2% (p-0.39) | 42.40% | 56.5% (p-0.01) | 11.10% | 2.8% (PcO.Ol) | 92.70% | 98.2% (p-0.03) | 98.60% | 100.0% (p-0.38) | 55.60% | 64% (P-0.11) |
| Pediatrics | 11.6 | 19.6 (pcO.Ol) | 56.10% | 55.8% (p=0.93) | 1.70% | 2.9% (p=0.15) | 30.10% | 18.3% (pcO.Ol) | 84.30% | 91.6% (pcO.Ol) | 97.20% | 99.3% (p=0.04) | 2.50% | 4.4% (p=0.09) |
| Sports | 36.1 | 45.0 (pcO.Ol) | 50.60% | 58.7% (pcO.Ol) | 8.50% | 18.2% (pcO.Ol) | 14.10% | 7.7% (cO.Ol) | 84.90% | 94.0% (pcO.Ol) | 97.90% | 99.2% (p=0.01) | 38.50% | 54.5% (pcO.Ol) |
| Spine | 53.9 | 64.8 (pcO.Ol) | 49.20% | 51.6% (p-0.42) | 35.70% | 56.7% (pcO.Ol) | 16.70% | 4.9% (pcO.Ol) | 92.30% | 96.0% (p-0.03) | 98.40% | 98.6% (p-0.83) | 55.50% | 63.4% (p-0.01) |
| Hand | 40.4 | 52.1 (pcO.Ol) | 49.20% | 61.3% (pcO.Ol) | 15.40% | 25.3% (pcO.Ol) | 21.90% | 9.8% (pcO.Ol) | 84.10% | 94.6% (pcO.Ol) | 96.90% | 98.8% (p=0.05) | 40.50% | 55.6% (pcO.Ol) |
| Arthroplasty | 55.8 | 58.5 (pcO.Ol) | 55.80% | 58.5% (p=0.18) | 45.30% | 58.8% (pcO.Ol) | 12.60% | 5.8% (pcO.Ol) | 90.60% | 95.9% (pcO.Ol) | 98.10% | 99.3% (p=0.03) | 57.50% | 67.4% (pcO.Ol) |
| Worker's Compensation | 47.3 | 51.1 (pcO.Ol) | 40.90% | 40.5 (P-0.92) | ** | ** | ** | ** | 8220% | 84.7% (P-0.5) | 95.70% | 96.9% (p-0.50) | 28.70% | 32.8% (p-0.31) |
Analysis of patients administered a survey from 90 day cohort
In order to evaluate patients included for survey administration, data was prospectively collected over a 90-day period. During 11/13/2017 to 2/16/2018, 10,421 patients generated 14,544 encounters. Due to exclusion criteria, 7,741 patients were not eligible to receive a survey Respondents (11,850 encounters). Thus, 81.5% of patient encounters were All Other excluded Patients from sampling, representing 74.3% of the patient population. Patients were excluded due to address error (2100 patients, 27.1%), duplicate visit within 90 days (5629 patients, 72.7%) or other exclusion criteria met (12 patients, 0.1%). There were no changes in the exclusion criteria during either time period studied.
To evaluate for selection bias, the demographics of all clinic patients were compared to the demographics of patients sent a survey. Patients significantly differed by age and gender. Those sent a survey were significantly younger and less likely to identify as female. They also were less likely to have Medicare. Patient demographics did not significantly differ with regards to race, marital status, or primary language (Table 3A). Furthermore, the rate of survey administration varied by department, ranging from 12.1% in trauma patients (99/820 patients) to 34.5% of hand patients (464/1347 patients).
Table 3A-C.
Demographic Characteristics of Patients Administered and Respondents During a 90-Day Period. A) Comparison of Patients who were Administered a Survey Versus Patients who were not Administered a Survey. B) Comparison of Respondents Versus Non-Respondents of the Administered Survey. C) Comparison of Respondents to all Other Patients.
| A | No survey administered | Survey administered | p-value |
|---|---|---|---|
| Age (years) | 44.6 (± 22.7) | 41.9 (±23.1) | <0.0001 |
| Gender (% female) | 52.3% | 49.7% | 0.023 |
| Payor | |||
| Medicare | 24.0% | 20.9% | 0.0006 |
| Medicaid | 16.9% | 18.4% | 0.0608 |
| Worker’s Compensation | 7.9% | 8.7% | 0.124 |
| Marital status (married) | 38.8% | 37.8% | 0.6445 |
| English native speaker | 98.0% | 98.1% | 0.6757 |
| White race | 87.4% | 87.4% | 0.1228 |
| B | Respondents | Non-respondents | p-value |
|---|---|---|---|
| Age (years) | 53.0 (± 22.2) | 39.8 (± 22.7) | <0.0001 |
| Gender (% female) | 56.0% | 48.5% | 0.0046 |
| Payor | |||
| Medicare | 33.3% | 18.5% | <0.0001 |
| Medicaid | 10.9% | 19.9% | 0.0003 |
| Worker’s Compensation | 5.7% | 9.4% | 0.0143 |
| Marital status (married) | 57.1% | 35.5% | <0.0001 |
| English native speaker | 99.8% | 97.7% | 0.0044 |
| White race | 95.5% | 87.7% | <0.0001 |
| C | Respondents | All Other Patients | p-value |
|---|---|---|---|
| Age (years) | 53.0 (± 22.2) | 44 (± 22.8) | <0.0001 |
| Gender (% female) | 56.0% | 52.0% | 0.1042 |
| Payor | |||
| Medicare | 33.3% | 23.7% | <0.0001 |
| Medicaid | 10.9% | 17.7% | 0.0003 |
| Worker’s Compensation | 5.7% | 6.7% | 0.3998 |
| Marital status (married) | 57.1% | 44.7% | <0.0001 |
| English native speaker | 99.8% | 97.8% | 0.006 |
| White race | 95.5% | 88.6% | <0.0001 |
To evaluate for non-response bias, the demographics of patients who responded to the survey and those who did not respond were similarly compared. Respondents were significantly older, more likely to identify as female, more likely to have Medicare, less likely to have Medicaid, more likely to be married, speak English as a native language and identify as White (Table 3B).
Similar to the 18-month cohort, patients in the 90 day cohort who responded to the survey versus patients for whom survey data was not available (all other patients) were significantly different with regards to multiple demographic characteristics queried. (Table 3C).
Determination of response rate
In the prospective 90 day cohort from 11/13/17 to 2/16/18, 2693 surveys were sent out of 10,426 patients (14,544 encounters), thus 25.8% of patients (18.5% of patient encounters) were surveyed. 425 surveys were returned, yielding a response rate of approximately 15.8%.
Discussion
The purpose of this study was to describe the methodology used by our institution to obtain patient satisfaction scores in the outpatient setting. This investigation was part of a quality improvement effort to identify ways to improve reporting rates as well as provide contextual information for patient satisfaction surveys. Our objective was to quantify, characterize, and compare the overall outpatient population, patients who were surveyed, and patient respondents to the survey.
From the 18 month cohort, we determined that the PG survey data at our institution were based on responses from 9.1% of the total patients seen on an outpatient basis. Since some patients had multiple visits, the survey data represented 3.5% of total clinical encounters during the study interval. The demographics of the responding population differed significantly from the population as a whole with regards to age, gender, payor status (Medicare and Medicaid), marital status, race, and native language. These differences were observed across all orthopaedic subspecialties. We had expected that some orthopedic specialties, such as joint arthroplasty, would be more resistant to demographic differences between the entire population and survey responders due to a relative homogeneity of the specialty population. However, we observed statistical differences in the demographics of respondents within each orthopedic specialty as well as when compared with the overall orthopedic population. Of all the orthopaedic patient populations, only Worker’s Compensation patients showed no statistical differences in demographics between survey respondents and the whole population with the exception of age. This may be due to Worker’s Compensation populations being more homogenous demographically, or perhaps due to a higher response rate (32.0% response rate).
Interestingly, PG does not retain a record of patients administered beyond 90 days. Thus, the total number of surveys administered during the 18 month study period was unknown. To determine the response rate, we prospectively collected data, including who was sent a survey, from a separate 90 day cohort of patients. The response rate (number of surveys returned/number of surveys administered) was 15.8%.
Given this response rate and the demographic differences between the respondent and non-respondent population, the possibility of nonresponse bias undermining survey validity is concerning. Although in survey based research there is no minimum acceptable response rate, the effects of survey non-response bias can be mitigated by increasing the response rate.6,12 Reasons why patients respond or do not respond to satisfaction surveys are speculative and likely multifactorial.6,9,13,14 The Journal of the American Medical Association editorial policy suggests that survey studies need a response rate of at least 60% to be considered for publication, and references the metrics proposed by the American Association for Public Opinion Research.15 The Office of Management and Budget, who reviews most survey protocols funded by the federal government, asks surveys being reviewed to “submit a plan for a non-response bias analysis if the expected unit response rate is below 80%.”16
In planning the survey with PG, institutions may customize their exclusion criteria. An individual may be categorically excluded from participating in the survey (for example, an institution may choose to exclude Worker’s compensation patients). Within our department, a patient sent a survey is excluded from being sent another survey for 90 days. In many clinical scenarios, frequent clinic visits are required and/ or a single patient requires multidisciplinary care. In the current study, 3041 patients, regardless of number of encounters (1-38) returned only 1 questionnaire (90.4%); however, 2217 patients (72.9%) had more than a single encounter. In our practice, a patient was most likely to be excluded due to duplication (they had attended another clinical visit within 90 days). Duplicates represented 54.0% of excluded patients in the time period. While the impact this has on PG scores for an individual provider is speculative, this practice likely influences survey data by altering the representation of high- versus low-health care utilization patients. Complex orthopedic patients requiring frequent evaluation and resource utilization may have the same survey representation as the patient seen once with a condition not warranting additional orthopaedic intervention.
Previously, non-modifiable patient characteristics such as distance traveled to facility, age and gender have demonstrated correlation with satisfaction ratings.17,18,19 Analysis of our excluded patients suggests that the population of patients selected for administration of a survey is different from the population on the whole with regards to age and gender, as well as proportion of Medicare patients. Selecting for a population that is significantly different from the overall population likely introduces bias into survey results. Furthermore, the effect of nonresponse bias discussed previously in the literature is likely underestimated because respondents are compared with the entire patient population instead of the patients administered a survey.20
In a similar study of PG respondents at an academic orthopedic university affiliated system, Tyser et al. found a response rate of 16.5% to the Press Ganey survey,21 although it is unclear if the 16.5% is of the total population, total number of patient encounters, or the population administered a survey. These authors raised the issue of non-response bias given this low response rate. In a study from the same institution, Abtahi et al. identified age, sex, insurance type, and orthopaedic subspecialty as variables that affected the odds of responding to the survey.8 Neither study addressed the potential of selection bias due to exclusion criteria applied to the population of patients generating an encounter; in this study, we found almost 75% of patients are not eligible to receive a survey. During our investigation of PG survey methodology, we found that the exclusion criteria amongst institutions are variable and customizable. Thus, respondent data cannot be generalized between institutions without a high level of transparency and understanding of each institutional contract.
In conclusion, our investigation revealed a low response rate to the PG survey with significant demographic differences between the outpatient orthopedic patient population, patients who were administered a survey and patients who responded. Although there was evidence of selection bias, non-response bias was more consistently evident across a variety of patient characteristics. This suggests non-representative sampling which may be the product of methodological shortcomings, and may explain the lack of correlation the PG score has with other outcome metrics. Although surveys administered to patients by PG are not required to meet the strict standards of survey science, the data generated are being used to compare physician and hospital performance by both consumers and third-party payors. It is also used within institutions to direct quality improvement measures. Moreover, given the customizable features of PG survey methodology, results cannot necessarily be generalized across institutions, nor can survey data be used for comparison against patient outcomes. Administers of healthcare, healthcare providers, and patients must exercise caution when interpreting and applying these data.
References
- 1.Poolman RW, Swiontkowski MF, Fairbank JC, Schemitsch EH, Sprague S, de Vet HC. Outcome instruments: rationale for their use. The Journal of bone and joint surgery American volume. 2009;91(Suppl 3):41–49. doi: 10.2106/JBJS.H.01551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Shirley ED, Sanders JO. Measuring Quality of Care with Patient Satisfaction Scores. The Journal of bone and joint surgery American volume. 2016;98(19):e83. doi: 10.2106/JBJS.15.01216. [DOI] [PubMed] [Google Scholar]
- 3.Manary MP, Boulding W, Staelin R, Glickman SW. The patient experience and health outcomes. The New England journal of medicine. 2013;368(3):201–203. doi: 10.1056/NEJMp1211775. [DOI] [PubMed] [Google Scholar]
- 4.Fenton JJ, Jerant AF, Bertakis KD, Franks P. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Archives of internal medicine. 2012;172(5):405–411. doi: 10.1001/archinternmed.2011.1662. [DOI] [PubMed] [Google Scholar]
- 5.Shirley ED, Sanders JO. Patient satisfaction: Implications and predictors of success. The Journal of bone and joint surgery American volume. 2013;95(10):e69. doi: 10.2106/JBJS.L.01048. [DOI] [PubMed] [Google Scholar]
- 6.Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the Consumer Assessment of Health Plans Study survey. Medical care. 2002;40(6):485–499. doi: 10.1097/00005650-200206000-00006. [DOI] [PubMed] [Google Scholar]
- 7.Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. Journal of clinical epidemiology. 1997;50(10):1123–1128. doi: 10.1016/s0895-4356(97)00166-2. [DOI] [PubMed] [Google Scholar]
- 8.Abtahi AM, Presson AP, Zhang C, Saltzman CL, Tyser AR. Association Between Orthopaedic Outpatient Satisfaction and Non-Modifiable Patient Factors. The Journal of bone and joint surgery American volume. 2015;97(13):1041–1048. doi: 10.2106/JBJS.N.00950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kim J, Lonner JH, Nelson CL, Lotke PA. Response bias: effect on outcomes evaluation by mail surveys after total knee arthroplasty. The Journal of bone and joint surgery American volume. 2004;86-a(1):15–21. [PubMed] [Google Scholar]
- 10.Johnson TP, Wislar JS. Response rates and nonresponse errors in surveys. Jama. 2012;307(17):1805–1806. doi: 10.1001/jama.2012.3532. [DOI] [PubMed] [Google Scholar]
- 11.Groves RM. Survey Nonresponse. New York: John Viley & Sons; 2001. [Google Scholar]
- 12.Sprague S, Quigley L, Bhandari M. Survey design in orthopaedic surgery: getting surgeons to respond. J Bone Joint Surg Am. 2009;91(Suppl 3):27–34. doi: 10.2106/JBJS.H.01574. [DOI] [PubMed] [Google Scholar]
- 13.Edwards P, Roberts I, Clarke M, et al. Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007. p. MR000008. [DOI] [PubMed]
- 14.Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009. p. MR000008. [DOI] [PMC free article] [PubMed]
- 15.Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 8:http://www.aapor.org/Publications-Media/AAPOR-Journals/Standard-Definitions.aspx. Accessed 9/23/2016, 2015.
- 16.Thompson JFLaKJ. Conducting Nonresponse Biase Analysis for Business Surveys. 2010. https://ww2.amstat.org/sections/srms/Pro-ceedings/y2010/Files/306113_55883.pdf. Accessed 1/1/2017, 2017.
- 17.Rogers F, Horst M, To T, et al. Factors associated with patient satisfaction scores for physician care in trauma patients. The journal of trauma and acute care surgery. 2013;75(1):110–114. doi: 10.1097/TA.0b013e318298484f. discussion 114-115. [DOI] [PubMed] [Google Scholar]
- 18.Bae JY, Kim JK, Yoon JO, Kim JH, Ho BC. Preoperative predictors of patient satisfaction after carpal tunnel release. Orthopaedics & traumatology, surgery & research : OTSR. 2018. [DOI] [PubMed]
- 19.Delanois RE, Gwam CU, Mistry JB, et al. Does gender influence how patients rate their patient experience after total hip arthroplasty? Hip international : the journal of clinical and experimental research on hip pathology and therapy. 2018;28(1):40–43. doi: 10.5301/hipint.5000510. [DOI] [PubMed] [Google Scholar]
- 20.Westreich D. Berkson’s bias, selection bias, and missing data. Epidemiology (Cambridge, Mass) 2012;23(1):159–164. doi: 10.1097/EDE.0b013e31823b6296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Tyser AR, Abtahi AM, McFadden M, Presson AP. Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC health services research. 2016;16(a):350. doi: 10.1186/s12913-016-1595-z. [DOI] [PMC free article] [PubMed] [Google Scholar]