Abstract
Objectives
Electronic forms of data collection have gained interest in recent years. In orthopaedics, little is known about patient preference regarding pen-and-paper or electronic questionnaires. We aimed to determine whether patients undergoing total hip (THR) or total knee replacement (TKR) prefer pen-and-paper or electronic questionnaires and to identify variables that predict preference for electronic questionnaires.
Methods
We asked patients who participated in a multi-centre cohort study investigating improvement in health-related quality of life (HRQoL) after THR and TKR using pen-and-paper questionnaires, which mode of questionnaire they preferred. Patient age, gender, highest completed level of schooling, body mass index (BMI), comorbidities, indication for joint replacement and pre-operative HRQoL were compared between the groups preferring different modes of questionnaire. We then performed logistic regression analyses to investigate which variables independently predicted preference of electronic questionnaires.
Results
A total of 565 THR patients and 387 TKR patients completed the preference question. Of the THR patients, 81.8% (95% confidence interval (CI) 78.4 to 84.7) preferred pen-and-paper questionnaires to electronic questionnaires, as did 86.8% (95% CI 83.1 to 89.8) of TKR patients. Younger age, male gender, higher completed level of schooling and higher BMI independently predicted preference of electronic questionnaires in THR patients. Younger age and higher completed level of schooling independently predicted preference of electronic questionnaires in TKR patients.
Conclusions
The majority of THR and TKR patients prefer pen-and-paper questionnaires. Patients who preferred electronic questionnaires differed from patients who preferred pen-and-paper questionnaires. Restricting the mode of patient-reported outcome measures to electronic questionnaires might introduce selection bias.
Cite this article: Bone Joint Res 2013;2:238–44.
Keywords: Health-related quality of life, Total hip replacement, Total knee replacement, Patient-reported outcome measure, PROM, Questionnaire mode
Article focus
Based on the recent literature, we hypothesised that patients would prefer electronic questionnaires to pen-and-paper questionnaires after total hip and total knee replacement
We hypothesised that elderly patients would prefer pen-and-paper questionnaires
Key messages
The vast majority of patients prefer pen-and-paper questionnaires
Patients who prefer electronic questionnaires are generally younger and have completed a higher level of schooling
Strengths and limitations
Strengths: a large sample size and a limited extent of optimism in our statistical models
Limitations: This study was performed using pen-and-paper questionnaires, which might lead to an overestimation of the actual preference of the pen-and-paper questionnaires
Introduction
Traditionally, the assessment of outcome in orthopaedics has focussed on technical aspects. In total hip (THR) or knee replacement (TKR), the cumulative incidence of revision surgery is often used to compare the outcome of different implants or surgical techniques.1 The underlying assumption of the traditional orthopaedic approach is that the technical aspects are the most important determinants of clinical success. However, a technically well-performed joint replacement does not guarantee clinical success, as no information is provided on functional status and pain. Additionally, the indication for revision surgery varies widely between orthopaedic surgeons.2 Patient-reported outcome measures (PROMs), defined as questionnaires that are completed by patients, provide complementary information as they give an impression of a patient’s experience of the surgical procedure and their concerns with regard to health status, health-related quality of life (HRQoL) and the results of the treatment received.3
PROMs can be measured using traditional pen-and-paper questionnaires or various electronic counterparts including touch screens,4 personal digital assistants,5,6 tablets or mobile phones.7 Expected advantages of electronic questionnaires include more complete data capturing, immediate availability of results and lower costs of administrating and entering data.6,8
On the other hand, electronic questionnaires may induce selection bias. A meta-analysis performed in 2008 showed that mail surveys had higher response rates than those based online.9 A recent randomised controlled trial, in which 2400 patients were randomised to receive either a pen-and-paper questionnaire or an internet-based questionnaire at four years after THR, revealed an enormous difference in response rate: 92% for the pen-and-paper group versus 49% for the internet-based group.10 Selection bias can occur if the association between exposure and outcome differs between participants and all eligible patients.8
To our knowledge, no study has investigated patient preference for electric questionnaires after THR and TKR. The majority of members of a senior citizens club prefers electronic to pen-and-paper questionnaires.11 Given the similar age of THR/TKR patients, we would expect a preference for electronic questionnaires. We aimed to estimate the proportion of patients who prefer pen-and-paper questionnaires to electronic questionnaires and to estimate predictors of electronic questionnaire preference.
Materials and Methods
The current study is part of a multi-centre cohort study of HRQoL after THR/TKR (NTR2190), performed from August 2010 to August 2011.12-15 Institutional review board approval was obtained from all participating centres and all patients gave written informed consent (CCMO-Nr:NL29018.058.09;MEC-Nr:P09.189). The data used in this report constitutes a subset of patients who underwent primary THR or TKR and who completed pre-operative HRQoL questionnaires along with a question regarding their preference for a mode of questionnaire at a mean of three years (1.5 to 6) after surgery.
We performed this study in order to investigate the preference for a mode of questionnaire for future studies in HRQoL after THR or TKR in a Dutch population. A prerequisite for such future studies is that patients can participate without outpatient department visits, thereby facilitating participation and forestalling the occurrence of selection bias. We selected a web-based questionnaire as the most feasible electronic option. At follow-up, we asked all THR and TKR patients which mode of questionnaire they preferred: pen-and-paper questionnaires or web-based electronic questionnaires, each completed at home.
In order to judge whether patients who preferred pen-and-paper questionnaires differed from patients who preferred electronic questionnaires, we compared age, gender, highest completed level of schooling, body mass index (BMI) categories (< 25 kg/m2, 25 to 30 kg/m2, 30 to 35 kg/m2, > 35 kg/m2), comorbidity, indication for joint replacement (osteoarthritis vs other indications) and pre-operative HRQoL between both groups.
We have aggregated the levels of schooling into an approximation of the social classes, on the assumption that level of schooling indexes the type of qualifications obtained, which in turn indicates the type of occupations available to the subject and hence their own adult social class.13 Thus: ‘university, higher vocational education and preparatory higher vocational and scientific education’ have been aggregated as indicating the professional and managerial social classes; ‘middle vocational education and preparatory middle vocational education’ have been aggregated as indicating the skilled non-manual and manual social classes; and ‘lower vocational education, elementary schooling and no formal education’ have been aggregated as indicating the semi- and unskilled manual social classes.
Comorbidity was measured using a patient-reported Charnley classification (A, patients in which the index operated hip or knee are affected only; B, patients in which the other hip or knee is affected as well; and C, patients with a hip or knee replacement and other affected joints and/or a medical condition which affects the patients’ ability to ambulate).16,17
HRQoL was measured two weeks before TKR/THR, using the Dutch version of the Short-Form 36 (SF-36).18,19 This questionnaire comprises 36 items covering eight domains (physical function, role physical, bodily pain, general health, vitality, social function, role emotional and mental health), for each of which a subscale score is calculated (100 indicating no symptoms and 0 indicating extreme symptoms). Additionally, these scales are incorporated into two summary measures: a physical component summary (PCS) and a mental component summary (MCS). Missing items were imputed whenever possible according to Ware et al.19 We compared pre-operative PCS and MCS between both preference groups.
Statistical analysis
We performed all analyses separately for THR and TKR patients, as clinically important differences vary considerably between these patient groups.20 We performed descriptive analyses of baseline patient characteristics. In order to predict which factors increased the probability of preference for electronic questionnaires, we performed multivariate mixed model logistic regression analyses. We considered the following potential predictors: age, gender, highest completed level of schooling category, BMI category, Charnley classification of comorbidity, indication for joint replacement and pre-operative PCS and MCS scores. In the mixed model regression analyses, patient preference was the dependent variable, all potential predictors were included as fixed effects and centre was included as a random effect. The explained variation was estimated using Nagelkerke’s generalised r2 and the discriminative ability was estimated using the area under the receiver operating characteristic (ROC) curve (AUC).21 The extent of optimism in the r2 and AUC estimates was estimated using bootstrap resampling (n = 1000 bootstrap samples).22-24 All analyses were performed using R v2.15.2 (R Development Core Team, Vienna, Austria).25
Results
Patient characteristics are shown in Table I. A total of 565 THR patients and 387 TKR patients completed the preference question. Pen-and-paper questionnaires were preferred by 462 THR patients (81.8% (95% confidence interval (CI) 78.4 to 84.7) and by 336 TKR patients (86.8% (95% CI 83.1 to 89.8)) (Table II).
Table I.
Patient characteristics (THR, total hip replacement; TKR, total knee replacement)
| Characteristic | THR (n = 565) | TKR (n = 387) | ||
|---|---|---|---|---|
| Mean (sd) age at THR/TKR (yrs) | 65.9 (10.6) | 68.9 (9.7) | ||
| Male (n, %) | 198 (35.0) | 126 (32.6) | ||
| Osteoarthritis (n, %) | 486 (86.0) | 346 (89.4) | ||
| Mean (sd) pre-operative Short-Form 36 | ||||
| Physical component summary | 38.9 (9.61) | 40.6 (9.53) | ||
| Mental component summary | 51.8 (10.8) | 51.5 (10.2) | ||
| Mean follow-up (yrs) (sd; range) | 3.20 (1.13; 1.5 to 6.0) | 3.14 (1.12; 1.3 to 6.0) | ||
| Body mass index at follow-up (n, %) | ||||
| < 25 kg/m2 | 194 (34.3) | 69 (17.8) | ||
| 25 to 30 kg/m2 | 242 (42.9) | 171 (44.1) | ||
| 30 to 35 kg/m2 | 97 (17.1) | 92 (23.8) | ||
| > 35 kg/m2 | 32 (5.7) | 55 (14.2) | ||
| Charnley comorbidity classification (n, %) | ||||
| A | 123 (23.3) | 54 (14.6) | ||
| B | 75 (14.2) | 39 (10.5) | ||
| C | 331 (62.6) | 278 (74.9) | ||
| Highest completed level of education (n, %) | ||||
| University, higher vocational education and preparatory higher vocational & scientific education | 115 (22.6) | 52 (15.4) | ||
| Middle vocational education and preparatory middle vocational education | 186 (36.6) | 120 (35.5) | ||
| Lower vocational education, elementary schooling and no formal education | 207 (40.7) | 166 (49.1) |
Table II.
Proportion of patients who prefer pen-and-paper questionnaires to electronic questionnaires (THR, total hip replacement; TKR, total knee replacement; CI, confidence interval)
| THR (n = 565) | TKR (n = 387) | ||||
|---|---|---|---|---|---|
| Preferred questionnaire | Patients (n) | Proportion (95% CI) | Patients (n) | Proportion (95% CI) | |
| Pen-and-paper | 462 | 81.8 (78.4 to 84.7) | 336 | 86.8 (83.1 to 89.8) | |
| Electronic | 103 | 18.2 | 51 | 13.2 | |
Patient characteristics per preference group are shown in Table III and Table IV for THR and TKR patients, respectively. THR patients who preferred electronic questionnaires tended to be younger, more often male, more often obese, less comorbid, more often highly educated and had worse pre-operative physical health. Age, gender and highest completed level of education remained associated with mode of questionnaire preference while adjusting for age and gender (Table III). TKR patients who preferred electronic questionnaires were younger, more often male, less often morbidly obese, less often Charnley class B and more often Charnley class C, more often highly educated and had worse pre-operative physical health. Age and highest completed level of education remained associated with mode of questionnaire preference while adjusting for age and gender (Table IV).
Table III.
Comparison of total hip replacement (THR) patients preferring pen-and-paper versus electronic questionnaires (OR, odds ratio; CI, confidence interval)
| Characteristic | Pen-and-paper | Electronic | Age- and gender- adjusted OR (95% CI) | |
|---|---|---|---|---|
| Patients (n) | 462 | 103 | ||
| Mean (sd) age at THR (yrs) | 67.5 (9.5) | 58.5 (12.2) | 0.93 (0.91 to 0.95) | |
| Male (n, %) | 140 (30.3) | 58 (56.3) | 0.35 (0.22 to 0.56) | |
| Osteoarthritis (n, %) | 406 (88.5) | 77 (75.5) | 0.75 (0.40 to 1.43) | |
| Mean (sd) pre-operative Short-Form 36 | ||||
| Physical component summary | 39.5 (8.9) | 35.8 (11.9) | 1.00 (0.97 to 1.03) | |
| Mental component summary | 51.2 (10.6) | 54.8 (10.9) | 1.01 (0.99 to 1.04) | |
| Mean (sd) follow-up (yrs) | 3.17 (1.13) | 3.31 (1.11) | - | |
| Body mass index at follow-up (n, %) | ||||
| < 25 kg/m2 | 157 (35.4) | 29 (29.3) | Reference | |
| 25 to 30 kg/m2 | 187 (42.1) | 46 (46.5) | 1.32 (0.75 to 2.32) | |
| 30 to 35 kg/m2 | 78 (17.6) | 15 (15.2) | 0.88 (0.42 to 1.83) | |
| > 35 kg/m2 | 22 (5.0) | 9 (9.1) | 2.18 (0.84 to 5.69) | |
| Charnley comorbidity classification (n, %) | ||||
| A | 100 (23.2) | 23 (23.5) | Reference | |
| B | 57 (13.2) | 18 (18.4) | 1.28 (0.59 to 2.79) | |
| C | 274 (63.6) | 57 (58.2) | 1.10 (0.61 to 1.98) | |
| Highest completed level of education (n, %) | ||||
| University, higher vocational education and preparatory higher vocational & scientific education | 78 (19.1) | 37 (37.0) | Reference | |
| Middle vocational education and preparatory middle vocational education | 140 (34.3) | 46 (46.0) | 0.82 (0.47 to 1.45) | |
| Lower vocational education, elementary schooling and no formal education | 190 (46.6) | 17 (17.0) | 0.24 (0.12 to 0.47) |
Table IV.
Comparison of total knee replacement (TKR) patients preferring pen-and-paper versus electronic questionnaires (OR, odds ratio; CI, confidence interval)
| Characteristic | Pen-and-paper | Electronic | Age- and gender- adjusted OR (95% CI) | |
|---|---|---|---|---|
| Patients (n) | 336 | 51 | ||
| Mean (sd) age at TKR (yrs) | 70.3 (8.9) | 59.9 (9.7) | 0.90 (0.86 to 0.93) | |
| Male (n, %) | 103 (30.7) | 23 (45.1) | 0.61 (0.31 to 1.18) | |
| Osteoarthritis (n, %) | 302 (90.4) | 40 (81.6) | 1.29 (0.48 to 3.45) | |
| Mean (sd) pre-operative Short-Form 36 | ||||
| Physical component summary | 41.1 (8.8) | 36.7 (12.9) | 1.02 (0.98 to 1.06) | |
| Mental component summary | 51.2 (10.4) | 53.5 (8.9) | 1.02 (0.99 to 1.06) | |
| Mean (sd) follow-up (yrs) | 3.10 (1.09) | 3.44 (1.32) | - | |
| Body mass index at follow-up (n, %) | ||||
| < 25 kg/m2 | 55 (17.5) | 10 (19.6) | Reference | |
| 25 to 30 kg/m2 | 136 (43.3) | 25 (49.0) | 0.86 (0.34 to 2.20) | |
| 30 to 35 kg/m2 | 75 (23.9) | 12 (23.5) | 0.65 (0.23 to 1.83) | |
| > 35 kg/m2 | 48 (15.3) | 4 (7.8) | 0.41 (0.11 to 1.57) | |
| Charnley comorbidity classification (n, %) | ||||
| A | 48 (14.9) | 6 (12.2) | Reference | |
| B | 36 (11.2) | 3 (6.1) | 0.83 (0.18 to 3.83) | |
| C | 238 (73.9) | 40 (81.6) | 1.35 (0.52 to 3.52) | |
| Highest completed level of education (n, %) | ||||
| University, higher vocational education and preparatory higher vocational & scientific education | 34 (11.7) | 18 (37.5) | Reference | |
| Middle vocational education and preparatory middle vocational education | 96 (33.1) | 24 (50.0) | 0.55 (0.24 to 1.26) | |
| Lower vocational education, elementary schooling and no formal education | 160 (55.2) | 6 (12.5) | 0.08 (0.03 to 0.25) |
Multivariate prediction of electronic questionnaire preference showed that lower age (p < 0.001), male gender (p < 0.001), higher completed level of schooling (p < 0.001) and higher BMI (p = 0.004) independently predicted preference of electronic questionnaires in THR patients (Table V). In TKR patients, multivariate prediction of electronic questionnaire preference showed that lower age (p < 0.001) and higher completed level of schooling (p < 0.001) independently predicted preference of electronic questionnaires (Table V). The prediction model for preference of electronic questionnaires in THR patients had an r2 of 0.31 with an optimism estimate of 0.04, yielding an optimism-corrected r2 estimate of 0.27. The AUC was 0.81, with an optimism estimate of -0.02, indicating absence of optimism. The prediction model for preference of electronic questionnaires in TKR patients had an r2 of 0.41 with an optimism estimate of -0.24, indicating absence of optimism. The AUC was 0.88, with an optimism estimate of -0.004, indicating absence of optimism.
Table V.
Multivariate prediction for preference of electronic questionnaires in total hip and knee replacement patients. Odds ratios > 1 indicate higher odds of preferring an electronic questionnaire, per increasing predictor unit (CI, confidence interval; AUC, area under curve)
| Characteristic* | Odds ratio (95% CI) | p-value | |
|---|---|---|---|
| TOTAL HIP REPLACEMENT | |||
| r2 = 0.31; AUC = 0.81 | |||
| Age at operation | 0.93 (0.90 to 0.96) | < 0.001 | |
| Male vs female gender | 0.31 (0.17 to 0.56) | < 0.001 | |
| Body mass index | |||
| 25 to 30 kg/m2 vs < 25 kg/m2 | 2.06 (1.03 to 4.11) | 0.04 | |
| 30 to 35 kg/m2 vs < 25 kg/m2 | 1.17 (0.48 to 2.81) | 0.73 | |
| > 35 kg/m2 vs < 25 kg/m2 | 5.49 (1.74 to 17.3) | 0.004 | |
| Other indications vs osteoarthritis | 0.59 (0.28 to 1.26) | 0.17 | |
| Charnley comorbidity classification | |||
| A vs B | 0.99 (0.40 to 2.42) | 0.98 | |
| A vs C | 0.87 (0.43 to 1.78) | 0.70 | |
| Education | |||
| U+HVE+PHVSE vs MVE+PMVE | 0.89 (0.45 to 1.77) | 0.74 | |
| U+HVE+PHVSE vs LVE+ES+NFE | 0.27 (0.12 to 0.59) | < 0.001 | |
| Short-Form 36 score | |||
| Physical component summary | 1.00 (0.97 to 1.03) | 0.80 | |
| Mental component summary | 1.00 (0.98 to 1.03) | 0.87 | |
| TOTAL KNEE REPLACEMENT | |||
| r2 = 0.41; AUC = 0.88 | |||
| Age at operation | 0.89 (0.84 to 0.94) | < 0.001 | |
| Male vs female gender | 0.53 (0.21 to 1.34) | 0.18 | |
| Body mass index | |||
| 25 to 30 kg/m2 vs < 25 kg/m2 | 1.05 (0.26 to 4.28) | 0.94 | |
| 30 to 35 kg/m2 vs < 25 kg/m2 | 1.27 (0.30 to 5.38) | 0.75 | |
| > 35 kg/m2 vs < 25 kg/m2 | 1.59 (0.28 to 8.91) | 0.60 | |
| Other indications vs osteoarthritis | 2.05 (0.53 to 7.89) | 0.30 | |
| Charnley comorbidity classification | |||
| A vs B | 1.40 (0.23 to 8.58) | 0.72 | |
| A vs C | 2.07 (0.58 to 7.31) | 0.26 | |
| Education | |||
| U+HVE+PHVSE vs MVE+PMVE | 0.33 (0.13 to 0.85) | 0.02 | |
| U+HVE+PHVSE vs LVE+ES+NFE | 0.04 (0.01 to 0.15) | < 0.001 | |
| Short-Form 36 score | |||
| Physical component summary | 1.01 (0.96 to 1.06) | 0.63 | |
| Mental component summary | 0.99 (0.95 to 1.04) | 0.75 | |
* U+HVE+PHVSE, university, higher vocational education and preparatory higher vocational and scientific education; MVE+PMVE, middle vocational education and preparatory middle vocational education; LVE+ES+NFE, lower vocational education, elementary schooling and no formal education
Discussion
The vast majority of THR and TKR patients prefer pen-and-paper questionnaires. THR patients who prefer electronic questionnaires are younger, more often male, have completed higher levels of schooling and are more often obese. TKR patients who prefer electronic questionnaires are younger and have completed higher levels of schooling.
A limitation of our study is the mode of questionnaire used to capture the data. In this study, we invited patients to participate by conventional mail. Additionally, all questionnaires consisted of pen-and-paper questionnaires. Patients willing to participate in this study might be more inclined to prefer pen-and-paper questionnaires than THR and TKR patients in general, thus leading to an overestimation of the proportion of patients preferring pen-and-paper questionnaires. However, we consider it unlikely that the entire preference for pen-and-paper questionnaires is based on such selection bias. Additionally, the identified predictors for electronic questionnaire preference, such as age and completed level of schooling, are plausible, thereby indirectly validating our results.
Strengths of our study include the large sample size, which allows precise estimation and multivariate prediction of patient preference. Although the low r2 values indicate that not all variance is explained by the predictors, the high AUC values indicate that the prediction models have a high discriminatory ability. The limited extent of optimism in r2 and AUC estimates indicate that overfitting did not play a role in our study.24 In other words, it is unlikely that the prediction models in this study have captured the peculiarities in this data set; conversely, it is likely that predictions, based on this data, will be generalisable to other, similar populations.
Unfortunately, we do not have any information on the availability of internet access of our patients. Although The Netherlands is rated as one of the most mature internet markets,26 recent evidence suggests that non-users of the internet are more likely to be elderly,27 which could explain the preference for pen-and-paper questionnaires. Practical advantages of electronic questionnaires are stressed in the current orthopaedic literature.6,28 Patients are sometimes considered to prefer electronic questionnaires without any evidence supporting this claim.28 Although electronic questionnaires certainly appear more efficient, our results reveal limitations in line with the findings of Rolfson et al.10 Future studies, which only measure PROMs using electronic questionnaires, might suffer from limited generalisability, as elderly and less educated patients are less likely to participate. Moreover, selection bias might occur if the association of interest is related to age or social class.
When planning a study in which PROMs will be completed by THR and TKR patients at home, we recommend using pen-and-paper questionnaires, despite their logistic limitations. Such studies should at least provide the option of pen-and-paper questionnaires, in order to prevent selection bias by questionnaire mode.
Funding Statement
Ongoing research grant of the Dutch Arthritis Association to Professor R. G. H. H Nelissen, Department of Orthopaedic Surgery, Leiden University Medical Center: grant number LLP-013 (http://www.reumafonds.nl/informatie-voor-doelgroepen/professionals/ongoing-research). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Footnotes
Author contributions:J. C. Keurentjes: Study conception and design, Data collection, Data analysis, Drafting of the article
M. Fiocco: Study conception and design, Data analysis, Critical revision of the article
C. So-Osman: Study conception and design, Data collection, Drafting of the article, Critical revision of the article
R. Onstenk: Data collection, Drafting of the article, Critical revision of the article
A. W. M. M. Koopman-Van Gemert: Data collection, Drafting of the article, Critical revision of the article
R. G. Pöll: Data collection, Drafting of the article, Critical revision of the article
R. G. H. H. Nelissen: Study conception and design, Data collection, Drafting of the article, Critical revision of the article
ICMJE Conflict of Interest:None declared
References
- 1.Keurentjes JC, Fiocco M, Schreurs BW, et al. Revision surgery is overestimated in hip replacement. Bone Joint Res 2012;1:258–262 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Nelissen RG, Brand R, Rozing PM. Survivorship analysis in total condylar knee arthroplasty: a statistical review. J Bone Joint Surg [Am] 1992;74-A:383–389 [PubMed] [Google Scholar]
- 3.Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technol Assess 1998;2:1–74 [PubMed] [Google Scholar]
- 4.Rolfson O, Kärrholm J, Dahlberg LE, Garellick G. Patient-reported outcomes in the Swedish Hip Arthroplasty Register: results of a nationwide prospective observational study. J Bone Joint Surg [Br] 2011;93-B:867–875 [DOI] [PubMed] [Google Scholar]
- 5.Kvien TK, Mowinckel P, Heiberg T, et al. Performance of health status measures with a pen based personal digital assistant. Ann Rheum Dis 2005;64:1480–1484 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Saleh KJ, Radosevich DM, Kassim RA, et al. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res 2002;20:1146–1151 [DOI] [PubMed] [Google Scholar]
- 7.Bellamy N, Wilson C, Hendrikz J, et al. Osteoarthritis Index delivered by mobile phone (m-WOMAC) is valid, reliable, and responsive. J Clin Epidemiol 2011;64:182–190 [DOI] [PubMed] [Google Scholar]
- 8.van Gelder MMHJ, Pijpe A. E-epidemiology: a comprehensive update. OA Epidemiology 2013;1:5 [Google Scholar]
- 9.Manfreda K, Bosnjak M, Berzelak J, Haas I, Vehovar V. Web surveys versus other survey modes: a meta-analysis comparing response rates. Int J Market Res 2008;50:79–104 [Google Scholar]
- 10.Rolfson O, Salomonsson R, Dahlberg LE, Garellick G. Internet-based follow-up questionnaire for measuring patient-reported outcome after total hip replacement surgery-reliability and response rate. Value Health 2011;14:316–321 [DOI] [PubMed] [Google Scholar]
- 11.Ryan JM, Corry JR, Attewell R, Smithson MJ. A comparison of an electronic version of the SF-36 General Health Questionnaire to the standard paper version. Qual Life Res 2002;11:19–26 [DOI] [PubMed] [Google Scholar]
- 12.Keurentjes JC, Fiocco M, So-Osman C, et al. Patients with severe radiographic osteoarthritis have a better prognosis in physical functioning after hip and knee replacement: a cohort-study. PLoS One 2013;8:59500. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Keurentjes JC, Blane D, Bartley M, et al. Socio-economic position has no effect on improvement in health-related quality of life and patient satisfaction in total hip and knee replacement: a cohort study. PLoS One 2013;8:56785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Keurentjes JC, Fiocco M, Nelissen RG. Willingness to undergo surgery again validated clinically important differences in health-related quality of life after total hip replacement or total knee replacement surgery. J Clin Epidemiol 2013:Epub. [DOI] [PubMed]
- 15.Evangelou E, Kerkhof HJ, Styrkarsdottir U, et al. A meta-analysis of genome-wide association studies identifies novel variants associated with osteoarthritis of the hip. Ann Rheum Dis 2013:Epub. [DOI] [PMC free article] [PubMed]
- 16.Charnley J. The long-term results of low-friction arthroplasty of the hip performed as a primary intervention. J Bone Joint Surg [Br] 1972;54-B:61–76 [PubMed] [Google Scholar]
- 17.Dunbar MJ, Robertsson O, Ryd L. What’s all that noise?: the effect of co-morbidity on health outcome questionnaire results after knee arthroplasty. Acta Orthop Scand 2004;75:119–126 [DOI] [PubMed] [Google Scholar]
- 18.Aaronson NK, Muller M, Cohen PD, et al. Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol 1998;51:1055–1068 [DOI] [PubMed] [Google Scholar]
- 19.Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 health survey: manual and interpretation guide Boston: New England Medical Center, 1993.
- 20.Keurentjes JC, Van Tol FR, Fiocco M, Schoones JW, Nelissen RG. Minimal clinically important differences in health-related quality of life after total hip or knee replacement: a systematic review. Bone Joint Res 2012;1:71–77 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Nagelkerke NJD. A note on a general definition of the coefficient of determination. Biometrika 1991;78:691–692 [Google Scholar]
- 22.Steyerberg EW, Eijkemans MJ, Harrell FE, Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 2000;19:1059–1079 [DOI] [PubMed] [Google Scholar]
- 23.Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol 2003;56:441–447 [DOI] [PubMed] [Google Scholar]
- 24.Steyerberg EW.Clinical prediction models: a practical approach to development, validation and updating New York: Springer, 2009.
- 25.No authors listed.R: a language and environment for statistical computing Vol. 1. Vienna: R Foundation for Statistical Computing, 2008.
- 26.Deutskens E, de Ruyter K, Wetzels M, Oosterveld P. Response rate and response quality of internet-based surveys: an experimental study. Marketing Letters 2004;15:21–36 [Google Scholar]
- 27.Brandtzæg PB, Heim J, Karahasanovic A. Understanding the new digital divide: a typology of internet users in Europe. Int J Human-Computer Studies 2011;69:123–138 [Google Scholar]
- 28.Lee SJ, Kavanaugh A, Lenert L. Electronic and computer-generated patient questionnaires in standard care. Best Pract Res Clin Rheumatol 2007;21:637–647 [DOI] [PubMed] [Google Scholar]