Socioeconomic deprivation and age are barriers to the online collection of patient reported outcome measures in orthopaedic patients

Abstract

Introduction

Questionnaires are used commonly to assess functional outcome and satisfaction in surgical patients. Although these have in the past been administered through written forms, there is increasing interest in the use of new technology to improve the efficiency of collection. The aim of this study was to assess the availability of internet access for a group of orthopaedic patients and the acceptability of online survey completion.

Methods

A total of 497 patients attending orthopaedic outpatient clinics were surveyed to assess access to the internet and their preferred means for completing follow-up questionnaires.

Results

Overall, 358 patients (72%) reported having internet access. Lack of access was associated with socioeconomic deprivation and older age. Multivariable regression confirmed increased age and greater deprivation to be independently associated with lack of internet access. Out of the total group, 198 (40%) indicated a preference for assessment of outcomes via email and the internet.

Conclusions

Internet access was not universal among the patients in our orthopaedic clinic. Reliance on internet collection of PROMs may introduce bias by not including results from patients in older age groups and those from the more deprived socioeconomic groups.

The assessment of outcome following orthopaedic procedures is vital for effective clinical governance, audit and research. Historically, function was judged using factors such as range of movement, complications and implant survival. Patient reported outcome measures (PROMs) have been used increasingly to complement these traditional measures or, in some cases, completely replace them. ¹ There has also been a growing emphasis on the evaluation of patient satisfaction, and an understanding of the relationship between expectation, function, psychological factors and outcome.

PROMs are usually collected through questionnaires.² The defining quality of a PROM is that it is a self-report from the patient. A PROM is designed to measure one or several characteristics, termed constructs. PROMs fall broadly into two categories: disease specific (tailored to the symptoms and functional impact of a specific condition) and generic (considering general aspects such as mobility and self-care). The PROMs approach to evaluating healthcare outcomes was adopted by the National Health Service (NHS) in 2009.³ Reports are released annually detailing the outcomes from hip and knee arthroplasty, along with groin hernia and varicose vein surgery. PROMs can drive change in healthcare delivery and underpin the development of evidence-based, cost effective treatments.⁴

There are, however, several barriers to the collection of PROMs.⁵ Non-response to PROMs can introduce the possibility of bias.⁶ PROMs are generally completed during a physical clinic attendance or via postal questionnaires. The increasing use of the internet has opened a further route of PROMs completion and several online systems are available to collect PROMs routinely.^7,8 Several orthopaedic specific systems are now marketed, such as Amplitude™ (http://www.amplitude-clinical.com/) and the Surgical Outcomes System™ (http://www.surgicaloutcomesystem.com/). Access to the internet may not be uniform throughout the orthopaedic population,⁹ and a reliance on this route of PROMs completion may also disadvantage certain population groups, possibly introducing bias.

Our institution is located in a city with the lowest internet access in Great Britain (57% in 2013).¹⁰ It has been suggested that this low uptake may be due in part to the socioeconomic profile. A previous study has demonstrated a lower response to emailed surveys¹¹ although internet access has increased greatly over the last decade, with growing acceptance of online completion of PROMs.⁸ Other routes of electronic collection have also been investigated.¹² The aim of our study was to describe the availability of internet access among orthopaedic patients and identify factors associated with limited access.

Methods

The study was performed over a two-week period in January 2013 in the orthopaedic outpatient department. Our institution is a metropolitan provider of secondary and tertiary orthopaedic care to a population of approximately 300,000. The clinics comprised general and specialist elective orthopaedic clinics, along with fracture and preoperative assessment clinics.

Patients were surveyed while waiting for their appointment. The survey was performed in a face-to-face manner, by one interviewer (SS). Patients could decline to participate. The data collected were sex, age, occupation, postcode, internet usage, method of internet access, access to an email account and frequency of email use. Patients who had access to an email account were asked their preferred method of completing PROMs from among three options: 1) attending a clinic to fill in a form, 2) having a paper form sent to their residence along with a prepaid envelope or 3) having an email sent to them with instructions for completing the form online.

Socioeconomic deprivation was measured using the Scottish Index of Multiple Deprivation (SIMD).¹³ This national tool has been recommended for use in Scotland since 2004. It can be used to identify areas suffering from deprivation. Scotland is divided into 6,505 datazones and individual patients were mapped to these datazones using their postcode.¹⁴ The SIMD 2012 dataset was used and this measures deprivation, providing a single index based on employment, income, health, education, geographic access to services, crime and housing. The datazones are ranked in order of deprivation and quintiles formed at a national level. Each patient was identified as coming from a datazone from one of these five deprivation quintiles.

Categorical data (internet access, age group, PROMs preference, SIMD group) were analysed with chi-squared tests. Age was normally distributed. The association of age and socioeconomic deprivation was assessed using analysis of variance (ANOVA). The independence of the association of age and deprivation with internet access was assessed using multivariate binary logistic regression. Variables predictive on univariate analysis (p<0.10) were entered into the regression equation. These were all entered in one step and no variable selection algorithm was used. A p-value of <0.05 was considered statistically significant and two-tailed tests were used throughout. Statistical tests were performed using Prism^® version 6 (GraphPad Software, La Jolla, CA, US) and SPSS^® version 20 (IBM, New York, US).

Results

A total of 497 patients (216 men, 281 women) were surveyed. The mean age was 53.6 years (standard deviation: 17.3 years, range: 16–89 years). The majority of patients were in the most deprived social quintile (Table 1). There was no variation of age with deprivation quintile (p=0.204, ANOVA).

Table 1.

Internet access across different demographic groups

	Access to internet	No access to internet	p-value
Sex			0.196
Male	162 (75%)	54 (25%)
Female	196 (70%)	85 (30%)
Socioeconomic deprivation (SIMD group)			<0.001
1 (most deprived)	131 (63.9%)	74 (36.1%)
2	53 (67.1%)	26 (32.9%)
3	47 (73.4%)	17 (26.6%)
4	53 (80.3%)	13 (19.7%)
5 (least deprived)	74 (89.2%)	9 (10.8%)
Age group			<0.001
<20 years	9 (100%)	0 (0%)
20–29 years	57 (100%)	0 (0%)
30–39 years	35 (92.1%)	3 (7.9%)
40–49 years	75 (90.4%)	8 (9.6%)
50–59 years	85 (78.7%)	23 (21.3%)
60–69 years	63 (57.8%)	46 (42.2%)
70–79 years	28 (41.8%)	39 (58.2%)
80–89 years	6 (23.1%)	20 (76.9%)
Occupation			<0.001
Manual	83 (87.4%)	12 (12.6%)
Office based	100 (97.1%)	3 (2.9%)
Retired	88 (47.6%)	97 (52.4%)
Sick leave	30 (73.2%)	11 (26.8%)
Student	14 (100%)	0 (0%)
Unemployed	43 (72.9%)	17 (27.1%)

SIMD = Scottish Index of Multiple Deprivation

Overall, 358 patients (72%) reported having access to the internet. There was a significant association between internet access and socioeconomic deprivation, with 64% of patients in the most deprived quintile reporting access compared with 89% in the least deprived quintile (odds ratio: 0.22, 95% confidence interval: 0.10–0.44, p<0.001, chi-squared test) (Table 1, Fig 1). There was also an association with age: all patients younger than 30 years had internet access but the proportion fell to 42% for 70–79-year-olds and 23% for 80–89-year-olds (p<0.001, chi-squared test) (Fig 2). The associations of age and deprivation with internet access were independent, after adjustment using multivariable regression (Table 2).

Figure 1.

The association between socioeconomic deprivation and access to the internet SIMD = Scottish Index of Multiple Deprivation

Figure 2.

The association between age group and access to the internet

Table 2.

Multivariable logistic regression with ‘access to internet’ as the dependent variable. The effect of age is controlled for and independently predictive (Nagelkerke R²=0.431).

Variable	OR (95% CI)	p-value
Age (per year)	0.91 (0.88–0.92)	<0.001
SIMD group
1 (most deprived)	Reference category
2	2.12 (1.05–4.27)	0.035
3	3.11 (1.46–6.64)	0.003
4	3.80 (1.68–8.57)	0.001
5 (least deprived)	12.20 (5.12–29.20)	<0.001

OR = odds ratio; CI = confidence interval; SIMD = Scottish Index of Multiple Deprivation

A significant association was also seen with occupation. Among students, 100% reported having access to the internet. For respondents on sick leave, the proportion was 73% and 48% of retired respondents had internet access. A variety of devices and combinations were used to access the internet (Fig 3).

Figure 3.

Methods used for internet access

The majority of patients with internet access had an email address (n=295, 82%). Of these, 169 (57%) checked their email every day, 52 (18%) every few days, 19 (6%) once a week and 55 (19%) less often.

Of the patients with access to email, 198 (67%) would be happy to complete PROMs online while 79 (27%) would prefer a postal questionnaire and 18 (6%) would prefer face-to-face evaluation. Consequently, out of the original study group of 497 participants, 198 (40%) would wish (or be able) to complete PROMs online.

Discussion

This study has demonstrated that access to the internet is not universal among orthopaedic patients at our institution. Patients who were older, more deprived and retired were less likely to report having access. Those aged under 30 years and students reported higher levels of access. The overall proportion of patients who would wish to complete PROMs online (following an email invite) was 40%.

In 2004, among orthopaedic patients, the response rate was lower to an email questionnaire than to a postal questionnaire.¹¹ Non-response to PROMs has been linked to male sex, younger age, ethnic minority status, socioeconomic deprivation, need for assistance in completing PROMs and co-morbidities.¹⁵ Non-response could bias outcomes from clinical audit and research.¹⁶

For lower limb arthroplasty patients, there has been shown to be a decay in response rates over time, with postal response rates reported at 99% preoperatively, and 90% at six weeks, 89% at six months, 83% at one year and 79% at two years.⁶ In the same study, following total hip replacement, non-response was associated with lower satisfaction, preoperative Oxford hip score and EQ-5D™ (EuroQol, Rotterdam, Netherlands) score. This study also demonstrated a demographic gradient in response, with the poorest response rate for male, younger, more deprived patients. Socioeconomic deprivation can also specifically adversely affect reported outcomes following orthopaedic surgery as it is associated with poorer preoperative health and disease severity.¹⁷

A 2013 study of internet completed PROMs reported only achieving data completion in 21 out of 61 individuals invited to participate.⁸ In 2002, a larger study of orthopaedic patients’ attitudes to the internet in London reported internet availability for 55% of respondents.⁹ Availability and willingness to use the internet correlated with age in these studies. A report on UK orthopaedic practice published in 2014 showed that a strategy combining reminders with using ‘tablet’ computers for the completion of PROMs in clinic achieved a 94% completion rate for preoperative PROMs.¹² It is not clear how this would translate to postoperative collection. A study in America demonstrated a poorer response rate for patients in the Medicare or Medicaid schemes, those with total knee replacement or revision arthroplasty, those aged over 75 years and those with ethnic minority status.⁵ The authors reported increased response rates during their study when electronic collection was introduced.

The collection of PROMs data is a part of current orthopaedic clinical practice, and the use of PROMs is likely to increase further with the establishment of more non-arthroplasty registers and the requirement of outcome data for revalidation of surgeons. The British Orthopaedic Association advocates long-term follow-up after joint replacement but traditional clinical review is not cost effective and new surveillance methods are urgently required. Moving away from traditional pen-and-paper methods to a computerised system of completing PROMs has many methodological benefits: entries would never be incomplete, scores would be computed automatically and accurately, and data would be entered correctly into a database. This negates the problems of incomplete questionnaires, errors during manual scoring of responses and mistakes in the entry of responses into a computer database.

The main strength of this study is the large number of patients surveyed over the study period. The deprivation profile is similar to that in a previous report from Glasgow.¹⁸ It is therefore likely that we have achieved a representative sample of our population. Our institution serves an area of high deprivation and as a result, this study has a greater power to detect an effect of deprivation on internet access.

A limitation of this study is the sampling methodology. As patients were approached randomly, there may be a combination of selection bias from the investigator and participation bias from the participant. There is a possibility that individuals who declined to participate possessed different access profiles and preferences.

Conclusions

This study presents data that will assist orthopaedic departments planning to collect PROMs. A strategy that uses just one method of collection is likely to be less successful and increase the potential for response bias. Reliance on online collection may disadvantage those from deprived areas and those in older age groups. Such an effect would increase inequality and would be undesirable. A multimedia strategy is therefore recommended, where patient communication preferences are recorded at the time of treatment and a variety of techniques are deployed to collect subsequent data.