Patient-reported outcome measurements in clinical routine of trauma, spine and craniomaxillofacial surgeons: between expectations and reality: a survey among 1212 surgeons

Abstract

Objective

To gain information about the advantages/disadvantages of an implementation of patient-reported outcome measures (PROM) into the clinical routine of trauma/orthopaedic surgeons, and to identify the technical constraints confronting a successful implementation of PROMs.

Design

Online survey.

Participants

Surgeons who are members of the AO Foundation.

Measures

Participants answered questions regarding demographics, their familiarity with specific and generic PROMs and the use of PROMs in clinical routine. Furthermore, reasons for/against using PROMs, why not used more often, prerequisites to implement PROMs into clinical routine and whether PROMs would be implemented if adequate tools/technologies were available, were solicited. Χ² tests and multivariable logistic regressions were conducted to evaluate the effect of the AO Region, surgeon specialisation, current position, clinical experience, and workplace on the familiarity with disease-specific PROMs, the familiarity with generic PROMs and the current use of PROMs. Exploratory factor analysis was used to identify issues underlying the extent of PROM usage.

Results

1212 surgeons completed the survey (response rate: 6.8%; margin of error: ±2.72%): 54.2% were trauma/orthopaedic surgeons, 16.6% were spine surgeons, 27.9% were craniomaxillofacial surgeons and 16 had no defined specialty. Working in a certain AO Region, surgical specialisation and current workplace were associated with a higher familiarity of disease-specific PROMs and the use of PROMs in daily clinical routine (p≤0.05). Exploratory factor analysis identified four categories important for the use of PROMs and two categories preventing the use of PROMs. In case of the availability of an adequate tool, 66.2% of surgeons would implement PROMs in clinical routine.

Conclusions

Our survey results provide an understanding of the use of PROMs in clinical routine. There is consensus on the usefulness of PROMs. User-friendly and efficient tools/technologies would be a prerequisite for the daily use of PROMs. Additionally, educational efforts and/or policies might help.

Strengths and limitations of this study.

• The sample of participating orthopaedic, trauma, spine and craniomaxillofacial surgeons could represent the general situation regarding the use of PROMs in clinical routine, with a margin of error of 2.72%.

• The large number of participants (1212 surgeons) is a strength of this study. Surgeons’ answers provide an understanding of the familiarity with and the use of PROMs in clinical routine.

• Reasons for the use of PROMs among surgeons already using PROMs and concerns against PROMs among surgeons not using PROMs were investigated separately; explorative factor analysis was applied.

• Limitations are the low response rate of 6.8%, and a risk of selection bias associated with it.

• It was assumed that surgeons interested in the topic responded to the survey. Therefore, the familiarity and current use of PROMs may have been rather overestimated.

Introduction

The collection of patient-reported outcome measures (PROM) is increasingly demanded in many healthcare systems for reasons of quality control, benchmarking and reimbursement.¹ It is proposed that the routine use of PROMs in clinical care could have the potential to help reform healthcare, its structure and its delivery. Healthcare providers could compare the healthcare provided with patient outcomes achieved in any patient population and use the gathered information to adjust their systems. Clinicians could use it for making treatment decisions for their patients.²

Individual physicians and/or hospitals increasingly use PROMs, however, in only a few countries, including England, Sweden and partially the USA PROMs are collected on a regional or national level.² In England, a limited mandatory collection of PROMs was initiated by the government in 2009.^{3 4} In Sweden, the initiative to collect disease-specific PROMs nationwide was clinician driven.²

In 2002, the National Institutes of Health (NIH) in the USA defined the need to develop state-of-the-art PROMs as a high priority scientific project to enhance clinical research in the 21st century.⁵ Starting in 2004, the NIH developed the Patient-Reported Outcomes Measurement Information System (PROMIS) together with an online assessment centre^{1 6} to provide generic measures of the most important health outcome domains for many diseases. This freely available system enables improved efficiency and practicability for implementing PROMs in the clinical setting, using computer adaptive testing (CAT).^{1 6–10}

A few investigations on clinicians’ attitudes to using PROMs in daily routine and/or on implications of using PROMs have been published, including surveys among paediatricians, oncologists, psychiatrists, spine surgeons and systematic literature reviews.^11–19 Little information on the familiarity of trauma and orthopaedic surgeons with PROMs is available. Also, there is limited information on the use of PROMs in daily routine. This study aimed to gain detailed information on the perceived advantages and disadvantages of PROM implementation, and the important constraints and reservations for a successful implementation of such procedures.

Methods

Survey

Information was gathered from members of the AO Foundation, a medically guided non-profit organisation led by an international group of surgeons specialising in the treatment of trauma and disorders of the musculoskeletal system (www.aofoundation.org). A questionnaire was sent to 17 931 surgeons (9567 trauma, 6054 spine and 2310 craniomaxillofacial (CMF) surgeons) via SurveyMonkey, together with an invitation email explaining the purpose of the survey and a secure web link. Participation in the survey was anonymous and voluntary, and no remuneration or reward was offered for participation. No reminders were sent. As the questionnaire was developed, based on similar published surveys that evaluated surgeons’ opinions on PROMs among other medical specialties,^{11 17 20} no pilot survey was conducted.

Local Institutional Review Board approval has not been obtained as no medical information was collected from the participants, and data were collected and analysed anonymously. However, all participants were informed that the survey results would be published.

The questionnaire is available in online supplementary file 1. It comprised seven general questions, eliciting demographic information, the region of residence (AO Regions Africa, Asia-Pacific, Europe, Latin America, Middle East and North America), surgical specialisation, current working position, clinical experience and current workplace. Another three questions solicited surgeons’ familiarity with generic or disease-specific PROMs, and surgeons’ current use of PROMs. Detailed statements either about surgeons’ frequent use of PROMs in clinical routine, or not using PROMs in clinical routine were to be made. Therefore, if surgeons responded to currently use PROMs regularly or infrequently in daily clinical work, or both, in daily clinical work and research, their opinion on 10 detailed statements (Question A, online supplementary file 1) regarding their familiarity and usage of generic and disease-specific PROMs in clinical routine was solicited. Whereas if surgeons responded to currently not use PROMs, or to just use them for research, their opinion on seven detailed statements regarding their reasons to not use PROMs was solicited (Question B, online supplementary file 1).

In addition, information on three further aspects of PROMs was collected: the reasons why PROMs are not used more often in clinical routine (four items); what would be the most important prerequisites to implement PROMs in clinical routine (five items); and whether surgeons would use PROMs in their clinical routine (assuming the availability of tools/technologies to overcome reported obstacles) (one item).

All questions were optional, and surgeons had the possibility to skip questions.

Patient and public involvement

There was no patient involvement in the presented survey. The participating surgeons were not involved in the development of this questionnaire. The survey results will be published and made available in this way. Once the survey results are published AOTrauma, AOSpine and AOCMF will be informed so that the results can be distributed among their members.

Statistical analyses

The response rate (including completed and partially completed questionnaires) and the margin of error at 95% confidence (expressing the amount of random sampling error) were computed. Percentages for all categorical variables (excluding missing responses) were computed. To examine associations between two categorical variables, Χ² tests were applied (univariable analyses). Multivariable logistic regression analyses were conducted to evaluate the effect of ‘region of residence’, ‘surgical specialization’, ‘current position’, ‘clinical experience’ and ‘workplace’ on (A) the familiarity with existing generic PROMs, (B) the familiarity with existing disease-specific PROMs, and (C) the current use of PROMs in clinical routine. Listwise deletion applied if a respondent stated ‘other’ for surgical specialisation, current position or workplace. The significance level was set at p<0.05.

Exploratory factor analyses (EFA) were performed for the questions relating to ‘why to use PROMs’ and for the questions ‘why to not use PROMs’ to identify broader issues underlying decisions.²¹ Factors were extracted using the principal component method and those factors with eigenvalues greater than 1 (Kaiser criterion) were retained.²² EFA with varimax rotation was used as a data reduction technique to create interpretable results from the retained factors.²³

All statistical analyses were performed using the software SAS V.9.4 (SAS Institute).

Results

One thousand two hundred and twelve surgeons of the AO Foundation network (17 931 surgeons; response rate: 6.8%) completed the online survey. The computed margin of error for the results of the survey was ±2.72%. Six hundred fifty-seven (54.2%) were trauma and orthopaedic surgeons, 201 (16.6%) were spine surgeons, 338 (27.9%) were CMF surgeons and 16 (1.3%) had no defined clinical specialty. Demographic data and details about the AO Region, surgical specialisation, current position, clinical experience and workplace are listed in table 1.

Table 1.

Summary of demographics and working experience of participating surgeons

Variable	n=1212
Gender, n (%)	1212
Female	126 (10.4)
Male	1086 (89.6)
Age, n (%)	1212
18–24	1 (0.1)
25–34	256 (21.1)
35–44	468 (38.6)
45–54	307 (25.3)
55–64	149 (12.3)
65–74	26 (2.1)
75 or older	5 (0.4)
AO Region, n (%)	1212
Africa	41 (3.4)
Asia-Pacific	299 (24.7)
Europe	394 (32.5)
Latin America	217 (17.9)
Middle East	133 (11.0)
North America	128 (10.6)
Specialisation, n (%)	1212
Orthopaedics	328 (27.1)
Trauma	329 (27.1)
Craniomaxillofacial	338 (27.9)
Spine	201 (16.6)
Other	16 (1.3)
Current position, n (%)	1212
Registrar	249 (20.5)
Consultant	611 (50.4)
Senior management/professorship	335 (27.6)
Other	17 (1.4)
Clinical experience, n (%)	1212
0–4 years	183 (15.1)
5–9 years	259 (21.4)
10–14 years	261 (21.5)
15–19 years	176 (14.5)
≥20 years	333 (27.5)
Workplace, n (%)	1212
University hospital	457 (37.7)
Private hospital	172 (14.2)
Public hospital	337 (27.8)
Private practice	62 (5.1)
Both university and non-university hospitals	169 (13.9)
Other	15 (1.2)

Senior management/professorship includes the following positions: head, deputy head, assistant professor, and professor.

Overall, 661 (54.5%) reported being familiar with respective disease-specific PROMs, and 547 surgeons (45.1%) reported being familiar with generic PROMs. The univariable analysis showed that surgeons in Europe and North America were more familiar with specific PROMs (Europe: 68.5%; North America: 72.7%) and generic PROMs (Europe: 60.4%; North America: 66.4%) than those in the remaining regions (p<0.001). Spine surgeons were most familiar with both, specific PROMs (85.6%) and generic PROMs (63.7%), whereas CMF surgeons were least familiar with PROMs in general (specific: 13.6%; generic: 31.7%). ‘Surgeons’ current position’, ‘clinical experience’ and ‘workplace’ were also associated with their familiarity of PROMs (p<0.05) (table 2).

Table 2.

Univariable analysis evaluating the familiarity, and current use of PROMs in clinical routine and/or clinical research, according to geographical region, surgical specialty, current position, clinical experience and workplace

Variable	n	Familiarity with disease-specific PROMs*			Familiarity with generic PROMs†			Current use of PROMs‡
		Yes	No	P values	Yes	No	P values	Yes	No	P values
AO Region
Africa	41	25 (61.0)	16 (39.0)	<0.001	20 (48.8)	21 (51.2)	<0.001	10 (24.4)	31 (75.6)	0.004
Asia-Pacific	299	126 (42.1)	173 (57.9)		95 (31.8)	204 (68.2)		72 (24.2)	225 (75.8)
Europe	394	270 (68.5)	124 (31.5)		238 (60.4)	156 (39.6)		128 (32.5)	266 (67.5)
Latin America	217	97 (44.7)	120 (55.3)		76 (35.0)	141 (65.0)		54 (24.9)	163 (75.1)
Middle East	133	50 (37.6)	83 (62.4)		33 (24.8)	100 (75.2)		21 (15.8)	112 (84.2)
North America	128	93 (72.7)	35 (27.3)		85 (66.4)	43 (33.6)		40 (31.3)	88 (68.8)
Surgical specialty
Orthopaedics	328	240 (73.2)	88 (26.8)	<0.001	162 (49.4)	166 (50.6)	<0.001	107 (32.6)	221 (67.4)	<0.001
Trauma	329	197 (59.9)	132 (40.1)		142 (43.2)	187 (56.8)		70 (21.3)	258 (78.7)
Craniomaxillofacial	338	46 (13.6)	292 (86.4)		107 (31.7)	231 (68.3)		52 (15.4)	285 (84.6)
Spine	201	172 (85.6)	29 (14.4)		128 (63.7)	73 (36.3)		91 (45.3)	110 (54.7)
Current position
Registrar	249	116 (46.6)	133 (53.4)	0.013	80 (32.1)	169 (67.9)	<0.001	63 (25.3)	186 (74.7)	0.665
Consultant	611	352 (57.6)	259 (42.4)		296 (48.4)	315 (51.6)		171 (28.1)	438 (71.9)
Senior management/professorship	335	184 (54.9)	151 (45.1)		164 (49.0)	171 (51.0)		88 (26.3)	247 (73.7)
Clinical experience (years)
0–4	183	75 (41.0)	108 (59.0)	0.001	59 (32.2)	124 (67.8)	<0.001	31 (16.9)	152 (83.1)	0.024
5–9	259	139 (53.7)	120 (46.3)		107 (41.3)	152 (58.7)		72 (27.8)	187 (72.2)
10–14	261	143 (54.8)	118 (45.2)		116 (44.4)	145 (55.6)		76 (29.2)	184 (70.8)
15–19	176	102 (58.0)	74 (42.0)		89 (50.6)	87 (49.4)		48 (27.3)	128 (72.7)
≥20	333	202 (60.7)	131 (39.3)		176 (52.9)	157 (47.1)		98 (29.5)	234 (70.5)
Workplace
University hospital	457	284 (62.1)	173 (37.9)	0.001	253 (55.4)	204 (44.6)	<0.001	120 (26.3)	336 (73.7)	0.899
Private hospital	172	85 (49.4)	87 (50.6)		65 (37.8)	107 (62.2)		43 (25.0)	129 (75.0)
Public hospital	337	165 (49.0)	172 (51.0)		121 (35.9)	216 (64.1)		92 (27.3)	245 (72.7)
Private practice	62	31 (50.0)	31 (50.0)		20 (32.3)	42 (67.7)		15 (24.2)	47 (75.8)
University and non-university hospitals	169	88 (52.1)	81 (47.9)		81 (47.9)	88 (52.1)		49 (29.2)	119 (70.8)

P values derived from Χ² test.

Due to extremely low frequencies, the categories ‘Other’ of surgical specialty, current position and workplace have been excluded from the current tabulation and comparison.

*For example, Oxford Knee Score, Oswestry Disability Index.

†For example, EuroQol-5D (EQ-5D) or 36-Item Short Form Health Survey (SF-36).

‡This variable derived after reclassifying ‘Current use of PROMs in your daily clinical work’ (No, I do not use PROMs/Yes for research purposes only vs the remaining three categories).

PROM, patient-reported outcome measure.

The multivariable analysis confirmed the regional differences, differences between surgeon specialties and workplaces in the familiarity with disease-specific PROMs (table 3). Of note, surgeons working either in a private or a public hospital were approximately half as likely to be familiar with disease-specific PROMs than surgeons working in a university hospital (p≤0.05). No evidence for an association between familiarity with disease-specific PROMs and ‘current position’ or ‘clinical experience’ was found (table 3).

Table 3.

Multivariable logistic regression analyses evaluating the effect of AO Region, specialisation, current position, clinical experience, and workplace on the familiarity with generic PROMs, the familiarity with disease-specific PROMs and the current use of PROMs

Variable	Category	Familiarity with disease-specific PROMs			Familiarity with generic PROMs			Current use of PROMs
		OR	95% CI	P values	OR	95% CI	P values	OR	95% CI	P values
AO Region	Europe	1.00			1.00			1.00
	Africa	1.12	(0.45 to 2.81)	0.805	0.58	(0.29 to 1.17)	0.126	0.46	(0.22 to 0.95)	0.036
	Asia-Pacific	0.35	(0.23 to 0.53)	<0.001	0.31	(0.22 to 0.45)	<0.001	0.66	(0.47 to 0.93)	0.017
	Latin America	0.39	(0.25 to 0.61)	<0.001	0.42	(0.29 to 0.63)	<0.001	0.48	(0.32 to 0.71)	<0.001
	Middle East	0.23	(0.14 to 0.38)	<0.001	0.22	(0.13 to 0.35)	<0.001	0.36	(0.23 to 0.57)	<0.001
	North America	1.60	(0.89 to 2.86)	0.113	1.23	(0.78 to 1.94)	0.378	0.77	(0.49 to 1.20)	0.254
Specialisation	Craniomaxillofacial	1.00			1.00			1.00
	Orthopaedics	23.7	(15.18 to 37.00)	<0.001	2.17	(1.53 to 3.07)	<0.001	3.41	(2.42 to 4.80)	<0.001
	Trauma	11.62	(7.57 to 17.85)	<0.001	1.45	(1.02 to 2.06)	0.039	1.91	(1.35 to 2.70)	<0.001
	Spine	53.66	(30.76 to 93.62)	<0.001	3.92	(2.62 to 5.87)	<0.001	6.96	(4.60 to 10.53)	<0.001
Current position	Registrar	1.00			1.00			1.00
	Consultant	1.39	(0.93 to 2.08)	0.106	1.68	(1.16 to 2.42)	0.006	1.15	(0.81 to 1.64)	0.427
	Senior management/professorship	1.52	(0.93 to 2.47)	0.096	1.76	(1.14 to 2.72)	0.011	1.36	(0.89 to 2.07)	0.156
Clinical experience (years)	≥20	1.00			1.00			1.00
	0–4	0.67	(0.39 to 1.15)	0.142	0.68	(0.42 to 1.09)	0.108	1.04	(0.66 to 1.66)	0.861
	5–9	1.04	(0.66 to 1.66)	0.859	0.87	(0.59 to 1.30)	0.498	1.12	(0.75 to 1.66)	0.578
	10–14	0.91	(0.59 to 1.42)	0.686	0.84	(0.57 to 1.22)	0.356	1.06	(0.73 to 1.55)	0.743
	15–19	1.11	(0.68 to 1.80)	0.672	1.00	(0.66 to 1.50)	0.994	1.03	(0.69 to 1.55)	0.882
Workplace	University hospital	1.00			1.00			1.00
	Private hospital	0.58	(0.36 to 0.92)	0.021	0.62	(0.41 to 0.93)	0.021	0.53	(0.36 to 0.80)	0.002
	Public hospital	0.48	(0.33 to 0.70)	<0.001	0.48	(0.35 to 0.66)	<0.001	0.48	(0.35 to 0.66)	<0.001
	Private practice	0.66	(0.31 to 1.41)	0.283	0.35	(0.18 to 0.69)	0.002	0.30	(0.16 to 0.59)	<0.001
	Both university and non-university hospitals	0.88	(0.54 to 1.43)	0.615	0.93	(0.62 to 1.39)	0.731	0.82	(0.55 to 1.22)	0.323

Due to low frequencies, those classified as ‘Other’ in specialisation, current position or workplace, have been excluded from the current model.

PROM, patient-reported outcome measure.

For the familiarity with generic PROMs, above results were also confirmed in the multivariable analysis. In addition, the respondents’ ‘current position’ was found to be a significant variable in this model. Compared with registrars, consultants and senior managers or professors were more familiar with generic PROMs (p<0.05) (table 3).

Only 97/1210 (8.0%) respondents used PROMs regularly in their clinical routine, 134 (11.1%) irregularly, 94 (7.8%) for both clinical routine and research purposes, and 253 (20.9%) for research purposes only, compared with 632 (52.2%) who did not use PROMs at all. As with ‘familiarity’, the univariable analysis showed a distinct difference in the clinical use of PROMs between the different regions and specialisations: PROMs were most frequently used in Europe (32.5% of surgeons used PROMs in clinical routine or clinical research) and North America (31.3%). Spine surgeons used PROMs the most (45.3%), whereas CMF surgeons used PROMs the least (15.4%). Moreover, ‘surgeons’ clinical experience’ seemed to be associated with the routine use of PROMs (p=0.024) (table 2).

The multivariable analysis again confirmed the results from the univariable analysis, regarding ‘AO Region’, and ‘surgeons’ specialization’. Contrary to the univariable analysis, ‘workplace’ instead of ‘surgeons’ clinical experience’ was associated with use of PROMs in daily clinical routine (table 3).

Of 325 surgeons who responded to currently use PROMs in clinical routine (irregularly or regularly) or both in daily clinical work and research, 292 provided their opinions to the 10 detailed statements regarding their familiarity and routine use of PROMs. Five of the 10 statements got more than 80% agreement (either strongly agreed or agreed), and four between 67% and 78% agreement. Only 34% of surgeons agreed that they were required to document quality of life (QoL) data by the government (last statement) (table 4).

Table 4.

Detailed reasons for and against collecting PROMs in clinical routine, provided by surgeons

Subgroup	Variable	n	Strongly agree	Agree	Neither agree nor disagree	Disagree	Strongly disagree
Pro-PROMs*	PROMs can help prioritise clinical problems, n (%)	292	66 (22.6)	183 (62.7)	30 (10.3)	11 (3.8)	2 (0.7)
	PROMs can help facilitate communication between the doctor and the patient, n (%)	291	71 (24.4)	171 (58.8)	38 (13.1)	10 (3.4)	1 (0.3)
	PROMs can screen for potential patient problem, n (%)	287	63 (22.0)	172 (59.9)	43 (15.0)	9 (3.1)	0 (0.0)
	PROMs can identify patient’s preferences, n (%)	291	50 (17.2)	146 (50.2)	74 (25.4)	19 (6.5)	2 (0.7)
	PROMs monitor changes/responses to treatment, n (%)	291	96 (33.0)	153 (52.6)	35 (12.0)	7 (2.4)	0 (0.0)
	PROMs monitor and assess the general healthcare status of patients and potential changes, n (%)	291	62 (21.3)	173 (59.5)	46 (15.8)	10 (3.4)	0 (0.0)
	PROMs can be helpful but are not substitutes to measure the clinical outcomes of patients, n (%)	290	79 (27.2)	145 (50.0)	45 (15.5)	18 (6.2)	3 (1.0)
	PROMs help to monitor the quality of healthcare provision, n (%)	290	52 (17.9)	152 (52.4)	70 (24.1)	13 (4.5)	3 (1.0)
	PROMs are useful for national/international comparison and benchmarking, n (%)	290	79 (27.2)	148 (51.0)	51 (17.6)	12 (4.1)	0 (0.0)
	I am required to document patient-reported quality of life data by the government, regulatory bodies or for insurance reasons, n (%)	291	24 (8.2)	76 (26.1)	66 (22.7)	80 (27.5)	45 (15.5)
Contra-PROMs†	I do not believe in the usefulness of quality of life measurements in orthopaedics and traumatology, n (%)	756	24 (3.2)	74 (9.8)	144 (19.0)	304 (40.2)	210 (27.8)
	I do not have sufficient information, knowledge or experience to use PROMs in daily clinical routine, n (%)	768	156 (20.3)	319 (41.5)	103 (13.4)	145 (18.9)	45 (5.9)
	It is too costly to implement PROMs in my daily clinical routine, n (%)	759	40 (5.3)	155 (20.4)	326 (43.0)	189 (24.9)	49 (6.5)
	It is too time consuming to implement PROMs in my daily clinical routine, n (%)	756	94 (12.4)	264 (34.9)	229 (30.3)	139 (18.4)	30 (4.0)
	The resistance of my patients to fill out patient-reported outcomes routinely is too high, n (%)	751	35 (4.7)	196 (26.1)	317 (42.2)	165 (22.0)	38 (5.1)
	I am interested in using PROMs but I have not yet had the possibility to do so, n (%)	761	200 (26.3)	337 (44.3)	136 (17.9)	66 (8.7)	22 (2.9)
	Resistance to PROMs within my hospital/department, n (%)	752	39 (5.2)	101 (13.4)	315 (41.9)	211 (28.1)	86 (11.4)

*Only participants who use PROMs (regularly in daily clinical work/infrequently in daily clinical work/both in daily clinical work and research) have been asked to provide reasons.

†Only participants who either do not use PROMs or use PROMs only for research purposes have been asked to provide reasons.

PROM, patient-reported outcome measure.

Available (non-missing) data/opinions of 281 surgeons who used PROMs in their daily routine, related to the 10 provided statements, were factor analysed. Four factors were identified with eigenvalues greater than 1 (figure 1) and accounted for 66% of the total variance. We labelled the factors as follows: factor 1: quality of care, factor 2: measurement of outcomes, factor 3: regulations and factor 4: limitations of PROMs. The main factor loadings on the 10 items are shown in figure 2.

Figure 1.

Scree plot showing the variance in the data (eigenvalues) of the explorative factor analysis for surgeons who used patient-reported outcome measures (PROM) in their daily routine. The line until factor 4 shows the four of 10 components with an eigenvalue >1 which were included in the explorative factor analysis. The first four factors accounted for 66% of the total cumulative variance.

Figure 2.

Factor analysis path diagram displaying the variance of agreement/disagreement for each statement (why patient-reported outcome measures (PROM) are used in clinical routine) explained by the different factors. Factor 1: quality of care, factor 2: measurement of outcomes, factor 3: regulations, factor 4: limitations of PROMs.

Of 885 surgeons who responded to currently not use PROMs or to only use PROMs in research, 768 provided their opinions to the seven detailed statements regarding their reasons to not use PROMs. Four of the seven statements got less than 31% agreement (either strongly agreed or agreed), one got 47% (PROMs are too time consuming) and two statements got more than 60% agreement (60%: I am lacking sufficient information, knowledge or experience to use PROMs; 70.6%: I am interested but did not yet have the possibility to use PROMs) (table 4). For the second factor analysis, all answers of 739 surgeons to the seven items were used: two factors were identified accounting for 51% of the cumulative variance (figure 3 and 4): factor 1: overall assessment of PROMs; factor 2: institutional responses regarding to PROMs.

Figure 3.

Scree plot showing the variance in the data (eigenvalues) of the explorative factor analysis for surgeons who did not use patient-reported outcome measures (PROM) in their daily routine. The line until factor 2 shows the two of seven components with an eigenvalue >1 which were included in the explorative factor analysis. The first two factors accounted for 51% of the total cumulative variance.

Figure 4.

Factor analysis path diagram displaying the variance of agreement/disagreement for each statement (why patient-reported outcome measures (PROM) are not used in clinical routine) explained by the different factors. Factor 1: overall assessment of PROMs, factor 2: institutional responses in regard to PROMs.

When surgeons were asked for further aspects considered important for an implementation of PROM-collecting instruments into the daily clinical routine, user-friendliness was very important or important for 95.2% of surgeons (938/986). But also, other aspects such as costs, time efficiency, comparability with existing software and the interpretability and clinical relevance of the results were considered important or very important (table 5).

Table 5.

Aspects considered important for an implementation of PROM-collecting instruments into the daily clinical routine

Variable	n=1212
User-friendliness (for hospital staff and patients), n (%)	986
Very important	605 (61.4)
Important	333 (33.8)
Neither important nor unimportant	42 (4.3)
Unimportant	4 (0.4)
Very unimportant	2 (0.2)
Costs (acquisition and maintenance), n (%)	985
Very important	389 (39.5)
Important	459 (46.6)
Neither important nor unimportant	115 (11.7)
Unimportant	20 (2.0)
Very unimportant	2 (0.2)
Time efficiency, n (%)	976
Very important	543 (55.6)
Important	370 (37.9)
Neither important nor unimportant	56 (5.7)
Unimportant	4 (0.4)
Very unimportant	3 (0.3)
Compatibility to existing software tools in my hospital (eg, electronic medical records), n (%)	984
Very important	449 (45.6)
Important	400 (40.7)
Neither important nor unimportant	110 (11.2)
Unimportant	20 (2.0)
Very unimportant	5 (0.5)
Interpretation and clinical relevance of results, n (%)	986
Very important	529 (53.7)
Important	404 (41.0)
Neither important nor unimportant	47 (4.8)
Unimportant	4 (0.4)
Very unimportant	2 (0.2)

PROM, patient-reported outcome measure.

When surgeons were asked whether they would implement PROMs into their daily routine, if tools/technologies to overcome barriers were available, 656/1212 (66.2%) agreed, whereas 335 (33.8%) were not sure yet. Of those 656 surgeons who agreed, 364 (75.4%) were using PROMs in any form at the time of the survey, and 292 (57.6%) were not using PROMs at the time of the survey (p<0.001). In a multivariable logistic regression model, surgeons from Africa were 2.67 (95% CI 1.05 to 6.78) times more likely to be willing to use a tool that would overcome barriers in collecting PROMs in clinical routine than surgeons from Europe; orthopaedic surgeons (OR: 2.11; 95% CI 1.44 to 3.09), trauma surgeons (OR: 1.90; 95% CI 1.30 to 2.79) and spine surgeons (OR: 1.74; 95% CI 1.14 to 2.66) were more likely than CMF surgeons. Surgeons working in a private hospital were half as likely (OR: 0.49; 95% CI 0.35 to 0.69) as surgeons working in a university hospital (p<0.05 for all).

Discussion

Our worldwide study surveyed the familiarity of trauma/orthopaedic, spine and CMF surgeons with PROMs, their current use in clinical routine, reasons for using or not using them and respondents’ perceptions of the important aspects to be considered in implementing PROMs in clinical routine. In a multivariable regression model, familiarity with existing generic and disease-specific PROMs was influenced by ‘geographic region’, ‘surgical specialization’, ‘current workplace’ and for generic PROMs, by ‘current working position’. The use of PROMs in clinical routine was again influenced by ‘geographic region’, ‘surgical specialization’ and ‘current workplace’.

In a survey published by the Dutch Pediatric Association, 69% of 303 paediatricians reported being familiar with patient-reported QoL data.¹¹ Another survey among the Latin-American AOSpine members reported familiarity with generic PROMs in 79% of 199 participants and familiarity with disease-specific PROMs in 86%.¹⁸ In our study, 45% of surgeons were familiar with generic and 55% with disease-specific PROMs. Interestingly, familiarity varied according to surgical specialisation and geographic region and was in general higher for disease-specific PROMs than for generic PROMs. A few European countries, that is, the UK and Sweden,^{3 24–26} and the USA (http://www.hosonline.org) have started collecting PROMs on a routine basis, which could be a reason for the higher local general awareness. The low overall familiarity of CMF surgeons with PROMs in our study can probably be explained by the lack of well-established PROMs for these patients. In contrast, well-established PROMs for patients with spinal disorders such as the Oswestry Disability Index²⁷ and the Neck Disability Index²⁸ may have contributed to the higher familiarity (of 86% of spine surgeons in our study) with PROMs within this specialty. The study by the Dutch Pediatric Association found that paediatricians working at a university hospital were significantly more familiar with the existence of QoL questionnaires than those working in a community hospital (p=0.007), and paediatric registrars were significantly less familiar with the existence of QoL questionnaires than experienced paediatricians (p=0.019).¹¹ These results were confirmed for the surgeons/respondents participating in our survey (p<0.05).

PROMs are rather new in the field of trauma surgery and, as such, clinicians’ awareness might well be expected to be lower than in other medical disciplines. This may explain why only 26.9% of surgeons in our study used PROMs in their daily routine and only 8.0% used PROMs regularly. Higher proportions are reported from surveys which involved practitioners dealing with patients with chronic illness.^{11 17 20} Interestingly, ‘current working position’ and ‘level of clinical experience’ did not have a significant influence on the use of PROMs. We speculate that less experienced clinicians are not familiar with PROMs but are obliged to collect them on behalf of a more senior clinician as first point of contact.

We found a high degree of agreement among surgeons who routinely use PROMs regarding the reasons for doing so, such as prioritisation of a clinical problem, facilitation of communication between the clinician and the patient, monitoring changes/responses to the treatment, and so on. These reasons are cited in the literature and indicate that with the help of PROMs patient care and treatment can be focused more on the individual patient rather than on the disease. Moreover, the quality of healthcare delivery can be monitored and improved.18 25 28 29 Our factor analysis of the questionnaire showed that surgeons see four broad categories of reasons for using PROMs: to improve the quality of care; to contribute to the measurement of outcomes; compliance with regulation and that PROMs should be limited to the assessment of individual patient outcomes.

For policymakers, governments, regulatory bodies and funders, awareness of these four categories is important to develop incentives to increase surgeons’ motivation for using PROMs in clinical routine. At the same time, it is as important to realise why PROMs are not used more often in clinical routine. Concerns mentioned by surgeons are: absence of institutional policies to guide the process and concerns about the length of PROM questionnaires, the time to get the results, the impression that PROMs are burdensome and time consuming, and the costs and infrastructural changes related to PROM implementation.2 17–19 30 Other reasons are the lack of knowledge and experience to interpret results and to use them in clinical practice, but also doubts on the compatibility of results with routinely collected clinical measurements.11 20 30–33 Furthermore, surgeons seem to be sceptical about PROMs, because they are thought to be ‘soft and subjective’.^{4 34}

In our study, 87.0% of all surgeons who did not routinely use PROMs disagreed with the statement that PROMs are not useful in orthopaedics and traumatology, and 61.9% agreed that they do not use PROMs due to insufficient information, knowledge or experience. Another 70.6% of surgeons stated that they would be interested in using PROMs but did not yet have the opportunity to do so. The factor analysis showed two broad constraints to the implementation of PROMs: cost and institutional ignorance and inertia. Developing policies to increase information and educational activities and improved tools/technologies for PROM use could help overcome these obstacles. Important prerequisites to implement and use PROMs were found among surgeons in our study: there was a belief in the usefulness of PROMs and a willingness to use PROMs if user-friendly technologies were available.

PROMIS instruments (http://www.healthmeasures.net) are one of the most important and promising developments in terms of advancement of PROMs. By using sophisticated algorithms, item selection is automatically matched to the health level of respondents, and the number of questions is reduced to a minimum.1 5–7 35 36 Available PROMIS instruments are useful for patients in the field of orthopaedics and traumatology (eg, the physical function CAT and pain interference CAT instruments), and even superior to established legacy instruments, due to various reasons (eg, taking less time to administer).^37–45

Developments to increasingly implement PROMs into clinical routine, and even into electronic health record systems are apparent. Additionally, stand-alone software to collect PROMs is becoming available for clinical routine. Clinicians immediately see the current health status of their patients and former scores and can monitor changes over time.

The current study has limitations, mainly the rather low response rate of 6.8%, which gives rise to selection/non-response bias. Comparing our demographic survey data with available internal data derived from a membership statistic of the AO Foundation from 2018, younger surgeons aged between 25 and 34  years were underrepresented in our survey (deviations within specialties and regions: 10%–20%), surgeons aged between 35 and 44, and between 45 and 54 were over-represented (deviations: 5%–11%). CMF surgeons were over-represented in all regions (deviations: 13%–34%) and balanced by spine and trauma surgeons who were under-represented (deviations: 5%–18%). The AO Asian Pacific region was under-represented among trauma and spine surgeons (deviations: 10%–12%), while Europe was over-represented (deviations: 6%–14%). The Middle East was over-represented by 5.5%, while Latin America was under-represented by 8% among CMF surgeons. The gender distribution is considered representative within regions and specialties.

In addition, response to the survey was voluntary, and it can be assumed that mainly surgeons interested in the topic responded, which in turn may mean that the current use of PROMs is rather overestimated, that is, PROMs are even less known and less used in clinical routine than reflected by the survey.

On the other hand, the margin of error was only 2.72%, implying that the random sampling error was low. Therefore, survey estimates can be considered precise and representative for the whole population of AO surgeons. Keeping in mind that the AO Foundation has nearly 18 000 members worldwide, results could reflect the general situation regarding the use of PROMs in clinical routine.

Conclusions

Our research survey among trauma surgeons provides an understanding of the familiarity with PROMs and their current use in clinical routine. There is consensus on the usefulness of PROMs. A majority of surgeons is willing to implement PROMs, if an adequate tool would be available. The development of user-friendly and efficient tools/technologies both for patients and clinical staff would be an important facilitator to increase the use of PROMs in clinical routine. Additionally, education and/or policies for PROM usage in clinical routine might help.