Benefits of and Barriers to Using Patient-Rated Outcome Measures in Athletic Training

Abstract

Context:

Patient-rated outcome measures (PROMs) are important for driving treatment decisions and determining treatment effectiveness. However, athletic trainers (ATs) rarely use them; understanding why may facilitate strategies for collection of these outcomes.

Objective:

To identify the benefits of and barriers to using PROMs in athletic training.

Design:

Cross-sectional study.

Setting:

Web-based survey.

Patients or Other Participants:

A total of 1469 randomly sampled ATs (age = 36.8 ± 9.8 years; 48% female) working in the college/university, 2-year institution, secondary school, clinic, hospital, or industrial/occupational setting.

Intervention(s):

An e-mail was sent to ATs inviting them to complete a survey regarding the use, benefits, and barriers of PROMs. Athletic trainers who indicated they used PROMs (AT-PRs) completed 65 questions about the benefits of and barriers to their use. Athletic trainers who indicated no use of PROMs (AT-NONs) completed 21 questions about barriers of use.

Main Outcome Measure(s):

Dependent variables were the endorsements for the benefits of and barriers to the use of PROMs.

Results:

A total of 458 ATs initiated the survey and 421 (AT-PR = 26%, AT-NON = 74%) completed it (response rate = 28.7%). The most frequently endorsed benefits by AT-PRs were enhancing communication with patients (90%) and other health care professionals (80%), directing patient care (87%), and increasing examination efficiency (80%). The most frequently endorsed barriers by AT-PRs were that PROMs are time consuming (44%), difficult (36%), and confusing (31%) for patients and time consuming for clinicians to score and interpret (29%). The most frequently endorsed problems by AT-NONs were that PROMs are time consuming for clinicians to score and interpret (31%), time consuming (46%) and irrelevant to patients (28%), and lacking a support structure for clinicians (29%).

Conclusions:

These results suggest that, although benefits to using PROMs exist, there are also barriers. Barriers are similar for AT-PRs and AT-NONs. Strategies to decrease barriers and facilitate the use of PROMs warrant investigation.

Key Points

• Patient-rated outcome measures (PROMs) are important in driving treatment decisions, determining effective treatments, and supporting patient-centered care.

• Benefits to the collection of PROMs as described by those who use them in clinical practice include enhancing communication with patients and other health care professionals, directing patient care, and increasing examination efficiency.

• Barriers to the collection of PROMs are similar between those who do and do not use them in clinical practice. Common barriers are that PROMs are time consuming for patients to complete and for clinicians to score and interpret.

• Strategies to overcome barriers and increase the use of PROMs in clinical practice are needed.

Over the last several years, there has been an increasing push to provide whole-person, patient-centered care in all health care fields, including athletic training.^1–4 The Institute of Medicine⁵ has suggested that all health profession education programs include patient-centered care in their curriculums. The athletic training profession has highlighted the importance of collecting patient outcomes for the advancement of the profession by identifying clinical outcomes research as a top professional priority⁶ and dedicating $1 million from the National Athletic Trainers' Association (NATA) for this research.⁷ Recently, the American Reinvestment and Recovery Act resulted in the Agency for Health Research and Quality receiving $300 million specifically for comparative effectiveness research, which focuses on patient outcomes.⁸ The importance of incorporating patient values and preferences into care has also been highlighted by the Patient-Centered Outcomes Research Institute's support of investigations that engage patients in health care research and dissemination.⁹ Taken together, these events suggest a clear trend toward patient care that involves evaluation of the whole person and collection of patient outcomes.

Whole-person health care is comprehensive care that considers and addresses all aspects of disability, from the impairment of body structures and functions to activity limitations and participation restrictions.^3,10,11 Most clinicians are familiar with clinician-based outcome measures, such as range-of-motion and strength assessments, and these measures are routinely part of patient evaluation and treatment progression. Less familiar components of patient evaluation and treatment progression are patient-based outcome measures, such as standardized patient-rated outcome measures (PROMs). Patient-rated outcome measures are instruments patients complete that provide information about the effect of their health condition or injury on their overall health status or health-related quality of life (HRQOL) and that highlight the patient perspective. Although both clinician-based and patient-based outcomes are valuable to patient care, patient-based outcomes are preferable because they have the ability to influence patient care to a greater degree and provide more meaningful information regarding the effectiveness of interventions.^12,13 Therefore, clinicians should use a combination of traditionally measured impairments of body function and information from the patient's point of view, including preferences for daily activities and life participation, because both outcomes provide different but complementary information.¹⁴

Several health care professions, such as psychology and physical therapy, have advocated the use of standardized outcome measures in clinical practice.^14,15 Studies^14,16 have identified clear benefits of using PROMs in patient care that contribute to the process of clinical reasoning, enhance communication with patients, direct the plan of care, improve patient outcomes, and motivate patients. Other reported benefits are identifying a need to alter treatment and following ethical practice.¹⁵

Despite these documented benefits for the use of outcome assessment, some health care professions still struggle to implement these measures as part of standard clinical practice. Physical therapists have cited numerous barriers to the use of PROMs, including length of time for patient completion and clinician analysis and patient difficulty in completing the instruments independently.¹⁴ In addition, some clinicians expressed concerns that outcome instruments were not helpful to their clinical practice, interfered with patient autonomy, and lacked confidentiality.¹⁵

Even though PROMs have existed for years, their routine implementation into clinical practice is new for many health care professions, including athletic training. Better understanding of the perceived benefits of and barriers to the use of these instruments could advance the profession, and identified barriers could be specifically targeted to increase the regular use of these instruments in practice. Therefore, the purpose of our study was to identify athletic trainers' (ATs') perceptions of the benefits of and barriers to using PROMs. We also aimed to identify the practice patterns of ATs who either use or do not use PROMs in clinical practice.

METHODS

Participants

We used the 2010 end-of-year NATA membership statistics to identify approximately 19 000 regular certified ATs and certified athletic training students in the college/university, 2-year institution, secondary school, clinic, hospital, and industrial/occupational settings. Based on an anticipated response rate of 40%, a 4% error rate, and a 95% confidence interval, we requested a random sample of 1500 ATs from the NATA member database representative of all 10 NATA districts. One premise of survey sampling is that larger populations do not require a particularly large sample size as long as the sample is random and the a priori level of confidence and acceptable error rates have been established.^17,18

Study Design

A cross-sectional survey research design was used to report on the perceived benefits of and barriers to the use of PROMs in athletic training clinical practice. Data were obtained through distribution of the survey to our targeted group of participants.

Survey Instrument

We developed our survey using a previously validated survey from a study by Jette et al¹⁴ that investigated the use, benefits, and barriers of outcome measures in physical therapy. Once the authors granted permission to use the survey, we slightly modified it to incorporate terminology specific to athletic training. The modifications were independently made by 3 investigators (A.S.V., J.P., L.V.), and consensus was reached for all changes. Additionally, before using the survey, we established content and face validity through expert panel review of all questions by 3 of the authors (A.S.V., J.P., L.V.) who have published in peer-reviewed journals in the area of clinical outcomes assessment.

The survey consisted of 86 questions split into 2 question sets. Athletic trainers who indicated they used PROMS (AT-PRs) completed 54 questions about demographics (n = 11); benefits (n = 12); barriers (n = 15); and policies, procedures, and selection requirements of these instruments (n = 16). Athletic trainers who indicated no use of PROMs (AT-NONs) completed 32 questions about demographics (n = 11) and the barriers of using these instruments (n = 21). We used Likert-style questions (5 points; 1 = strongly agree, 5 = strongly disagree) to assess the benefits and barriers of PROMs and a 5-point adjectival scale (1 = always, 5 = never) to assess how PROMs were used by the participants. Descriptive data regarding policies, procedures, and instrument selection criteria associated with the use of PROMs were captured through multiple-choice questions.

We assessed the reliability and factor structure of the modified instrument by using data from a pilot survey of 295 randomly sampled NATA members categorized as regular certified ATs and certified athletic training students in the college/university, 2-year institution, secondary school, clinic, hospital, and industrial/occupational settings. Eighty-three ATs responded (response rate = 28%) to the pilot e-mail survey. The pilot study required that participants complete the survey 1 time, and the data from the pilot study were used to establish the factor structure of the survey as well as the internal consistency (ie, reliability) of the items in each factor. The reliability and validity of the instrument were considered established based on the work of Jette et al.¹⁴ Because we made only minor changes to the original survey, no further reliability or validity testing was necessary.¹⁹ We used a principal component exploratory factor analysis with varimax rotation in SPSS (version 18.0; SPSS Inc, Chicago, IL) and identified a 3-factor structure for the survey: benefits (n = 11 questions), barriers (n = 11 questions), and language/culture (n = 2 questions). A scree plot and an eigenvalue greater than 1 were used to identify the factor cutoff, and 62.8% of the total variance was accounted for by the 3-factor structure. Each of the factors displayed high internal consistency (Cronbach α = 0.89, 0.90, 0.71, respectively).²⁰ We retained all survey items and made no additional changes to the survey.

Procedures

We recruited participants via e-mail invitation to complete our Web-based survey through Survey Monkey (http://www.SurveyMonkey.com, Portland, OR). E-mail addresses were provided through an application process offered by the NATA. The study was approved by 2 local institutional review boards, and participants implied consent by accessing and completing the survey. A reminder e-mail was sent to all potential participants 2 weeks after the initial invitation.¹⁷

Data Analysis

The primary dependent variables were the endorsements for the benefits of (AT-PRs only) and barriers to (AT-PRs and AT-NONs) using PROMs. Benefit and barrier Likert-style questions were classified as endorsed when a participant selected a rating of agree or strongly agree. Dependent variables also included data regarding the reasons why AT-PRs used PROMs (eg, quality assurance, patient comparison). Ratings of sometimes, usually, and always for these questions were classified as endorsed. Data are reported as percentage endorsed (%). We used Spearman ρ correlations (r_s) to determine the relationship between clinician and practice characteristics and whether or not a clinician used PROMs. Correlation coefficients were interpreted as little to no relationship (0–0.25), fair relationship (0.26–0.50), moderate to good relationship (0.51–0.75), and good to excellent relationship (0.76 and higher).²¹ Also, we ran t tests to determine whether differences existed between the AT-PR and AT-NON groups in terms of practice characteristics (eg, number of patients seen in an 8-hour period). The level of significance was set at P < .05. All data analyses were performed using SPSS statistical software, version 18.0.

RESULTS

E-mail invitations were sent to 1500 ATs. Thirty-one e-mails were returned as undeliverable, leaving 1469 invitations. Of these, 458 ATs initiated the survey (response rate = 31%) and 421 completed the survey (response rate = 29%). For those who completed the survey, 26% indicated that they collected PROMs, and 74% indicated that they did not. The descriptive characteristics for both AT-PR and AT-NON groups are shown in Table 1.

Table 1.

Participant Characteristics for the Current Study (N = 421)

Variable
	Athletic Trainers
	Used Patient-Rated Outcome Measures	Did Not Use Patient-Rated Outcome Measures
	No. (%)
Sex
Male	48 (44)	153 (49)
Female	62 (56)	158 (51)
Athletic training practice, y
<3	15 (14)	25 (8)
3–5	11 (10)	50 (16)
6–10	19 (17)	92 (30)
11–20	36 (33)	85 (27)
>20	29 (26)	59 (19)
Professional (entry-level) degree
Bachelor's	73 (66)	220 (71)
Master's	37 (34)	91 (29)
Highest earned degree
Bachelor's	32 (29)	85 (27)
Entry-level master's in athletic training	7 (6)	15 (5)
Advanced master's in athletic training	6 (6)	39 (13)
Master's in related field	59 (54)	159 (51)
Doctoral	6 (6)	13 (4)
Postdoctoral	0 (0)	0 (0)
Practice setting
High school	33 (30)	130 (42)
College/university	37 (34)	120 (39)
Two-year institution (college)	2 (2)	16 (5)
Clinic/outreach	14 (13)	35 (11)
Clinic	20 (18)	6 (2)
Hospital	2 (2)	1 (0.3)
Industrial/occupational	2 (2)	3 (1)
National Athletic Trainers' Association district
1	15 (13)	29 (9)
2	11 (10)	31 (10)
3	10 (9)	17 (6)
4	11 (10)	36 (12)
5	18 (16)	42 (14)
6	9 (8)	35 (11)
7	10 (9)	29 (9)
8	7 (6)	31 (10)
9	12 (11)	29 (9)
10	7 (6)	32 (10)
	Mean ± SD
Age, y	38.1 ± 10	36.4 ± 10
Treatment sessions per 8-h d	19.9 ± 13	19.7 ± 14

The AT-PR participants were asked how they learned to use PROMs in a question for which respondents could select all that apply. The most frequent sources of learning were professional education programs (30%), followed by postprofessional education programs (20%) and professional colleagues (20%), continuing education workshops/conferences (24.5%), and other (15%). The AT-NON participants were not asked about their training regarding the use of PROMs but were asked if lack of training was a barrier to their use of PROMs. Only 8% of AT-NONs endorsed lack of training as a barrier to PROMs use.

Overall Perceptions of Patient-Reported Outcome Measures

The most frequently endorsed benefits by the AT-PR group were enhancing communication with patients (90%) and other health care professionals (80%), helping to direct the plan of care (87%), and increasing efficiency of examinations (80%). Complete data related to the frequency of endorsed benefits for the use of PROMs by the AT-PR group are listed in Table 2. The most frequently endorsed problems noted by the AT-PR group were that PROMs are time consuming (44%), difficult (36%), and confusing (31%) for patients to complete and time consuming for clinicians to analyze (29%).

Table 2.

Endorsed Benefits of Patient-Rated Outcome Measures by Athletic Trainers Who Used Them (n = 110)

Endorsed Benefits	No. (%)
Enhance communication between athletic trainer and patient	96 (90)
Help to direct plan of care	93 (87)
Enhance communication with other health care professionals	86 (80)
Increase efficiency of examinations	86 (80)
Help to focus choice of interventions	85 (79)
Patients feel clinicians are thorough in their examination	83 (78)
Enhance communication with physicians	82 (77)
Attain better patient outcomes	76 (71)
Motivate and encourage patients	66 (62)
Decrease rate of insurance denials	33 (31)
Enhance marketing	29 (27)

The most frequently endorsed problems cited by the AT-NON group were that outcomes instruments are time consuming for clinicians to analyze (31%), time consuming (46%) and irrelevant to patients (28%), and lacking a support structure for clinicians (29%). Complete data related to the frequency of endorsed barriers for using PROMs by AT-PRs and AT-NONs are listed in Table 3. Participants in both the AT-PR and AT-NON groups reported that a lack of time was a common barrier for using patient-reported questionnaires. Interestingly, the groups did not differ in the reported number of patients treated in an 8-hour day (19.87 ± 12.86 compared with 19.68 ± 13.92, respectively; t₄₁₈ = 0.127, P = .90). In other words, patient load did not explain the reported time barriers.

Table 3.

Endorsed Barriers to Using Patient-Rated Outcome Measures

Endorsed Barriers	No. (%)
By athletic trainers who used them (n = 110)
Patient completion time too long	46 (44)
Difficult for patients to complete	37 (36)
Confusing to patients	32 (31)
Too much time for clinician to analyze	30 (29)
Make patients anxious	24 (23)
Provide information too subjective to be useful	23 (22)
Difficult to interpret	23 (22)
No relevance to patients	18 (17)
Not worth the effort	16 (15)
Require too high of a reading level	15 (14)
Not sensitive to cultural/ethnic concerns	12 (12)
Language in which patients are not fluent	9 (9)
No direction to plan of care	8 (8)
Confusing to clinicians	7 (7)
By athletic trainers who did not use them (n = 311)
Patient completion time too long	144 (46)
Too much time for clinician to analyze	97 (31)
Lack of support	91 (29)
No relevance to patients	87 (28)
Not worth the effort	77 (25)
Questions not relevant to my patients	65 (21)
Difficult for patients to complete	53 (17)
No direction to plan of care	44 (14)
Confusing to patients	36 (12)
Difficult to interpret	28 (9)
Lack of training	25 (8)
Provide information too subjective to be useful	24 (8)
Cost too much	21 (7)
Do not get completed at discharge; not useful	18 (6)
Used for research purposes only	14 (5)
Will use in future	12 (4)
Make patients anxious	11 (4)
Require too high of a reading level	11 (4)
Language in which patients are not fluent	4 (1)
Confusing to clinicians	4 (1)
Not sensitive to cultural/ethnic concerns	2 (1)

Characteristics and Practice Patterns of AT-PRs

The majority (81%) of the AT-PR group used PROMs to examine and document the status, progress, and outcomes of individual patients. Other commonly endorsed reasons for using PROMs were to determine treatment effectiveness (79%), to communicate with other health care providers or referral sources (76%), and for quality improvement/assurance activities (70%). Fewer AT-PRs indicated that they used PROMs to compare average outcomes of patients within a practice or athletic training room (58%) or to answer clinical questions through a traditional research approach (55%).

In the AT-PR group, 46% were mandated to use PROMs by their employer, and an additional 31% reported that they were encouraged but not mandated to do so. A combination of patient self-report and clinician observation health status questionnaires were used by 61% of the AT-PR group. Those in the AT-PR group most often reported that they used PROMs for all injuries, regardless of severity (51%); that they completed health status questionnaires on a monthly (36%), biweekly (17%), or weekly (22%) basis; and that all ATs at their facility used the same types of questionnaires (54%).

Most of the AT-PR group (56%) used raw information from paper questionnaires rather than calculating summary scores; an additional 24% used paper questionnaires and calculated summary scores. Only 7% used computerized questionnaires.

Half of the AT-PR group learned to use health status questionnaires in entry-level (30%) or postprofessional (20%) education. The most commonly endorsed criteria for selecting a health status questionnaire were that the instrument was easy for the patient (55%) or clinician (36%) to understand, could be completed quickly (54%), and was valid and reliable (36%). The body regions most commonly assessed with PROMs and instruments used most frequently by the AT-PR group are shown in Table 4.

Table 4.

Body Regions Assessed with Patient-Rated Outcome Measures by Athletic Trainers

Body Region	No. (%)	Instruments
Lumbar spine	72 (66)	Oswestry Low Back Pain Questionnaire, Quebec Back Pain Disability Scale
Head	72 (66)	Sport Concussion Assessment Tool (SCAT, SCAT2), Standardized Assessment of Concussion (SAC), Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT), Graded Symptoms Checklist
Cervical spine	71 (65)	Neck Disability Index
Knee	68 (62)	Lysholm Knee Scoring Scale, Single Assessment Numeric Evaluation
Shoulder	65 (59)	Shoulder Pain and Disability Index
Ankle	64 (58)	Functional Ankle Ability Measure
Hip	59 (54)
Elbow	56 (51)
Wrist	55 (50)
Hand	54 (49)
Foot	54 (49)	Foot Health Status Questionnaire
Region specific	NA	Lower Extremity Functional Scale, Disabilities of the Arm, Shoulder and Hand
Generic	NA	McGill Pain Questionnaire, pain scale, Subjective Injury Assessment, Short Form-36, Health Status Questionnaire
Other	NA	Homegrown questionnaires, medical history,b generic satisfaction questionnaire, Faust Injury Report,b Physical Activity Readiness Questionnaire, Loss of Productivity Assessment, Perceived Improvement Questionnaire, Get Up and Go Test,b Tinetti Balance Assessment Tool,b Balance Error Scoring Systemb
Outcomes-measurement systemsa	NA	Clinical Outcomes Research Education for Athletic Trainers; Focus on Therapeutic Outcomes, Inc; Therapeutic Associates Outcomes Systems; SportsWareb

Abbreviation: NA, not available.

^aOutcomes-measurement systems integrate multiple outcomes-assessment instruments that are generic, region specific, and disease specific.

^bThese items were reported but are not considered patient-rated outcomes measures.

The majority of clinician and practice characteristics had little to no relationship (r_s < 0.25) with whether or not a clinician used PROMs. The correlation values for each relationship are listed in Table 5.

Table 5.

Relationship Between Clinician and Practice Characteristics and Whether a Clinician Used Patient-Rated Outcome Measures (N = 420)

Variable	rs
Sex	−0.05
Years as practicing athletic trainer	−0.08
Highest degree earned	−0.02
Additional health care certifications	−0.08
Practice setting	−0.18
Patients treated in an 8-h d	−0.02

DISCUSSION

The primary objective of our study was to report on the benefits of and barriers to using PROMs in athletic training practice as perceived by athletic training clinicians. Secondarily, we aimed to identify athletic training practice patterns related to the use of PROMs in patient care. To our knowledge, this is the first survey to measure the use of PROMs in athletic training clinical practice.

Overall Perceptions of PROMs

Most (74%) clinically practicing ATs did not routinely use PROMs during patient care. Moreover, only 4% of the AT-NON group indicated that they would use these types of measures in the future. These data suggest that only a small percentage (26%) of ATs actually collect the type of patient information that is meaningful to patients and can address the effectiveness of services provided. This finding is not surprising because the push to use these outcomes in the profession is relatively new. However, it is troubling that only a very small percentage of ATs who do not currently use outcome measures intend to use them in the future, especially considering the increased frequency of studies reporting deficits in HRQOL after athletic illness or injury^22–25 and the calls from within the profession to routinely measure patient outcomes.

Our results regarding the use of PROMs are similar to those reported by other health care professions, including physical therapy and physiotherapy, occupational therapy, and speech therapy.²⁶ According to Jette et al¹⁴ in 2009, more than half of physical therapists in the United States reported not using standardized outcome measures, and only 9% planned to implement them in the future. The use of PROMs was even less in psychiatry. A decade ago, Garland et al²⁷ reported that the frequency with which psychologists used standardized measures to evaluate treatment effectiveness was much less than other methods, such as clinician intuition or feelings or proxy reports of client functioning. More recently, Hatfield and Ogles¹⁵ surveyed psychologists in 2007 and found that more than 60% reported no use of outcome measures, whereas Zimmerman and McGlinchey²⁸ in 2008 observed that 80% of psychiatrists did not routinely use outcome measures.

Taken together, these findings highlight how other professions have been working for years to address the concept of measuring patient outcomes, and they suggest athletic training lags in the implementation of outcomes assessment into practice. As the literature above suggests, complete implementation across a profession and its various practice settings takes time, but progress can be made through targeted efforts.

For example, researchers in Australia^16,29,30 demonstrated an increased use of outcomes assessment by physiotherapists over a 6-month period when several strategies to promote use were implemented (pre = 30%; post = 66%). Increased use of outcomes instruments has been attributed to positive attitudes toward outcomes assessment, education initiatives, professional support, and the perception that these assessments are beneficial.^16,29,30 Therefore, efforts should be targeted at improving attitudes toward outcomes assessment, providing education regarding outcomes assessment in clinical practice, offering professional support for clinicians measuring outcomes, and creating outcomes-assessment instruments pertinent to a physically active population. This approach may be especially important in athletic training given the variety of practice settings and patient populations across the profession.

Another interesting finding of the current study is related to the benefits endorsed by those in the AT-PR group. The most frequently endorsed benefits included enhanced communication between the AT and patient (90%) and enhanced communication with other health care professionals (80%). Clearly, improvements in communication seem to be important factors for practicing ATs, as well as other health care professionals. These findings are supported by other studies. In 2000, the Department of Health in the United Kingdom published the National Health Service Cancer Plan, which stated that good communication between oncologists and patients is central to the management of cancer patients.³¹ Also, according to Garland et al,²⁷ more than 60% of psychologists in their study felt that standardized outcome measures helped them communicate information to patients and their parents.

Communication between health care providers and patients is important. Patients often do not understand the terminology used to describe their injury or illness. As a result, health care providers must accurately describe and communicate health-related concerns to patients in an easy-to-understand language. Patient-rated outcomes measures can help communicate to patients the progress of their condition. Further, these instruments often provide visual cues, such as 5-to-10-point Likert scales or visual analogue scales (eg, 10-cm line). Thus, these visual aids can assist patients in understanding their current health status, and when status is measured over time, the visual aids can show patients their change in health status (eg, improvement, no change, or deterioration).

Although PROMs can help clinicians communicate information to patients, they also communicate important patient information to clinicians. Patient completion of outcomes instruments can facilitate the detection of subjective or unseen psychological, emotional, or social problems that require treatment to improve overall quality of care. Velikova et al³² reported that routine communication of outcomes measure information between oncologists and patients had a positive effect on communication and resulted in more frequent discussions of nonspecific symptoms. Future researchers in athletic training should investigate the specific benefit of improved communication between patients and practitioners because this information may help to illustrate the clinical benefit of outcomes assessment in patient care.

In addition to the benefit of communication, most of the AT-PR group indicated that outcome measures promoted several aspects of patient care, which included directing the plan of care (87%), making patients feel as though clinicians are thorough in their examination (78%), attaining better patient outcomes (71%), and motivating and encouraging patients (62%). Jette et al¹⁴ reported similar endorsed benefits, with response rates ranging from 78.9% to 93.6%.

One key patient care benefit to the use of PROMs is better patient outcomes. This perceived benefit has been endorsed in studies^16,33 of physical therapists in the Netherlands and Israel, who reported that using measurement instruments improved the quality of treatment, helped focus treatment management, and facilitated patient discharge planning. In a prospective study³² involving 28 oncologists, 286 cancer patients were randomly assigned to an intervention group (regular completion of 2 outcomes measures with feedback), an attention-control group (completion of 2 outcomes measures with no feedback), or a control group (no HRQOL measurement). A greater number of patients in the intervention group showed clinically meaningful improvement in HRQOL than patients in either of the control groups. These findings suggest that using outcomes measures in clinical practice may result in better patient outcomes. However, future investigators need to address the relationship between evaluating patient outcomes and improving HRQOL in athletic training.

We are not surprised by the finding that most of the AT-PR group reported that outcomes assessment benefits several aspects of patient care. Clinicians have been well educated and trained to use clinician-based measures (ie, decrease in swelling or pain, increase in range of motion) to progress care, and PROMs allow a similar evaluation of progress from the perspective of the patient. Through evaluation with PROMs, a clinician may learn that the patient is not psychologically or physically prepared to move on to the next phase of rehabilitation or return to play. Further, this information can position the clinician to direct the plan of care by incorporating other methods of treatment that are not focused on impairments, which may not be the root of the problem. For example, having the patient spend the hour of treatment on the field with teammates to regain social acceptance, being referred to a psychologist to treat concerns related to sleep deprivation, or spending the afternoon being tutored to help with falling grades are all interventions that may be warranted and identified through the measurement of patient outcomes. Using outcomes measures provides the clinician with objective evidence that can motivate the patient to continue care and to mentally prepare for the physical challenges of the next phase of treatment.

Although we identified several benefits to collecting PROMs, participants endorsed several barriers as well. The most frequently endorsed barriers by the AT-PR group were that PROMs are time consuming (44%), difficult (36%), and confusing (31%) for patients and time consuming for clinicians to analyze (29%). These were also the top barriers reported by Jette et al,¹⁴ suggesting that the problems ATs face in terms of outcomes implementation may also be experienced by other health care professions. A recent systematic review²⁶ confirms that barriers, such as perceived value of outcomes assessment, practical considerations (eg, time, infrastructure), and perceived relevance of the instruments, are common in many allied health care fields. Russek et al³⁴ reported that physical therapists found a standardized data-collection system for outcomes was inconvenient. Additionally, clinicians were uncomfortable with the technical aspects of using outcome measures. Garland et al²⁷ noted that psychologists had feasibility problems and interpretation difficulties with outcomes measures. Therefore, a lack of perceived relevance and difficulties in technical aspects, such as scoring and interpreting these measures, may deter implementation and use. However, these common barriers were cited by clinicians who still found ways to implement outcomes instruments into patient care. Thus, the perceived barrier did not completely negate the clinician's ability to measure patient outcomes.

Unfortunately, most ATs we surveyed did not use PROMs. The AT-NON group identified barriers that included too much patient completion time (46%), too much time for clinicians to analyze (31%), lack of support (29%), irrelevance to patients (28%), and unworthiness of the effort (25%). All of these barriers, along with patient difficulty completing the forms, were also top barriers reported by physical therapists.¹⁴ Further, psychiatrists and psychologists who do not routinely use outcomes measures have reported barriers that include not believing outcomes measures were helpful in clinical practice, believing that completing measures takes too much time, and perceiving challenges related to implementation of an outcomes-collection system.^15,28

Interestingly, even though a large percentage of our respondents did not use these instruments, only a small number of them endorsed the barriers. Jette et al¹⁴ reported a similarly low level of barrier endorsement in physical therapists. Together, these data may reflect that clinicians who currently do not use PROMs perceive few barriers to their use. Although we provided a definition and examples of PROMs in our survey, many participants may have been unfamiliar with patient-rated outcomes assessment and unable to make judgments regarding barriers. The problem of unfamiliarity with outcomes assessment concurs with the findings from others that suggest a clinician's knowledge of outcomes assessment is a major influence on his or her use of outcomes instruments.²⁶ However, another explanation for our results may be that the listed barriers did not address the concerns nonusers have with the use of PROMs.

Another interesting finding is the similarities between the barriers that were endorsed by AT-PRs and AT-NONs. Two common barriers were that PROMs take too much time for the patient to complete and too much time for the clinician to analyze. As mentioned earlier, in a 2012 systematic review, Duncan and Murray²⁶ indicated that time is a noted barrier in other health professions, including occupational therapy, physical therapy, physiotherapy, and communication disorders; athletic training is not alone in this concern. Furthermore, the amount of time to complete the instruments by patients, the number of patients seen by the clinician, and external factors that limit the time available to spend with patients were all common time barriers. Yet Duncan and Murray noted a trend toward the linkage of time barriers with apprehension about the applicability of the outcomes instrument. For example, the authors^30,34 of 2 studies of physical therapists identified concise and applicable outcomes instruments as increasing the likelihood of using the instrument in clinical practice.

Based on our findings, we are not able to explain why the AT-PRs overcame the time and analysis barriers but the AT-NONs did not. Duncan and Murray²⁶ provided possible explanations from related fields. In addition, the theory of planned behavior may be used to better understand the factors that influence a clinician's intention to systematically measure outcomes in clinical practice. The theory of planned behavior posits that the combination of attitudes toward a behavior, social norms, and perceived behavioral control all contribute to a person's intention to perform a behavior.³⁵ Preliminary research into the perceived benefits and barriers of outcomes assessment has provided insight into how the theory of planned behavior can be used to explain clinician behaviors. Previous research^14,16,33,36 has confirmed that a positive attitude toward the value of outcomes assessment plays a large role for those health care professionals who choose to measure outcomes systematically. Changes in policies and job site culture may affect social norms and increase willingness to use standardized outcomes measures.^29,36 Constraints in practice settings (perceived behavioral control), such as time and access to support, also affect a clinician's decision to use standardized measures.²⁶

Moreover, clinicians may not use PROMs because they do not believe the available instruments meet their clinical needs. Thus, instead of using instruments perceived to be inadequate for their patient population (eg, active individuals), they choose to not use them at all. More research needs to be performed to understand the behaviors of ATs in this regard. In the meantime, we can use strategies developed in other fields to address some of the deterrents of outcomes assessment. For example, a systematic implementation plan with a tailored-strategy approach aimed at identified barriers and facilitators to outcomes-assessment practices was successful in changing attitudes, increasing knowledge, and changing behaviors of physical therapists.³⁷ The tailored-strategy approach includes a problem analysis, literary search, interviews, and sounding-board meetings to develop specific strategies for implementing a behavior change. This method could be used in different athletic training practice settings to identify best practices for creating a positive culture, infrastructure, and resources for outcomes assessment.

Athletic Training Practice Patterns and Clinician Characteristics Related to Use of PROMs

Several of our findings relate to the practice patterns of PROM use reported by ATs, including employer-mandated use, which had a positive effect on the frequency of use by clinicians. We found that 46% of ATs in the AT-PR group were mandated by their employer to collect PROMs, and an additional 31% were encouraged to do so. These findings are in contrast to those of Hatfield and Ogles,¹⁵ who reported that most psychologists used outcomes measures for treatment-related reasons as opposed to external pressures such as workplace or payer requirements.

Although external pressures frequently present as mandates at the employer level, mandates can also come at the profession level. One type of profession-level mandate is professional publications, including clinical practice guidelines, position statements, and policy statements on the use of outcomes instruments in clinical practice. This type of mandate indirectly compels the use of outcomes measures by asserting that they are a component of best practices and therefore a requirement for meeting an established standard of care.

A second type of mandate stems from the development of specific education standards and related practice competencies, such as those used in education accreditation processes. This type of mandate compels use, or at least the instruction of use, as a standardized component of professional education. For example, in a study²⁹ completed in Australia, support for standardized outcomes assessment was added to a physiotherapy practice guideline and was combined with a multifaceted educational intervention that included seminars, educational materials (via hard copy and Web sites), and guidance from a peer. This series of interventions increased compliance with using standardized outcomes instruments from 30% to 66%.²⁹

Creating profession-level mandates in athletic training will require a collaborative effort between the major organizations of the athletic training profession, including the Board of Certification, the Commission on Accreditation of Athletic Training Education, and the NATA. The mandates could also be supplemented by a support toolkit of online resources. Other educational interventions, including small-scale educational opportunities and peer support to improve provider competence with using PROMs, have also had positive results.¹⁶

We investigated other practice characteristics related to the format in which ATs administered PROMs and the use of total and subscale scores. Most ATs were collecting PROMs through paper-based methods as opposed to computerized or electronic methods. In fact, computerized methods were used by only 7% of the AT-PRs, a value that corresponds with the findings of Jette et al.¹⁴ One reason for this finding may be that most PROMs are available in paper format, which is an easy, simple method of collecting the information, especially when support staff is limited or funding is low. However, of the individuals who reported using paper forms, only 25% calculated summary scores to assist in their analysis of the information. Lack of use of outcomes-measure scores has been reported by others, with 1 group²⁷ from mental health services noting that 92% of clinicians did not use scores from mandated standardized measures in their clinical practice. Failure to use summary scores warrants further investigation because these scores provide helpful information for objectively and systematically measuring patient progress over time and can be used for making clinical decisions. Clinicians may prefer and be more likely to use an outcome measure or system that has simplified scoring and interpretation, such as a narrative interpretation that accompanies scale scores.²⁷ Electronic systems may be useful for increasing the use of PROMs because they can be designed both to prompt patients to complete the measures and to autocalculate summary scores for clinicians, thereby streamlining the administration process. In 1 study,³² computerized data entry and analysis made the process of administering and analyzing outcomes measures feasible and effective, potentially decreasing the burden of time for the patient and clinician. Yet electronic systems may pose additional monetary (eg, training, technical support, software) or time (eg, training) costs to the clinician or clinical site, especially in the early stages of implementation.

Another clinician characteristic is whether a health care provider's degree or setting influences the use of PROMs. In New Zealand, physical therapists with a master's degree were more likely than those with a bachelor's degree to use outcomes measures in patients with low back pain.³⁰ Physical therapists in outpatient settings and home care settings were more likely to use standardized outcomes measures than therapists in acute care settings.¹⁴ However, we did not find a relationship between clinician or practice setting characteristics and the use of PROMs in our study.

Limitations

One limitation of our study is the low response rate. Compared with similar studies and suggestions in the literature,^14,21 a response rate closer to 40% is preferred. As the first survey study of its kind in the athletic training profession, though, we believe our findings are a resource for current practice patterns. Our survey asked specific questions about the use of PROMs and did not include clinician-rated outcomes, so no direct comparisons on usage patterns can be made between these outcomes. Additionally, because we used a previously validated survey, there were limitations to the structures of the questions, including the concern that the section related to the barriers of using PROMs for those who used and did not use them differed slightly in wording and number of questions. However, the benefit of being able to directly compare our results with the work of others who implemented the same survey was a necessary compromise. Another limitation is that we recruited only NATA members; therefore, the results may not represent the entire athletic training profession. Further, although our sample was designed to represent all 10 NATA districts, it was not constructed to reflect the percentage of ATs in each of the included practice settings. Finally, clinician competence with standardized measures is a common barrier cited by other studies,²⁶ but we did not investigate it directly. As a result, we do not know if clinician knowledge of PROMs affected our findings.

Future Research

More research is needed to further explore the concerns related to the use of PROMs in clinical practice. We identified a trend in that more practitioners in the clinic setting used PROMs (18%) than not (2%). This same general trend occurred for clinicians in the clinic/outreach, hospital, and industrial/occupational settings. All other settings (high school, college/university, and 2-year institution) showed the reverse trend: a greater percentage of clinicians in these settings did not use PROMs than did use them. Certain settings may be more conducive for collecting patient outcomes because outcomes collection can be tied to reimbursement for services provided. Future investigators should study this trend to determine if practice setting affects the use of PROMs in patient care.

Another area for further research is identifying strategies to enhance the use of PROMs. Although we did not investigate strategies for implementation, other disciplines have made practice changes to encourage the collection of patient outcomes. For example, 2 events related to physiotherapy practice in Australia led to an increased use of standardized outcome measures. First, the Australian Physiotherapy Association adopted a treatment justification position statement that included the professional requirement to measure patient outcomes.²⁹ Second, a clinical justification model relevant to physiotherapy practice was released that required monitoring outcomes for individual patients.^29,38 Together, these professional changes led to increased collection of patient outcomes. Whether or not practice guideline changes related to outcomes assessment would positively influence the collection of patient outcomes in athletic training is unknown, but efforts should be made to investigate approaches that have been successful in other disciplines.

Finally, since both groups in the current study reported similar barriers to using PROMs, future authors should research areas related to clinician motivation, such as the theory of planned behavior, to increase the use of PROMs.

CONCLUSIONS

The majority of respondents in our study did not use PROMs in clinical practice. Of the respondents who did use them, most reported the benefits as enhanced communication with patients and other health care providers and improved plans of care for patients. However, among clinicians who did and did not use PROMs, many believed these measures were confusing to patients and too time consuming for both the patient and clinician. The routine use of PROMs in athletic training is a tangible, profession-wide goal that will require a cultural and practice change in the profession and in individual ATs. By understanding the barriers to routine use of PROMs in clinical practice, such as time, we can implement training and education programs that provide tools to practicing ATs to overcome these barriers. Encouraging the routine implementation of PROMs into clinical practice is also necessary to advance the profession. A change in practice may require strategies beyond training, education, and encouragement though. The reality is that our practice must evolve with the change in the overall health care system, which requires guidance and leadership from our professional organization. For example, recent changes in health care reimbursement resulting from health care reform are partially based on the assessment of patient outcomes. As in other professions, mandates or position statements from the NATA that endorse the collection of PROMs as the standard of care are needed to remain current with the evolving health care system. The evolution of our profession to one that embraces the collection of patient outcomes as central to our clinical practice will take time and is an effort we must not delay.