Modified Outpatient Physical Therapy Improvement in Movement Assessment Log (mOPTIMAL): A Responsive and Reliable Tool for Patients with Non-Operative Shoulder Pain

Abstract

Background:

Medicare regulations require that physical therapists report functional limitations and severity modifiers utilizing a claims-based data collection tool. The Modified Outpatient Physical Therapy Improvement in Movement Assessment Log (mOPTIMAL) captures key constructs about patient confidence and difficulty but has not been evaluated for responsiveness/ reliability during a routine clinical encounter with patients who have shoulder pathology. The purposes of this retrospective study are to 1) explore if mOPTIMAL changes after a single session with a physical therapist, and 2) determine if the tool is reliable among people with non-operative shoulder pain.

Methods:

We included 106 individuals (58% female; mean age 45.8; range: 18-94 yrs.) with “non-operative” shoulder pathology who were seen in outpatient physical therapy from 2011 to 2012. Subjects completed a mOPTIMAL survey and a pain scale before and immediately after the initial physical therapy visit. The mOPTIMAL is a patient-centered instrument that assesses how much “Difficulty” and “Confidence” a client has in performing a battery of functional tasks. T-tests, Cronbach’s Alpha, and Intra-class Correlations were used to assess responsiveness, internal consistency, and reliability, respectively.

Results:

After a single visit, participants reported improved Confidence with sleeping, dressing/ bathing, throwing, carrying, and lifting (adjusted for ceiling effects; p<0.002) but no change in pain. Cronbach’s Alpha and Intra-class Correlations were excellent (0.821-0.923; 0.967, respectively).

Conclusions:

mOPTIMAL is a reliable and responsive tool with excellent internal consistency. This observational study revealed that patient Confidence may change independent of Pain after a single physical therapy visit. Taken together, the mOPTIMAL appears to be an excellent tool to report severity modifiers in compliance with Medicare regulations.

Level of Evidence: IV

Introduction

Quality improvement initiatives and pay for performance reimbursement strategies emphasize the need for high quality patient centered measures when completing the electronic medical record.^1,2 Medicare regulations require that physical therapists report functional limitations and severity/complexity modifiers utilizing “a claims-based data collection strategy.”^3,4 Based on its ease of use, the Outpatient Physical Therapy Improvement in Movement Assessment Log (OPTIMAL)⁵ may be an acceptable tool to meet this need.⁴

The OPTIMAL is a patient centered instrument that assesses how much “Difficulty” and “Confidence” a client has in performing each of 24 functional tasks and three cognitive tasks using a generic assessment scale. The developers of the OPTIMAL suggest that patient Confidence is a construct based upon Bandura’s self-efficacy theory which would indicate a person’s “sense” of mastery over the ability to perform actions.⁵ It is not clear, however, what patient Confidence actually measures. Some suggest Confidence may represent an underlying psychological construct related to the concern a client has about her/ his impairment or disability.⁶ Many clients receiving physical therapy treatments have an underlying “fear” associated with their condition^7-11 which may be predictive of those who develop chronic pain conditions and disability^12-14 or benefit from improved education about how to manage their condition.^15-17 Investigators have reported the effects of long term interventions on decreasing “fear” about an impairment or disability.^7,18-22 However, to our knowledge, no study has examined the responsiveness and reproducibility of a single physical therapy treatment session on patient confidence in people with non-operative shoulder pain.

Although our primary goal in exploring these retrospective data was to assess the psychometric properties of the mOPTIMAL, we also questioned whether Confidence would be sensitive to change after a single visit with a physical therapist. We reasoned that a patient’s Confidence may improve when the patient either becomes educated about his/her condition or believes he/ she is on the “right track” in terms of management. We would not expect change after a single visit on various items of functional difficulty associated with long term adaptations and secondary impairments (i.e. muscle atrophy),^23,24 but may observe a change in Confidence after a single visit.

Because 21 percent of all Medicare patients seeking outpatient physical therapy have shoulder pathology (second only to the lumbar area²⁵) we sought to examine this dataset as part of our standards of practice and quality assurance program. At our tertiary healthcare center, we previously modified the OPTIMAL, as described in this report, and developed a consensus-based intervention as part of our standards of practice initiative to minimize variation. Importantly, many patients with non-operative shoulder conditions are treated with a single physical therapy visit within our healthcare center.

Accordingly, the purposes of this retrospective study are to: 1) explore if certain domains of the mOPTIMAL systematically change after a single session with a physical therapist, and, 2) determine if the tool is internally consistent and reliable among people with non-operative shoulder pain. Our underlying premise for mining these data was that any systematic change after the single visit may be more obvious in a patient’s perception of Confidence, while patient perception of Difficulty in performing a task may be less likely to change. We also expected that the assessments for most domains would be highly reproducible, even if separated by a single encounter with a physical therapist.

Methods

Subjects

The use of the retrospective de-identified data explored in this study was approved by the University of Iowa Human Subjects Institutional Review Board. We report on 106 adult patients (62 females) with an average age of 45.8 years (range 19-84 years), average body mass index (BMI) of 29.0 kg/ m2 (range 19.1-57.1 kg/ m2), average height of 1.69 m, and average mass of 81.52 kg. Participants were patients with “non-operative shoulder pain” referred for outpatient physical therapy at our academic medical center over the course of one year (2011-12). We included patients who had received past physical therapy or corticosteroid injections to the shoulder. We excluded subjects who had prior surgery to their involved shoulder or the shoulder was not the source of the pain. In total, 106 subjects completed the Pre and Posttests with the mOPTIMAL. The most common medical diagnosis was “Shoulder Pain” (n=54) followed by “Impingement” (n=18) and “Rotator Cuff Tendonitis” (n=11).

Instrument Modification

The changes in Medicare reporting guidelines necessitated a review of the patient reported functional outcome tools used across our department. Our Physical Therapy Department has a long history of developing clinical rehabilitation measurements²⁶ to be incorporated into the electronic medical record for observational outcomes research.¹ We also strive to develop consensus based physical therapist interventions to minimize variation in practice.²⁶ The mOPTIMAL outcome tool was developed from the template of the original OPTIMAL. Both the Confidence and Difficulty subscales were retained. Only six of the original 24 OPTIMAL items were specific to upper extremity activities.⁵ Because of our interest in patients with shoulder pathology, we eliminated the items which were meant to analyze trunk and lower extremity function (Items 1-18). We added four items that were determined to be pertinent information regarding upper extremity function as determined by eight expert clinicians (physical therapists) with an average of 18 years of experience treating people with shoulder pathology. The result was a list of 10 functional items (the original six: pushing, pulling, reaching, lifting, grasping, and carrying, plus the four items we added: dressing/ bathing, sleeping, throwing, and driving) on which patients would assess their Confidence and Difficulty. The activity, sleep, is a common cause of concern for clients with shoulder pain and was considered important by the clinical experts.

We assessed Confidence and Difficulty using the 5-point Likert Scale as originally designed for the OPTIMAL. Because the OPTIMAL does not include pain, we added a standard 10-point Likert Scale with 0 representing “no pain” and 10 representing a “very painful” condition. The patients were asked to assess their pain for that day. The modified OPTIMAL form used in this study is included in Supplemental Digital Content (Appendix 1).

Physical Therapy Visit Procedure

The initial physical therapy interaction was, on average, 45 minutes in length. When a patient with “non-operative shoulder pain” checked in for their appointment, a receptionist administered the mOPTIMAL form and provided an informational sheet describing the importance of patient centered outcome measures as a standard assessment in our department. The patient was led to a private exam room where the therapist performed their standard physical therapy evaluation consisting of observation, palpation, range of motion, strength measures as appropriate, and special testing as needed. Each assessment verified that the patient’s shoulder complex was the origin of their discomfort as opposed to radicular spinal pain or pain of a referred nature. Fourteen patients were excluded from this retrospective analysis because of radicular signs or uncertainty about the source of the shoulder pain.

Our consensus based single physical therapist treatment plan consists of four distinct phases: Pain Relief, Early Strengthening, Advanced Strengthening, and Return to Activity. Our guidelines did not discourage therapists from customizing their prescribed treatments and perform specific interventions as deemed necessary under the general treatment categories. The frequency of scheduled physical therapy follow-up visits was not dictated by these guidelines but was left to the discretion of the treating therapist.

As verified in our retrospective evaluation forms, all initial physical therapy interactions consisted of a patient interview, evaluation, and some initial form of treatment, most often a form of treatment the patient could replicate independently at home. Therapists answered questions the patients had regarding their condition. At the conclusion of the session, patients returned to the reception desk where they completed a post-treatment mOPTIMAL survey.

Data Collection

Upon review we identified 106 patients who met our inclusion criteria that completed the mOPTIMAL before and immediately following their single physical therapy encounter.

Data Analysis

The mOPTIMAL scale ranges from 1 (“Able to do without any Difficulty”) to 5 (“Unable to Do”). Similarly, a number 1 on the Confidence scale indicates poor Confidence with a specific task. The average Difficulty and average Confidence scores were calculated for each patient across the 10 functional tasks. If patients did not feel an item was relevant to their situation, they indicated that it was not applicable. In all, only 142 (3.3%) of the 4240 measurements were scored as “does not apply”.

Responsiveness to a single PT visit

A Students Dependent T-test with a Bonferroni adjustment was used to determine if there was a systematic change in the patients’ Difficulty and Confidence with each task after the single visit with a physical therapist. Because 20 comparisons (10 functional tasks, Difficulty and Confidence scales) were carried out, we adjusted our p-value to 0.0025 (0.05/20). We excluded two items that had over 30% of the sample showing a ceiling effect on initial evaluation.

Reliability of the mOPTIMAL

Pearson Product-moment Correlations were calculated to determine the relationship between the Pre and Posttest conditions. Intra-class correlations (ICC, 2,k)²⁷ were calculated to assess the degree of agreement in the repeated measures. In addition, a Cronbach’s Alpha was calculated to assess the internal consistency of the 10 test items within the mOPTIMAL. All Data analysis was carried out using SPSS .

Results

Single Visit Change in Confidence, Difficulty, and Pain

The aggregate descriptive changes in Confidence and Difficulty after the single physical therapy visit are presented in Table 1. The overall mean Confidence score was significantly improved (p=0.041) after a single visit, while the overall Difficulty and Pain scores were not systematically changed (p=0.244).

Table 1.

Difficulty, Confidence, and Pain Scores Before and After the Single Physical Therapy Visit

	Difficulty (1-5)		Confidence (1-5)		Pain (0-10)
	Pre	Post	Pre	Post	Pre	Post
Mean (SD)	2.45 (1.07)	2.39 (1.05)	2.60 (1.24)	2.44 (1.27)	4.29 (2.38)	4.18 (2.41)
Average Change		0.06		0.16		0.11
Effect Size		0.06		0.13		0.05
t Test		0.10		0.04*		0.06

Difficulty and Confidence values are mean (SE) for all ten OPTIMAL task items. Pain was rated on a 10-point Likert scale. (* = p < 0.05)

An analysis of each of the 10 task items, adjusted for the number of tests, revealed that the patients’ Confidence with sleeping demonstrated the greatest improvement after the single visit (t=3.44, DF=105, p = 0.0008). When we accounted for the subjects who were fully confident with sleeping in their pre-assessment (ceiling effect), the change remained highly significant (t=3.43, DF=82, p<0.0001; Table 2). In addition, the patients’ Confidence for lifting, carrying, throwing, and dressing/ bathing significantly improved after adjusting for pre-assessment ceiling effects (Figures 1A, 1B; p=0.001, p<0.0001, p<0.0001, and p<0.0001, respectively). Prior to adjusting for ceiling effects, there were no items significantly changed for the Difficulty scale. However, after we adjusted for ceiling effects, the change in patients’ Difficulty scores for lifting and carrying were significantly improved (p=0.001 and 0.0001, respectively) (Figure 1C, 1D).

Table 2.

Analysis of Individual OPTIMAL Task Items Before and After Ceiling Effect Correction

Function	Subscale	Effect Size (N=106)	Significance (N=106)	Effect Size (adjusted for ceiling)	Significance (adjusted for ceiling)
Pushing	Difficulty	-0.01	NS	0.52	NS
	Confidence	0.05	NS	0.55	NS
Pulling	Difficulty	0.04	NS	0.53	NS
	Confidence	0.06	NS	0.48	NS
Reaching	Difficulty	0.11	NS	0.30	NS
	Confidence	0.18	NS	0.49	NS
Lifting	Difficulty	0.16	NS	0.55	S*
	Confidence	0.12	NS	0.60	S*
Carrying	Difficulty	0.17	NS	0.79	S*
	Confidence	0.10	NS	0.70	S*
Bathing	Difficulty	0.03	NS	0.48	NS
	Confidence	0.14	NS	0.87	S*
Sleeping	Difficulty	0.05	NS	0.39	NS
	Confidence	0.29	S*	0.88	S*
Throwing	Difficulty	0.05	NS	0.49	NS
	Confidence	0.11	NS	0.80	S*

Values are the mean effect size for post-visit versus pre-visit scores. S*=significant at p < 0.0025; NS= not significant.

Figure 1.

OPTIMAL data before and immediately after a single session of physical therapy.

We analyzed Pain using a 10-point Likert Scale. Paired t-tests did not demonstrate any significant differences between Pre (Mean= 4.29 + 2.38) and Post (Mean=4.18 + 2.41) pain measures (p=0.741) after an encounter with a physical therapist.

Reliability of the mOPTIMAL Assessment Tool

A total of 106 patients with “non-operative shoulder pain” completed the Pre and Post assessments separated by a single encounter with a physical therapist. Pain ratings demonstrated a high degree of agreement between Pre and Post assessments with an overall intraclass correlation (ICC (2,1)) value of 0.962 (95% CI=0.942-0.973). The mOPTIMAL subscale for Difficulty was highly reproducible with an ICC (2,k) value of 0.943 (95% CI=0.912-0.963). The lowest ICC (2,k) value was 0.824 (95% CI=0.731-0.87) for Confidence, suggesting a lower level of reliability or the presence of a systematic change after the single physical therapy visit. Pearson correlations comparing patient ratings before and after the physical therapy visit largely mirrored ICC results. Correlations for the 10 Difficulty tasks varied from 0.765 to 0.831; while Pre to Post correlations for Confidence were again the lowest (0.594 to 0.713). The Pearson correlation for Pain was high (0.964) suggesting that pain was not changed after the single visit with a physical therapist.

The internal consistency for the mOPTIMAL was excellent for Confidence and Difficulty (Cronbach’s Alpha ranged from 0.821 to 0.904).

Discussion

Our analysis suggests that Confidence, with the ability to sleep, showed a systematic change (improvement) after a single encounter with a physical therapist for patients with non-operative shoulder pain; while Difficulty in performing functional tasks did not change. When we adjusted for ceiling effects people did perceive they would have less Difficulty lifting and carrying objects. Both systematic changes in perception may reflect the educational component of a session with a physical therapist. The level of pain showed no change after the single session. Taken together, this retrospective analysis supports that the mOPTIMAL is a reliable and internally consistent tool to assess 10 key functional activities (including sleeping) in people with non-operative shoulder pain. A trend that certain domains, like patient confidence, may be impacted by a single visit with a physical therapist is interesting, but not conclusive without further assessment with control conditions.

Nonetheless, the observation that there was a systematic change, primarily with patient confidence, is an interesting concept. There is strong evidence in the literature that patients develop an alliance with their physical therapist, and this alliance impacts patient outcome after eight weeks of treatment.²⁸ Recently, a single session with a physical therapist has been shown to alter movement strategies patients use for lifting, with a corresponding decrease in pain back pain.^29,30 But to our knowledge this is the first report that a single physical therapy encounter may influence patient perception of an impending outcome in response to treatment. Because this is a retrospective observational study, the therapists and patients were not biased by factors that may influence clinical behaviors when both the patient and provider are informed that a study is being undertaken through the consent process. However, because we have no control condition, we cannot be assured that a meeting with a lay person, or merely asking the patient to read information about sleep, would trigger a similar change in confidence with sleep. Importantly, this study does support that the mOPTIMAL is responsive to a systematic change, regardless of the underlying cause; and the mOPTIMAL is a highly reliable outcome assessment tool with excellent internal consistency. Indeed, this survey may be ideal for documenting complexity as recommended by the Medicare standards.

The underlying psychological constructs that influence patient responses to single encounters are not fully developed in the literature and warrant review. For example, Mintken et al.^31,32 found a single session of mobilization, an active, hands-on event, produced a significant reduction in pain, fear avoidance, and kinesiophobia for patients with shoulder pain. The construct “patient Confidence” was not examined. While evaluating the reliability of outcome questionnaires for patients with low back pain, neither Williams and Myers⁶ nor Yamada et al.³³ found any changes in patient Confidence as the result of a single physical therapy encounter. This lack of change in Confidence for patients with spine pathology may be related to the greater interplay between patient psychology and disorders of the spine.^22,34,35 A single bout of exercise testing and instruction was shown to improve self-efficacy scores and predict adherence to a general exercise program in a group of older adults.^36,37 In patients with musculoskeletal pain, Mosely et al.^38,39 demonstrated a single educational session, focused on the neurophysiology of pain, can alter pain and pain attitudes in patients with chronic low back pain. We also know that pre-operative educational sessions are commonly used to moderate patient expectation prior to total joint arthroplasty.¹⁵ Although these studies demonstrate the effect of a single interaction, none of these previous studies measured the patient’s Confidence in performing specific tasks.

Disordered sleep has a significant effect on health care utilization and costs.^40,41 The ability to sleep was identified by our group and others⁴² as an integral component in promoting health in people with shoulder pathology. Our independent panel of physical therapists agreed that pathology to the shoulder directly disrupts the quality of sleep for many clients. Importantly, there are strong correlations among anxiety, depression, pain, and sleep disturbances in patients with shoulder pathology.⁴³ From a mechanical perspective, passive tension on the rotator cuff at night or changes in sub acromial pressure in various positions common during sleep are important considerations for people with shoulder pathology.⁴⁴ Our expert panel of physical therapists appear to have been clinically astute when recommending that we add sleep to the mOPTIMAL during our quality assurance development program. Interestingly, confidence with the ability to sleep, was the one item that had a moderate effect size,⁴⁵ after just a single session with a physical therapist. Previous reports using the general OPTIMAL detected small effect sizes,⁵ however, those studies included all musculoskeletal conditions and were not limited to people with shoulder pathology. Moreover, we modified the original OPTIMAL to better focus on activities related to the upper extremity. Because this is a retrospective analysis and the first report using this mOPTIMAL in people with shoulder pathology, our outcomes are not directly comparable to many previous reports using the OPTIMAL.^5,46

We also explored other variables that may assist us in understanding the perceived response to improved ability to sleep. We found no relationship between sleep Confidence scores and gender, shoulder pathology, treating therapist, or initial Difficulty and Confidence scores. We did discover that participants who demonstrated the most improved Confidence with sleep had a slightly lower BMI (27.1 kg/ m2 vs 29 kg/ m2) and had slightly lower initial pain scores (3.90 vs 4.29). Among this subset, sleeping confidence improved more than 1 point (a 20% improvement) as the result of a single physical therapy visit. In the absence of a strict control group, we are unable to state that the intervention offered by the physical therapist was critical to this outcome and therefore further investigation is warranted.

As part of this report, we identify three features that are new to the OPTIMAL assessment tool that we adopted as part of our standard of care program. First, the belief that participants commonly gain Confidence because of the skills/traits of health care providers offers an important area of future investigation. The “fear of the unknown,” once addressed, may ultimately impact a client’s Confidence and expectations to improve in various health related constructs. In this respect, even the initial visit may have a profound impact on easing patient fears and improving confidence that his/ her condition will improve with treatment. Second, we re-instituted Pain as a separate construct so we could evaluate the extent to which difficulty and pain are reproducibly measured over a single physical therapy visit, i.e. a time frame over which improvements would be unlikely to occur. Third, we added additional test items that we believed were more germane for patients with upper extremity pathology.

Based on the outcomes of this analysis, we believe that mOPTIMAL is intuitive and easy to implement in the clinical center. In addition, mOPTIMAL allows the Difficulty and Confidence subscales to be applied to any functional task important to the patient, thus the opportunity for the assessment to become patient centered rather than physician, physical therapist, or institution centered.

In summary, the mOPTIMAL is a psychometrically sound measurement tool that is responsive to systematic change after an encounter with a physical therapist. The mOPTIMAL may be well suited to meet the Medicare severity/complexity documentation requirement for people with non-operative shoulder impairment.

Appendix 1: Modified Optimal Survey

Instructions: Please circle the level of Difficulty you have for each activity today.	Able to do without any Difficulty	Able to do with little Difficulty	Able to do with moderate Difficulty	Able to do with much Difficulty	Unable to do	Not Applicable
Pushing	1	2	3	4	5	9
Pulling	1	2	3	4	5	9
Reaching	1	2	3	4	5	9
Grasping	1	2	3	4	5	9
Lifting	1	2	3	4	5	9
Carrying	1	2	3	4	5	9
Bathing/Dressing	1	2	3	4	5	9
Sleeping	1	2	3	4	5	9
Throwing	1	2	3	4	5	9
Driving	1	2	3	4	5	9

Instructions: Please circle the level of Confidence you have for doing each activity today.	Fully confident in my ability to perform	Very confident	Moderate Confidence	Some Confidence	Not confident in my ability to perform	Not Applicable
Pushing	1	2	3	4	5	9
Pulling	1	2	3	4	5	9
Reaching	1	2	3	4	5	9
Grasping	1	2	3	4	5	9
Lifting	1	2	3	4	5	9
Carrying	1	2	3	4	5	9
Bathing/Dressing	1	2	3	4	5	9
Sleeping	1	2	3	4	5	9
Throwing	1	2	3	4	5	9
Driving	1	2	3	4	5	9

Please rate your overall shoulder pain today.

No Pain										Very Painful
0	1	2	3	4	5	6	7	8	9	10