THE ADOLESCENT MEASURE OF CONFIDENCE AND MUSCULOSKELETAL PERFORMANCE (AMCAMP): DEVELOPMENT AND INITIAL VALIDATION

Abstract

Background

Although the relationship of self-efficacy to sports performance is well established, little attention has been paid to self-efficacy in the movements or actions that are required to perform daily activities and prepare the individual to resume sports participation following an injury and associated period of rehabilitation. There are no instruments to measure self-confidence in movement validated in an adolescent population.

Purpose

The purpose of this paper is to report on the development of the AMCaMP, a self-report measure of confidence in movement and provide some initial evidence to support its use as a measure of confidence in movement.

Methods

The AMCaMP was adapted from OPTIMAL, a self-report instrument that measures confidence in movement, which had been previously designed and validated in an adult population. Data were collected from 1,115 adolescent athletes from 12 outpatient physical therapy clinics in a single healthcare system.

Results

Exploratory factor analysis of the 22 items of the AMCaMP using a test sample revealed a three factor structure (trunk, lower body, upper body). Confirmatory factor analysis using a validation sample demonstrated a similar model fit with the data. Reliability of scores on each of three clusters of items identified by factor analysis was assessed with coefficient alpha (range = 0.82 to 0.94), Standard Error of Measurement (1.38 to 2.74), and Minimum Detectable Change (3.83 to 7.6).

Conclusions

AMCaMP has acceptable psychometric properties for use in adolescents (ages 11 to 18) as a patient-centric outcome measure of confidence in movement abilities after rehabilitation.

Level of Evidence

IV

INTRODUCTION

Return to sport is a critical goal in rehabilitating adolescents engaged in athletics. Although these athletes may be most concerned with their abilities to execute the movements essential to their sport, physical rehabilitation of an injury begins with the individual's ability to execute the actions of daily activities (e.g., bending, running, hopping, lifting, reaching) that will eventually support accomplishment of sport-specific tasks (e.g., turn a handspring, shoot a lay-up, throw a pass) properly after the patient returns to athletic engagement.

Although the relationship of self-efficacy to sports performance is well established,^1-3 little attention has been paid to self-efficacy in the movements or actions that are required to perform daily activities. These movements, once restored, form the basic “vocabulary” of the sport-specific movements that will eventually allow the individual to resume sports participation. The proper execution of these movements depends in part on an individual's confidence in being able to perform the movement, which may be particularly challenging if the individual has suffered a re-injury. Derived from the broader social psychology literature, self-efficacy was first conceptualized by Bandura,⁴ who proposed that situation-specific beliefs about one's capabilities to perform specific tasks will help to determine what tasks individuals will choose to do, the energy and attention they will devote to doing it, and the perseverance that will be displayed to execute a specific level of performance when confronted by barriers to success. If one lacks confidence in the ability to move correctly to perform everyday activities, then it is highly unlikely that the athlete will perform more demanding movements of a sport.

Although there is a plethora of infant and child development scales as well as standardized assessments for adults, especially older adults, few instruments have been expressly developed for and validated in adolescent athletic populations.^5-7 Such is also the case for instruments that measure self-efficacy, despite the importance of self-efficacy to effective rehabilitation.⁸ After searching the literature for such an instrument that would be relevant to the movements of daily life and support the various movements required by the particular sport in which these adolescent patients participated, it was concluded that the confidence scale of OPTIMAL⁹ was a suitable candidate for a general instrument measuring movement self-efficacy that might be validated on adolescent athletes. OPTIMAL has a specific focus on movement, and had known psychometric properties for populations as young as 18 years of age. The purpose of this paper is to report on the development and initial validation of the Adolescent Measure of Confidence and Musculoskeletal Performance (AMCaMP), a self-report measure of confidence in movement.

METHODS

Instrument Development Process

The OPTIMAL confidence scale is a 22-item self-report measure on an individual's confidence in performing 22 basic movements such as rolling, sitting, standing, walking, running, lifting, and carrying. Its psychometric properties were evaluated on 360 individuals over the age of 18. All items were rated on a 1-5 scale (fully confident to no confidence) at initial visit and at discharge. The preliminary draft of the AMCaMP used the same Likert-type scale for each item. Designed as a patient-centric measure that would aid treatment planning for adults, OPTIMAL also contains an item in which respondents are given the opportunity to identify three movements or actions that they would like to perform better. This item was also carried over as a single question to the preliminary draft of AMCaMP.

Although OPTIMAL used a visual analogue scale to make a global assessment of confidence in movement, this question was re-written for the AMCaMP's intended population by eliciting a global estimation of self-efficacy at discharge as it related to return to sport. This global item, “Are you ready to return to your previous level of physical activity?” had a dichotomous response (i.e., yes/no).

Study Sites and Participants

We gathered data from patients who were referred to physical therapy for sport-related injuries from 12 outpatient clinics in the Children's Healthcare of Atlanta (CHOA) system. The institutional review board of CHOA approved the data collection process with exemptions from the institutional review boards of George Mason University and the Ohio State University. Traditionally the onset of puberty is accepted as the beginning of adolescence, which was arbitrarily operationalized as 11 years of age. All therapists at each site were instructed on who was eligible to participate in the study and how to collect the data. Any new patient was eligible to participate in the study if the patient was: (1) 11 to 18 years of age; (2) spoke and read English; and (3) had the cognitive ability to complete the questionnaire independently. Demographic data were also collected on the first visit.

Pilot Testing for Reading Level and Comprehension

The reading level of the draft instrument was assessed by CHOA's Learning Center and deemed appropriate for this age group. The OPTIMAL's confidence scale was administered as it had been originally published to 217 adolescents drawn from two pediatric clinics to gather feedback on the readability of the instrument by directly debriefing subjects after each administration to identify items which they did not understand. From these preliminary tests, it was learned that many of the adolescents in our test sample did not know the meaning of “stooping,” which had been used in OPTIMAL on one item (“bending/stooping”). Therefore this word was eliminated from the subsequent version of the instrument.

OPTIMAL measures self-confidence in movement in everyday (i.e., non-sports related) activities. Because adolescent athletes may not perceive a clear “boundary” between every-day life and their sports-related activities an explicit distinction was subsequently made about context in the instructions for study participants based on the questions they asked in filling out the AMCaMP (see Appendix for the full instrument). Subjects were instructed to think only of their non-sports-related activities.

Data Analysis

Descriptive statistics were calculated to assess the demographic and clinical characteristics of study participants. The entire sample was divided into a test sample and a validation sample (i.e., potential models were generated using half of the sample and then these models were evaluated in the half not used to generate them) to help guard against over-fitting sample-based error during the factor analyses.¹⁰ Exploratory factor analysis (EFA)was used to generate plausible hypothetical models which were then tested in the confirmatory factor analysis (CFA) phase. These analyses were used to understand: 1) how many constructs were assessed by the 22 items; and 2) which items were related to which constructs. Once a model which adequately explained the observed data was identified, a replication of that model in the validation sample was attempted to assess reproducibility of the results. As the data are categorical in nature (i.e., five-point Likert-type responses), polychoric correlation matrices¹¹ were analyzed in both EFA and CFA. EFA model parameters were estimated using the CEFA software package.¹² Ordinary least squares estimation in CEFA was used and, when applicable (i.e., when estimating models with more than one factor), an oblique CF-Quartimax rotation was employed.¹³ CFA model parameter estimates were obtained using Mplus.¹⁴ A mean- and variance-adjusted weighted least squares estimator (WLSMV)¹⁵ was used in Mplus.

Once the factor analyses were complete and a satisfactory latent structure was established, the newly formed scales were tested using standard psychometric methods from classical test theory including coefficient alpha,¹⁶ standard error of the measurement (SEM) calculated as

$$SEM=\widehat{\sigma }\sqrt{1-\widehat{\alpha }},$$

where σis the estimated standard deviation for test scores and α is an estimate of the reliability (coefficient alpha, in our case), and minimal detectable change (MDC)¹⁷ calculated as

$$MDC=SEM*1.96\sqrt{2}.$$

It is important to note that both SEM and MDC can also be calculated using repeated measures-type information, but that is not the route that pursued here. Also, it is important to remember that MDC is about detectable change, not necessarily important or meaningful change. There are suggested methods to address the meaningfulness of change, but they can be complex, involve variables outside the measure, and are beyond the scope of this project.

RESULTS

Sample

Demographic information on the sample is presented in Table 1. During the period of data collection, 1,115 adolescents with sports-related needs for rehabilitation were eligible for study. The population for this study was predominantly female, and 14.3 years old on average. The majority of participants were full-time students in middle and high school, who had been participating in sports for almost four years.

Table 1.

Study Population Demographics and Characteristics

STUDY POPULATION	N = 1,115
	n	%
Gender
Male	495	44.39
Female	620	54.60
Mean Age (years)
	14.3	Range = 11-18
School Grade
Elementary (less than 6th)	22	1.97
Middle (6th through 8th)	256	22.95
High (9th through 12th)	381	34.17
Greater than 12th	11	.986
Missing	445	39.91
Average Years Sports Participation
	3.8	Range = 1 to 15
Sport Played
Basketball	120	10.76
Soccer	113	10.13
Football	103	9.23
Baseball	97	8.69
Other	227	20.35
Missing	455	40.80

Of these 1,115 patients, 829 individuals had an initial examination. For factor analyses, only intake data were used. List-wise deletion (where any individual with a missing value is deleted) was used on the data before factor analyses were performed yielding a final sample of 661 subjects with complete data. This total sample was then split into a test sample and a validation sample (sample sizes of 331 and 330, respectively).

Exploratory Factor Analyses

Figure 1 displays a scree plot, which contains the eigenvalues associated with the sample polychoric correlation matrix computed using the test sample. There are a wide variety of “rules” governing interpretation of scree plots. Rather than using such information in a dogmatic fashion, the data from the scree plot was used to identify potentially viable solutions. One can often look for an “elbow” in a scree plot, which represents an inflection point after which the subsequent eigenvalues are all very similar. In order to keep the model as simple as viable, 1-, 2-, 3-, and 4-factor solutions were explored. For reasons of interpretation, the 3-factor model was preferred over the 1- or 2-factor model. Estimation of a fourth factor yielded a “walking” factor, which related primarily to the three items which reference walking. Although this extra covariance might have been worth modeling in some respects, the research team did not regard it as an interesting common factor.

Figure 1.

Scree Plot for the 22-item Core Set.

Confirmatory Factor Analyses

In the CFA phase of analysis, a decision was made to test a 1-factor model, a 3-factor model with independent clustering (where each item relates to only the common factor it showed the strongest association with in the 3-factor EFA solution), a 3-factor model with cross-loadings (all of the loadings from the independent clustering model plus any additional loadings that were greater than 0.3 in absolute magnitude in the 3-factor EFA solution), and a 3-factor independent clustering model with correlated residuals among the three “walking” items. These four models are designated 1F, 3FIC, 3FCL, and 3FIC-W, respectively. In addition, a more data-drive fifth model, which was selected by addressing the largest modification indexes identified by Mplus, was explored. This model, called 3FIC-MI, contained all the same parameters as 3FIC-W, but added three additional correlated residuals between items 14 and 15, items 17 and 18, and items 21 and 22. The results from fitting these five models are summarized in Table 2.

Table 2.

CFA Model Fit Summary

Model	chi^2	df	RMSEA	90% CI	TLI
1F	2670.9	209	0.189	(0.182, 0.195)	0.859
3FIC	975.1	206	0.106	(0.1, 0.113)	0.955
3FCL	897.4	201	0.102	(0.096, 0.109)	0.958
3FIC-W	866.8	203	0.099	(0.093, 0.106)	0.961
3FIC-MI	582.7	200	0.076	(0.069, 0.083)	0.977
V 3FIC	697	206	0.085	(0.078, 0.092)	0.973
V 3FIC-W	620.7	203	0.079	(0.072, 0.086)	0.977
V 3FIC-MI	503.3	200	0.068	(0.06, 0.075)	0.983

Note. df = degrees of freedom, RMSEA = root mean square error of approximation, CI = confidence interval, TLI = Tucker Lewis Index, 1F = 1-factor model, 3FIC = 3-factor independent clustering model, 3FCL = 3-factor model with cross-loadings, 3FIC-W = 3-factor independent clustering model with correlated residuals among the walking items, 3FIC-MI = 3-factor independent clustering model with three model driven correlated residuals, V 3FIC = cross-validation run of 3FIC model, V 3FIC-W = cross-validation run of 3FIC-W model, V 3FIC-MI = cross validation run of 3FIC-MI model.

The 1-factor model fit poorly (RMSEA = 0.19, TLI = 0.86), which is consistent with the EFA results. The 3-factor model with independent clustering (3FIC) fit the data significantly better (RMSEA = 0.11, TLI = 0.96). Although the TLI value was acceptable,¹⁰ the RMSEA value was on the border of values usually considered representative of acceptable model fit.¹⁸ Allowing additional parameters to be estimated in 3FCL and 3FIC-W improved model fit slightly, but the 90% confidence intervals around the RMSEA values were overlapping. Adding the three additional parameters identified by Mplus as having the largest corresponding MIs did significantly improve the model fit (RMSEA = 0.08, TLI = 0.98).

Cross-validation

Three models (3FIC, 3FIC-W, and 3FIC-MI) were explored in the validation sample to evaluate the extent to which these models fit a (relatively) new data set as well as the test sample. As the test and validation samples were chosen at random from the original sample, differences in the fit of these models help us understand sampling variability and avoid over-fitting models to sample-specific error. These results are also summarized in Table 2, where the models have the same name with a new “V” prefix to identify those results as coming from fitting a model in the validation sample. All three models showed slightly better fit in the validation sample. This is an encouraging result which suggests that over-modeling sampling error in the original sample did not occur.

When considered in their totality, these results led to the conclusion that there were three common factors being measured by the 22 items in the core set. The first factor reflects the trunk (items 1-4, 7), one reflects the lower extremities (items 5, 6, 8-16), and the third comprises items concerning the upper extremities (17-22). Therefore, we calculated point estimates and standard errors for the 3FIC model using the full combined sample, which are presented in Table 3, were calculated.

Table 3.

CFA Point Estimates and Standard Errors from Preferred Model (3FIC)

Item	Content	Trunk	Lower Extremities	Upper Extremities
1	Lying Flat	0.82 (0.04)
2	Rolling Over	0.86 (0.04)
3	Moving - Lying to Sitting	0.88 (0.03)
4	Sitting	0.74 (0.03)
5	Standing		0.81 (0.03)
6	Squatting		0.84 (0.02)
7	Bending Over	0.87 (0.03)
8	Balancing		0.69 (0.03)
9	Kneeling		0.74 (0.03)
10	Walking - Short Distance		0.93 (0.02)
11	Walking - Long Distance		0.89 (0.02)
12	Walking - Outdoors		0.91 (0.02)
13	Climbing Stairs		0.85 (0.02)
14	Hopping		0.97 (0.01)
15	Jumping		0.97 (0.01)
16	Running		0.82 (0.02)
17	Pushing			0.95 (0.01)
18	Pulling			0.94 (0.01)
19	Reaching			0.86 (0.02)
20	Grasping			0.86 (0.03)
21	Lifting			0.90 (0.01)
22	Carrying			0.93 (0.02)

Note. Correlation between Trunk and Lower Body was 0.71 (0.04), between Trunk and Upper Extremities was 0.64 (0.04), and between Lower Body and Upper Extremities was 0.38 (0.05). In table and note point estimates for 3FIC model are presented followed by estimated standard error in parentheses.

Reliability Analyses

Once a satisfactory factor structure was identified, about a goal was set to provide additional evidence about the reliability of the resulting scores, including assessment of coefficient alpha, SEM, and MDC. Based on the factor analyses, three scores for each subject were constructed and the resulting reliability properties were evaluated. Table 4 contains the summary information about each of the three scales.

Table 4.

Classical Test Theory Results

Scale	Items	# of items	Rawc Mean	Raw SD	Raw Min	Raw Max	Scaled Mean	Scaled SD	Alpha	SEMa	MDCb
Trunk	1-4, 7	5	7.01	3.29	5	25	10%	16.4%	0.82	1.38	3.83
Lower Body	5,6,8-16	11	21.96	10.83	11	55	24.9%	24.6%	0.94	2.74	7.6
Upper Body	17-22	6	10.7	5.68	6	30	19.5%	23.7%	0.91	1.68	4.65

^aStandard Error of Measurement is calculated as SE, where sd is the sample standard deviation and r is the sample estimate of coefficient alpha.

^bMinimum Detectable Change (MDC) is calculated as

^cRaw descriptives include reverse coding, but no change to percentile.

Score Reporting

All study subjects were offered the opportunity to comment verbally to administering clinicians on the interpretability of the instrument. Most subjects did not comment, but among those comments shared verbally by clinicians with the research team, a consistent theme that the scoring system seemed counter-intuitive was noted. Following the originally published scoring of OPTIMAL, the responses for each item were displayed left to right from “fully confident” to “not at all confident” using the same arbitrary value assignment of 1-5 arrayed left to right. In this scheme, a high or “best” score on, say, the trunk subscale was five while 25 was actually a low or “worst” score possible. Not surprisingly, adolescents found this scoring confusing. Therefore, the values assigned to each level of response was reversed so that higher numbers represented greater confidence. To ease interpretation further, the score was calculated as a percentage of the available points achieved. Because the subjects in this study were also students who were used to being graded academically, they grasped the difference between scoring 0% and 100% on an assessment without any difficulty. This allows scores for the three scales, which have different lengths, to be expressed in a comparable metric. This ease of interpretation held true for cases with missing values where the ratio was calculated as the total actual points awarded across all items answered (minus the minimum possible score) divided by the total possible points achievable across all items answered (minus the minimum possible score), multiplied by 100%.

DISCUSSION

On a very large sample drawn from multiple sites, the data indicate that the AMCaMP separated itself into three factors: lower body, upper body, and core, and performed adequately when subjected to classic psychometric testing. All three subscales demonstrated good internal consistency and an MDC threshold which is reasonable for clinical practice and documentation of clinical progress and outcomes. As was the case with OPTIMAL, its progenitor instrument, AMCaMP items reflecting the neuromusculoskeletal core do not lend themselves to a scale that is quite as unidimensional as the scales for either the upper or lower body. From the perspective of clinicians for whom movement is the primary phenomenon of interest, this finding is clinically axiomatic. While the focus on intervention may be primarily directed at segmental or intersegmental movement, each factor must make its full contribution to the overall whole body movement or action. Limb mobility must combine successfully with trunk stability to produce sustainable movement or action that is safe, effective, and efficient. Thus an instrument which captures these three factors has great potential to document sources of variability in movement-related outcomes.

Beyond its value as a psychometrically acceptable instrument, AMCaMP possesses two distinct attributes. The first is its patient-centricity. By measuring confidence in movement as a critical outcome of rehabilitation, this instrument can describe the impact of injury or illness on movement from the patient's point of view. Furthermore, because self-efficacy is highly predictive of what a person might actually do once leaving clinical care, it may serve as a proxy measure of carry-over in proper movement after returning to sport.

On a methodological note, although the 3FIC-MI model was superior when evaluated using the proposed model fit criteria, there are no substantively different conclusions one would reach in accepting this model over the simpler a priori 3FIC model. As it has been proposed to score these scales using summed scores, the additional parameters, while improving model fit, will not impact any conclusions that are likely to be made. However, if one were to use factor scores in the future (or a unidimensional item response theory analysis), the correlated residuals that were identified here may prove to be strong enough to require attention.

While self-report instruments can provide a necessary patient-centric perspective, it is not in itself a sufficient basis for sound clinical judgment and treatment planning by health professionals. A parallel set of instruments for assessing functional performance objectively should be used to complement the data provided by AMCaMP. Capturing both perspectives should lead to patient-centric goal-setting, professionally competent treatment planning, and outcomes relevant to all stakeholders. Future research should also explore the predictive validity of AMCaMP, especially with respect to recurrent injury.

SUMMARY

Primarily relying on factor analysis, the latent structure of AMCaMP was established. This 22-item self-report instrument measuring confidence in performing particular movements or actions revealed a three factor structure comprising the trunk, upper body and lower body. These three scales demonstrated good internal consistency and an acceptable MDC. These psychometric properties and the instrument's patient-centricity and ability to provide an ecological context for a respondent's answers recommend its use for adolescent athletes, especially when self-efficacy regarding confidence in movement in daily activities prior to returning to sport is a primary clinical or research outcome.

Appendix

Confidence-Evaluation

Name:_________________________________Date:______________________

Date of Birth:__________________ Gender (circle one): Male Female

Grade in school:_________ Years participating in current sport(s):_______

How CONFIDENT are you about completing these activities?

These are questions about how well you believe you can complete each of the following activities. Please respond to each question while thinking about your non-sports activities.

Please circle the number that matches your answer for each activity listed.

Staff Use Only ID #	Fully confident in my ability to complete	Very confident	Moderately confident	Somewhat confident	Not confident in my ability to complete	Does not apply to me
1. Lying flat	5	4	3	2	1	9
2. Rolling over	5	4	3	2	1	9
3. Moving–lying to sitting	5	4	3	2	1	9
4. Sitting	5	4	3	2	1	9
5. Standing	5	4	3	2	1	9
6. Squatting	5	4	3	2	1	9
7. Bending over	5	4	3	2	1	9
8. Balancing	5	4	3	2	1	9
9. Kneeling	5	4	3	2	1	9
10. Walking–short distance	5	4	3	2	1	9
11. Walking–long distance	5	4	3	2	1	9
12. Walking–outdoors	5	4	3	2	1	9
13. Climbing stairs	5	4	3	2	1	9
14. Hopping	5	4	3	2	1	9
15. Jumping	5	4	3	2	1	9
16. Running	5	4	3	2	1	9
17. Pushing	5	4	3	2	1	9
18. Pulling	5	4	3	2	1	9
19. Reaching	5	4	3	2	1	9
20. Grasping	5	4	3	2	1	9
21. Lifting	5	4	3	2	1	9
22. Carrying	5	4	3	2	1	9

23. Given the things you have problems with in the list above, which 3 would you like to complete with more confidence? (for example, if you would most like to be able to climb stairs, kneel, and hop with complete confidence, you would choose: (1. 13 2. 9 3. 14) 1.____ 2.____ 3.____