Validation of the MUSIC Model of Motivation Inventory for Use with Cognitive Training for Schizophrenia Spectrum Disorders: A Multinational Study

Abstract

Aim:

Low motivation is a core symptom of schizophrenia which significantly impacts successful engagement in and benefit from psychosocial treatments. Therefore, it is important for clinicians to design psychosocial treatments to effectively motivate and engage patients during the treatment. The MUSIC® Model of Academic Motivation Inventory (MMI) is an 18-item scale with five indices that assess students' motivation during academic tasks. The objective of the current study was to validate the MMI for use with schizophrenia-spectrum patients undergoing cognitive training.

Methods:

Participants included 181 people with schizophrenia spectrum disorders enrolled in cognitive training in four countries. A confirmatory factor analysis (CFA) assessed construct validity. Quality of fit was determined using the Comparative Fit Index (CFI), the Standardized Root Mean Square Residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA). Pearson's correlation coefficients assessed construct validity and Cronbach's alphas assessed reliability. Furthermore, we examined factor loadings for each inventory item and assessed predictive validity by analyzing MMI subscales with attendance outcomes.

Results:

Consistent with the original MMI validation studies used in academic settings, we found CFI values indicated a good fit, as did the SRMR and RMSEA values. The scales were correlated yet distinct. Cronbach's alpha values ranged from good to excellent and factor loadings showed that all items loaded very well onto their intended factors. The MMI had a positive relationship to treatment intensity.

Conclusion:

The MMI is a valid and reliable tool to use with individuals with schizophrenia spectrum disorders undergoing a cognitive training intervention.

1. Introduction

Motivational impairment is a core negative symptom of schizophrenia spectrum disorders and has been found to play a key role in an individual’s participation in and benefit from psychosocial interventions (Medalia & Saperstein, 2011). For example, individuals with higher motivation show greater gains from cognitive remediation training (Medalia & Richardson, 2005) and motivation contributes to the heterogeneity of outcomes for cognitive behavioral therapy for psychosis (CBTp) (Menon, Andersen, Quilty, & Woodward, 2015). Although there is a high prevalence of amotivation/apathy in schizophrenia populations (Kirkpatrick, Fenton, Carpenter, & Marder, 2006), motivation in individuals with schizophrenia is not a static entity, but rather, a dynamic process that is malleable (Choi & Medalia, 2010; Medalia & Saperstein, 2011). Therefore, enhancement of patient motivation should be considered a primary target of psychosocial treatment interventions (Fiszdon, Kurtz, Choi, Bell, & Martino, 2016).

Researchers in the fields of education and psychology have documented that learning environments can have a significant impact on learners’ motivation within that environment (e.g., Spearman & Watt, 2013). Some researchers refer to this as a situative perspective that “interprets individuals’ beliefs and behaviors as arising through participation in social, cultural, and historical contexts or systems” (Turner & Nolen, 2015, p. 168). It is possible, therefore, to design learning environments that motivate learners to engage in learning tasks (e.g., Hulleman & Harackiewicz, 2009). In many forms of cognitive remediation, the learning environments utilize educational and instructional techniques, such as interactive coaching, confidence-building, group discussion, and practice exercises (See: Medalia and Bowie, ed 2016; Medalia, Herlands, Saperstein, & Revheim, 2017). In these cognitive training settings, the role of therapists in training people with schizophrenia-spectrum disorders is comparable to the role of instructors who design learning environements for students. Therefore, it should be possible for therapists to intentionally design a cognitive remediation environment to increase patients’ motivation for the training tasks.

The MUSIC^® Model of Motivation (Jones, 2009, 2018) is a multidimensional framework that can be used by instructors to design instruction consistent with current motivation research and theories. The model posits five components of motivation that educators should consider when designing instruction: 1) eMpowerment, 2) Usefulness, 3) Success, 4) Interest, and 5) Caring (MUSIC is an acronym for these five components). Empowerment refers to students’ sense of control over their learning environment. Specifically, students should feel that they have some control over some aspect of their learning in order to feel autonomy within in the classroom. Usefulness refers to students’ beliefs that the material they are learning is relevant to their lives and/or their personal goals. Success is students’ perceived sense of competence and their perception that they have the ability to succeed in the learning environment. Interest refers to students’ sense of enjoyment and curiosity while learning and Caring refers to students’ beliefs that their instructor and others in the learning environment care about their learning and about their well-being.

The MUSIC^® Model of Academic Motivation Inventory (MMI; Jones, 2017) is a self-report measure that assesses students’ perceptions of the five MUSIC components in an educational setting. The MMI does not assess individuals’ more stable, trait-like motivation; rather, it assesses individuals’ perceptions of the learning environment, which can change if the instruction is altered (e.g., McGinley & Jones, 2014). The measure has been validated for use with college undergraduates (Jones & Skaggs, 2016), elementary school students (Jones & Sigmon, 2016), U.S. and Icelandic middle and high school students (Chittum & Jones, 2017; Jones, Sahbaz, Schram, & Chittum, 2017b; Schram & Jones, 2016; Parkes, Jones, & Wilkins, 2015), Chinese- and Spanish-speaking university students (Jones, Li, & Cruz, 2017a), and student pharmacists (Pace, Ham, Poole, & Wahaib, 2016).

Prior research has also validated the use of other educational motivational scales for use in schizophrenia populations. For example, the Intrinsic Motivation Inventory (IMI) has been adapted for use in schizophrenia research (Choi, Mogami, & Medalia, 2010) and focuses on interest/enjoyment, perceived choice, and value/usefulness. The Perceived Competency Scale (PCS), which has also been used with schizophrenia populations (Choi, Fiszdon, & Medalia, 2010), measures participants’ perceived ability to succeed at a task. In comparison, the MMI includes the elements in both the IMI and PCS, as well as perceptions of the clinician as caring, which also has been shown to predict learners’ motivation and engagement (Quin, 2016). Furthermore, the MMI can be administered easily and quickly in any type of educational setting using any type of instruction (Pace et al., 2016).

The purpose of this study was to assess the psychometric properties of the MMI for use with individuals with schizophrenia spectrum disorders undergoing cognitive remediation. Therefore, we adapted the MMI for use with cognitive training programs and collected data to examine the extent to which the MMI was valid for use with this population. In addition, it has previously been hypothesized that the instructional strategies related to the five MUSIC Model components can also be considered for use with cognitive training programs as a method to enhance client motivation (Medalia, Saperstein, Hansen, & Lee, 2016). If the MMI was shown to produce valid scores, it could be useful during cognitive training to help clinicians develop service-user informed approaches for enhancing motivation in their programs. For example, knowing that a client does not perceive an intervention as useful, would inform the clinician that it is necessary to better demonstrate why learning the information is relevant. Alternatively, if a client indicates he or she does not feel able to succeed at the tasks, adjusting difficulty level and providing CBT oriented interventions to address thinking errors, might improve their sense of competence. In sum, the central aim of this study was to examine the psychometric properties of the MMI and validate its use with schizophrenia spectrum populations undergoing cognitive training.

2. Methods

2.1. Participants

Participants included 181 people with schizophrenia spectrum disorders currently enrolled in IRB-approved cognitive training research studies. Participants were located in four countries: 104 from the United States, 16 from Canada, 27 from Denmark, and 34 from Japan. Participants were aged 18 to 65 with a mean age of 37 years. The sample was 70% male. Trained research assistants confirmed participants’ diagnoses: 56% were diagnosed with schizophrenia, 24% schizoaffective disorder, 14% ultra high risk, 0.6% bipolar disorder I with psychosis, 0.6% psychosis NOS, and 0.6% major depression with psychotic features. The sample was multiethnic with 32% Caucasian, 25% Black/African, 23% Asian, and 14% Hispanic Latino/a. Years diagnosed spanned from 0 (ultra high risk) to 44. See Table 1 for descriptions of the samples at each site and Table 2 for symptom profiles of the samples.

Table 1.

Description of Samples at Each Site

	Site #1 (n=52)	Site #2 (n=8)	Site #3 (n=25)	Site #4 (n=9)	Site #5 (n=27)	Site #6 (n=18)	Site #7 (n=8)	Site #8 (n=34 )
Gender (%)
Male	61	87	84	100	51	66	75	73
Female	36	12	16	0	48	33	25	26
Age
Mean	45.94	46.88	37.32	41.44	22.56	33.33	26.37	38.09
SD	10.65	11.91	11.65	13.47	4.22	8.91	5.26	8.95
Range	23-65	30-57	20-63	21-57	18-34	21-52	18-32	22-58
Diagnosis (%)
Schizophrenia	65	75	36	55	0	61	75	91
Schizoaffective	32	12	56	33	0	22	12	8
Other Psychotic
Disorder	0	12	4	11	0	0	0	0
Ultra High Risk	0	0	0	0	100	0	0	0
Years Diagnosed
Mean	NR	12.25	14.5	22	N/A	7	6.25	14.1
SD	NR	6.82	13.44	14.84	N/A	3.03	5.66	9.8
Range	NR	1-21	5-24	4-44	N/A	2-14	1-16	<1-40
Race/Ethnicity (%)
Black	55	0	40	11	0	16	25	0
Hispanic	28	0	32	22	0	0	12	0
White	11	87	20	44	77	66	37	0
Asian	0	0	4	11	7	16	0	100
Other	1	1	4	11	14	0	25	0
Education
Mean	11.25	11.5	12.26	13.55	13.2	13.8	13.5	13.64
SD	2.27	3.33	1.98	1.94	2.17	1.69	2.77	2.56
Range	5-16	7-18	9-16	12-17	10-18	12-16	11-19	7-19
WAIS-IV IQ Estimate
Mean	84	94	NR	97	103	105	NR	110
SD	10.37	15.46	NR	14.12	12.58	9.47	NR	10.04
Range	71-118	70-119	NR	78-117	83-127	86-124	NR	92-126

Site #1= Columbia University (US); Site #2= McGill University (Canada); Site #3 Manhattan Psychiatric Institute (US);

Site #4= University of Minnesota (US); Site #5= Mental Health Centre Copenhagen (Denmark);

Site #6 University of Pittsburg (US); Site #7= Center for Addiction & Mental Health (Canada); Site #8= National Center of Neurology & Psychiatry (Japan)

NR= Not Reported; N/A= Not Applicable

Table 2.

Symptom Profiles from Select Sites

	Site #1 M (SD)	Site #2 M (SD)	Site #3 M (SD)	Site #4 M (SD)
BPRS	NR	NR	42.04 (6.72)	38 (9)
PANSS +	13.54 (4.91)	NR	NR	NR
PANSS −	13.24 (3.93)	NR	NR	NR
PANSS General	24.81 (6.33)	NR	NR	NR
SANS	20 (9.09)	30.9 (9.78)	1.68 (0.91)*	NR

Notes.

Site #1= Columbia University (US); Site #2= McGill University (Canada); #3 Mental Health Centre Copenhagen (Denmark); Site #4= Center for Addiction & Mental Health CAMH (Canada)

BPRS= Brief Psychiatric Rating Scale

PANSS= Positive and Negative Syndrome Scale

SANS= Scale for the Assessment of Negative Symptoms

NR= Data Not Reported

^*SANS total calculated by averaging global scores excluding the Attention subscale

2.2. Measures

MUSIC^® Model of Motivation Inventory, Cognitive Training version.

The Cognitive Training version of the MMI measures individuals’ motivation-related perceptions of the clinical training environment. It consists of 18 items that are divided into five scales: a four-item empowerment scale, a three-item usefulness scale, a four-item success scale, a three-item interest scale, and a four-item caring scale (Jones, 2017). Response options range from 1 to 6 on a Likert-format scale: 1 = Strongly disagree, 2 = Disagree, 3 = Somewhat disagree, 4 = Somewhat agree, 5 = Agree, 6 = Strongly agree. The items in each scale are averaged to create a mean scale score. The Cognitive Training version is similar to the Middle/High School version of the MMI except that the subject area (e.g., science) is replaced by “Cognitive Training,” the words “class work” are changed to “activities,” and references to the “teacher” are replaced with “Cognitive Training instructor.” The complete inventory is available at Jones (2017) and example items include the following: “I have choices in what I am allowed to do in Cognitive Training” (empowerment), “In general, the Cognitive Training activities are useful to me” (usefulness), “I am confident that I can succeed in the Cognitive Training activities” (success), “The Cognitive Training activities are interesting to me” (interest), and “My Cognitive Training instructor is respectful of me” (caring).

In the present study, the items were translated into Danish and Japanese. For both languages, the MMI was translated by researchers whose native language was the target language. The original translations were then translated back into English (back-translated) by other individuals whose native language was the target language (Villagran & Lucke, 2005). The individuals who conducted the back-translations had not seen the original English version of the MMI. The back-translations were then compared by an individual whose native language was English, who had worked as a professor at U.S. universities for 18 years, and who was familiar with the constructs in the MUSIC model and MMI. The native English speaker found only a few discrepancies in the back-translation and worked with the translator and back-translator to resolve the issues until the inventory items were deemed acceptable by the English speaker and the translators.

2.3. Procedures

All participants were participating in cognitive training that used computer-based cognitive exercises and support from a clinician, including instruction and strategy coaching. The majority of participants (90%) were involved in group, rather than individual, cognitive training sessions. Participants were asked to complete the MMI once (after the third training session and before the midpoint of training). The duration of cognitive training ranged across sites from 3 to 18 months. Measures were administered and collected by individuals who were not the instructors of the cognitive training.

2.4. Analysis and Interpretation of Values

We assessed the internal reliability of the MMI by calculating Cronbach’s alpha values for each of the sub-scales using SPSS (version 23). We judged the quality of the values using the following criteria (Kline, 2005): greater than .9 was considered excellent, between .7 and .9 was considered good, and between .6 and .7 was considered acceptable (Kline, 2005).

We assessed the construct validity of the MMI by conducting confirmatory factor analyses (CFAs) with maximum likelihood estimates using LISREL (version 8.80) to examine how the items in the MMI fit the five-factor structure of the MUSIC model. We judged the quality of the fit using three fit indices: the Comparative Fit Index (CFI), the Standardized Root Mean Square Residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA). Although the CFI can range between 0 and 1, values closer to 1 indicate a better fit (values above .90 represent reasonable fit and above .95 represent good fit; Hu & Bentler, 1999). The SRMR also ranges from 0 and 1, but values closer to 0 indicate a better fit (less than .05 indicates good fit [Byrne, 2001] and less than .10 indicates reasonable fit [Kline, 2005]). Finally, the RMSEA can vary between 0 and 1 with values closer to 0 indicating better fit (values between .1 and .08 indicate mediocre fit, less than .08 indicate reasonable fit, and values less than .05 indicate good fit; Browne & Cudeck, 1993; Byrne, 2001; Kline, 2005).

The fit indices provide a means to assess the factorial construct validity of the MMI scales because the model has been shown to consist of five different constructs (Jones et al., 2017a; Jones & Skaggs, 2016). To further assess the construct validity of the scales, we computed the coefficients of determination (r²) to examine the relationships between the MMI constructs. In addition, we examined the factor loadings for each MMI item on the appropriate factor (e.g., the empowerment items should load adequately on the empowerment factor). We judged the factor loadings to be acceptable if they were greater than 0.32 (Tabachnick & Fidell, 1996). Test–retest reliability was not assessed because it was not appropriate for this study due to the nature of the inventory. The MMI measures individuals’ perceptions of their environment (e.g., usefulness of the material); therefore, a change in the learning environment (e.g., a different cognitive remediation module or exercise) could change the way a participant responds. For example, a respondent’s scores on the Interest or Usefulness subscale scores may change as the focus of cognitive training changes (e.g., exercises shift from problem-solving to processing speed tasks) and in turn, the perceived interest or usefulness of the intervention to the respondent. Therefore, a change in respondents’ scores over time could indicate that the instruction during that time changed respondents’ perceptions rather than indicating a lack of reliability of the scale items. As a result, we did not compute test-retest reliability.

We assessed predictive validity by using independent samples t-tests to compare dropout rate and MUSIC subscale scores for a subset of the sample (N = 112). Pearson’s product-moment correlation coefficients were conducted to assess the relationship between session intensity (days per week attended cognitive training) and MUSIC subscale scores (N = 73).

Finally, we assessed the relationship between negative symptoms and the MMI through Pearson’s product-moment correlation coefficients using the Avolition-Apathy subscale of the Scale for the Assessment of Negative Symptoms (SANS) for a subset of the sample (N =27) and the five MUSIC scales. We also conducted Pearson’s product-moment correlation coefficients for a second subset of the sample (N = 51) Positive and Negative Syndrome Scale (PANSS) total negative symptom scores and the five MUSIC scales.

3. Results

3.1. Missing Data

A response to one item was missing from two of the 181 participants. For these two participants, we used the average the other items in the scale for that participant to complete the missing data. Because two other questionnaires were missing responses to three or more items, we excluded those questionnaires from our analysis and they were not included with the 181 participants.

3.2. Descriptive Statistics

We computed descriptive statistics to assess the variability among responses. Responses on all five MMI scales ranged from 1.0 to 6.0 with the means and standard deviations shown in Table 3. Although all of the means are rather high, scores on each scale varied, which demonstrates that the scales are sensitive to individual differences. The only exception is that the mean for the caring scale was quite high at 5.47 and fairly consistent across languages, which could result in a ceiling effect that could limit the clinical utility of the measure. However, if the clients perceived their therapist as highly caring, then the scores would still be accurate and reliable.

Table 3.

Means and Standard Deviations for the MUSIC Inventory Scales

MUSIC Inventory language		M (SD)
	n	M	U	S	I	C
English, Danish, Japanesea	181	4.89 (0.83)	4.95 (0.92)	4.78 (0.94)	4.95 (0.97)	5.47 (0.68)
English only	120	4.94 (0.91)	5.06 (0.91)	4.99 (0.94)	5.13 (0.91)	5.49 (0.73)
Danish only	27	4.79 (0.56)	4.62 (0.97)	4.54 (0.69)	4.28 (1.15)	5.45 (0.55)
Japanese only	34	4.82 (0.70)	4.80 (0.82)	4.23 (0.88)	4.87 (0.81)	5.42 (0.55)

Note. M = eMpowerment, U = Usefulness, S = Success, I = Interest, C = Caring

^aEnglish (n = 120), Danish (n = 27), Japanese (n = 34)

3.3. Internal Consistency Reliability

The Cronbach’s alpha values for each of the MMI scales are shown in Table 4. Most of the alpha values ranged from .73 to .91, indicating that the reliability of the scales was good to excellent (Kline, 2005). One exception was that the alpha value for the Danish-only sample was .61 for empowerment, indicating that it was “acceptable” based on the criteria suggested by Kline (2005). As a means of comparison, the bottom row of Table 4 displays some of the Cronbach’s alpha values that were reported for the Middle/High School version of the MMI in Chittum and Jones (2017, p. 7) with fifth-, sixth-, and seventh-grade students in science class. In all samples and languages, the alpha values for the Cognitive Training version of the MMI were the same or higher than the values for the Middle/High School version, except that the values for empowerment and caring were lower for the Danish-only sample.

Table 4.

Cronbach’s Alpha Values and Fit Indices

		Cronbach’s alpha values
MUSIC Inventory language	n	M	U	S	I	C	χ2(df)	CFI	SRMR	RMSEA
English, Danish, Japanesea	181	.81	.81	.88	.85	.85	249.1* (125)	0.98	0.051	0.069
English only	120	.83	.78	.86	.83	.87	--	0.96	0.059	0.094
Danish only	27	.61	.84	.84	.88	.73	--	--	--	--
Japanese only	34	.83	.87	.91	.80	.85	--	--	--	--
Middle/High School versionb	321	.72	.78	.83	.77	.84	--	0.97	0.052	0.069

Note. M = eMpowerment, U = Usefulness, S = Success, I = Interest, C = Caring, CFI = Comparative Fit Index, SRMR = Standardized Root Mean Square Residual, RMSEA = Root Mean Square Error of Approximation

^aEnglish (n = 120), Danish (n = 27), Japanese (n = 34)

^bValues for the Middle/High School version were reported in Chittum and Jones (2017) with fifth-, sixth- , and seventh-grade students in science class and were not based on data collected in the present study.

^*p < .01

3.4. Factorial Construct Validity

Costello and Osborne (2005) noted that there are no strict rules regarding sample size for factor analyses but suggest that a common rule of thumb appears to be a participant-to-item ratio of at least 10 to 1. To meet this general criterion, the 18-item MMI requires at least 180 participants (180:18). Therefore, we found it to be acceptable to run the CFA for the combined English, Danish, and Japanese samples. We also conducted a CFA on the English only sample because the ratio for that sample was 120:18, which is a 6.7:1 ratio. We did not compute the fit indices for the Danish only sample (ratio of 1.5:1) or the Japanese only sample (ratio of 1.9:1) because the ratios were far lower that what we deemed to be acceptable.

Table 4 displays the fit indices from the CFA results and some of the values from the Middle/High School version presented in Chittum and Jones (2017, p. 7). For the combined English, Danish, and Japanese sample, the CFI values indicate a good fit (Hu & Bentler, 1999), the SRMR value indicates a reasonable fit (Kline, 2005), and the RMSEA value indicate a reasonable fit (Browne & Cudeck, 1993; Byrne, 2001; Kline, 2005). For the English only version, the CFI values indicate a good fit (Hu & Bentler, 1999), the SRMR value indicates a reasonable fit (Kline, 2005), and the RMSEA value indicate a mediocre fit (Browne & Cudeck, 1993; Byrne, 2001; Kline, 2005). The fit indices for the Cognitive Training version are comparable to those of the Middle/High School version shown in the bottom row of Table 2.

The coefficients of determination (r²) for the MMI scales ranged from .27 to .54 (see Table 5). In other words, as little as 27%, and no more than 54%, of the variance is shared between any two scales, indicating that the scales measure distinct constructs.

Table 5.

Coefficient of Determination for the Study Constructs

Scale	Empowerment	Usefulness	Success	Interest
Empowerment
Usefulness	.37, .44
Success	.28, .28	.33, .36
Interest	.33, .37	.50, .54	.46, .52
Caring	.31, .34	.27, .34	.34, .43	.28, .41

Notes. p ≤ .001 for all coefficients. The first number represents the sample with the combined English, Danish, and Japanese scores (n = 181) and the second number represents the sample with the English scores only (n = 120).

All of the standardized factor loadings from the CFAs ranged from .68 to .89, indicating that the items loaded very well on their intended factors (Tabachnick & Fidell, 1996). As seen in Table 6, all of the values were well above the .32 value recommended by Tabachnick and Fidell (1996) and even much higher than .5, which indicates a “solid factor” according to Costello and Osborne (2005). The patterns of values for the combined English, Danish, and Japanese sample are very similar to the values of the English only sample, with the difference between each pair of values for each item no greater than 0.06.

Table 6.

Standardized Factor Loadings from the CFA

Item	Empowerment	Usefulness	Success	Interest	Caring
M1	.70, .73
M2	.73,.74
M3	.69, .75
M4	.76, .76
U1		.76, .78
U2		.81, .76
U3		.72, .68
S1			.74, .71
S2			.72, .69
S3			.86, .86
S4			.89, .85
I1				.83, .78
I2				.84, .83
I3				.76, .75
C1					.71, .76
C2					.77, .79
C3					.81, .83
C4					.82, .82

Note. The two numbers in each cell are the standardized coefficients from two different analyses. The first number represents the sample with the combined English, Danish, and Japanese scores (n = 181) and the second number represents the sample with the English scores only (n = 120).

3.5. Predictive Validity

Independent samples t-tests were conducted in a subset of the sample (N = 112) to compare MUSIC subscale scores for individuals who dropped-out (N =19) and completed (N = 61) cognitive training. There were no significant differences in any of the five MUSIC scale scores for drop-outs versus completers. In a subset of the sample for which session intensity (i.e., number of sessions attended per week) was available (N = 73), there was a positive relationship between session intensity and all of the MUSIC scales except caring [r = .136; p = .252]: eMpowerment scale [r = .250; p = .033], Usefulness scale [r =.325; p = .005], Success scale [r = .397;p = .000], and Interest scale [r = .332;p =.004].

3.6. Relationship of the MMI to Negative Symptoms

Pearson’s product-moment correlation coefficients were computed to assess the relationship between each of the five MUSIC subscales and the SANS Avolition-Apathy Scale (N = 27). There was no correlation between the SANS Avolition-Apathy Scale with any of the MUSIC scales: eMpowerment scale [r = .032,p = .873], Usefulness scale [r = −.311; p = .114], Success scale [r = −.170; p = .397], Interest scale [r = −.155; p = .441], and Caring scale [r = −.074; p = .714], In addition, Pearson’s product-moment correlation coefficients were conducted with a second subset of the sample (N = 51) and PANSS total negative symptom scores. There was no correlation between PANSS total negative symptom scores and any of the MUSIC scales: eMpowerment scale [r = −.216; p = .128], Usefulness scaled [r= −.246; p = .082], Success scale [r= −.158, p = .269], Interest scale [r = −.214; p = .132] and Caring scale [r = −.018; p = .898], Therefore, whether measured by the SANS Avolition-Apathy scale (N = 27) or PANSS negative symptom scale (N = 51), negative symptoms were not significantly related to the five MUSIC subscales.

4. Discussion

4.1. Overall Findings and Future Research

The purpose of this study was to assess the psychometric properties of the MUSIC^® Model of Motivation Inventory for use with individuals with schizophrenia spectrum disorders undergoing a skills training intervention, specifically, cognitive remediation. In addition, we translated the items to Danish and Japanese in order to assess the validity of the scores in those settings. Our results show that the MMI is a valid and reliable self-report measure that can be used to assess the motivation-related perceptions of individuals with schizophrenia spectrum disorders who are engaged in a skills training learning experience. This is the first study of which we are aware that has provided validity evidence for the use of the MMI in mental health clinics.

Perhaps the greatest strengths of this study are that it demonstrates that the items in each MUSIC scale reliably assess the intended motivation construct (as evidenced by the alpha values) and that learners participating in cognitive remediation can distinguish among the five components of the MUSIC model. That is, the items loaded on the constructs as hypothesized. We also found that motivation in a learning environment (as assessed by the MMI) operates independently from negative symptoms, suggesting that the MMI can be used effectively with people who experience deficits in trait-level motivation. We also provided evidence that the MUSIC components are related to the intensity with which people attend CR sessions, suggesting the inventory scales have some degree of predictive validity. We did not find a significant relationship between the MUSIC components and drop-out rates, suggesting that no one motivating factor is salient in the decision to stay in treatment. In addition, we did not find a relationship between the Caring subscale and treatment intensity, which may be an artifact of the fairly uniform and high degree of clinician Caring perceived in this sample.

Because we administered the MMI in the U.S., Canada, Denmark, and Japan, these results demonstrate that the Cognitive Training version of the MMI can be used in a variety of cultures and countries. These findings are consistent with research studies using the MMI with students in countries such as China (Jones et al., 2017a), Colombia (Jones et al., 2017a), Egypt (Mohamed, Soliman, & Jones, 2013), Iceland (Schram & Jones, 2016), and Spain (Mora, Anorbe-Diaz, Gonzalez-Marrero, Martin-Gutierrez, & Jones, 2017). Thus, this study contributes to a broader body of evidence demonstrating the potential to use the MUSIC model in many different cultures and contexts.

4.2. Limitations

The results of this study must be interpreted within the context of its limitations. First, participants were drawn from multiple sites in different countries, which allowed us to increase our sample size to meet the minimum statistical requirements. However, further studies with larger samples in each country would be needed to better examine the extent to which cultural differences impact the psychometric properties of the MMI. Second, not all participants were undergoing the same type of cognitive training with the same instructors, and therefore, differed in terms of the cognitive training approach used, software, and level of educational training of the instructor. But given the fact that the MMI assesses clients’ perceptions of the environment regardless of instructor or type of instruction, this feature of the study provides evidence of the generalizability of the measure across varying training programs. Third, convergent and discriminant construct validity could be further examined by correlating the MMI scales with other scales to ensure that they correlate as expected. Fourth, sensitivity to change is also an area in need of examination and a limitation of the current study. Further research should determine how responsive the inventory is to changes in participant motivation which occur in reaction to changes in their learning environment, or as a result of symptom fluctuation when the learning environment is stable.

4.3. Conclusions

The MMI could be useful in both research and clinical settings. The inventory could help researchers articulate whether and how interventions affect individuals’ motivation-related perceptions of the intervention. Because the MMI consists of five scales, it allows researchers to assess the impact across a range of motivation-related constructs. Clinicians can administer the inventory to facilitate better clinical care by assessing the motivational perceptions of clinical tasks. They can then use these results to personalize treatments to create a learning environment that clients perceive to be motivating and engaging, which should increase their learning.

Treatments could also be tailored to enhance a particular facet of motivation (for examples, see Jones, 2018). As examples in cognitive training, a sense of empowerment can be increased if clients are provided with meaningful choices in the cognitive exercises they engage in, thereby helping to inform the effectiveness of clinical techniques. Clients’ usefulness perceptions can be enhanced through bridging exercises that connect computer-based exercises to client’s goals (e.g., returning to school or work). Usefulness perceptions can also be increased through group discussions in which clients share how they use the training in their lives. Clients’ perceptions of success can be improved by choosing cognitive exercises of varied levels of personal difficulty and providing clients with accurate, constructive feedback. Difficult exercises can be broken down into smaller, manageable steps which ensure participant success through “errorless learning.” CBT can be used to correct cognitive errors about learning behaviors. For example, a statement such as “I always forget the instructions my boss gives me” can be challenged via cognitive biases (e.g., black and white thinking). Interest can be enhanced through developing novel activities to use during cognitive training and capitalizing on clients’ specific interests. For instance, if a client is particularly interested in acquiring a retail job, cognitive training exercises and group bridging can incorporate retail themes. Clients’ perceptions of caring can be increased through a therapist’s interest in clients’ successes and failures regarding reaching rehabilitation goals. Therapists can also demonstrably value clients’ opinions during bridging exercises and one-on-one coaching.

Although these examples are specific to cognitive remediation, MUSIC model strategies can be expanded to other forms of skills training. Further studies should adapt the MMI for use with other interventions, for example, Cognitive Behavioral Therapy for Psychosis, Social Skills Training, and Vocational Training. Importantly, the predictive validity of the measure should be assessed based on symptom ratings and treatment outcomes. In addition, future studies should examine the MMI in relation to other measures that purport to assess the same constructs.