Development of the Patient Activation Measure for Mental Health (PAM-MH)

Abstract

Objective

To adapt the physical health Patient Activation Measure (PAM) for use among people with mental health conditions (PAM-MH).

Research Design

Data came from 3 studies among people with chronic mental health conditions and were combined in Rasch analyses.

Results

The PAM-MH’s psychometric properties equal those of the original 13-item PAM. Test-retest reliability and concurrent validity were good, and the PAM-MH showed sensitivity to change.

Conclusions

The PAM-MH appears to be a reliable and valid measure of patient activation among individuals with mental health problems. It appears to have potential for use in assessing change in activation.

Introduction

Recommendations for improving the quality of physical and mental health services suggest changing from more paternalistic models of care to those that are person-centered and person-directed (Institute of Medicine, Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders & Board on Health Care Services, 2006; Institute of Medicine & Committee on Quality Health Care in America, 2001). Such models assume that individuals will actively participate in treatment, will manage their physical and mental health and symptoms, and will benefit from doing so. Individuals vary, however, in their interest, capacity, and skills for such participation, as do the training and support available to facilitate such participation (Von Korff M., Gruman & Schaefer, 1997). Thus, finding ways to measure and improve activation are becoming increasingly important as research shows that more activated individuals tend to have better health and functioning, report higher quality of life and tend to be more satisfied with their care (Hibbard, Mahoney, Stockard & Tusler, 2005; Hibbard, Mahoney, Stock & Tusler, 2007). Studies also indicate that activation predicts a range of important health-related practices, including self-management (e.g., medication adherence/monitoring, diet, exercise), disease prevention practices (e.g., screenings or immunizations), and health-information seeking (Hibbard, Stockard, Mahoney & Tusler, 2004). Additionally, when activation changes, health-related practices also change, and preliminary evidence suggests that interventions can increase activation (Alegria, Polo, Gao, Santana, Rothstein, Jimenez, Hunter, Mendieta, Oddo & Normand, 2008). Thus, clinically useful activation measures to assess individual differences in “activation” (requisite knowledge, skill, confidence, beliefs for managing personal health and care) are important in tailoring care and intervening when care needs change—e.g., a less-activated patient may need additional information and encouragement compared to one more activated.

The original 13-item Patient Activation Measure (PAM) is an interval-level, unidimensional Guttman-like measure that contains items measuring self-assessed knowledge about chronic conditions, beliefs about illness and medical care, and self-efficacy for self-care. The original PAM focused on physical conditions and was designed to measure activation as a broad construct.

To address the need for a similar, clinically relevant, activation measure for individuals with mental health conditions, we have modified and tested a mental health version of the PAM. Our measure, the PAM-Mental Health (PAM-MH), adapted the 13-item PAM to specifically assess mental-health-related activation. This paper evaluates the PAM-MH’s psychometric properties among individuals with serious mental disorders, and includes a psychometric analysis using Rasch analysis. Rasch methodology provides advantages in measurement development because it allows use of ordinal items to create interval-level measures (Rasch, 1960; Wright & Masters, 1982; Wright & Stone, 1979). Our analyses also examine the PAM-MH’s test-retest reliability, concurrent validity, and sensitivity to change. We examined the pattern of correlations between the PAM-MH and concepts believed to be congruent with activation and those related to activation. We expected to see the strongest correlations with congruent concepts because they are measuring very similar concepts. The related concepts are likely to be influenced by other factors in addition to activation, thus we expected to see smaller correlations for these concepts than for those with congruent concepts. Based on previous findings from the original PAM, we hypothesized that increased activation would be associated with recovery from serious mental illness, better mental-health self-care, mental and physical health, and quality of life.

Methods

We piloted the PAM-MH in three studies, all reviewed and monitored by Kaiser Permanente Northwest’s (KPNW) Institutional Review Board. Data were combined for Rasch analyses. The three studies, and their data contributions, are described below; Table 1 contains descriptive statistics for all studies. Participants received a complete study-specific description before giving written informed consent.

Table 1.

Participant Characteristics

	Study 1	Study 2	Study 3
	N=170	N=30	N=30
Age at baseline, mean (sd)	49.2 (14.5)	47.2 (11.9)	44.3 (9.8)
Male gender, %	47.6	23.3	43.3
Education level, %
Less than high school graduate	6.5	13.3	0.0
High school graduate or GED	22.4	36.7	16.7
Some college or technical school	39.4	36.7	50.0
College graduate or higher	31.1	10.0	33.4
Missing	0.6	3.3	0.0
Race and Ethnicity, %1
White	94.1	53.3	96.7
African-American	5.9	30.0	0.0
Native American/Alaska Native	2.9	6.7	0.0
Asian or Pacific Islander	1.8	0.0	0.0
Mixed or other	4.7	3.3	3.3
Missing	0.0	6.7	0.0
Hispanic Ethnicity (across racial groups)	0.6	10.0	3.3
Primary Mental Health Diagnosis, %
Schizophrenia spectrum diagnosis	42.9	21.4	63.3
Bipolar disorder	46.5	17.9	36.7
Affective psychosis	10.6	35.7	n/a
PTSD	n/a	17.9	n/a
Depression	n/a	7.1	n/a
Employment Status, %1
Working	46.5	3.3	43.3
Homemaker	9.4	10.0	6.7
Student	6.5	3.3	0.0
Retired	22.9	3.3	6.7
Unemployed/laid off	2.9	10.0	6.7
Disabled	19.4	66.7	33.3
Missing	0.0	3.3	3.3
Marital Status, %1
Never married	22.9	20.0	33.3
Widowed	8.2	6.7	3.3
Divorced	18.2	40.0	16.7
Separated	4.1	20.0	3.3
Married	45.9	10.0	40.0
Living with partner	8.8	3.3	3.3
PAM-MH, mean (sd)	61.8 (14.1)	57.4 (13.1)	60.6 (11.8)

¹Total exceeds 100.0% because participants could check more than one category.

Study 1

STARS, a longitudinal study of recovery among people with serious mental illness, collected in-depth interview and paper-and-pencil questionnaire data at baseline, 1 year, and 2 years. Participants were 177 KPNW members, 93 women (52.2%) and 85 men (47.8%), with schizophrenia or schizoaffective disorder [42.1%], bipolar disorder [47.8%] or affective psychosis [10.1%]). Data from the 1-year questionnaire (n=170) were used for Rasch modeling and tests of concurrent validity. Data from the 2-year questionnaire were used to test concurrent validity for the modular survey items (see below).

Study 2

Study 2 piloted a consumer/counselor-led group recovery intervention in a large community mental health center. Participants were 7 men (23%) and 23 women (77%) (n=30), ranging in age from 24 to 73 years (mean age = 47.2 [sd = 11.9]). Overall, 48.2% were members of racial or ethnic minority groups and 51.8% were white. Post-traumatic stress disorder was the most frequently reported diagnosis (35.7%), followed by schizophrenia (21.4%), bipolar disorder (17.9%), depression (17.9%), and other disorders (7.1%). All socio-demographic characteristics and diagnoses were self-reported. The study’s goal was to determine the best length for the Pathways to Recovery intervention, so no control condition was included. Data for the 13 items of the PAM-MH were collected at baseline and used for Rasch modeling.

Study 3

Study 3 was a pilot randomized clinical trial of the Pathways to Recovery intervention described in Study 2. Participants (n=30) were randomized to intervention (n=12) or control conditions (n=18). The intervention group met weekly for 10 two-hour sessions. Data were collected at baseline and post-intervention follow-up (about 14 weeks post-baseline). Participants were recruited from KPNW and ranged in age from 25 to 64 years (mean age = 44.3 years [sd = 9.8]), 57% were women, and all but one reported white race/ethnicity. About two-thirds (63.3%) had a schizophrenia spectrum diagnosis, with the remainder having bipolar disorder. Diagnoses were collected from health plan records and socio-demographics were collected by paper-and-pencil questionnaire. The PAM-MH was administered at baseline and follow-up. Data were used for the Rasch analysis, to evaluate test-retest reliability among control group participants, and to assess sensitivity to change over time in the intervention and control groups.

Measures

Patient Activation Measure—Mental Health (PAM-MH)

We adapted the existing PAM items so that they applied to mental health conditions. Figure 1 shows the adapted items; the original PAM items are available in Hibbard et al. (2005). The PAM-MH was included in longer paper-and-pencil questionnaires in all three studies. Total PAM-MH scores were computed if at least 10 items were completed. PAM-MH total was available for 98% of Study 1 and 100% of Study 2 and 3 participants.

Figure 1.

Patient Activation Measure—Mental Health (PAM-MH) Items

Socio-demographic Information

We obtained age and gender from health plan records for studies 1 and 3, and self-report for Study 2. Education level was assessed by questionnaire and coded ordinally from grade school to some high school, high-school graduate or GED, some college or technical school, college graduate, and post-graduate. Participants were also asked their race/ethnicity and instructed to code all categories that applied (White, African-American, Native American/Alaska Native, Asian or Pacific Islander); all were asked about Hispanic ethnicity. We also asked participants about employment status and marital status.

Mental Health

Diagnoses

We obtained primary mental health diagnosis from health plan records for studies 1 and 3. Study 2 participants self-reported primary mental health diagnoses.

Recovery from Mental Illness

Recovery Assessment Scale (RAS)

The RAS (Corrigan, Giffort, Rashid, Leary & Okeke, 1999) is a relatively brief, easy-to-complete measure of recovery, developed for use with individuals who have serious mental illnesses. It produces an overall score that has good test-retest reliability (r = 0.88) and high internal consistency (Cronbach’s alpha = 0.93). In addition to the overall score, five subscales measure willingness to ask for help, personal confidence and hope, goal and success orientation, symptom domination, and reliance on others.

Mental Health Symptoms

Our primary measure of mental health symptoms in Study 1 was the Colorado Symptoms Inventory (CSI), a brief self-report measure of psychiatric symptom status (Shern, Wilson, Coen, Patrick, Foster, Bartsch & Demmler, 1994) that measures anxiety/depression, psychoticism, and dangerousness to self/others. The CSI has high internal consistency (alpha = .87) for consumer self-report and has been found to detect differences between severely mentally ill individuals who did and did not use emergency services, between outpatients and inpatients, and between those with serious functional impairments and those classified as “adapted.”

We also used the SF-12 Mental Health, Role-Emotion, and Social Functioning subscales, which assess general mental health, emotional, and social functioning (Ware, Kosinski & Keller, 1995; Ware, Jr., Kosinski, Turner-Bowker & Gandek, 2002). The SF-12 is a short version of the SF-36 Health Inventory designed as a general indicator of health status for use in population-based surveys and health-policy evaluations. It can be used to assess health among individuals with widely ranging conditions/severities and is particularly useful for assessing the health status of individuals with multiple health conditions and for comparing the health of different populations (McDowell & Newell, 1996). Additionally, the SF-36 has been shown to be a reliable and valid measure of perceived functioning and well-being among people with serious mental illnesses (Tunis, Croghan, Heilman, Johnstone & Obenchain, 1999). The SF-12 reproduces the eight SF-36 subscales with >90% accuracy.

Quality of Mental Health Care

Modular Survey Common Performance Measures

The Centers for Substance Abuse Treatment and the Substance Abuse and Mental Health Service Administration are developing a common set of performance measures for mental health and substance abuse services. These 10 items, called the Modular Survey, assess quality of care (Bartlett, Chalk, Manderscheid & Wattenberg, 2006) (see Table 2) rated on a 5-point scale from strongly disagree to strongly agree, with an option of “does not apply.”

Table 2.

Concurrent Validity of the PAM-MH.

	Pearson’s r
Congruent Concepts
Recovery Assessment Scale (RAS)
RAS Overall Score	.63***
Willingness to Ask for Help	.55***
Personal Confidence and Hope	.52***
Goal and Success Oriented	.51***
Not Dominated by Symptoms	.43***
Reliance on Others	.38***
Related Concepts
Modular Survey Quality of Care Items
I am given enough information to effectively handle my condition	.46**
My clinicians explain things in a way I understand	.45**
I help to develop my service/treatment goals	.42**
I am given information about my rights as a consumer	.36**
I am given information about available services	.34**
My clinicians are sensitive to culture background	.32**
My clinicians spend enough time with me	.28**
I am treated with respect	.26**
Mental Health Self-care
Manage mental health symptoms by eating right	.27**
Manage mental health symptoms by listening to music	.23**
Manage mental health symptoms by getting right amount of sleep	.22**
Manage mental health symptoms with meditation	.21**
Manage mental health symptoms with self-injury	−.18*
Psychiatric Medications
How satisfied with taking psychiatric medications?	.33**
How well psychiatric medications help control symptoms?	.32**
Frequency of taking psychiatric medications as prescribed?	.24**
Physical & Mental Health
SF-12 Vitality	.37**
SF-12 Mental Health	.34**
Colorado Symptoms Inventory	−.34**
SF-12 Physical Functioning	.30**
SF-12 Role Emotional	.27**
SF-12 General Health	.26**
SF-12 Role Physical	.26**
SF-12 Social Functioning	.23**
SF-12 Bodily Pain	.21**
Quality of Life
W-QLI General Life Satisfaction	.30**

^*p ≤ 0.05,

^**p ≤ 0.01,

^***p ≤ .001

Physical Health

Measures of physical health in Study 1 included the SF-12 Physical Component Summary, SF-12 Physical Functioning, SF-12 Role Physical, SF-12 Bodily Pain, SF-12 General Health, and SF-12 Vitality subscale. Psychometric properties are described above.

Mental Health Self-care

We also asked Study 1 participants about their mental health self-care practices. Binary items included managing mental health symptoms by getting the right amount of sleep, eating right, listening to music, meditating, and managing symptoms with self-injury.

Quality of life

In Study 1, we measured quality of life using the Wisconsin Quality of Life Index (W-QLI) (Becker, Diamond & Sainfort, 1993). W-QLI scales are importance-weighted; the general life satisfaction subscale, used here, assesses satisfaction with neighborhood, housing, food, clothing, transportation, sex life, personal safety, and how time is spent during the day. The W-QLI was developed for paper-and-pencil use with severely mentally ill individuals. It measures fairly-independent preference-weighted quality-of-life dimensions, each of which has objective and subjective components (Becker, 1998; Diamond & Becker, 1999). Test-retest reliability is good, with percentage agreements from 82% to 87% across domains. Psychometric properties were examined in a Canadian version that adapted background information for applicability in Canada; test-retest reliability was good, with a correlation of .74 for general life satisfaction level. The correlation with Spitzer’s quality of life index (Spitzer, Dobson, Hall, Chesterman, Levi, Shepherd, Battista & Catchlove, 1981) was also good (r = .72).

Psychiatric Medications

In Study 1, we also asked participants to report how frequently they took their psychiatric medications as prescribed, how well psychiatric medications helped control their symptoms, and how satisfied they were with their psychiatric medications (from very dissatisfied to very satisfied on a 7-point scale).

Results

We used Rasch Analysis to examine the psychometric characteristics of the PAM-MH. The original PAM has a calibrated scale range from 39–53 (on a theoretical 0–100 point scale), while the adapted PAM-MH has a range from 39–59. Thus, the PAM-MH has a slightly larger difficulty range than the PAM. In addition, all of the infit and outfit statistics for the 13-item PAM-MH are well within the 0.5–1.5 acceptable range, with infit and outfit scores very similar to the original. Finally, the PAM-MH person-reliability and the item-reliability scores compare favorably with the original 13-item PAM. Person-item reliability for the PAM-MH is .84 (original PAM = .82), and item reliability is .97 (original PAM = .99). Thus, the adapted PAM-MH appears to have psychometric properties as strong as the original 13-item PAM.

Table 1 presents socio-demographic characteristics of participants in the three studies and mean PAM-MH scores. Bivariate analyses assessing the relationship between Study 1 socio-demographic characteristics and PAM-MH scores showed only one significant correlation—between working or being a student and PAM-MH (r =.19, p < .05)

Test-Retest Reliability

We assessed test-retest reliability with data from Study 3 control group participants (n = 18), who received only usual care. The mean interval between administrations was about 14 weeks. Because they received no intervention, we did not expect to see a change in activation across the 14-week period for the control group. Test-retest reliability was good (Pearson’s r = .74).

Concurrent Validity

Table 2 presents Study-1-based correlations between the PAM-MH and two sets of measures used to examine concurrent validity. We classified variables as either Congruent Concepts (those closest to measures of activation) or Related Concepts (those we expected would be related to activation, but not as strongly because they are influenced by other factors). Rows are sorted, within subgroups, by strength of the relationship.

We hypothesized that greater activation would be most strongly associated with greater recovery and less strongly associated with better mental health self-care, medication satisfaction, mental health, physical health, and quality of life. As expected, the strongest relationships were with Recovery Assessment Scores (r = .63 for overall RAS score), with scores on subscales closely tied to activation (Willingness to Ask for Help, Personal Confidence and Hope, and Goal and Success Oriented) the highest (r = .51 to r = .55), and with those less clearly tied to activation (Symptom Domination, Reliance on Others) having weaker relationships (r = .38 to r = .43).

As expected, quality of mental health care was more moderately associated with activation than recovery. Notably, relationships were seen between the PAM-MH and clinicians explaining things in ways the patient understands, being involved in developing treatment goals, clinician sensitivity to cultural background, and receiving information about available services and consumer rights (r = .32 to r =.46).

Several mental health self-care practices also showed relationships with the PAM-MH. Individuals with higher activation were more likely to manage mental health symptoms by getting the right amount of sleep, listening to music, meditating, and eating right, and less likely to manage symptoms by self-injury. In addition, greater activation was associated with better adherence to psychiatric medications, better symptom control, and greater satisfaction with medications.

Finally, higher activation was also associated with fewer mental health symptoms on the CSI (r = −.34) and better mental and physical health across all SF-12 subscales (r =.21 to r = .34). Similarly, greater activation was associated with better general life satisfaction (r =.30). As expected, these associations were weaker than associations with congruent concepts.

Sensitivity to Change

We assessed sensitivity to change in Study 3, comparing change across time in PAM-MH for intervention and control groups. If the intervention was effective and the PAM-MH was sensitive to change, we expected a greater change in the intervention than control group. The mean PAM-MH score at baseline for the intervention group¹ was 65.09 (sd = 13.08), which increased to 72.84. The mean PAM-MH score at baseline for the control group was 58.49 (sd = 11.17), which increased to 61.71. The effect size for change over time in the intervention group (d = 0.74) was substantially greater than in the control group (d = 0.29).

Discussion

Our results suggest that the PAM-MH is a valid and reliable measure of activation among individuals with mental health problems, with psychometric properties equal to the original 13-item PAM. Our results are consistent with findings examining activation for medical conditions: we found that greater activation was related to higher levels of recovery, better quality mental health care, better physical and mental health, and fewer mental health symptoms. In addition, more activated individuals engaged in better mental health self-care, were more likely to take psychiatric medications as prescribed and reported greater satisfaction and symptom control from their psychiatric medications. More activated respondents also reported better quality of life.

Another important result was that, in general, socio-demographic measures were not related to activation. This pattern of findings indicates that factors amenable to change are more closely related to activation than are more stable personal characteristics or traits that either cannot be changed or are more difficult to change. Perhaps most significant, activation was strongly related to recovery across several dimensions. Such findings, in combination with indications that the PAM-MH is sensitive to change and associated with better quality mental health care, suggest the possibility that efforts to enhance activation and improve care quality may also facilitate recovery from serious mental illness. More research is needed, however, to assess these linkages.

Limitations

Our findings are limited to people with more serious mental conditions; additional analyses are needed to explore the PAM-MH’s use among individuals with less serious problems. In addition, although we included several samples to increase the generalizability of our findings, the total sample size remains relatively limited and additional research should explore the PAM-MH in broader populations.

Conclusions

The PAM-MH appears to be a reliable and valid measure of patient activation among individuals with serious mental health problems. More activated individuals were more likely to score higher on measures of recovery, adhere to psychiatric medication directions, be satisfied with medications and their effectiveness, engage in better mental health self-care, have better mental and physical health, and better quality-of-life. They were also more likely to have received better quality mental health care. These findings mirror those found with the original PAM and suggest that the PAM-Mental Health is a valid measure for use with populations with mental health diagnoses. These results, combined with lack of associations with stable socio-demographic factors, also suggest that activation related to mental health is amenable to change. Results showing that the PAM-MH is sensitive to change suggest that the PAM-MH could be a useful instrument for measuring change in activation.