Development and Validation of the Consumer Health Activation Index

Abstract

Background

While there has been increasing interest in patient engagement, few measures are publicly available and suitable for patients with limited health literacy.

Objective

We sought to develop a Consumer Health Activation Index (CHAI) for use among diverse patients.

Methods

Expert opinion, a systematic literature review, focus groups and cognitive interviews with patients were utilized to create and revise a potential set of items. Psychometric testing guided by item response theory was then conducted among 301 English-speaking, community-dwelling adults. This included differential item functioning analyses to evaluate item performance across participant health literacy levels. To determine construct validity, CHAI scores were compared to scales measuring similar personality constructs. Associations between the CHAI, physical and mental health established predictive validity. A second study among 9,478 adults was used to confirm CHAI associations with health outcomes.

Results

Exploratory factor analyses revealed a single factor solution with a 10-item scale. The CHAI showed good internal consistency (alpha=0.81) and moderate test-retest reliability (ICC=0.53). Its reading grade level was found to be at the 6^th grade. Moderate to strong correlations were found with similar constructs (Multidimensional Health Locus of Control; r=0.38, p<0.001; Conscientiousness, r=0.41, p<0.001). Predictive validity was demonstrated through associations with functional health status measures (depression, r=−0.28, p<0.001; anxiety, r=−0.22, p<0.001; and physical functioning, r=0.22, p<0.001). In the validation sample, the CHAI was significantly associated with self-reported physical and mental health (r=0.31 and r0.32 respectively, both p<0.001).

Conclusions

The CHAI appears to be a valid, reliable, and easily administered tool that can be used to assess health activation among adults, including those with limited health literacy. Future studies should test the tool in actual use and explore further applications.

An increasing number of references have been made in recent years about the need for greater “patient engagement” throughout the continuum of healthcare.^1,2 While helping patients assume more active roles in managing their personal health has been a longstanding public health priority, health systems continue to struggle to find effective, sustainable ways to involve patients in their care.³ Since the advent of the Affordable Care Act (ACA) in 2010, healthcare providers and payers alike have now taken on a more vested interest in patient engagement.^4,5 This is due in part to certain provisions mandating the routine assessment of consumers’ health-related activation. Yet emerging reimbursement models that promote quality over fee-for-service, such as value-based care delivery, offer additional incentives for health systems and insurers to improve their outreach to patients and optimize health and healthcare outcomes.⁶ The underlying premise is that a less-engaged or motivated individual will be poorly adherent to a prescribed treatment regimen or inappropriately use healthcare services, driving down reimbursement in such a new payment system.

This growing need for better patient engagement has created a demand for the ability to measure it among healthcare consumers to inform health system and/or payer strategies. As a result, the Patient Activation Measure (PAM), which was developed more than a decade ago by Hibbard and colleagues has taken on considerable prominence in this space.^7,8 The PAM measures health activation, defined as “an individual’s willingness to take on the role of managing their health and healthcare”. It is the first assessment tool to operationally define the construct of activation in a health context, and now has been widely used to near exclusivity as an assessment of how engaged a patient is in their own health. Several studies have even documented strong associations between PAM scores and a range of outcomes, including medication adherence, preventive services use, hospitalization, and indicators of chronic disease self-management.^9-12 Approaches have been proposed that tailor interventions based on the PAM’s levels of activation, suggesting that different forms of remediation may be necessary based on one’s baseline level.¹³

The value of measuring health-related activation among patients is still being determined, but without question the PAM has played an important role in keeping patient engagement a prominent issue to be addressed. Yet there is still a need to revisit innovation around the measurement of this important construct. The PAM itself has some specific limitations, including concerns it may not be appropriate among lower health literate audiences based on item readability, and its proprietary status makes its use costly and subject to restrictions.¹³ But it is also important, on principle, to explore alternative measures. Continued exploration of the construct of patient activation and its measurement can help further define what is meaningful engagement in healthcare. This, in turn, may inform the development of interventions designed to increase patients’ involvement in their personal health.

In this study, we sought to develop a new, brief measure of healthcare activation, herein referred to as the Consumer Health Activation Index (CHAI). Our goal was to devise a valid instrument that was publicly available, easy to administer and score, and appropriate for use among individuals across health literacy levels. As health systems and payers alike are increasingly in need of such a tool, special emphasis was placed on optimizing a minimum number of items that could accurately measure the construct.

METHODS

The goals of the study were to (1) create a preliminary bank of items with broad theoretical coverage of the CHAI construct and (2) evaluate the psychometric and qualitative properties of this bank. Research activities included readability analyses, cognitive interviewing, focus groups, and differential item functioning (DIF) analyses (to evaluate the appropriateness of the items for use among lower health literate consumers) as well as exploratory factor analyses to evaluate the psychometric properties of the item set. A follow-up study was also performed among a larger, age-diverse sample to confirm CHAI performance and to test its predictive validity across a range of outcomes.

CHAI Item Generation and Refinement

To generate potential CHAI items, we engaged in an extensive and iterative process that incorporated multiple diverse perspectives. First, a list of potential CHAI domains and items were generated by the study team through an in-depth review of the scientific literature, existing measures and expert opinion from physicians, health services researchers, behavioral scientists, and health plan leaders. From these sources, five key domains emerged for the CHAI, including consumers’: (1) knowledge, (2) self-efficacy, (3) motivation and beliefs, (4) actions, (5) internal locus of control. We also included a subdomain, chronic illness, to capture aspects of consumer activation specific to managing a chronic disease. Ten items per domain were developed for further testing and refinement.

Following health literacy best practices established by Wolf and colleagues previously, all CHAI candidate items were positively worded to avoid changing item valences.¹⁴ In addition, a 6-point Likert scale was used to ascertain item agreement (strongly disagree (1), disagree (2), slightly disagree (3), slightly agree (4), agree (5), strongly agree (6)). A ‘two-step’ approach to instructing participants ensured accurate responses by separating out the valence from the intensity of items. For example, after each statement an interviewer would ask “do you agree or disagree?”, followed by a second question “do you (dis)agree a little, somewhat, or a lot?” Responses could then be dispersed accordingly across the 6 categories.

We then sought patient feedback on potential CHAI items through two discussion groups with adult, English-speaking patients (N=17 subjects). Participants were recruited from an ongoing observational study conducted among primary care patients 60 years of age and older living with asthma, among other chronic conditions.¹⁵ Using existing data collected as part of the study, we targeted individuals with limited health literacy to ensure items were easily understood. Discussion groups focused on reviewing potential CHAI items and generating ideas for explicit improvement. Patients also provided opinions on the concept of activation and how it might affect their health. Specifically, participants were asked to discuss attributes that best described an “engaged and activated patient” through a variety of exercises. Patient input was used to revise potential CHAI questions for subsequent testing. Additional items were also generated based upon patient suggestions.

Item Readability

We further sought to ensure items were appropriate for patients with limited literacy skills by applying three readability formulas (Lexile, Fleisch-Kincaid, Gunning Fog Index) to item sets in order to verify they were written at a reading grade level that was appropriate for diverse audiences. Lexile analysis, specifically, was performed on individual items as a prior study found that even with short text statements, Lexile scores were significantly associated with patients’ comprehension of the text.¹⁶ Items were then tested in cognitive interviews with adult learners from an urban adult basic education center (N=13). The cognitive interviews and iterative editing resulted in 35 easy-to-understand items to advance with further item performance testing.

Item Performance Testing

We prospectively administered the proposed pool of items to an existing sample of older adults who were recruited from a longitudinal study of aging, known as LitCog (R01AG030611).¹⁷ Patients in the LitCog study were recruited from one academic general internal medicine clinic and five federally qualified health centers in Chicago, IL. They were eligible if they were (1) between the ages of 55 and 74; (2) had a primary care physician associated with the academic general internal medicine clinic or the federally qualified health center they were recruited from; (3) did not have an uncorrectable hearing or vision problem; (4) had fluency in the English language; and (5) did not have severe cognitive impairment. For the purpose of this study, eligible LitCog participants were purposively sampled in order to ensure representative groups from both the academic medical center and the federally qualified health centers.

Eligible patients were contacted by phone and asked to complete a brief interview that included the potential CHAI items and the Multidimensional Health Locus of Control (Form A).¹⁸ This measure was considered to be conceptually aligned with activation, and therefore would serve as a means to assess construct validity since it may capture similar traits as the CHAI. Data collected as part of recent in-person LitCog interviews was also used, specifically, patient socio-demographic characteristics, health literacy skills as measured via the Newest Vital Sign (NVS)¹⁹, and health status (as measured by single item self-report of overall health, and number of chronic conditions). The LitCog battery also included the NEO personality inventory; the internal conscientiousness subscale was used as a second construct validation measure.²⁰ Patient-reported outcomes including mental health (depression, anxiety) and physical functioning were measured via validated PROMIS tools.^21,22 Patients were given a $15 credit card for their participation. The Northwestern Institutional Review Board approved all research procedures.

Psychometric Analyses

Exploratory factor analysis (EFA) was used to evaluate the structure of the 35 CHAI items. Specifically, these analyses were conducted by extracting from 1 to 10 factors, allowing for oblique (oblimin) rotations and using the “psych” package in R.²³ Fits for these factor solutions were evaluated using a large number of fit statistics, including the root mean square error of approximation (RMSEA), the Bayesian Information Criterion (BIC), Velicer’s MAP criterion (MAP), and mean item complexity values. While we expected that a solution with as many as five factors (matching the five theoretical domains) might provide an optimal statistical fit, we aimed to identify a solution that was both efficient and highly replicable (i.e., contained many items with high primary loadings and low secondary loadings).

Using the polychoric correlations between a unidimensional subset of items identified from the EFA, two-parameter Item Response Theory-based analyses were conducted.^23-26 This was done in order to report item calibrations (difficulty and threshold parameters) that would enable computer adaptive testing of the final measure, as this allows for more brief assessment without loss of precision.²⁵

Cronbach’s alpha was used to examine reliability (internal consistency). We also conducted test-retest reliability, examining the intraclass correlation coefficient (ICC). Differential item functioning (DIF) analyses examined the extent to which items in a questionnaire perform similarly or differently across different groups of patients, helping to ensure that all items are related to the concept of interest in a consistent way across groups. Specifically, DIF was investigated across gender, race (black vs. other and white vs. other), and health literacy level (adequate vs. limited).

Construct and Predictive Validity

To assess construct validity, we examined bivariate correlations between the Multidimensional Health Locus of Control and the Conscientiousness subscale of the gold standard NEO personality assessment. Given the proprietary nature of the PAM, we could not also use it as a means of construct validation since researchers are restricted from using the tool as part of a process to develop alternative, competing measures. To explore discriminant validity, we examined correlations with the NVS, as Smith and colleagues previously found health literacy and patient activation to be distinct constructs.¹⁰

Associations between the CHAI total score and depression, anxiety, physical functioning, self-rated overall health, and number of self-reported chronic conditions were examined for predictive validity. Self-rated overall health was measured by asking participants the question, “how would you rate your overall health” and given the choices, “excellent”, “very good”, “good”, “fair”, or “poor”. Additionally we examined the association between CHAI total score with meaningful decline in physical health over three years.²⁷ This variable was calculated using the effect size method, which calculates a standardized measure of change over time. The difference between baseline and follow-up scores is divided by the standard deviation at baseline. The resulting effect size represents change in terms of the number of baseline standard deviation³⁶. A medium effect size corresponded to an absolute decline on the 0-100 score of approximately 5 points. We conducted correlations, t-tests or ANOVA analyses, as appropriate.

Scoring

Thresholds for the CHAI were established by first reviewing the distribution of scores. Stratum-specific likelihood ratios (SSLRs) were then calculated for three proposed scoring categories that mapped their association to meaningful decline in physical health over three years. These ratios provide additional confirmation of each threshold by offering estimates of the likelihood of having a significant decline per category.

Validation Sample

Members of the study team (SR, GK, VH) were able to disseminate the 10 final CHAI items to a second sample of employees at a for-profit financial organization affiliated with UnitedHealth Group. The CHAI items were embedded in an online “health and productivity assessment”. Employees were incented to complete the CHAI online using an existing health portal. Although the assessment was available for completion throughout the year, for this analysis only those who completed the CHAI between January 1, 2016 and March 31, 2016 were included. In addition to the CHAI, demographic variables (gender, age, race, marital status) were collected. To confirm predictive validity, the following outcomes were available: 1) self-reported overall health and mental health via a single item each using a 5-point Likert scale (excellent, very good, good, fair, poor); 2) health risk status, as measured by a previously developed model that factors in medical biomarkers (body mass index, blood pressure, cholesterol, triglycerides, blood glucose).^30,31 A summary count variable of the number of risk factors present for a consumer was calculated (0-12 possible). Spearman or Pearson correlation coefficients were calculated on health status and risk outcomes, accordingly.

RESULTS

The initial validation sample included 315 LitCog participants who were identified as eligible and contacted via telephone. A total of 301 individuals completed interviews (96% response rate). Participants were on average 66.7 years old (SD=5.3); over two thirds (69.1%) were female. Eighty patients (26.7%) were recruited from federally qualified health center. Table 1 provides more description of the study sample.

Table 1.

Primary Sample Demographic Characteristics (N=301).

Variable	%
Age
< 60	13.5
60 – 64	29.3
> 65	57.2
Gender
Male	30.9
Female	69.1
Income
< $10K	11.7
$10K – $24.9K	19.4
$25K – $49.9K	22.3
>$50K	46.6
Race
White	48.5
Black	44.5
Other	7.0
Health Literacy Level
Limited	52.2
Adequate	47.8
Education
Less than high school	8.7
High school graduate	16.1
Some college	20.4
College graduate	22.1
Graduate degree	32.8

Pilot items

After reviewing response distributions for the 35 pilot items, those with less than five responses in a given category were collapsed for analyses. For example, if only three respondents answered “slightly agree” to a specific item, answer choices were combined with the response option nearest, in this case “agree.” Twenty-one items required this adjustment. Additionally, the research team reviewed all items and removed two of them based on redundant content.

Exploratory Factor Analysis

Exploratory factor analyses of the 35 CHAI items suggested good fits for solutions where 1 to 4 factors were extracted. The 4 factor solution suggested optimal fit based solely on the fit statistics (MAP = .0123; BIC = −1,606; RMSEA = .067; mean item complexity = 1.53); solutions with more than 5 factors consistently demonstrated less optimal fit. However, the organization of items based on the primary factor loadings for the 4 factor solution did not produce an easily-interpretable and efficient solution in clinical settings. While the four factor solution may prove useful for future research with respect to underlying mechanisms (see supplementary Appendix A), this solution was considerably less efficient than the single factor solution for use in clinical settings.

With the exception of the first factor (in the 4 factor solution) the organization of items was highly inconsistent with the theorized domains. Fit statistics for the single factor solution (MAP = .018; BIC = −1,126; RMSEA = .098) were less optimal than the four factor solution but sufficient to warrant the identification of item parameters using analyses based in item response theory (for the full item set, alpha = .92 and unidimensionality based on the ratio of the squared residual correlations to the squared correlations = .73). The lack of parsimony for the four factor solution is also contrasted to the substantial efficiency gained by the item-response based assessment benefits offered by the single-factor solution.

Factor loadings based on the single factor solution are shown for all items in Table 2. On the basis of both the loadings and the item content, the research team identified a 10-item subset that allows for brief assessment and is representative of the theorized domains. Fit and unidimensionality statistics for the single factor EFA solution with this 10-item subset provided evidence for adequate unidimensionality (omega hierarchical = 0.64, unidimensionality = 0.71, MAP =0.03; BIC = −28; RMSEA =0.11).

Table 2.

Exploratory Factor Analysis (EFA) and Lexile Analysis.

Item	Factor Loading	Lexile Score
Know1*: I always know how to make myself feel better†	0.42	450
Know2: I always know when to call a doctor	0.46	290
Know3*: I always know what steps to take when I have a health problem	0.49	680
Know4: I always know how to follow my doctor’s instructions	0.50	520
Know5: I always know which doctor I should call based on the problem I am having	0.48	860
Know6*: I always know where to look for information before making decisions about my health	0.53	840
Know7: I always know the difference between good and bad health information	0.54	670
Selfeff1*: It is very easy for me to follow my doctor’s instructions	0.63	640
Selfeff2*: It is very easy for me to understand my doctor’s instructions	0.59	630
Selfeff3: It is very easy for me to ask my doctor questions	0.56	600
Selfeff4*: It is very easy for me to make changes to my daily life to improve my health	0.68	950
Selfeff5: It is very easy for me to get a second opinion if I need it	0.52	750
Selfeff6: It is very easy for me to disagree with my doctor or nurse	0.42	760
Selfeff7: It is very easy for me to use technology to take care of my health	0.36	820
Selfeff8: It is very easy for me to find health information	0.40	550
Selfeff9: It is very easy for me to use healthcare	0.39	420
Beliefs1: It is very important that I try to make positive changes to my health	0.55	790
Beliefs2: It is very important that I follow my doctor’s instructions	0.55	590
Beliefs3: It is very important that I take steps to improve my health every day	0.59	820
Beliefs4*: It is very important that I treat my health as my top priority	0.60	810
Beliefs5: It is very important that I fight for my health	0.57	500
Beliefs6: It is very important that I work hard to maintain and improve my health	0.58	810
Beliefs7: It is very important that I make changes to my life now, so I can have the best health when I’m older.	0.55	1070
Action1: I always follow all of my doctors advice	0.58	460
Action2: I always make healthy changes to the way I live my life	0.69	650
Action3*: I always attend all of my doctors appointments	0.39	540
Action4: I always fill all of my prescriptions on time	0.37	520
Action5: I always update my doctor about any major health changes between my visits	0.44	1000
Action6: I always look for answers to my health questions from places like the internet, pamphlets or magazines	0.28	1080
Action7: I always bring a list of questions to ask my doctor to my visits	0.34	800
Action8*: I always make the health changes I should, even if I do not feel well	0.58	810
Locus1: I can always avoid getting sick if I take care of myself	0.48	730
Locus2: I can always control my health	0.55	250
Locus3: I can always improve my health	0.43	300
Locus4*: I can always take care of myself	0.51	270
All 36 items		700
Final 10 items		680

^†Final CHAI items in bold

Item Response Theory-Based Analyses

Table 3 shows the discrimination and threshold parameters based on IRT calibrations using the 10 item subset identified from the EFA. All of the items had moderate to strong discrimination values though the three “self-efficacy” items (items 6 through 8 in Table 3) were notably more discriminating. The high positive values at β₅ for items 1 and 9 suggested that these items were rarely endorsed at extreme levels; none of the participants endorsed the lowest response category for either of these items.

Table 3.

Item Response Theory Analyses

Items		Item Parameter Estimates
	α	β1	β2	β3	β4	β5
It is very important that I treat my health as my top priority	0.79	−2.40	−1.98	−1.48	−0.17	Inf
I always know what steps to take when I have a health problem	0.72	−2.15	−1.24	−0.76	−0.32	0.74
I always know how to make myself feel better	0.61	−2.26	−1.50	−0.61	−0.32	0.62
I always know where to look for information before making decisions about my health	0.75	−2.49	−1.54	−1.00	−0.54	0.79
I can always take care of myself	0.68	−2.48	−1.30	−0.99	−0.51	0.47
It is very easy for me to understand my doctor’s instructions	1.13	−4.09	−2.84	−2.23	−1.70	0.14
It is very easy for me to make changes to my daily life to improve my health	0.93	−2.33	−1.37	−0.95	−0.25	0.99
It is very easy for me to follow my doctor’s instructions	1.34	−3.89	−2.80	−2.14	−1.68	0.18
I always attend all of my doctor’s appointments	0.72	−3.34	−2.11	−1.82	−0.22	10.10
I always make the health changes I should, even if I do not feel well	0.72	−2.73	−1.65	−1.03	−0.34	0.96

Notes: α = discrimination parameter; β₁, β₂, β₃, β₄, β₅ = threshold parameters

Reading Level

All 10 items, without response options, were analyzed for reading grade level and related statistics. Average reading grade level across the Flesh Kincaid, Lexile and Gunning Fog rating scales was 6.2.

Reliability

Cronbach’s alpha for the 10-item CHAI was α = 0.81 which indicates good internal consistency. The items of the CHAI were re-administered to 45 participants after approximately two weeks (average time between administrations = 17 days). Intra-class correlation coefficient (ICC) is indicative of moderate degree of reliability (ICC=0.53).

Differential Item Functioning

Using the pre-defined cut-off of 0.13, an accepted value within the literature³⁴, no differential item functioning was found for gender, race, or health literacy.

Scoring

For ease of interpretation and analysis, we chose to transform the CHAI total score (theoretical range of 10 – 60) to a more interpretable rubric. We used linear transformation to put CHAI total scores onto a 0-100 point scale⁴⁰. The average score for the sample was 80.3 (SD=12.5), with a range of 43.3 to 100.

Associations with demographic variables

CHAI score by demographic variables are shown in Table 4. No differences were identified by age, gender, income, race/ethnicity, or educational attainment. Significant differences were found between race/ethnicity groups with African Americans scoring highest (p<0.001).

Table 4.

Validity Analyses

Variable	CHAI	P-Value
Construct Validity
Multidimensional Health Locus of Control – Internal	rp =0.38	<0.001
Conscientiousness (NEO)	rp =0.41	<0.001
Discriminant Validity
Health Literacy, Mean (SD)		0.57
Adequate	76.7 (15.2)
Limited	75.6 (14.8)
Predictive Validity
Depression (PROMIS)	rp =−0.28	<0.001
Anxiety (PROMIS)	rp =−0.22	<0.001
Physical Functioning (PROMIS)	rp =0.22	<0.001
Decline in Physical Functioning (PROMIS), Mean (SD)		0.04
No Decline	77.4 (14.8)
Decline	73.3 (15.3)
Self-rated Overall Health	rs =−0.16	0.01
Self-report # Chronic Conditions	rs =−0.10	0.07

r_p =Pearson’s r, r_s =Spearman’s rho

Validity analyses

The CHAI score was moderately positively correlated with the MHLC – Internal subscale (r=0.38, p<0.001) and the conscientiousness subscale of the NEO (r=0.42, p<0.001). Relationships with health outcomes were also assessed. High CHAI scores were associated with fewer depressive (r=−0.27, p<0.001) and anxiety symptoms (r=−0.22, p<0.001) and with greater physical functioning (r=0.27, p=0.002). In analysis examining change in physical health over 3 years, a higher CHAI total score was significantly associated with less physical decline (p=0.05). The CHAI was not related to overall self-reported health (p=0.09) or health literacy (p=0.64), shown in Table 4.

Preliminary Cut-Points

Using the variable of meaningful decline in physical functioning over three years, patients who scored at or below 79 on the CHAI had a nearly three times worse physical decline over three years, compared with those scoring 95 and above on the CHAI (OR =2.99; 95% C I=1.09 – 8.19; p=0.03). Adults scoring between 80 and 94 on the CHAI had over two times worse physical decline over three years compared to the same referent group (OR =2.24; 95%CI= 0.8 – 6.21; p=0.12). Based on these findings, we proposed CHAI scores be preliminarily segmented into “low” (CHAI score 0-79), “moderate” (CHAI score 80-94), and “high” (CHAI score 95-100) activation levels. A final version of the CHAI instrument, with instructional guidance and scoring, is presented in Supplementary Appendix B.

Validation Sample

The second validation sample included 9,478 adults. They were primarily female (62.8%), middle-aged (Mean (M) = 42.7 years, Standard Deviation (SD) = 10.9), white (78.8%), non-Hispanic/Latino (89.0%), married or living with a domestic partner (63.0%). The sample was generally healthy; only 558 respondents (5.9%) had one or more chronic conditions. The mean CHAI score was 84.3 (SD = 12.11). All items were moderately to strongly correlated with each other (correlation coefficients ranging from 0.27 to 0.72).

The Cronbach’s alpha was used to determine internal consistency, which was high (ω = 0.91). CHAI scores were significantly associated with both perceived physical (r=0.31, p<0.001) and mental health status (r=0.33, p<0.001). In addition, higher CHAI scores were significantly associated with lower multiple health risk status as indicated by the presence of fewer elevated medical and lifestyle risk factors (r=−0.19, p<0.001). Table 5 presents the associations between the CHAI and each individual medical and lifestyle risk factor investigated.

Table 5.

Relationship of the CHAI with Medical and Lifestyle Risk Factors

Medical Risk Factor1	n	r3	p2
Weight/BMI	6542	−0.13	< 0.001
Blood Pressure	6283	0.04	< 0.008
Cholesterol	6285	0.08	< 0.001
Triglycerides	5159	0.08	< 0.001
Blood Glucose	5187	0.06	< 0.002
Lifestyle Risk Factor1	n	r3	p2
Tobacco Use	9477	0.10	< 0.001
Alcohol Use	9478	0.10	< 0.001
Physical Activity	9478	0.24	< 0.001
Dietary Fat	9478	0.23	< 0.001
Fruits/Vegetables	9478	0.17	< 0.001
Stress/Coping	9478	0.29	< 0.001
Seat Belt Use	9478	0.04	< 0.001

¹With the exception of Weight/BMI all risk data are ordinal and are based on risk status (low, medium, high).

²With 12 comparisons, significance was set a p < 0.004

³Type of correlation used: BMI = Pearson; All other risk factors = Polyserial

DISCUSSION

Our study aimed to develop a viable method of assessing a consumer’s health activation level that can be easily administered to patients of varying health literacy levels in both broad research and healthcare settings. We conceptualized the construct of patient activation with consultation from the literature, experts and patients, constructing items through an iterative process that incorporated feedback from adults with limited health literacy. Analyses indicated an acceptable model fit with a brief, 10-item measure. The final CHAI tool appears to be a reliable, valid measure of consumer health activation with easy-to-understand items. Its readability achieved the lowest recommended grade levels for health materials (6^th grade level)^32,33, and our findings suggest the final items’ performance did not differ by health literacy level. It also demonstrated exceptional construct validity, as it was found to be strongly and significantly associated with measures assessing similar content (locus of control and conscientiousness). CHAI scores were also associated with patient-reported outcomes including mental health and physical function, and clinically meaningful decline in physical health over three years. Results were replicated in a second, larger and more age-diverse sample. Overall, we believe these results suggest the CHAI is a viable alternative to existing activation measures but with several advantages. Our tool is publicly available and free to use, easy to administer, score and interpret. Hopefully the accessibility of the CHAI may encourage researchers, clinicians, and others in healthcare to include the variable of health activation in projects and in clinical settings.

As health activation research expands, more precise generalizations can be made about individuals with lower and higher activation and in particular, how these groups differ with regards to their self-care behaviors and outcomes. Understanding a healthcare consumer’s view of their role in the context of their health is critical, as healthcare providers often make assumptions about their patient’s motivation or ability to carry out recommendations.^34,35 Volpp and Mohta concluded from a survey of clinicians and healthcare executives that improving patients’ engagement in their health is critical for achieving better outcomes, and this is nearly universally recognized.³⁶ Similarly, improving the ability to measure engagement has now become paramount. A measure such as the CHAI may play an important role moving forward both as a screening tool to identify those at risk and intermediary outcome for the increasing number of initiatives, ranging from patient education and counseling endeavors to ‘mhealth’ activities that monitor behavior, which are already being launched in healthcare systems to promote patient engagement.

A novel emphasis in our development of the CHAI was the specific inclusion of health literacy issues in its validation. To date, very few psychometric tools have taken such explicit steps to validate measures among adults with limited health literacy skills. Yet an estimated 18-25% of the adult U.S. population is thought to have low health literacy and/or health numeracy skills placing them at high risk for misunderstanding items and consequently providing inaccurate responses.^37,38 Wolf and colleagues previously established a process for developing low health literacy-appropriate measures, used here.¹⁴ This includes the consult and iterative feedback from those with limited health literacy and experts in the field, but also robust readability analyses, using a common valence scheme, simplification of the response scale by using the 2-step approach, and performing differential item functioning by health literacy level. While these efforts cannot guarantee suitability of the CHAI or another measure for use among individuals with varying proficiencies, certainly these additional steps should be considered as a worthwhile investment for acquiring the most accurate, valid responses. Some if not all of these activities should be recommended in any psychometric tool development endeavor.

Clearly there are limitations to this study. Our samples may not be entirely representative of the general population, particularly by income. Second, the most logical construct validation would have occurred with the current gold standard measure of activation, the PAM, however this was not possible due to prohibitions on its use. Predictive validity analyses were limited to patient-reported outcomes within our initial sample, and while expanded in our validation sample those associations were cross-sectional. Additional evidence demonstrating causal links between the CHAI and a range of health and healthcare outcomes would further solidify the value of the measure. This would also provide greater support for the current proposed threshold cut-off scores, or possibly justify changes. Next directions should continue to extend the CHAI’s psychometric evaluation, including a more robust investigation of test-retest reliability and discriminant validity. Evidence of sensitivity to change will be necessary to eventually determine whether the CHAI is an applicable evaluative tool for health system interventions.

With the CHAI, we offer a valid, more user-friendly alternative for the measurement of healthcare activation. Hopefully, this new addition to a measurement-limited field will catalyze research and inform interventions. While the CHAI has been designed to be applicable to most adults, eventually an item bank may also be constructed to allow for a tailoring of assessments based on the self-care roles individuals are expected to assume as part of their healthcare. For example, a CHAI assessment for a patient with type 2 diabetes and hypertension might include specific items that pertain to blood sugar monitoring, medication adherence, engagement in lifestyle behaviors, among others. This could possibly enhance the predictive traits of the tool and provide more discrete guidance to health systems on how to respond to specific patient groups demonstrating limited activation.

Appendix A. Exploratory Factor Analysis (EFA)

Item	Factor 1 Loading	Factor 2 Loading	Factor 3 Loading	Factor 4 Loading	Com	H2
Locus2: I can always control my health	0.76	0.01	−0.07	−0.01	1.02	0.54
Locus1: I can always avoid getting sick if I take care of myself	0.63	0.06	−0.03	−0.07	1.05	0.38
Know7: I always know the difference between good and bad health information	0.61	−0.03	0.03	0.09	1.05	0.42
Action2: I always make healthy changes to the way I live my life	0.54	0.26	0.05	0.03	1.48	0.53
Locus4*: I can always take care of myself	0.54	0.09	0.06	−0.06	1.11	0.35
Know3*: I always know what steps to take when I have a health problem	0.48	−0.21	0.12	0.28	2.21	0.44
Selfeff4*: It is very easy for me to make changes to my daily life to improve my health	0.47	0.14	0.13	0.17	1.61	0.5
Action8*: I always make the health changes I should, even if I do not feel well	0.47	0.1	0.16	0	1.33	0.37
Locus3: I can always improve my health	0.45	0.23	−0.05	−0.08	1.59	0.29
Know1*: I always know how to make myself feel better	0.45	−0.08	0.06	0.12	1.25	0.26
Action7: I always bring a list of questions to ask my doctor to my visits	0.19	0.12	0.08	0.07	2.53	0.12
Beliefs7: It is very important that I make changes to my life now, so I can have the best health when I’m older.	0.01	0.82	−0.01	0.02	1	0.68
Beliefs6: It is very important that I work hard to maintain and improve my health	0.02	0.76	0.03	0.06	1.02	0.63
Beliefs1: It is very important that I try to make positive changes to my health	0.01	0.73	0.02	0.07	1.02	0.57
Beliefs3: It is very important that I take steps to improve my health every day	0.15	0.67	0.06	−0.05	1.13	0.58
Beliefs4*: It is very important that I treat my health as my top priority	0.27	0.46	0.05	0.05	1.66	0.43
Beliefs5: It is very important that I fight for my health	0.07	0.45	0.23	0.05	1.59	0.4
Selfeff1*: It is very easy for me to follow my doctor’s instructions	0.11	0.01	0.68	0.02	1.06	0.56
Know4: I always know how to follow my doctor’s instructions	−0.05	−0.05	0.61	0.19	1.23	0.45
Beliefs2: It is very important that I follow my doctor’s instructions	−0.1	0.36	0.61	−0.13	1.82	0.55
Action4: I always fill all of my prescriptions on time	0.07	−0.04	0.54	−0.13	1.17	0.27
Action3*: I always attend all of my doctors appointments	−0.01	0.03	0.53	−0.03	1.01	0.27
Action1: I always follow all of my doctors advice	0.22	0.09	0.5	−0.07	1.47	0.41
Selfeff2*: It is very easy for me to understand my doctor’s instructions	0.06	0.05	0.47	0.24	1.56	0.43
Selfeff3: It is very easy for me to ask my doctor questions	−0.02	0.2	0.42	0.18	1.85	0.37
Know2: I always know when to call a doctor	0.23	−0.19	0.39	0.19	2.66	0.35
Know5: I always know which doctor I should call based on the problem I am having	0.12	−0.17	0.38	0.37	2.57	0.42
Action5: I always update my doctor about any major health changes between my visits	0	0.18	0.31	0.12	1.99	0.22
Selfeff9: It is very easy for me to use healthcare	0	0.11	0.26	0.2	2.29	0.18
Selfeff8: It is very easy for me to find health information	−0.05	0.11	−0.05	0.7	1.06	0.47
Know6*: I always know where to look for information before making decisions about my health	0.1	−0.01	0.09	0.69	1.08	0.61
Selfeff7: It is very easy for me to use technology to take care of my health	−0.02	0.04	0.02	0.56	1.01	0.32
Action6: I always look for answers to my health questions from places like the internet, pamphlets or magazines	0	0.22	−0.19	0.46	1.83	0.23
Selfeff6: It is very easy for me to disagree with my doctor or nurse	0.16	0.07	0.04	0.34	1.57	0.23
Selfeff5: It is very easy for me to get a second opinion if I need it	0.15	0.12	0.19	0.29	2.64	0.3