Development and validation of the positive affect and well-being scale for the neurology quality of life (Neuro-QOL) measurement system

Abstract

Purpose

To develop and validate an item-response theory-based patient-reported outcomes assessment tool of positive affect and well-being (PAW). This is part of a larger NINDS-funded study to develop a health-related quality of life measurement system across major neurological disorders, called Neuro-QOL.

Methods

Informed by a literature review and qualitative input from clinicians and patients, item pools were created to assess PAW concepts. Items were administered to a general population sample (N = 513) and a group of individuals with a variety of neurologic conditions (N = 581) for calibration and validation purposes, respectively.

Results

A 23-item calibrated bank and a 9-item short form of PAW was developed, reflecting components of positive affect, life satisfaction, or an overall sense of purpose and meaning. The Neuro-QOL PAW measure demonstrated sufficient unidimensionality and displayed good internal consistency, test–retest reliability, model fit, convergent and discriminant validity, and responsiveness.

Conclusion

The Neuro-QOL PAW measure was designed to aid clinicians and researchers to better evaluate and understand the potential role of positive health processes for individuals with chronic neurological conditions. Further psychometric testing within and between neurological conditions, as well as testing in non-neurologic chronic diseases, will help evaluate the generalizability of this new tool.

Introduction

Positive affect and well-being (PAW) have been largely under-studied concepts in the context of chronic illness when compared with their “negative” counterparts, such as depression, anxiety, or distress [1]. This has significantly limited our ability to fully characterize the impact of certain diseases and their treatment effects in relation to one’s level of adaptation, impairment, and disability. This is especially germane as advances in medicine result in more people surviving and even thriving despite significant illnesses [2]. Models of adaptation to acute and chronic illness have expanded beyond an exclusive focus on disease and distress to include positive well-being and healthy functioning. Much of this work has been catalyzed by the science of positive psychology with an emphasis on constructs such as hope, optimism, and positive affect [3], but it also reflects a more global socio-political zeitgeist as nations examine indicators of well-being and factors predictive of happy, healthy, successful lives (e.g., the United Kingdom, the Gallup World Poll, the Center for Disease Control, the OECD, and the Commission on the Measurement of Economic Performance and Social Progress) [4-8]. Collectively, these initiatives underscore the burgeoning interest in well-being and health.

Components of psychological well-being have been associated with better health outcomes in a number of studies. For example, positive emotions have been associated with longevity [9, 10], better cardiovascular health [11, 12], lower prevalence of diabetes [12], fewer respiratory tract infections [12], lower medical morbidity in general [10], and decreased symptoms and pain [10]. Cross-sectional survey data show that people who report greater happiness tend to function better in life than less happy people, are typically more productive, more socially engaged, and tend to have higher incomes [13-15]. In addition, having a purpose in life may serve as a protective factor in several health outcomes, including decreased mortality from cardiovascular disease [16], increased quality of life among rheumatoid arthritis patients [17], and prevention of depressive symptoms [18].

Self-report measurement of positive emotions and psychological well-being has historically employed adjectival descriptors of positive mood states or a finite set of questions asking about sense of well-being, life appreciation, or meaning. Advances in measurement, including item response theory (IRT), provide the opportunity to assemble and organize an inventory of questions relating to these important adaptive processes. Through the development of item banks (i.e., a set of carefully selected items that provide an operational definition of a trait or construct), IRT methods make it possible to estimate a total score for a given construct using any subset of items in an item bank. This in turn enables the development of robust short forms. From the perspective of measuring PAW, this offers greater breadth of content coverage, improved precision and flexibility, ability to individualize assessment, and reduced respondent burden.

Research in positive psychological processes has been conducted in broad areas of behavioral medicine and health psychology [2, 19], but few studies have been conducted with neurology patient samples [20]. Thus, there is a need for more studies that incorporate health-related quality of life (HRQoL) and well-being outcome measures, as well as greater instrument refinement [21]. As part of an NINDS-funded study (Neuro-QOL) to develop a patient-reported outcomes measurement system for use with a variety of neurological diseases [22], several important domains of HRQoL were identified and subsequent item banks were developed and validated in independent samples (see [23-26]). Of these domains, PAW was included as a high priority item bank.

The PAW domain and item content were identified through a larger multi-step process to identify all Neuro-QOL domains, which included (1) a comprehensive literature review and keyword search, (2) two waves of expert interviews with neurology professionals (n = 44 and n = 63, respectively) [27], and (3) patient and caregiver focus groups (N = 11 groups) [28]. In our Medline literature review of 24 neurological conditions (from 1996 to 2006), the most common emotional health-related sub-concepts that emerged were general emotional distress, depression, anxiety, and positive responses to chronic illness, the latter of which we subsequently have called PAW. We have defined PAW as “aspects of a person’s life that relate to a sense of well-being, life satisfaction, or an overall sense of purpose and meaning.”

Subsequent individual interviews were used to solicit opinions from neurology experts about the most important HRQoL concepts to measure in people with neurological conditions. PAW was spontaneously mentioned in the majority of these interviews. Next, during 11 focus groups (eight with patients and three with caregivers) (see Perez et al. [28]), all of the previous PAW concepts were mentioned and discussed. Finally, an Ovid key word search was used (1996–2007) to estimate the frequency with which certain concepts and sub-concepts have been studied in neurology research. Data from these different approaches were examined systematically [29-31], and independent raters summarized and aggregated individual codes into higher order themes, followed by numerous discussions with team investigators and outside experts to decide upon the final group of concepts. On the basis of these efforts, we created a conceptual model to illustrate the most important concepts of emotional health for people with major neurological conditions (Fig. 1). The purpose of this manuscript is to describe the (1) item calibration and (2) clinical validation phases of PAW development for the Neuro-QOL measurement system.

Fig. 1.

Emotional health conceptual model

Methods

Participants

Calibration sample

Working with an online health panel organization, Toluna (previously Greenfield Online), we conducted a large-scale calibration testing of our item pools with a general population sample. There were 2,113 English-speaking participants in this sample. Of those, 513 participants were randomly assigned to complete the emotional health form which included the PAW item bank. The average age of the 513 respondents was 48.7 years (SD = 15.6). 50 % were male; 47 % were married; 28 % had a high school education or less; 37 % self-reported as full-time employed; and 19 % self-reported as retired. 10 % self-reported as Hispanic, and 89 % self-reported as White. 73 % reported none of the neurological conditions listed. The most frequently reported condition was migraines (13 %). The most frequently reported comorbidities were as follows: hypertension (42 %) depression (31 %), anxiety (25 %), migraines (22 %), sleep disorder (20 %), and diabetes (17 %).

Validation sample

Participants were English-speaking adults (>18 years of age) and diagnosed with stroke, multiple sclerosis, amyotrophic lateral sclerosis (ALS), Parkinson’s disease, or epilepsy. Prospective participants with cognitive impairment that would prevent them from providing informed consent and/or completing test items with the assistance of an interviewer were deemed ineligible. Participants (N = 581) with the following conditions (stroke, n = 101; epilepsy, n = 119; multiple sclerosis, n = 161; Parkinson’s, n = 120; ALS, n = 80) were recruited from several clinical sites: Cleveland Clinic Foundation, Dartmouth-Hitchcock Medical Center, NorthShore University HealthSystem, Northwestern University Feinberg School of Medicine, Rehabilitation Institute of Chicago, University of California—Davis, University of Chicago, University of Puerto Rico, and University of Texas Health Science Center. Demographic characteristics of both the calibration and validation samples are provided in Table 1.

Table 1.

Socio-demographic and clinical characteristics of the calibration and validation samples

	Calibration sample: adult general population		Validation sample: adult clinical sample
	Full Sample	Emotional Health Subsample
N	2,113	513	581
Age average (SD)	52.7 (15.5)	48.7 (15.6)	55.2 (14.3)
Gender
Male	50 %	50 %	46 %
Female	50 %	50 %	54 %
Race
White	91 %	89 %	87 %
Black/African American	5.5 %	7.4 %	12 %
American Indian/Alaskan Native	1.5 %	1.4 %	2 %
Asian	3.3 %	3.5 %	2 %
Native Hawaiian/Pacific Islander	1.0 %	0.8 %	–
Occupation
Homemaker	12 %	11 %	8 %
Unemployed	8 %	11 %	9 %
Retired	31 %	19 %	30 %
Disability	10 %	11 %	34 %
Leave of absence	<1 %	<1 %	1 %
Full-time employed	31 %	37 %	21 %
Part-time employed	12 %	13 %	10 %
Full-time student	3 %	5 %	1 %
Marital status
Married	52 %	47 %	62 %
Widowed	7 %	6 %	5 %
Living with someone	7 %	7 %	5 %
Separated	3 %	3 %	2 %
Never married	17 %	21 %	16 %
Income
>$20,000	18 %	18 %	16 %
$20–$49,000	45 %	44 %	35 %
$50–$99,000	31 %	29 %	28 %
<$100,000	11 %	9 %	21 %
Education
Some high school or less	2 %	3 %	3 %
High school or equivalent	22 %	28 %	19 %
Some college	40 %	38 %	29 %
College degree	24 %	25 %	29 %
Advanced degree	11 %	9 %	20 %

Instruments

Calibration testing

Each participant was randomly assigned to complete items included in one of four static forms created from the larger Neuro-QOL item pools. One group completed the Ability to Participate in Social Roles and Activities and Satisfaction with Social Roles and Activities items, N = 549. A second group completed Lower Extremity (Mobility) items and Upper Extremity (Fine Motor, ADL) items, N = 518. A third group completed the Applied Cognition—General Concerns items, N = 533, and a fourth group completed the Depression, Anxiety, and PAW items, N = 513. Along with 30 depression and 28 anxiety items, we tested 27 PAW items.

Validation testing

In addition to completing a calibrated short-form of the PAW, participants completed external validation measures that included assessments administered across all diseases in areas of general function, cognitive function, HRQoL, and pain. Guided by recommendations of study investigators and experts in the field of neurology, these measures were selected because of their common utility in clinical practice and research with patients who have neurological conditions.

General function

Barthel Index

The Barthel Index [32] assesses the degree of independence a patient has in performing various self-care and mobility activities of daily living (ADLs). The weighted ordinal scale assesses 10 items of ADL in the following subgroups: personal care (including eating), dressing, personal hygiene and bathing, continence of urine and stool, mobility (including transfer from a bed and toilet), walking, and steps. We administered this instrument by standardized interview.

Instrumental Activities of Daily Living Scale

The Lawton Instrumental Activities of Daily Living Scale [33] is an interviewer-administered measure which includes 8 items: telephoning, shopping, food preparation, house-keeping, laundry, transportation, medications, and handling finances. Each task is graduated on a 3- or 4-level scale. The scale measures performance in contrast to ability.

Karnofsky Performance Status Scale (KPSS) [34]

The KPSS is a rating of functional impairment and offers a simple if coarse breakdown of activity level across patients regardless of diagnosis. KPSS criteria are based on descriptive categories from 0 to 100. Ratings were made by providers.

Cognitive function

Oral Digit Symbol Modalities [35]

This is a test of information processing speed, but is also thought to assess visual acuity and figural memory. Examinees pair specific numbers (0–9) with designated geometric figures that are matched in a coding key, attempting to complete as many matches as possible in 90 seconds. Written and oral forms are highly correlated (in normal adults >0.78). Because some participants may have greater motor deficits compared to others, we administered the oral version.

Symbol Search [36]

This is a test of mental speed and is a timed orthographic measure of visual attention, scanning, and motor speed. Participants must determine whether a target nonsense figure is present in a string of figures and mark a corresponding “yes” or “no” box presented at the end of each item.

Digit Symbol Coding [36]

This is a timed paper/pencil symbol substitution task of mental, visual, and motor speed. Using a key of paired numbers and symbols, participants must draw corresponding nonsense symbols below rows of numbers.

Health preference and pain

EQ-5D [37, 38]

This is a 5-item self-report measure of health status developed by the EuroQoL Group in order to provide a simple, generic measure of HRQoL for clinical and economic appraisal. Applicable to a wide range of health conditions and treatments, it provides a simple descriptive profile and a single preference value (utility) for current health. Domains include mobility, self-care, usual activities, pain/discomfort, and anxiety/depression.

PROMIS Global Health Scale [39]

Global health refers to evaluations of health in general rather than specific elements of health. The PROMIS global health items include global ratings of the five primary PROMIS domains (physical function, fatigue, pain, emotional distress, and social health) and general health perceptions that cut across domains. It can be scored into a Global Physical Health component and Global Mental Health component. Global health items have been found to be consistently predictive of important future events such as healthcare utilization and mortality.

Global HRQoL Question [40]

A single item from the Functional Assessment of Chronic Illness Therapy (FACIT), “I am content with the quality of my life right now,” was used as a global measure of quality of life.

Pain Question

A single (0–10) item that asks patients to rate, from “none” (0) to “the worst pain you can think of (“10”), the severity of their worst pain during the past week.

Responsiveness

Global Rating of Change

This measurement strategy assumes that a patient can judge whether over the course of a specified period, their self-reported health status has changed. Typically, such questions require patients to remember a prior health state and compare it to how they are currently feeling [41, 42]. In this study, participants were asked to rate how much their physical, emotional, cognitive, social/ family, and symptomatic well-being and their overall quality of life had changed over the past 6 months according to the following scale: +3 = “Very much better” to −3 = “Very much worse.” Such global transition ratings have the advantage of being easy to interpret, and they enhance the interpretability of HRQoL scores when found to be correlated with the target instrument. For instance, if the correlation between a global rating of change and the change score on a target instrument is over 0.5, the validity of the target instrument is supported. Global transition ratings have been widely used in HRQoL outcome assessments to augment the interpretation of HRQoL scores [43-45].

Procedures

Calibration

Creating and refining item pools

First, we created item pools through a process of reviewing existing measures and writing new items. We began this by creating an item library of over 750 emotional health items (Neuro-QOL Library contained over 3,000 items across all domains). Three investigators independently binned or grouped items to their respective conceptual areas (e.g., anxiety, depression, stigma, PAW). When binning decisions were inconsistent between two raters, the third reviewer reconciled differences. Next, emotional health co-chairs were chosen, each of whom engaged in a process of winnowing, or pairing down item pools into smaller sets. Outside content experts were also consulted throughout this process and helped in reducing redundancies and revising items to conform to a similar format. Decisions were made to keep items either in the simple present or in simple past tense and to limit response options to no more than two different types per item pool. All emotional health items used a 5-point Likert scale (never, rarely, sometimes, often, and always). With these item pools, investigators conducted 63 telephone-based cognitive interviews with neurological patients and proxies in English and Spanish. These interviews helped assess patients’ perceptions and understanding of the item’s meaning and allowed for a final opportunity to make modifications prior to testing. Institutional Review Board approval was obtained for these interviews and subsequent data collection activities from all participating sites. All participants provided informed consent prior to their inclusion in this study.

Creating calibrated item banks

Following a screening process to ensure eligibility (based on age, current English-speaking), Toluna sent emails to invite potential participants from their databases to enroll in the current study. Of the eligible participants, 513 were randomly assigned to complete the form containing the 27 PAW items. All participants who completed the survey were eligible for prize or incentive-based compensation through Toluna. The calibration sample was used primarily for calibrating item parameters and for establishing the midpoint of the score range for the PAW item bank, enabling comparison of scores to general population values.

Analyses for PAW item bank calibrations

The data analysis strategy included evaluation of unidimensionality and estimation of item parameters using IRT models. All of the psychometric analyses for the PAW item bank were conducted using data from the calibration sample (Note: also described as “Wave 1b” testing in other NeuroQOL publications). In addition to descriptive statistics and item-total correlations, factor analytic approaches were used to evaluate dimensionality of items of each domain. We first conducted an exploratory factor analysis (EFA), followed by confirmatory factor analysis (CFA). Since a one-factor model is rejectable based on the chi-square statistic when a large sample size is used, the focus was on practical fit indices such as the comparative fit index (criterion: CFI > 0.90), RMSEA (criterion: <0.10), factor loadings (criterion: >0.3), and average absolute residual correlations (criterion: <0.15). Samejima’s Graded Response Model (GRM) as implemented in MULTILOG was used for IRT-related parameter estimations for items that met the unidimensionality requirements. GRM is a polytomous IRT model which is specifically designed for use with items with ordered categories.

We evaluated parameter stability related to differences in gender, education, and age groups. An item displays differential item functioning (DIF) when probabilities of responding in different categories differ by population for the same underlying level of the attribute. Items can be evaluated for DIF by contrasting the IRT difficulty or location and slope parameters between two groups, which in this context relates to the major demographic groups represented in the Neuro-QOL sample: age (<50 vs 50+), gender (male vs female), and education (some college and lower vs college degree and higher). IRT-based ordinal logistic regression (OLR) approach as implemented in LORDIF [46] was used for evaluation of DIF. Items were flagged when the probability associated with the chi-square test was <0.01 and the effect size measures (McFadden’s pseudo R²) were >0.02 (a small but non-negligible effect).

Clinical validation

Testing of calibrated PAW short form

Once the PAW item bank was created, we assembled a calibrated PAW short form by selecting the most informative and discriminating items and weighing their clinical relevance and conceptual uniqueness against one another. Along with other Neuro-QOL short forms, the PAW short form was then tested in a large-scale clinical validation study at sites across the United States and Puerto Rico to validate them against existing criteria and to assess test–retest reliability (7 days) and responsiveness to change (180 days). Using an NIH-funded online assessment tool called Assessment Center^SM (see www.assessmentcenter.net) as the primary data capturing system, testing occurred at baseline, 7- to 10-day and 5- to 7-month follow-up.

Analyses for clinical validation testing

Means, standard deviations, and other distributional statistics were calculated for all scores at the baseline and follow-up assessments for all clinical groups. Internal consistency analyses were performed for each short form using Cronbach’s alpha coefficients. Intraclass correlation coefficients and corresponding 95 % confidence intervals were calculated to assess the test–retest reliability of the PAW-SF using the baseline and 7-day assessments. Concurrent validity was assessed at baseline by Spearman’s rho correlations between the PAW-SF and cross-disease measures.

To demonstrate the sensitivity of the PAW-SF for detection of change, we evaluated general linear models using each patient’s change score. We conducted responsiveness analyses on the PAW-SF using several criteria for change. One criterion used across all adult conditions was the Karnofsky Performance Status, and another was the self-reported Global Rating of Change (GRC) described above. Beginning with the 7-level GRC (range: +3 = very much better; 0 = about the same; −3 = very much worse), we collapsed the three “better” categories into one, and the three “worse” categories into one, leaving three categories (“better;” “about the same;” “worse”). These three categories were compared using one-way analysis of variance followed by least significant difference testing of adjacent groups when the overall F statistic was significant.

Results

Psychometric analysis

The PAW item pool contained 27 items. All but 2 of these items had more than 70 % of the sample selecting the top 3 categories (sometimes, often, and always). Preliminary analysis was conducted to examine internal consistency and the existence or impact of missing data, sparse response option data, and violations of monotonicity. For the 27 items, Cronbach’s alpha was 0.984, and item/total correlations ranged from 0.60 to 0.91. No cases were deleted due to excessive missing data since all cases had missing responses for 5 or fewer items. No items had sparse data (fewer than 5 responses) in the lowest (never) or highest (always) categories. One item, “I felt happy about the future,” had a category inversion (average raw score for persons selecting category 2 (rarely) was lower than the average for person selecting category 1 (never)).

Next, we examined whether the items could be fit to a unidimensional model. On the EFA based on polychoric correlations, eigenvalues for the first and second factors, respectively, were 20.89 and 0.88. The eigenvalue ratio (first to second) was greater than 23, suggesting a substantial, dominant factor. We conducted a parallel analysis based on polychoric correlations [47]. Comparing the observed eigenvalues (without communality estimates) of the correlation matrix with those from random data, the parallel analysis suggested 1 factor, that is, only the first factor had a larger observed eigenvalue than the corresponding eigenvalue based on random data. The first three eigenvalues based on random data were 1.50, 1.43, and 1.36, respectively. CFA of the full set of 27 items yielded a CFI = 0.978 which suggested good fit; however, the RMSEA of 0.116 was unacceptable. All item loadings had R² values greater than 0.60, except “I felt happy about the future” (R² = 0.45). Two item pairs were identified as locally dependent with residual correlations >|0.15|: “I felt happy about the future” with “I was able to enjoy life” (r = 0.17), and “I had good control of my thoughts” with “I had good control of my emotions” (r = 0.18).

After initial calibration, 4 items were suppressed due to misfit and/or local item dependence (“I felt loved and wanted,” “I felt happy about the future,” “I was able to relax,” “I had good control of my emotions”). We re-ran a CFA on the reduced (23-item) bank, and the fit statistics improved (CFI = 0.99, RMSEA = 0.105). We also re-ran an EFA on the reduced set. The eigenvalues for the first and second factors, respectively, were 18.4 and 0.75. The eigenvalue ratio was 24.5 (higher than 23.7, based on the original 27-item set). The subsequent IRT parameter estimates revealed slopes ranging from 2.66 to 6.61 and thresholds ranging from −1.89 to 1.47 (Table 2). Information was good between thetas of −2.2 and 1.8 (see Fig. 2 for the scale information function). The shaded area represents the score range where a conventional reliability of approximately 0.95 can be attained. The S-X² model fit statistics were examined using the IRTFIT macro program. All but one of the 23 items had adequate or better model fit statistics (p > 0.05), and the marginal reliability was 0.98. Following an examination of item calibration statistics (e.g., slope, theta, item characteristic curves) for the PAW bank, we identified the most informative and discriminating items and balanced these psychometric considerations with clinical content review to ensure representativeness of items assessing positive affect, life satisfaction, and meaning and purpose. As a result, 9 items met these criteria and were selected from the PAW bank to be used in the calibrated short form.

Table 2.

Item response theory model fit parameters

Item stem	Slope	Threshold 1	Threshold 2	Threshold 3	Threshold 4
I was able to enjoy life	2.85836	−1.6382	−0.84296	0.13591	1.23909
I felt a sense of purpose in my life	3.69836	−1.3651	−0.6816	0.1953	1.03512
I could laugh and see the humor in situations	2.73128	−1.86336	−1.26057	−0.1628	0.79273
I was able to be at ease and feel relaxed	3.03956	−1.6355	−0.84738	0.02593	1.27938
I looked forward with enjoyment to upcoming events	3.43471	−1.54605	−0.90628	0.10454	1.03551
Many areas of my life were interesting to mea	4.01345	−1.46801	−0.66877	0.17609	1.07496
I felt emotionally stable	2.65589	−1.63191	−1.04796	−0.18002	0.77892
I felt lovable	3.04784	−1.6718	−0.82091	0.09937	0.98844
I felt confident	3.43655	−1.54569	−0.81654	0.01393	0.96405
I felt hopefula	4.96074	−1.64871	−0.8302	0.12341	0.87771
I had a good life	5.20641	−1.50093	−0.88238	0.01107	0.69634
I had a sense of well-beinga	6.60979	−1.40631	−0.7089	0.07246	0.81527
My life was satisfyinga	5.82867	−1.37871	−0.69551	0.17156	0.89189
I had a sense of balance in my lifea	4.91912	−1.39207	−0.6029	0.20218	0.96279
My life had meaninga	5.60018	−1.38904	−0.845	0.00082	0.68678
My life was peaceful	3.19238	−1.64183	−0.79829	0.07381	1.16794
My life was worth livinga	4.15583	−1.88684	−1.05718	−0.28998	0.30601
My life had purposea	5.09529	−1.52156	−0.90209	−0.12017	0.53111
I was living life to the fullest	3.6484	−1.12527	−0.44023	0.3556	1.13119
I felt cheerfula	4.58501	−1.65267	−0.8785	0.09314	1.11936
In most ways my life was close to my ideal	3.63087	−0.84065	−0.26787	0.47792	1.46623
I had good control of my thoughts	2.82777	−1.87484	−1.03534	−0.10592	0.75892
Even when things were going badly, I still had hope	3.19493	−1.88618	−1.08212	−0.09792	0.74487

^aA short-form item

Fig. 2.

Scale information function (23 items)

DIF was examined for age, gender, and education. Five items were flagged for negligible age DIF (“I was able to be at ease and feel relaxed,” “I felt emotionally stable,” “I felt lovable,” “My life was peaceful,” “I was living life to the fullest”), and three items were flagged for negligible gender DIF (“I felt emotionally stable,” “I felt confident,” “My life was peaceful”). No items were flagged for negligible education DIF, and no items were flagged for nonnegligible DIF for age, gender, or education.

Clinical validation testing

A succeeding round of data collection was used to examine the reliability and validity of the PAW-SF. Table 3 shows that the internal consistency was invariably very high and 1-week test–retest reliability of the short forms was good for all but the ALS sample, with Cronbach’s alphas ranging from 0.94 to 0.95 and test–retest intraclass correlation coefficients ranging from 0.59 to 0.86. Table 4 provides correlations between Neuro-QOL PAW and external validity criteria. The strongest effect sizes were found for correlations between the PAW-SF and HRQoL, including global HRQoL (range = 0.52–0.81), PROMIS Global Physical (range = 0.31–0.61) and PROMIS Global Mental (range = 0.66–0.81). Although PAW-SF scores were largely uncorrelated with indices of cognitive function, modest correlations were obtained between PAW-SF scores and indices of general function, Karnofsky Performance Scale (range = −0.05 to 0.38), and between PAW-SF scores and Pain (range = −0.20 to −0.40).

Table 3.

Descriptive and reliability statistics for Neuro-QOL short-form T-scores

Neuro-QOL PAW short form	N	M	SD	Internal consistency	Test–retest
Stroke	100	54.92	8.02	0.94	0.83
Amyotrophic lateral sclerosis	76	53.90	7.70	0.94	0.59
Multiple sclerosis	161	53.61	7.72	0.95	0.76
Parkinson’s disease	120	54.40	7.53	0.94	0.86
Epilepsy	118	53.80	8.20	0.95	0.81

Coefficient alpha was our index of internal consistency reliability, and an intraclass correlation coefficient was our index of test–retest reliability

Table 4.

Correlations for the Neuro-QOL positive affect and well-being short-form (PAW-SF) T-scores with cross-disease measures

	Barthel Index	Karnofsky Performance Scale	Lawton IADL Scale	Symbol Digit Modalities # Correct	Symbol Search Raw Score	Digit Symbol Coding # Correct	PROMIS Global Physical	PROMIS Global Mental	Pain Scale 0−10	EQ-5D Index Score	Global HRQoL (0−4)
Multiple  Sclerosis	0.22**	0.28***	0.27***	0.01	0.05	0.12	0.61***	0.81***	−0.40***	0.48***	0.81***
Parkinson’s  Disease	0.24**	0.35***	0.17	0.16	0.20*	0.13	0.45***	0.74***	−0.20*	0.41***	0.64***
Amyotrophic	−0.15	−0.05	0.05	0.08	−0.03	−0.14	0.31**	0.68***	−0.24*	0.13	0.56***
Lateral
Sclerosis
Stroke	0.36***	0.38***	0.24*	0.28**	0.22*	0.14	0.46***	0.66***	−0.26*	0.38***	0.52***
Epilepsy	0.20*	0.30**	0.17	−0.10	−0.05	0.00	0.47***	0.73***	−0.38***	0.49***	0.59***
TOTAL	0.18***	0.25***	0.15***	0.03	0.05	0.04	0.48***	0.73***	−0.32***	0.40***	0.65***

^*p < 0.05;

^**p < 0.01;

^***p < 0.001

Responsiveness

A parallel set of one-way ANOVAs was conducted using responsiveness to change categories (better, same, worse) as the grouping variable and PAW-SF change scores over the past 6 months as the dependent variable. For these analyses, disease groups were combined because responsiveness to change categories was inadequately represented for all diseases with some cells containing sparse and unrepresented cases. Patients who reported changes in physical (F(2,483) = 19.51, p < 0.001), emotional (F(2, 483) = 25.84, p < 0.001), cognitive (F(2, 481) = 11.14, p < 0.001), social/family (F(2, 482) = 15.66, p < 0.001), and symptomatic well-being (F(2, 483) = 8.32, p < 0.001), and their overall quality of life (F(2, 482) = 23.18, p < 0.001) also reported corresponding changes in PAW (see Fig. 3). Least significant difference post hoc t test comparisons of the three groups found statistically significant differences in PAW (p < 0.001) between patients who reported getting better compared to those who reported staying the same or worsening across all global responsiveness categories except for physical well-being and symptomatic well-being. For those categories, the comparison between patients who reported getting better and patients who reported staying the same was not statistically significant, with p = 0.085 and p = 0.180 for physical well-being and symptomatic well-being, respectively.

Fig. 3.

ANOVA for PAW responsiveness to change

Discussion

Our rigorous mixed-methods approach to measurement development resulted in a 23-item calibrated bank and a 9-item short form of positive affect and well-being (PAW) for neurological conditions. The PAW demonstrated sufficient unidimensionality and displayed good internal consistency, test–retest reliability, IRT model fit, convergent and discriminant validity, and responsiveness. In addition, DIF analyses provided preliminary evidence of item equivalence across demographic variables of age, gender, and education. Further psychometric testing and comparisons across non-neurological conditions will help evaluate the generalizability of the PAW to other acute and chronic health conditions.

Our approach emphasized the importance of early qualitative input from neurology professionals, patients, and caregivers. Interview and focus group participants spontaneously reported positive responses to chronic illness, including themes of spirituality, meaning, mastery, and control. This input was supplemented with a comprehensive literature review and keyword search to help guide initial item selection. Rigorous psychometric analyses reduced the initial pool of items to a unidimensional bank of 23 locally independent PAW items.

Situated within an empirical and theoretical context, psychological well-being has been conceptualized as having hedonic and eudaimonic components [48]. Hedonic aspects of well-being are often more experiential in nature and emphasize pleasure (e.g., positive affect), whereas eudaimonic aspects of well-being are more evaluative in nature and emphasize human flourishing (e.g., meaning, life satisfaction). Similarly, Diener [49] proposed a construct of subjective well-being which is comprised of related but distinct components of high positive affect, low negative affect, and high life satisfaction. Several models of positive psychological functioning also include experiential and evaluative dimensions of well-being [50-53], which are key dimensions when examining links between well-being and improved health. Although we developed the PAW to encompass multiple aspects of psychological well-being with items assessing both positive affective states (“I felt cheerful”) and cognitive evaluations of desirable states (“My life had purpose”), all items loaded on the same dimension, suggesting a single-factor model was appropriate for this measure. In other words, theoretically and conceptually distinct facets of well-being can be modeled psychometrically in a unidimensional fashion due in part to the presence of a strong underlying general well-being factor.

Convergent and discriminant validity supported the construct validity of the PAW-SF in a clinical sample of individuals with multiple sclerosis, Parkinson’s disease, Stroke, epilepsy, and amyotrophic lateral sclerosis. Convergent validity was displayed through positive correlations with global perceptions of mental and physical health, yielding strong and moderate to strong effect sizes, respectively. Discriminant validity was displayed through negative correlations with self-reports of pain, yielding moderate effect sizes, and through largely non-significant correlations with cognitive function measures.

The PAW item bank demonstrated very high internal consistency, largely due to the number of items, but also suggesting that the item content was similar and a short form of the PAW could be created without sacrificing reliability. Indeed, the 9-item PAW-SF maintained excellent internal consistency reliability, comparable to the full 23-item bank administration. Test–retest reliability was also good, except for ALS patients, some of whom may experience true change in their status, even during a short time frame. In terms of the stability of individual item parameters across age, gender, and education, very little DIF was observed, and when it was observed, it was negligible and thus probably inconsequential to the overall measurement of PAW.

Finally, there was initial evidence of responsiveness to change for the PAW-SF. Self-reported changes in physical, emotional, cognitive, social/family, symptomatic well-being, and overall health-related quality of life were reflected in PAW-SF scores between those who reported improvement, worsening, or staying the same over time. Given that responsiveness is a type of clinical validity characterized by the level of concordance between changes in clinical or therapeutic effects and changes in PRO scores over time [54], these data suggest that the PAW-SF is responsive to longitudinal score changes that are expected as a result of a treatment or therapeutic gains. In addition to our use of anchor-based methods of global patient ratings of change, future studies will also benefit from evaluating responsiveness using distribution-based indicators, such as effect size, standardized response mean, or standard error of measurement [54].

This study is not without limitations. The adult general population sample was predominantly White with racial minorities underrepresented in the calibration testing. It is plausible that response rates for and interpretation of items assessing PAW would vary as a function of race. Future research with racial minority patients who have acute and chronic health conditions can better inform the generalizability and applicability of this measure to other clinical populations. The general population sample provided responses that were negatively skewed with the majority of the sample endorsing the top three response categories to indicate higher levels of PAW. This pattern of responding is not unusual in general population samples for items assessing well-being [55], and there are no apparent ramifications for the stability of the IRT parameter estimates.

In summary, we developed the PAW item bank and short form to aid clinicians and researchers to better evaluate and understand the potential role of positive health processes for individuals with chronic neurological conditions. Our use of IRT in the measure development process permits briefer assessments, more efficient assessments, and assessment of more symptoms and domains of interest. Moreover, future applications of this work may include computerized adaptive testing (CAT). CAT exams are, on average, half as long as paper-and-pencil measures with equal or better precision [56-58]. Additional study of the impact of positive affect and related aspects of psychological well-being may yield important findings to enhance healthy adaptation to acute and chronic health conditions and may suggest potential avenues for interventions. The PAW item bank and SF provide psychometrically promising tools to enable and enhance this future work.