Development of Long and Short Forms of the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV

Abstract

Understanding resilience in relation to HIV-related outcomes may help address racial/ethnic disparities, however, significant gaps in its measurement preclude in-depth study. Thus, this research aims to develop and evaluate the psychometric properties of long and short forms of the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV. To develop the items, we conducted a mixed methods study (N = 48) and reviewed published resilience measures. We completed content validity index analyses to ensure the items reflected the resilience construct. Next, we conducted 20 cognitive interviews and a field survey (N = 400). The long and short forms demonstrated acceptable to excellent psychometric properties based on factorial validity, internal consistency and convergent validity and on measurement invariance (conducted for the short form only). These measures provide a comprehensive framework to examine resilience and HIV-related outcomes and can inform resilience-building interventions to reduce racial and ethnic health disparities.

INTRODUCTION

Although progress in addressing the HIV epidemic in the United States (US) has been made, including reductions in the number of new HIV cases, substantial disparities in HIV incidence, prevalence, morbidity and mortality persist (1, 2). Of particular concern, African American/Black (AA/B) adults and individuals living in the southern US experience pronounced and persistent disparities across the HIV care continuum (1, 2). Adverse social determinants of health at multiple system levels undermine AA/B people living with HIV’s (PLWH) access to care, with significant adverse downstream effects on health. Examples of these adversities include: (1) policy level factors such as inconsistent funding for HIV prevention and treatment services and lack of Medicaid expansion in many states where AA/B adults reside, (2) structural racism and other types of discrimination (e.g., gender and sexual orientation), and long distances to HIV services at the community/neighborhood level, (3) poorer quality of care and stigma at the organizational level, (4) experiences of patient mistreatment and racism at the interpersonal level, and (5) substance use and mental health challenges at the individual level, which are due in part to structural level factors (e.g., policy, neighborhood, and discrimination) (3–20).

Despite the considerable structural, interpersonal, and individual-level adversities that AA/B PLWH encounter in their daily lives, AA/B communities display strategies of resistance, such as resilience, which may help them overcome some of these adversities (21, 22). Resilience refers to “positive adaptation in the face of ongoing daily stressors and highly taxing, yet still common events” (23) whereby one draws upon resilience resources by leveraging their own capacity (i.e., assets) as well as familial and community resources (7, 24). In the context of PLWH, resilience comprises processes buffering the multilevel adversities to HIV management that allow for achievement of favorable HIV outcomes (e.g., clinic attendance, medication adherence and viral suppression) (25, 26). Some resilience resource examples include safety and access to services (neighborhood level), social support and social cohesion (interpersonal level), and optimism and self-efficacy (individual level) (27–30). Identifying the multilevel resilience resources associated with improved HIV management among AA/B PLWH and developing interventions to enhance them, could potentially overcome some of the intermediary determinants of racial and ethnic health disparities in HIV (31). Although resilience may be a viable approach, research and policy efforts should also be directed toward dismantling racist, structural systems that give rise to racial and ethnic inequalities. If the abovementioned research and policy efforts are not implemented as well, AA/B PLWH will be expected to draw upon resilience resources more than other groups, which could potentially tax their stores of resilience resources and ultimately lead to even poorer health outcomes (3, 32–34).

Fostering resilience may be a promising approach to HIV management, however, significant gaps in its measurement preclude in-depth study. A recent literature review described these gaps among adult PLWH for the outcomes of retention in HIV care, adherence to antiretroviral therapy (ART), and virological suppression (22). This review documented that the majority of studies relied on general measures of resilience that are not tailored for PLWH. The few studies that did tailor resilience resource measures did not include PLWH in the measure development. Also, most of the studies focused only on individual- and some interpersonal-level resilience resources. A resilience measure, The People Living with HIV Resilience Scale, tailored for PLWH has been published since the recent literature review (35). This measure however focuses primarily on individual-level resilience resources, perhaps because it is adapted from the well-established Connor-Davidson Resilience Scale (CD-RISC), which similarly measures resilience at an individual level (35). Moreover, this new measure examines resilience in response to living with HIV as opposed to contextualizing the many adversities that PLWH face and it is developed for use in West and East-Central African countries. Thus, additional work is needed to develop a measure of resilience for PLWH that is multilevel, broadens the scope of adversities and includes insights from PLWH, especially subgroups who are impacted disproportionately by HIV disease like AA/B PLWH in the southern US. Therefore, the objectives of this study are to: 1) summarize a previously completed mixed methods study and literature review that informed item content and draft of the multilevel resilience measure; 2) describe the cognitive interview and behavior coding assessments for the draft measure; and 3) report the psychometric properties of the self-report Multilevel Resilience Resource Measure for AA/B Adults living with HIV, henceforth referred to as the Multilevel Resilience Measure (MRM). In subsequent sections, we present results for the long and short forms of the MRM. We present results for both forms because there are situations where a short form version is preferred (e.g., reduced participant burden, time constraints).

METHODS

In Figure 1, we summarize the measure development steps used to inform the MRM. We adhered to the Checklist for Measure Development and Validation Manuscripts (36) by reviewing foundational papers on resilience (e.g., (37–39), completing a literature review of resilience measures and engagement in the HIV care continuum (22), conducting a mixed methods study to generate multilevel resilience items (40), and performing cognitive interviews and testing the MRM with AA/B PLWH. In the field test, we examined the psychometric properties (e.g., validity and reliability) of the measure.

Figure 1.

Steps to develop the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV

Note. Generate item content and develop draft of the measure activities are published elsewhere (see manuscript text for more details).

Participants and methods for each study that informed development and testing of the MRM

For all of the studies described below, we recruited patients from two HIV clinics in the southeastern US. The general eligibility criteria across the studies required that participants be: AA/B, at least 18 years of age and enrolled in either the University of Alabama at Birmingham (UAB) 1917 Clinic Cohort (41) or the University of North Carolina Center for AIDS Research HIV Clinical Cohort (UCHCC) (42, 43). To be eligible, patients also had to be able to speak, read, and understand English sufficiently to complete consent procedures and needed to have a scheduled clinic appointment on the date of study enrollment. Additional study specific eligibility criteria are presented in the subsequent paragraphs where relevant. Institutional Review Boards at the University of Alabama at Birmingham, University of North Carolina at Chapel Hill and Brown University approved this human subjects research. Clinical research staff at each of the clinics read and reviewed consent forms with potential participants and answered all questions; individuals who agreed to participate in the study provided their informed written consent.

Study 1 Previously Completed: Mixed Methods Study and Literature Review that informed the Item Content and Draft Measure (40). In this study, we adhered to the Checklist for Measure Development by conducting a mixed methods study to generate multilevel resilience items. We specifically recruited patients who may have resilience resources because they demonstrated progression through the stages of the HIV care continuum. We defined progression as attendance of ≥ 75% of scheduled clinic appointments and < two consecutive detectable HIV-1 viral load measurements for a given assay in the two years prior to the start of study enrollment. Across the clinic cohort sites, the assays that were used to measure HIV-1 RNA level of patients who enrolled in the mixed methods study had a lower limit of detection equal to 20 copies/mL or equal to 40 copies/mL. We included the criteria described above to align with the definition of resilience (e.g., positive adaptation to ongoing stressors that allows for favorable behavioral/health outcomes).

Participants also had to complete a patient reported outcomes survey assessment (i.e., a clinic survey of demographics, behavioral and health measures) within 6 months prior to enrollment. We used concept mapping methodology to generate the item pool for the MRM. Concept mapping incorporates both qualitative and quantitative methods and key stakeholder engagement to depict group-generated concepts and their relationships via two-dimensional maps and graphs (23–25). Forty-eight AA/B PLWH participated in one or more steps of the concept mapping; the median age of participants was 53 years and 56% of participants were male. At study enrollment, the median (IQR) CD4 cell count cells/mL was 769 (476) and almost 98% of participants had RNA copies/ml <40 and 2.1% had copies >40 and <200. We included a total of 116 statements in the analyses. Additional details about the concept mapping methodology, retrieval and review of published resilience measures, content validity index analysis and study participants’ characteristics and results have been described elsewhere (40).

Study 2 Current Study: Conducting Cognitive Interviews with Behavioral Coding. In this study, we adhered to the Checklist for Measure Development by conducting cognitive interviews with behavioral coding. Additional eligibility criteria required that participants complete the patient reported outcomes survey assessment within 6 months prior to enrollment. Clinical research staff approached 20 patients and all (n = 10 per clinic) participated in these interviews. The interviews included concurrent verbal probing (i.e., where participants provide feedback on each item, one at a time) and behavioral coding. We performed concurrent verbal probing to determine ease of understanding of the survey instructions, understanding of item wording and content and preferred event sampling-frame (i.e., one month, three months or six months) for life’s adversities. During the cognitive interview, clinical research staff also conducted behavior coding to detect frequency of potential problems with the items (e.g., request for clarification). Due to potential time burdens of administering the 55 items during a single cognitive interview, we divided the instrument into two surveys, the non-individual level resilience resource items and the individual-level resilience resource items. Participants received $25 for their participation.

Clinical research staff completed the interviews in two rounds. In the first round, five participants completed interviews for the non-individual level resilience resource items (n = 2 participants at clinic 1 and n = 3 at clinic 2) and five participants completed interviews about the individual level resilience resource items (n = 3 participants at clinic 1 and n = 2 at clinic 2). Given that there were few minor revisions needed after the first 10 interviews, we administered the same item set in the subsequent set of 10 cognitive interviews. In round two, a total of five new participants across both clinic sites completed the individual-level resilience resource items and a total of five new participants across both sites completed the non-individual level resilience resource items. Based on the results from all 20 interviews, we refined the survey items and tested them in the larger field survey (which is described next).

Study 3 Current Study: Performing Psychometric Testing. We adhered to the Checklist for Measure Development by examining the psychometric properties of the measure in a large field test of the survey. Potential participants had to meet the general eligibility criteria only. Clinical research staff approached 428 patients, of these, 400 enrolled and completed the survey. Participants received $25 for their participation.

Data Ascertained Via Medical Records or the Survey

Demographics

Clinical research staff reported participants’ demographic characteristics based on information obtained from participants’ medical records including participants’ self-reported race and ethnicity (i.e., Black or African American and non-Hispanic/Latino or Hispanic/Latino) and year of birth. We used two measures to determine participants’ gender identity and sexual orientation. (1) Clinical research staff reported participants’ self-reported gender based on information obtained from participants’ medical records. The gender categories included (male, female, transgender male to female, transgender female to male, or unknown). (2) In addition, participants self-reported their gender identity using the Lesbian, Gay, Bisexual and Transgender (LGBT) Identity Measure (44). For this measure, participants selected all identities that applied (i.e., straight, gay or lesbian, bisexual, or transgender, transsexual, male to female, transgender or transsexual, female to male, gender non-conforming). From these data, we derived gender identity (transgender male, transgender female, male, female or missing) and sexual orientation (heterosexual, gay, lesbian, bisexual or missing).

Participants also reported their housing status using the measure developed by Marshall et al. (45); which aligns with the housing status categorizations described in a recent systematic review (46). We created a four option categorical variable to reflect current housing status with the following categories: 1) Stably housed –living in an apartment or house that the respondent rents or owns, a parent or family member’s apartment or house, or a college dorm, 2) Unstably housed – living in someone else’s apartment or house short-term (like couch-surfing), a recovery or residential treatment center, a transitional housing program, a corrections halfway house, or a drop-in center or emergency shelter, 3) Homeless –living in a car, abandoned building, some other enclosed public place, or on the street or other outdoor place like a park, and 4) Other –responses that did not specify the other type of place where the participant was living or the response could not be mapped onto response options 1 - 3.

MRM

Participants responded to 55-items measuring resilience resources at the individual, interpersonal, organizational level items presented in random order and the neighborhood levels. The measure included one set of instructions for the 47 individual, interpersonal, and organizational resilience resource items and separate instructions for the 8 neighborhood-level items. In the cognitive interviews, we asked participants to define neighborhood. The most common response was “the area where I stay, or live in.” In the instructions, participants were asked to provide responses based on “the neighborhood where you currently live.”

In brief, we asked participants to think about their life’s challenges over the past three months and to rate how much they agreed that the statements helped them handle these life challenges so that they could stick with their HIV care. Participants rated the statements using a 5-point Likert scale (5 = strongly agree; 4 = agree; 3 = uncertain/not sure; 2 = disagree; and 1 = strongly disagree) or 99 = refused. The readability of the item-set and instructions was a 5.2 grade level with a reading ease of 72.6% (measured by Flesch-Kincaid).

Established measures used to evaluate convergent validity (see Table 1)

Table 1:

Measures Utilized in Field Study to Assess Convergent Validity of the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV (MRM) with Established Measures

Measure	Scales	Higher scores/tertiles represent	Internal consistency (Cronbach’s α)
			Prior research	Study sample
Connor–Davidson Resilience Scale 10 (CD-RISC 10)	Individual-level resilience (10-items); Total resilience score	Greater resilience	0.85	0.96
Eight-item modified Medical Outcomes Social Support Survey (8-item mMOS-SS)	Emotional social support (4-items) and instrumental social support (4-items); Total social support score	Greater social support	0.88 – 0.93	0.95
Perceived Support Structures in Community Measure	Access to services that are accepting/supportive of PLWH (10-items); Total community support score	Greater community support	0.96	0.95
Brief Resilient Coping Scale	Individual-level adaptive coping (4-items); Total coping score	Greater coping capacity	0.69	0.87
Neighborhood Disadvantage Index	Census-tract level unemployment, poverty and high school education data combined and categorized by tertile	Increased neighborhood-level risk	---	---
Neighborhood Murder Index	Census-tract level murder rate data categorized by tertile	Increased neighborhood-level risk	---	---
Neighborhood Assault Index	Census-tract level assault rate data categorized by tertile	Increased neighborhood-level risk	---	---

PLWH: People living with HIV

The neighborhood-level indices are objective measures of United States census tract level neighborhood socioeconomic status (Neighborhood Disadvantage Index) and Esri-derived murder and assault rate indexes.

We included the Connor-Davidson Resilience Scale 10 (CD-RISC 10) and the Brief Resilient Coping Scale (BRCS) which both measure resilience at the individual level (47, 48). The established measure at the interpersonal level was the Eight-Item Modified Medical Outcomes Social Support Survey (8-item mMOS-SS) (49). At the neighborhood level, we used the Perceived Support Structures in the Community Measure (PSSCM) which is a measure of resilience, and three objective neighborhood measures of risk, including a Neighborhood Disadvantage Index developed using US census-tract derived indicators and Esri [Geographic Information System (GIS) Software] derived Assault Rate Index and Murder Rate Index indicators categorized by tertile (i.e., high-risk, medium-risk, low-risk) (40, 50, 51).

Statistical Analyses

We computed descriptive statistics (these included as applicable means and standard deviations, medians and interquartile ranges, and frequencies and percentages) for participants’ demographics and MRM items. We also conducted non-parametric and parametric tests to determine if there were differences in demographic characteristics between a) study participants (n = 400) versus decliners (n = 28) and b) participants with complete data on each of the 55 MRM items (n = 329) versus those missing a response on any of the MRM items (n = 71). We used both IBM SPSS Statistics version 25 and SAS version 9.4 (SAS Institute, Inc. Cary, NC) for these analyses.

Psychometric testing analyses for field survey data: We randomly split participant data (N = 400) within each of the two participating clinic sites and re-combined the data to create two site-weighted sub-samples of 200 participants each. Two participants had missing data on all or most of the 55 resilience items and on the convergent validity scales so we excluded these two participants from further analyses. To develop the long form MRM, we used subsample 1 data (n = 199) for exploratory dimensional analyses (EDA) and analyzed these data using principal component analysis. We conducted these analyses within each of the resilience resource levels (i.e., individual, interpersonal, organizational, and neighborhood) using listwise-deleted sets of data from participants who responded to all items in the variable groups. Solutions for the number of underlying components within each level were guided by conceptual theory, implementation of the parallel analysis (PA) procedure, and the visual scree (VS) test (52–54).

We used subsample 2 (n = 199) data to examine the EDA solutions of items representing each resilience level within a structural equation modeling (SEM) confirmatory factor analysis (CFA) framework. We evaluated the CFA models based on the minimum fit function chi-square statistic, and with several alternative fit indices commonly used to help evaluate structural equation models, including the comparative fit index, the Tucker-Lewis Index, and the standardized root mean square residual (55–57). Reduction of the long form item sets to form short scales was conducted within a CFA framework. To create the short form scales, we examined modification index values reported by the Mplus software and iteratively reduced the long form item sets, while preserving construct breadth for each resilience domain. We examined measurement invariance of the short forms across gender (i.e., male and female) at the configural, metric and scalar levels of invariance (58). There was a small group (n = 11) of male-to-female transgender participants and their responses were combined with the female gender group (n = 141) for these analyses. Measurement invariance was evaluated across invariance models using the Chi-square difference test and several alternative indices of invariance model change including the change in the comparative fit index, the change in the root mean square error of approximation and the change in gamma hat (59, 60). We conducted the CFAs and measurement invariance analyses using the Mplus version 8.4 program for structural equation modeling (61).

We computed internal consistency reliability and coefficient alpha for each long form MRM subscale after initial development within subsample 1 and also for the long and short form versions using the full available sample for each MRM subscale from the combined subsample 1 and 2 data. We conducted convergent validity analyses using responses for the full available sample for each of the new long and short form MRM subscales to examine their relationships with previously established measures of individual, interpersonal, and neighborhood-level resilience. As some MRM variables and scales were skewed negatively with participants tending to respond in a more positive direction, we used both the Pearson r and Spearman rho correlations to examine and compare these relationships. For all psychometric analyses, we examined data from only those participants with complete data on the specific resilience level (e.g., individual- and interpersonal-level).

RESULTS

Descriptive Characteristics of Study Participants (N = 400)

Among the study participants, twenty-five percent of participants were from site 1 and 75% were from site 2. The mean age of the sample was 48.1 (SD = 13.4), median age was 49 (Q₁ = 37, Q₃ = 58), and regarding gender, 62% (n = 248) male, 35% (n = 141) female and 3% (n = 11) transgender female. With respect to sexual orientation, 52% (n = 209) were heterosexual, 43% (n=172) were gay/lesbian/bisexual, and 5% (n = 19) were coded as missing. The majority of participants were stably housed (91%), 6% were unstably-housed, 1% was homeless, 1% responded other, and 1% refused to respond or were missing a response. Among the clinic cohort participant population overall, the majority are male (76% at site 1 and 67% at site 2), African American or Black (65% at site 1 and 67% at site 2) and have achieved virological suppression (81% at site 1 and 89% at site 2).

Test for Differences in Characteristics between Study Enrollees vs. Decliners and Enrollees with Missing vs. No Missing Data

Enrollees and decliners did not differ on gender [(p = 0.13) or age (p=0.26), Fisher’s Exact test, no test statistic available]. However, compared to site 2, a larger proportion of site 1 invitees declined to enroll [(p < 0.001), Fisher’s Exact test, no test statistic available]. Enrolled individuals with any missing versus no missing data did not differ on gender [(p = 0.50), Fisher’s Exact test, no test statistic available] or age [(p=0.26), U = 4888] for enrollees with any item missing versus enrollees with no missing items. Also, there were no differences between clinic sites 1 and 2 for participants with any missing versus no missing data [(p = 0.18), Fisher’s Exact test, no test statistic available]. See Supplementary Tables 1 and 2 for a more detailed presentation of these comparisons.

Initial Exploratory Dimensional Analyses (EDA) with Subsample 1 (n = 199) to Develop the Long Form MRM

An initial EDA of the individual level resilience items (29 items) was conducted on data from 187 participants. The parallel analysis and visual scree heuristic methods suggested that up to 3 components might best represent the data. Upon examination of the 3-component varimax-rotated solution, we conducted sequential analyses and removed the lowest loading items (<0.40 on all components) and the complex loading items (>0.40 on two or three components). This process resulted in a final solution with one component. Further EDA analyses reduced the single component to 16 items with loading greater than 0.50 as noted in Table 2a. Coefficient alpha (internal consistency reliability) for the resulting 16-item scale was 0.91.

Table 2.

Initial Exploratory Dimensional Analyses (EDA) with Subsample 1 (n = 199) to Develop the Long Form Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV (MRM)

2a. MRM–Individual subscale after EDA (n=187)		Item # in Survey	Item Loadings
I keep a positive attitude.		2	0.72
I move forward with handling my daily tasks.		9	0.72
I am determined to live a long healthy life.		10	0.67
I develop solutions that work for my problems.		21	0.63
I get involved in my health care.		22	0.66
I make taking my medications part of my routine.		23	0.67
I want to look healthy.		25	0.69
I am determined to fulfill my goals, hopes, and dreams.		34	0.76
I love myself.		35	0.74
I want to see my loved ones achieve their goals.		36	0.69
I tend to bounce back quickly after hard times.		37	0.64
I take time for myself.		38	0.58
I have confidence in myself.		42	0.75
I keep a positive attitude about living with HIV.		43	0.63
I make attending my medical appointments part of my routine.		44	0.61
I accept that I am living with HIV.		45	0.53
Coefficient Alpha			0.91
2b. MRM–Interpersonal subscale after EDA (n=189)
I have someone who I can ask for help.		5	0.75
I have someone who helps me keep a positive attitude about living with HIV.		13	0.70
I have someone who listens to me when I have life challenges.		18	0.81
I have someone who keeps me in their prayers.		19	0.79
I have someone who encourages me to stay alive.		29	0.82
I have a strong support system, for example, family, friends, spiritual leaders, or peer mentors.		30	0.85
I have someone who motivates me to keep going.		32	0.78
I have someone who helps with my HIV care.		33	0.66
I have someone who tells me how good I look.		40	0.67
Coefficient Alpha			0.89
2c. MRM–Organizational subscale after EDA (n=193)
I have good communication with my doctor.		6	0.82
The doctor gives me positive feedback when I make healthy lifestyle changes.		14	0.82
The healthcare staff call me with reminders for appointments or medications.		15	0.67
The clinic, hospital, provider, or pharmacy makes sure that I get my medications on time.		24	0.53
The clinic or hospital employees treat me with kindness and compassion.		27	0.82
The case manager or social worker helps me with medical and other needs.		31	0.58
Coefficient Alpha			0.78
2d. MRM–Neighborhood subscales after EDA (n=192)	Item # in Survey	Services Item Loadings	Order Item Loadings
My neighborhood has healthcare services.	48	0.78	0.19
My neighborhood has religious services.	49	0.87	0.11
My neighborhood has police services.	50	0.79	0.05
My neighborhood does not have a lot of crime.	52	0.09	0.88
My neighborhood does not have drugs or alcohol being used in public places.	53	0.03	0.87
My neighborhood is peaceful.	54	0.17	0.89
My neighborhood is clean.	55	0.20	0.83
Coefficient Alpha		0.75	0.89

Next, we conducted an initial EDA of the interpersonal level items (11 items) using data from 189 participants. Both the parallel analysis and visual scree heuristic methods suggested a single component represented the data best. Examination of the single component supported this solution, and after removing two low loading items (<0.50), a single component of 9 well-defined items was found with all loadings greater than 0.60 (see Table 2b). Coefficient alpha for the 9-item scale was 0.89.

The next EDA examined organizational level items (8 items) using data from 193 participants. Both the parallel analysis and visual scree heuristic methods suggested a single component would be the best solution. After removing 2 low loading items (<0.50), a single component with 6 well-defined items was found with all loadings greater than 0.50 (see Table 2c). Coefficient alpha for the 6-item scale was 0.78.

The last EDA was conducted on data from 192 participants on the 8 items that measured resilience at the neighborhood level. Both the parallel analysis and visual scree heuristic methods suggested that a 2-component solution would best represent the data. Examination of the 2-component solution supported this recommendation. After removal of 1 complex item (>0.40 on more than one component), 2 well-defined scales remained with all item loadings greater than 0.70 (see Table 2d). Coefficient alpha for the 3-item Neighborhood Services scale was 0.75 and was 0.89 for the Neighborhood Order scale. (see Table 3)

Table 3.

Confirmatory Factor Analyses (CFA) of Exploratory Dimensional Analysis Results with Subsample 2 (n = 199) and Full Sample (N = 398) to Develop the Short Form Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV (MRM)

3a. MRM–Individual subscale (short form) after CFA	Item # in Survey	Item Loadings (N=192)	Item Loadings (N=382)
I am determined to live a long healthy life.	10	0.67	0.70
I develop solutions that work for my problems.	21	0.69	0.62
I am determined to fulfill my goals, hopes, and dreams.	34	0.80	0.81
I love myself.	35	0.85	0.73
I want to see my loved ones achieve their goals.	36	0.80	0.76
I accept that I am living with HIV.	45	0.71	0.59
Coefficient Alpha		0.88	0.83
3b. MRM–Interpersonal subscale (short form) after CFA	Item # in Survey	Item Loadings (N=191)	Item Loadings (N=383)
I have someone who listens to me when I have life challenges.	18	0.66	0.70
I have a strong support system, for example, family, friends, spiritual leaders, or peer mentors.	30	0.83	0.84
I have someone who motivates me to keep going.	32	0.79	0.77
I have someone who helps with my HIV care.	33	0.54	0.55
I have someone who tells me how good I look.	40	0.67	0.65
Coefficient Alpha		0.81	0.81
3c. MRM–Organizational subscale (short form) after CFA	Item # in Survey	Item Loadings (N=193)	Item Loadings (N=392)
I have good communication with my doctor.	6	0.61	0.73
The doctor gives me positive feedback when I make healthy lifestyle changes.	14	0.86	0.85
The healthcare staff call me with reminders for appointments or medications.	15	0.54	0.56
The clinic or hospital employees treat me with kindness and compassion.	27	0.69	0.72
Coefficient Alpha		0.75	0.78
3d. MRM–Neighborhood subscales after CFA	Item # in Survey	Services Item Loadings (N=194)	Order Item Loadings (N=194)
My neighborhood has healthcare services.	48	0.62
My neighborhood has religious services.	49	0.83
My neighborhood has police services.	50	0.60
My neighborhood does not have a lot of crime.	52		0.66
My neighborhood does not have drugs or alcohol being used in public places.	53		0.71
My neighborhood is peaceful.	54		0.93
My neighborhood is clean.	55		0.85
Disattenuated Correlation between Services & Order		0.16
Coefficient Alpha		0.71	0.88
3e. MRM–Neighborhood subscales after CFA on Full Available Sample	Item # in Survey	Services Item Loadings (N=384)	Order Item Loadings (N=384)
My neighborhood has healthcare services.	48	0.65
My neighborhood has religious services.	49	0.85
My neighborhood has police services.	50	0.61
My neighborhood does not have a lot of crime.	52		0.70
My neighborhood does not have drugs or alcohol being used in public places.	53		0.70
My neighborhood is peaceful.	54		0.94
My neighborhood is clean.	55		0.84
Disattenuated Correlation between Services & Order		0.24
Coefficient Alpha		0.73	0.89

Confirmatory Factor Analyses (CFA) of EDA Results with Subsample 2 (n = 199) and Full Sample (N = 398) to Develop the Short Form MRM

We entered the 16 individual-level resilience items into a CFA using data from 192 participants. The initial model item loadings were generally satisfactory (i.e., 15 of the 16 items > 0.50), but the overall model fit was poor for the minimum fit function chi-square statistic (p <.0001) and 2 alternative fit indices [Comparative Fit Index (CFI) = .837; Tucker-Lewis Index (TLI) = .812]. We conducted a sequential series of re-analyses first removing the lowest loading item and then using modification index suggestions to remove additional items. We reached a short form 6-item set to measure individual-level resilience that displayed excellent model fit, χ²(9) = 12.00 (p = .213), CFI =.995, TLI = .991, and standardized root mean square residual (SRMR) = .020. We also fit the model using the full available sample (N = 382), and again there was excellent model fit, χ²(9) = 13.44 (p = .144), CFI = .995, TLI = .991, and SRMR = .019. Table 4a presents the individual item loadings and coefficient alphas for the CFA on subsample 2 and the full available sample.

Table 4.

Examination of Model Fit for the Configural, Metric and Scalar Models of Measurement Invariance for Males (n=238) and Females (n=144) on the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV

4a. Individual-Level Subscale
Model	Chi-Square (df)	CFI	RMSEA (90% CI)	Gamma Hat
Configural	X2(18) = 25.28 (p=0.12)	0.992	0.046 (0.00, 0.085)	0.987
Metric	X2(23) = 27.77 (p=0.23)	0.994	0.033 (0.00, 0.071)	0.992
Scalar	X2(28) = 29.99 (p=0.36)	0.998	0.019 (0.00, 0.060)	0.997
Change from Configural to Metric Model	X2(5) = 2.49 (p=0.78)	+0.002	−0.013	+0.005
Change from Metric to Scalar Model	X2(5) = 2.22 (p=0.82)	+0.004	−0.014	+0.005
Table 4b. Interpersonal Level Subscale
Model	Chi-Square (df)	CFI	RMSEA (90% CI)	Gamma Hat
Configural	X2(10) = 5.58 (p=0.85)	1.000	0.000 (0.00, 0.014)	1.000
Metric	X2(14) = 7.15 (p=0.93)	1.000	0.000 (0.00, 0.021)	1.000
Scalar	X2(18) = 11.77 (p=0.86)	1.000	0.000 (0.00, 0.035)	1.000
Change from Configural to Metric Model	X2(4) = 1.57 (p=0.82)	0.00	0.00	0.00
Change from Metric to Scalar Model	X2(4) = 4.62 (p=0.33)	0.00	0.00	0.00
4c. Organizational Level Subscale
Model	Chi-Square (df)	CFI	RMSEA (90% CI)	Gamma Hat
Configural	X2(4) = 3.91 (p=0.42)	1.000	0.000 (0.00, 0.107)	1.000
Metric	X2(7) = 7.66 (p=0.36)	0.999	0.022 (0.00, 0.092)	0.998
Scalar	X2(10) = 8.98 (p=0.53)	1.000	0.000 (0.00, 0.072)	1.000
Change from Configural to Metric Model	X2(3) = 3.76 (p=0.29)	−0.001	0.022	−0.002
Change from Metric to Scalar Model	X2(3) = 1.31 (p=0.73)	+0.001	−0.022	+0.002
4d. Neighborhood Level Subscale
Model	Chi-Square (df)	CFI	RMSEA (90% CI)	Gamma Hat
Configural	X2(24) = 34.32 (p=0.08)	0.992	0.047 (0.00, 0.081)	0.985
Metric	X2(29) = 36.52 (p=0.16)	0.994	0.037 (0.00, 0.070)	0.989
Scalar	X2(34) = 38.90 (p=0.26)	0.996	0.027 (0.00, 0.061)	1.993
Change from Configural to Metric Model	X2(5) = 2.20 (p=0.82)	+0.002	−0.010	+0.004
Change from Metric to Scalar Model	X2(5) = 2.38 (p=0.79)	+0.002	−0.010	+0.004

Invariance Standards: No Significant difference for change using Chi Square Difference Test.

Maximum change value limits for CFI, RMSEA, and Gamma Hat based on Cheung & Rensvold, and/or Chen: CFI = −.010; RMSEA = +.015; Gamma Hat = −.001.

Next, the 9 interpersonal-level resilience items were entered into an initial CFA using data from 191 participants. The item loadings from the initial model were generally good, with 8 of the 9 items 0.57 or greater, but the overall model fit was poor for the minimum fit function chi-square statistic (p <.0001), and 2 alternative fit indices [Comparative Fit Index (CFI) = .810; Tucker-Lewis Index (TLI) = .746)]. We completed similar re-analysis steps as described for the individual-level items and a short form 5-item set was found to measure interpersonal resilience and displayed excellent model fit, χ²(5) = 10.19 (p = .070), CFI = .984, TLI = .969, and standardized root mean square residual (SRMR) = .027. We also fit the model using the full available sample data (N = 383) and again found excellent model fit, χ²(5) = 3.08 (p = .688), CFI = 1.000, TLI = 1.000, and SRMR = .010. See Table 4b for interpersonal item loadings and coefficient alpha for the CFA on subsample 2 and the full available sample.

Subsequently, the 6 organizational-level items were entered into an initial CFA using data from 193 participants. The item loadings from the initial model were generally good, with all 6 items having a value of 0.54 or greater, but the overall model fit was poor for the minimum fit function chi-square statistic (p = .0002) and 2 alternative fit indices [Comparative Fit Index (CFI) = .927; Tucker-Lewis Index (TLI) = .878)]. Using the same re-analysis protocol described for the individual-level items, a short form 4-item set was found to measure organizational-level resilience and displayed excellent model fit, χ²(2) = 4.31 (p = .116), CFI = .988, TLI = .965, and standardized root mean square residual (SRMR) = .024. We also fit the reduced model using the full available sample data (N = 392) and again there was excellent model fit, χ²(2) = 1.69 (p = .430), CFI = 1.000, TLI = 1.000, and SRMR = .009. See Table 4c for organizational item loadings and coefficient alpha for the CFA on subsample 2 and the full available sample.

Last, we entered the 7 neighborhood resilience items into an initial CFA using data from 194 participants. The estimated model was a correlated 2-factor model. The item loadings from the initial model were generally good, with all 3 items on the Neighborhood Services factor having a value of 0.60 or greater and all 4 items on the Neighborhood Order factor having a value of 0.66 or greater with a disattenuated correlation between the factors of 0.16. One adjustment was made to the Neighborhood Order factor based on a modification index value, namely the residuals between 2 items were allowed to correlate. No items were removed from these already short scales and the correlated 2-factor model with one correlated residual fit very well as judged by the minimum fit function chi-square statistic, χ²(12) = 18.70 (p = .096), the [Comparative Fit Index (CFI) = .989, the Tucker-Lewis Index (TLI) = .980], and standardized root mean square residual (SRMR) = .043. We also fit this same correlated 2-factor model to the full available sample data (N = 384) and again found excellent model fit, with χ²(12) = 19.69 (p = .074), CFI = .994, TLI = .989, and SRMR = .029. See Table 4d for the subsample 2 neighborhood resilience scale item loadings and coefficient alpha value and see Table 4e for the 2 neighborhood resilience scale item loadings and coefficient alpha value for the full available sample.

Measurement Invariance Analyses on the Short Form MRM

We conducted measurement invariance analyses across gender groups on each of the short form resilience measures. These analyses included examining 3 increasingly restrictive models, the configural (least restrictive), metric, and scalar (most restrictive) models (58). The configural model requires that any identified factors are associated with the same items across the groups that are tested for configural invariance. This form of invariance is described by the overall fit and pattern of loadings for each factor in the analysis. The second level of invariance, the metric model, postulates that all factor loading parameters are equal across groups on all factors in the analysis, and if supported indicates that the meaning of the latent factor is equivalent across groups. The third level of invariance we examined is the scalar model which examines that the groups share both the same factor loadings and the same intercept, a requirement for comparing the latent mean differences across groups. The results of the measurement invariance analyses across gender are reported in Tables 4a - 4d. In each case, the short form measures described in the CFA analyses were determined to be invariant across genders based on the more conservative chi-square test and chi-square difference test, and also 3 alternative measures of model fit and comparisons of model change (59, 60).

Convergent Validity Analyses

Table 5a and 5b contain the Pearson r and Spearman rho correlations, respectively, examining correlations between the new MRM with several established measures of resilience and risk. Correlations of 0.10, 0.30, and 0.50 are generally regarded as small, medium, and large effect sizes, respectively (62). For comparison purposes, we present the long and short form correlations of each measure next to each other. In general, the long and short forms correlate similarly to the established measures, and the Pearson r and Spearman rho correlations are also generally similar in magnitude. The highest correlations generally match resilience constructs that are more similar (convergence) and non-matching constructs are usually lower (divergence). Although there is some overlap for constructs that are more novel measures of resilience, significant correlations are primarily in the small to medium range in terms of effect size.

Table 5.

Convergent Validity of the Multilevel Resilience Resource Measure for African American/Black Adults Living with HIV (MRM) with Established Measures of Resilience and Risk with P-Values and Sample Sizes

5a. Pearson r correlations of long and short form multilevel resilience measures with the more established measures
Established Measure of Resilience or Measure of Riska	Individual		Interpersonal		Organizational		Neighborhood Services	Neighborhood Order
	Long (N)	Short (N)	Long (N)	Short (N)	Long (N)	Short (N)	Single Form (N)	Single Form (N)
Connor-Davidson Resilience Scale	0.267*** (377)	0.219*** (380)	0.118* (378)	0.112* (381)	0.089 (384)	0.071 (389)	0.122* (381)	0.115* (381)
Eight-item modified Medical Outcomes Social Support Survey	0.317*** (371)	0.301*** (374)	0.457*** (373)	0.447*** (376)	0.165** (378)	0.155** (383)	0.246*** (375)	0.212*** (375)
Brief Resilient Coping Scale	0.335*** (374)	0.295*** (377)	0.235*** (375)	0.240*** (378)	0.172** (381)	0.125* (386)	0.243*** (378)	0.130* (378)
Perceived Support Structures in Community Measure	0.136** (378)	0.113* (381)	0.135** (379)	0.144** (382)	0.100* (385)	0.110* (391)	0.311*** (383)	0.054 (383)
Neighborhood Disadvantage Indexa	−0.061 (345)	−0.048 (347)	−0.102 (344)	−0.068 (347)	−0.010 (351)	−0.039 (357)	0.000 (351)	−0.343*** (351)
Neighborhood Murder Indexa	−0.021 (345)	−0.023 (347)	−0.064 (344)	−0.021 (347)	0.050 (351)	−0.002 (357)	0.009 (351)	−0.273*** (351)
Neighborhood Assault Indexa	−0.067 (345)	0.023 (347)	−0.020 (344)	0.002 (347)	0.013 (351)	0.026 (357)	0.055 (351)	−0.269*** (351)
5b. Spearman rho correlations of long and short form multilevel resilience measures with the more established measures
	Individual		Interpersonal		Organizational		Neighborhood Services	Neighborhood Order
	Long (N)	Short (N)	Long (N)	Short (N)	Long (N)	Short (N)	Single Form (N)	Single Form (N)
Connor-Davidson Resilience Scale	0.392*** (377)	0.351*** (380)	0.208** (378)	0.207** (381)	0.189*** (384)	0.183*** (389)	0.208*** (381)	0.195*** (381)
Eight-item modified Medical Outcomes Social Support Survey	0.367*** (371)	0.368*** (374)	0.476*** (373)	0.474*** (376)	0.241*** (378)	0.225*** (383)	0.303*** (375)	0.227*** (375)
Brief Resilient Coping Scale	0.413*** (374)	0.395*** (377)	0.316*** (375)	0.296*** (378)	0.225*** (381)	0.186*** (386)	0.276*** (378)	0.170*** (378)
Perceived Support Structures in Community Measure	0.144** (378)	0.116* (381)	0.161** (379)	0.177** (382)	0.115* (385)	0.140** (391)	0.319*** (383)	0.033 (383)
Neighborhood Disadvantage Indexa	−0.120* (345)	−0.118* (347)	−0.112* (344)	−0.093 (347)	−0.011 (351)	−0.065 (357)	−0.028 (351)	−0.322*** (351)
Neighborhood Murder Indexa	−0.066 (345)	−0.021 (347)	−0.048 (344)	−0.022 (347)	0.082 (351)	0.013 (357)	0.027 (351)	−0.229*** (351)
Neighborhood Assault Indexa	−0.091 (345)	−0.042 (347)	−0.029 (344)	−0.002 (347)	0.028 (351)	0.015 (357)	0.029 (351)	−0.255***(351)

^***<.001

^**<.01

^*<.05

^aMeasure of risk

DISCUSSION

To our knowledge, this is the first study to report the psychometric properties of a multilevel resilience measure for PLWH that centers AA/B adults as key stakeholders in the generation of measure items. Results of the psychometric analyses provide support for a multi-dimensional, multilevel measure that assesses resilience at individual-, interpersonal-, organizational-, and neighborhood-levels. The long and short forms of the MRM (See Appendix A for measures) should be used to examine whether multilevel resilience resources are associated positively with progression through the stages of the HIV care continuum for AA/B PLWH. If evidence supports the above-mentioned associations, subscales and items from this measure could inform development of multilevel resilience building interventions which has implications for approaches to reduce racial/ethnic health disparities. Specifically, the focus on resilience resources related to clinic attendance and viral load suppression as intervention touchpoints could potentially address disparities in low rates of retention in care over the long-term and higher AIDS-related death for AA/B PLWH relative to all other racial and ethnic groups. Additionally, these interventions could support progress toward the 95-95-95 target by 2030 relating to diagnosis, receipt of ART, and virological suppression (63–67).

Both the long and short forms of the MRM demonstrate strong psychometric properties. In general, component patterns (a measure of validity) with moderate to high item loadings (i.e., .60 to .80) found with samples > 150 are recommended for interpretation and are expected to generalize to similar samples; both the long and short form measures achieve these moderate to high item loading thresholds (68–70). Importantly, all of the long and short form measures meet or exceed established cut-points for reliability (i.e., internal consistency with Coefficient alphas ≥ 0.70). Most of the MRM subscales had a reliability level considered very good (i.e., Coefficient alpha ≥ 0.80) (71, 72). The internal consistency reliability estimates for each of the MRM subscales indicate that the items within each relate to each other and support the interpretation that the item sets measure the same level of resilience (73). Additional psychometric testing to create the short form of each measure indicates excellent model fit within the confirmatory factor analysis framework across a variety of fit indices, further supporting the factor structure of the MRM short forms.

Ample evidence supports the validity of the measures. Specifically, we examined the convergent validity of the MRM with established measures and measurement invariance. In general, results provide evidence of convergent validity with correlation values in the small to medium range. Specifically, the long and short forms of the individual-level subscales of the MRM exhibit the strongest correlations with established individual level resilience measures (i.e. CD-RISC 10 and BRCS). In contrast, the long and short forms of the interpersonal level subscale exhibit the strongest correlations with the interpersonal level resilience measure (i.e., mMOS-SS). The two neighborhood resilience measures indicate some differential relationships to established measures, with the Neighborhood Order scale displaying moderate correlations with objectively measured neighborhood measures (i.e., Neighborhood Disadvantage Index, Assault Rate Index and Murder Rate Index), while the Neighborhood Services scale relates more strongly to the Perceived Support Structures in the Community Measure. The organizational resilience measure is a novel resilience construct with no clear established measure to compare it to, however, it may have other relationships to measures not included in this study. The aforementioned correlations suggest that while the new MRM subscales share some variance with established measures, the MRM may also measure aspects of resilience not captured by these measures and this should be investigated further. Last, we examined each of the new measures in a series of measurement invariance analyses across gender groups (i.e., comparing females and transgender females versus males). Measurement invariance across gender groups is strongly supported for each of the resilience measures, but future studies should aim to recruit larger groups of transgender women and men to explore how well the measures perform for them.

Limitations and Future Directions

Although novel and informative, this study is not without limitations. First, the data informing the measure development and psychometric testing are obtained from participants at two clinic based-cohorts. Findings from this study should be re-examined in a community sample of AA/B PLWH and across a larger cross-section of clinic cohort sites. Also, some of the items skewed toward the higher end of the response range; this could be due to several reasons, including characteristics of a clinic-based cohort of people who are engaged in care and the ordering of item response options all in the same direction. Due to the recruitment of clinic patients only, it is possible that the range of scores are higher than those of AA/B PLWH generally. Of note, however is that in some community and clinic-based studies, high average resilience scores in studies of resilience and HIV outcomes are observed for other measures such as the CD-RISC 10 and the 8-item mMOS-SS [e.g., (50, 74, 75)]. Given the demonstrated psychometric properties of the measure, future studies should examine and/or compare results between non-clinic and clinic-based samples. Some components of the neighborhood-level items (e.g., my neighborhood has health care services) measured the presence of resources but not whether the amount was sufficient. However, this is consistent with prior well-established neighborhood environment measures [e.g., (76, 77)]. Future studies could potentially modify the wording of neighborhood environment items to assess sufficiency of resources and reexamine the measure using the Checklist for Measure Development and Validation Manuscripts (36). Although missing data prevented some participants’ responses from contributing to the development of each of the measures at each level of resilience, 398 of 400 participants did contribute responses used in the development of at least some of the measures, which provides strong support for the validity and reliability of the developed measures. Overall, for the final short form versions that used the complete available sample, 90% (n = 359) of participants contributed data that were used in the development of each of the 4 levels of resilience measures, an additional 7% (n = 28) contributed data that was used in the development of 3 of the 4 levels of resilience measures, and an additional 2.5% (n = 10) contributed data that was used in the development of 2 of the 4 levels of resilience measures, with 1 participant’s data used in the development of 1 level of the resilience measure.

The current findings have implications for future research. We were unable to test the psychometric properties of the MRM as an overall summary score measure because at the time of data collection, there was no established multilevel measures of resilience available to compare with the MRM overall. Despite this, future studies could potentially create an overall summary score to explore associations between the MRM and HIV outcomes. Additional research is needed to test whether clinic-based cohorts across multiple clinic sites and with non-clinic based AA/B PLWH yield greater variability in scores. Additionally, future studies could explore whether changing to a 7-point response option also leads to greater variability in scores. Also, studies should examine longitudinal relationships between the MRM and HIV outcomes and potential effect modification of adversities on relationships of resilience with HIV outcomes. Moreover, the MRM may have the potential to be adapted to other chronic diseases more broadly and researchers should consider adapting and testing its psychometric properties among people living with other chronic diseases, where multi-level resilience may be expected or hypothesized to have a potential impact on these diseases. Better understanding of multilevel resilience in relation to disease outcomes could lead to potential resilience-building intervention strategies to improve disease management.

Data/Code Availability:

N/A