Fifteen Years of Progress in Measurement and Methods at the Resource Centers for Minority Aging Research

The National Institute on Aging (NIA) Health Disparities Strategic Plan (2009-2013) for reducing racial/ethnic health disparities in our aging population includes (a) efforts to explore aging as a web of genetic, biochemical, physiological, economic, social, and psychological factors; (b) provision of resources to promote high-quality research to reduce health disparities; and (c) training of a diverse investigator workforce to conduct aging and health disparities research. The Resource Centers for Minority Aging Research (RCMAR) program is a critical mechanism toward achievement of these goals, and was one of the first coordinated efforts to create a research infrastructure toward that end. NIA established the RCMARs in 1997, with support from the National Institute of Nursing Research (NINR) and the National Center for Minority Health and Health Disparities (NCMHD). The overall RCMAR mission is to “improve the health and well being of older minority populations by identifying mechanisms for reducing health disparities.” The primary mechanism for this is investigator development, that is training and career development of minority investigators. The RCMAR effort has also funded Measurement and Methods Cores (MMCs) because conducting minority aging research requires use of appropriate and specialized methods and measures. The MMCs support the RCMAR mission and NIA priorities by developing and testing specific measures appropriate for research in ethnically diverse older populations, providing methodological training to investigators interested in conducting minority aging research, developing and refining specialized methods, and disseminating culturally appropriate measures and methods broadly.

Prior to the RCMAR program, relatively little research dealt with measurement methodology in minority populations, and even less attention was focused on the issue of cultural sensitivity of measures. Over the 15 years of the RCMAR program, MMC investigators contributed substantially in advancing this field by convening experts, addressing measurement issues, and disseminating this work through publications. A brief history of RCMAR, culminating in the work presented in this issue of the Journal of Aging and Health, is provided.

RCMAR 1 (1997-2002)

Presentations and publications of reviews of measures and measurement methodology

In 2001, the RCMAR MMCs published a volume (Skinner, Teresi, Holmes, Stahl, & Stewart, 2001) of state-of-the-art reviews of measures and methodological issues related to ethnically diverse populations. Included were methods articles that discussed statistical approaches that should be avoided, and that provided guidance regarding the use of item response theory (IRT) and confirmatory factor analytic (CFA) approaches for examining measurement invariance. Review articles of measures used in cross-cultural assessment focused on the following domains: acculturation, socioeconomic status, social support, cognitive status, health, mental health and functional status, health locus of control, health-related quality of life, and religiosity.

A RCMAR MMC conference was convened that included a group of measurement and cross-cultural research specialists, with the goal of advancing the scientific basis of measurement across racial/ethnic groups. One product was a framework for considering the conceptual and psychometric adequacy and equivalence of measures for use in health disparities research (Stewart & Nápoles-Springer, 2003). Conceptual and measurement issues pertaining to four concepts known to be key determinants of health disparities were reviewed systematically: socioeconomic status, discrimination, acculturation, and quality of care. Recommendations were made for advancing the quality of the measures of these concepts, and an agenda was produced for accomplishing the needed measurement research.

Measurement evaluation grids (MEGs)

One of the most critical issues in the measurement of health, social, and behavioral constructs is the comparability of concepts and measures across race/ethnic groups. The need was identified for reviews of measures, in the form of MEGs, which summarize the properties of measures from the perspective of cross-cultural equivalence. The MMCs began creating and posting MEGs on the RCMAR Coordinating Center measurement website (www.rcmar.ucla.edu; http://www.rcmar.ucla.edu/rcmar_wiki/home.html; www.research-hhar.org/SubMeasure). Thus far, measures of cognition, affective suffering, physical health and function, and quality of life were reviewed. Unlike other measurement data bases, the focus of the MEGs is on the use and psychometric qualities of the measures with respect to ethnically diverse groups. Expert Reviews summarize information contained in the MEGs (described in the appendix). The purpose was to (a) select the measures that have been used among minority elders, (b) summarize and critically evaluate available information on the measurement characteristics, (c) discuss issues related to generalizability (external validity) of results, (d) suggest areas in which ethnic differences may impact the measurement process, (e) propose future research in minority elders for the measurement of the domains analyzed, and (f) provide recommendations for researchers wishing to use existing measurement instruments.

Summaries of instruments and annotated bibliographies

In addition to MEGs, MMC investigators contributed annotated bibliographies on several methods and domains: cognitive interviews, focus groups, causal inference in health disparities, IRT and differential item functioning (DIF), translations, measuring race and ethnicity, cognition, depression, health, and health literacy. These are available at http://www.rcmar.ucla.edu/rcmar_wiki/home.html.

RCMAR 2 (2002-2007)

Measurement symposia and preconference workshops

During RCMAR 2, the Coordinating Center began a series of pre-conference workshops at the Gerontological Society of America (GSA), funded by the NIA (R13 AG023033). These provided a forum for sharing the expertise acquired by RCMAR researchers with the broader group of scientists interested in the science of diversity. Several critical measurement-related topics were covered, including the conceptual and psychometric adequacy and equivalence of self-report measures in studies of health disparities; approaches for reviewing the adequacy and equivalence of existing measures in diverse groups; use of qualitative methods to pretest potential measures in diverse groups; and CFA, IRT, and DIF methods for evaluating measures within and across groups.

During RCMAR 2, the first RCMAR cross-site collaborative study was organized by the MMCs, and presented at a GSA symposium. A comparison of four methods of DIF analyses showed the relative merits and disadvantages of each. Four junior scholars from four sites participated and all six RCMAR sites contributed over 20 senior mentors (Yang et al., 2011). A list of contributors can be found in the acknowledgment section of the appendix.

Reviews of state-of-the art measurement methods with examples

A major goal for the RCMAR 2 MMCs was to publish articles from the 2001 RCMAR conference, as well as other articles from prominent scholars in qualitative and quantitative measurement issues. Led by Teresi, Stewart, Morales, and Stahl (2006), internationally known experts including the developers of the methods were invited to prepare papers. State-of-the-art methods for qualitative analyses were presented including frameworks for cross-cultural measurement, use of cognitive interviews, and issues of deconstructing race. The quantitative section included presentation of methods for the conduct of CFA, IRT, DIF, item banking, and computerized adaptive testing (CAT) in the context of health disparities research. These presentations set the stage for the work of RCMAR 3 and beyond, described later. RCMAR measurement publications are available in the Appendix and at: http://www.rcmar.ucla.edu/references.php.

RCMAR 3 (2007-2012)

Reviews of possible solutions to measurement problems identified in RCMARs 1 and 2

In this issue of the Journal of Aging and Health, three papers are included that are the product of the RCMAR MMCs. Although the authors of these papers do not claim to resolve all of the issues and problems identified in prior RCMAR efforts, they attempt to provide recommendations and guidelines based on the body of scientific work produced collectively through the RCMAR projects. The first paper in the series (Stewart, Thrasher, Goldberg, & Shea, this issue) forms the foundation for the other two papers and introduces qualitative considerations in modification of measures. Although information is increasingly available on various problems using self-report measures in diverse groups, there is little guidance on how to modify the measures if necessary. A framework of issues to consider when modifying measures for diverse populations is provided, including reasons for considering modifications, the types of information that can be used as a basis for making modifications, and modifications researchers have made. The authors make suggestions for how to report modifications in publications using the measures. The intent of presenting these issues is to open a dialogue about guidelines for adapting measures for use in ethnically diverse populations.

Based on the framework in the Stewart et al. article, the second article (Thrasher, Clay, Ford, & Stewart (2012) illustrates the process of reviewing measures for appropriateness in health disparities research and identifies areas requiring modifications. The authors focus on research about how discrimination contributes to health disparities among older adults. They review four widely used measures of discrimination in terms of their ability to address questions based on theoretical frameworks relevant to the study of perceived discrimination and health. Potential areas of modification are suggested in each context, providing an example of the process of determining the adequacy of specific measures for answering research questions.

Modifications of measures may impact the comparability of scores across groups and settings. As reviewed in the third article by Teresi, Ramirez, Jones, Choi, and Crane (2012), IRT permits well-calibrated items to be used interchangeably. Exactly similar is not technically correct in this context. The exact same item does not have to be administered to each respondent, theoretically permitting wider latitude in terms of modification. The authors review methods for examining the impact of DIF and provide illustrations of DIF related to age, race or ethnicity. Recommendations regarding modifications are given in the context of research, clinical decision making and high stakes testing, which may require item removal or separate calibrations to ensure accurate assessment. Guidelines for modification based on DIF analyses and illustrations of the impact of adjustments are presented.

RCMAR 4 And Beyond

To conclude, the RCMAR MMCs are planning a pre-conference at GSA in November 2012 on item banking and CAT. This conference will be linked to the National Institutes of Health Patient Reported Outcomes Measurement Information System (NIHPROMIS) (www.NIHPROMIS.org) methodologies and measures (Reeve et al., 2007) and to NIHTOOLBOX (www.nih-toolbox.org). In many ways, this effort merges the pioneering activities of the RCMAR MMC teams with the advances made in patient reported outcomes assessment, both of which were built upon landmark work over the past century in educational testing. Collectively, these activities have advanced the science of measurement and methods related to physical and mental health, cognitive assessment, social and behavioral functioning, health disparities, and minority aging research.