Development and Dissemination of the El Centro Health Disparities Measures Library

Abstract

Purpose

This report describes the development and dissemination of a library of English measures, with Spanish translations, on constructs relevant to social determinants of health and behavioral health outcomes. The El Centro Measures Library is a product of the Center of Excellence for Health Disparities Research: El Centro, a program funded by the National Institute on Minority Health and Health Disparities of the U.S. National Institutes of Health. The library is aimed at enhancing capacity for minority health and health disparities research, particularly for Hispanics living in the United States and abroad.

Design

The open-access library of measures (available through www.miami.edu/sonhs/measureslibrary) contains brief descriptions of each measure, scoring information (where available), links to related peer-reviewed articles, and measure items in both languages. Links to measure websites where commercially available measures can be purchased are included, as is contact information for measures that require author permission. Links to several other measures libraries are hosted on the library website. Other researchers may contribute to the library.

Methods

El Centro investigators began the library by electing to use a common set of measures across studies to assess demographic information, culture-related variables, proximal outcomes of interest, and major outcomes. The collection was expanded to include other health disparity research studies. In 2012, a formal process was developed to organize, expand, and centralize the library in preparation for a gradual process of dissemination to the national and international community of researchers.

Findings

The library currently contains 61 measures encompassing 12 categories of constructs. Thus far, the library has been accessed 8,883 times (unique page views as generated by Google Analytics), and responses from constituencies of users and measure authors have been favorable.

Clinical Relevance

With the paucity of availability and accessibility of translated measures, behavioral nursing research focused on reducing health disparities can benefit from repositories of research instruments such as the El Centro Measures Library.

This report describes the El Centro Measures Library, which is housed in the University of Miami School of Nursing and Health Studies (SONHS). We describe the process, challenges, and approaches of curating measures translated to Spanish and used in health disparities research with Hispanic-Americans. We also present metrics regarding access and feedback from constituencies of users and measure authors. Ultimately, we aim, through this vehicle, to inform the research community of the availability of the library, and of other resources for translated measures that can be used in research to address health disparities in vulnerable populations.

The National Institute on Minority Health and Health Disparities defines health disparities as differences in health status that adversely affect socially disadvantaged groups, including ethnic minorities, sexual and gender minorities, and individuals of low socioeconomic status (Perez-Stable, 2016). Health disparities scientists document disparity outcomes such as incidence, prevalence, and burden of disease, as well as health-related quality of life. The science of health disparities is also devoted to discovering the factors that contribute to risks for and protection from poor health outcomes among vulnerable groups. These factors include social determinants of health at the individual, interpersonal, community, and societal level. Understanding the determinants of health disparities is necessary for developing interventions that can eliminate or reduce these disparities. Nursing research has been instrumental in addressing health disparities (NINR, 2016), benefiting both individual and public health, as well as shaping the advancement of the nursing profession (McFetrudge, 2014; NINR, 2016; Polit & Beck, 2014; Van Herk & Andrew, 2011).

While Hispanics in the United States enjoy longer life expectancy than other minority groups, they experience disproportionate prevalence in specific health problems, including HIV/AIDS, cervical cancer, diabetes, obesity, psychological distress, suicidality, health risk behaviors among youth, and teenage childbearing (National Center for Health Statistics [NCHS], 2016). In addition, Hispanics have considerable challenges in regard to social determinants of health, such as educational attainment, poverty, access to health care, and low rates of health insurance coverage (NCHS, 2016; U.S. Department of Health and Human Services [USDHHS], 2015). In the United States, persons of Hispanic origin constitute nearly 17% of the total population and by 2060 are expected to comprise 28% of the population (U.S. Census Bureau, 2015a); thus, the disparities experienced by this community constitute a significant public health concern (USDHHS, 2015).

Behavioral research to address health disparities experienced by Hispanic-Americans requires approaches that are linguistically appropriate for the diverse Hispanic population. Hispanics in the United States trace their national origins to over 20 Spanish-speaking countries (Lopez, Gonzalez-Barrera, & Cuddington, 2013). Because of the variations in language used by people from different national or regional origins, investigators conducting research with Spanish speakers sometimes translate measures specifically for their local study populations when a translation they deem suitable for their specific study population is not available. The profusion of ad hoc translations raises the likelihood that the same measure applied in different settings may differ not only with respect to language, but potentially to differences in the construct being measured, limiting the comparability of findings across studies. Furthermore, not all Hispanic-Americans are fluent in Spanish. It is estimated that of Hispanics living in the United States, 36% are bilingual, 38% are primarily or exclusively Spanish speakers, and 25% are primarily or exclusively English speakers (Krogstad & Gonzalez-Barrera, 2015). Therefore, research addressing health disparities experienced by Hispanic-Americans requires research instruments that are available in both English and a version of Spanish that at least aspires to be universally accessible (NCHS, 2016).

Measure Translation and Dissemination Approaches

Developers and publishers of research instruments are increasingly attuned to the need to conduct rigorous and universal translations to Spanish as well as other languages given increasing trial internationalization and the predominant development of instruments in the English language. Four examples follow and demonstrate varying approaches and standards of rigor for translation. The first example is the Mapi Research Trust, a leading curator and distributor of clinical outcome assessments, which aims to provide access to information about these instruments and disseminate them to the global research community. The library houses 4,000 patient-centered outcome questionnaires, which includes over 250 for which they are the official distributor. The Mapi Language Services Division has expertise in linguistic validation and has authored the Linguistic Validation Manual for Health Outcome Assessments (Mapi Health Research & Commercialization, 2016). Mapi’s approach to translation involves working in close collaboration with measure authors to ensure conceptual equivalence and international harmonization of all language versions. Mapi’s translation methodology includes (a) conceptual analysis of source questionnaire, (b) forward translation, (c) backward translation, (d) interviews with patients or clinician review, and (e) proofreading and finalization. Mapi also provides translatability assessment, which is defined as the evaluation of the extent to which a measure can be meaningfully translated into another language (Mapi Health Research & Commercialization, 2016).

A second example is Psychological Assessment Resources (PAR), a leading publisher of psychological assessment materials that offers products in a variety of formats. All the translations available through PAR have been approved using translation followed by back-translation by an individual unfamiliar with the English version of the test, with the back-translation forwarded to the author and PAR for review and approval.

A third example is the National Institutes of Health (NIH)—Supported Patient-Reported Outcomes Measurement Information System (PROMIS), a set of person-centered measures that evaluates and monitors physical, mental, and social health in adults and children and is open access. The PROMIS methodology for translation includes (a) translation from English to Spanish by one individual, (b) back-translation of the Spanish translation to English by a second individual, (c) review by multiple experts, and (d) cognitive debriefing with a sample of native speakers. A universal approach to translation ensures that, whenever possible, one language version is created for multiple countries instead of country-specific versions of the same language (PROMIS, 2016).

Finally, Mind Garden, a company that aims to enable access to validated psychological assessments and instruments to the global community, offers a full range of psychological assessments. Many of the translations available on Mind Garden have been (a) conducted by individual researchers, (b) may be partial translations, and (c) typically do not have validation data. Although the translation quality is not assured, they seek to facilitate work with non-English-speaking populations by making translations available in their current form. Mind Garden has a process for granting permission to researchers to translate measures and a requirement that the translations are returned to them and made available on their site free of charge to other researchers. They also include a link to the International Test Commission Guidelines for Translating and Adapting Tests (Mind Garden, 2015).

Access to measures collected in the above examples can range from open-access public domain materials to copyrighted materials with access limited by paywalls; the four libraries referenced above illustrate this spectrum of access. PROMIS provides open and free access to all their research instruments. The questionnaires that Mapi distributes on behalf of their copyright holders represent conditions of use that are specific to each questionnaire. For example, a fee may be charged for commercial users, a nominal fee for funded academic research studies, and no fee for nonfunded academic users. Mapi may offer to handle the licensing process for questionnaires that they house but do not distribute. The instruments available through both PAR and Mind Garden are for purchase and require permission for use as well as for any modifications.

Navigating copyright law within the context of commercial publishing operations presents a challenge in creating openly accessible online repositories such as the El Centro Measures Library. As Thomas Jefferson observed, there is a tension in copyright law between protecting the knowledge creators’ rights and the rate of innovation in society: copyright laws that are too strong slow the rate of knowledge production (Stiglitz, 1999). This tension creates a legal environment that is constantly being renegotiated. Instead of strict enforcement, copyright law is interpreted through the analysis of four factors in the use of copyrighted material: what is the purpose of the use (e.g., commercial or nonprofit), what is the nature of the copied work (e.g., fiction or nonfiction), how much of the original work was copied, and finally the effect of the use on the market (Crews, 2006). Some observers note that current copyright law in the United States is too strong and thus stifles innovation as well as society’s ability to steward knowledge as a public good (Farb, 2006; Samuelson, 1996). PubMed Central shows exemplary collaborative efforts between publishers, online content providers, and regulators to overcome this difficulty.

How scientists discover prior research in their field has also changed dramatically since large scale open-access publishing efforts began. All research is fundamentally based on drawing from past work to create new knowledge. Libraries as repositories of historical knowledge are an integral part of this process, and over the course of millennia they have created organizational systems to serve the purposes of discovery and access (Battles, 2003). As scientific work continues to adopt digital tools and methods in the research process, the academy has started to build digital libraries in addition to traditional paper-based collections (Bowker, 2005). Indeed, all researchers build pdf document libraries on their computers as well as book-based libraries in their offices. A principal function of contemporary academic libraries is to reproduce this process at scale.

The Beginnings and Impetus for the El Centro Measures Library

At the inception of El Centro in 2007, the center investigators elected to use a common set of measures across studies to assess demographics, culture-related variables (e.g., acculturation, familism), proximal outcomes of interest (e.g., HIV knowledge, family cohesion), and major outcomes (e.g., substance use). Thus began the process of collecting a library of measures. The collection was expanded to include all the measures used in each study conducted by researchers associated with El Centro, not only the ones funded by the center. At the beginning, the library of measures was loosely structured and measures were shared by investigators through an informal workplace network along with anecdotal experiences with use of the measures. Some measures and existing translations were modified by El Centro investigators for our research populations, which consist of Hispanic-Americans of various national origins, reflecting the population of our city. All measures were empirically supported for use with Hispanics, and the translations were approved for use by the university’s institutional review board (IRB).

The decision to broaden the scope of measures included in the library and expand the curation and dissemination of measures more widely across the institution came from the experience of the first author (V.B.M.) as a member of the Social and Behavioral Sciences IRB, where it was noted that although the university is in a community where a sizable proportion of the population speaks Spanish as a primary language, many of the research protocols being reviewed were only open to participants who could read and speak English. An informal review conducted in 2007 by personnel at the university’s Human Subjects Research Office found that only 15% to 20% of behavioral studies approved by the IRB included non-English speakers. This is out of proportion with the population of Miami-Dade County where over 70% of families speak a language other than English at home (U.S. Census Bureau, 2015b), contributing to the problem of underrepresentation of immigrants in biomedical and behavioral research.

Methods

Building the Library

We initiated a formalized process for developing the El Centro Measures Library in 2012. The goal was to organize, expand, and centralize the library in preparation for a gradual process of dissemination—first to faculty and doctoral students at our SONHS—next to the larger community of investigators at the university, and finally to the national and international community of researchers. We identified three publicly available web-based measures libraries as exemplars to inform our design: (a) the Oregon Research Institute–Center on Early Adolescence Measures Repository, funded by the National Institute on Drug Abuse (http://measures.earlyadolescence.org/); (b) the Grid-Enabled Measures Database (GEM), funded by the National Cancer Institute (https://www.gem-beta.org/Public/Home.aspx); and (c) PROMIS Measures–Northwestern University, funded by the NIH (http://www.healthmeasures.net/explore-measurement-systems/promis).

The measures library team consisted of the director and coordinator of the El Centro Research Core, a PhD student, and undergraduate student workers. The team collected each measure in English and Spanish, collected or wrote brief descriptions of each measure, and gathered peer-reviewed publications describing the development, translation, and psychometric properties of the measure. The team also traced any modifications that were made to the measure to adapt the measure for a specific population as well as the source and method used for the Spanish translation. Spanish translations that are part of the library come from a variety of sources: translations conducted by measure originators, by other investigators who have published research using their own translation, by El Centro study teams using translation-back translation-reconciliation methods, or by a University of Miami IRB-approved vendor. The process of tracing modifications and translations presented a significant challenge because in some cases the measures being used in El Centro studies had been passed down from study to study, some dating back more than a decade. Considerable effort was required to gather the information regarding translations and modifications. Team members interviewed researchers who had used the measures, and in some cases read IRB protocols and publications to find clues when the researcher could not confirm the information. Measures were not included in the library until a thorough verification process was completed to insure accuracy.

Translations performed within El Centro and by vendors approved by the University of Miami Human Subjects Research Office used the translation an back-translation method, with reconciliation by a third party. Back-translation allows for comparison of the translation with the original text for quality and accuracy and helps to evaluate the equivalence of meaning between the source and target texts. It involves taking the translated version of a document or file and then having a separate independent translator (who has no knowledge of or contact with the original text) translate it back into the original language. During the reconciliation step, the original source material is compared with the back-translation to look for and revise instances where the meaning of the translated item is unclear or different from the original meaning. El Centro translators, who were often members of study teams, were fully bilingual and signed a document attesting that they were qualified to translate, based on their proficiency in both languages, and that their translations were conducted independently.

Approach to Intellectual Property

As we moved towards disseminating the library to a larger audience, we encountered the next challenge, which was to determine the copyright or public domain status of measures as well as the terms of use. The approach to intellectual property rights involved in hosting a publicly available repository required an understanding of copyright and intellectual property laws and the strategies needed to comply with those laws. In the first phase of dissemination to investigators within our immediate school, the library was accessible via a secure, internal sign-on. A paragraph appeared on the library’s landing page advising users that the library was for informational purposes only and that measure authors and copyright holders must be contacted for permission to use.

In preparation for wider dissemination, we consulted with specialists in digital strategies and data curation at the University of Miami libraries who provided insights into copyright and intellectual property, fair use, and common research practice. Of note is that practice among researchers does not always align with publisher policy. Publishers will often overlook informal sharing among researchers of copyrighted research materials but have much stricter standards for libraries. Additionally, measure authors and authors often sign over rights to publishers and therefore do not have the right to grant permission for use. As a collaborative effort with the University of Miami Libraries, we drafted copyright and intellectual property and disclaimer language and designed a strategy for contacting measure authors and authors to seek permission to include their measure in the library. The strategy assured that we performed due diligence to gain permission before including a measure in the library. With due diligence and the disclaimer, the risk for copyright infringement is reduced. The final language advising users to contact copyright holders on the landing page of the El Centro library is similar to what was used in the first phase of dissemination to the SONHS, as described above.

The process to contact measure authors for permission to include their measure in the library included several steps. First, a team member located e-mail contact information for all measure authors. Next, we sent a web-based survey that asked the measure authors to select from three options to indicate the level of inclusion of their measure in the library: (a) full posting (post description, link to source article or abstract, and measure items in English and Spanish), (b) limited posting (brief description and link to source article), or (c) refuse to post. The message that accompanied the survey explained the mission of the library and asked for nonexclusive rights to disseminate the measure, translate it to Spanish, copy it for preservation (back-up), and migrate it to different formats (for preservation). The initial mailing went out to 62 authors in late February of 2016. A second mailing was sent to those who did not respond to the first e-mail approximately 6 weeks later. The response rate after both mailings was 58%. Of the 36 authors who responded, 22 permitted full posting, and 14 opted for limited posting. None of the respondents refused to allow their measure to be posted in the library. Those who approved only limited posting were no longer the copyright holders and provided contact information for the copyright holders or reached out to them on our behalf. In keeping with standards imposed for due diligence, as advised by the University of Miami Library, we kept those measures from authors from whom we did not receive a response after two mailings. However, as clearly stated in the El Centro Measures Library website, we will respect requests from measure authors or copyright holders to remove posted measures.

Results

Library Contents

The library currently contains 61 measures, 56 of which came from El Centro–associated studies. The remaining five came from other university investigators. Twenty-five additional measures are being prepared for inclusion in the library: 13 from El Centro investigators and 12 from other university investigators. Spanish translations included in the library came from several sources. Twenty-nine were translated by El Centro–associated investigators, 3 were translated by other University of Miami investigators that contributed to the library, 16 were translated by the measure authors, 4 were translations reported by other researchers, and 9 are available through the copyright holder. The level of access to measures currently included in the library varies. Thirty-three of the measures are publicly available, 12 require author permission (contact information is included in the library), and 16 are available for purchase (in one case the English version is available for purchase while the Spanish translation is publicly available). Twelve constructs have been identified for categorizing the measures, allowing for more than one construct or measure. Table 1 shows the constructs and the number of measures currently categorized under each XXXXX.

Table 1.

Categories of Constructs Represented in the Measures Library

Child/adolescent	10
Culture	5
Experience with Health care services	4
Family	6
Gender/sexuality	3
Health	5
HIV/AIDS	8
Psychological distress	6
Substance use	6
Stress/coping	1
Social support/relationship	6
Violence	4

Dissemination

In June of 2016, the library became live and publicly accessible on the El Centro website. At that time, an e-mail was sent to researchers across the university via the Research Listserv announcing the rollout of the library as a resource to university investigators and inviting them to contribute measures with Spanish translations used in their research. Additionally, information about the El Centro Measures Library was disseminated at scientific conferences, including two important nursing conferences, the 2016 Sigma Theta Tau International Nursing Research Conference and the 2016 Council for the Advancement of Nursing Science Conference.

From June 22, 2016, through March 22, 2017, a Google Analytics report indicated that there were 8,883 unique IP address views of the library. The average time on the landing page of the library was 1 min and 52 s, indicating that visitors were taking some time to look at the list of measures. The number of page views spiked dramatically at two time points, one in July and another in September, corresponding with dissemination at the two nursing conferences referenced above.

Feedback From Measure Authors and Users

Among the authors who responded to the survey, 14 provided additional information about their measure such as copyright status, suggestions for inclusion of additional measures they authored, information on Spanish translations, and scoring. One author offered to provide clean copies of the measure as well a list of related scientific article citations. Quotes from authors included “Thank you for including our work” and “Happy to support this effort!”

Local users of the library have let us know that they have found it to be a useful resource. Several users from outside of our institution, representing national as well as international doctoral students, medical doctors, and PhD level researchers, have reached out for information about measures they were interested in using in their research.

Discussion

Advancing the science of minority health and health disparities requires resources that can support rigorous research and facilitate the inclusion of participants from disparity populations. The El Centro Measures Library aims to contribute to the advancement of health disparities science by facilitating access to a set of instruments that have been used with Spanish-speaking study samples. Increased access to Spanish translations of measures and to information about the pedigree of these translations is critical to conducting studies that can develop knowledge on social, psychological, and cultural determinants of health. Hispanics are a growing demographic in the United States such that health disparities in this population are of high public health significance. Further, having Spanish-language measures serves the needs of researchers in Spanish-speaking countries or other countries with large numbers of immigrants of Hispanic origin. Spanish-language measures that have a universalist approach to translation facilitate the use of common measures across sites, which can help to disentangle commonalities and differences between Hispanic subgroups.

Access to measurement resources is limited by cost and lack of an efficient means of discovery of these compendia of translated measures. To find and access standardized research output from different locations in time and space remains difficult. While the recent development of open-access repositories is remarkable, there remains much work to be done to transform these initial efforts into tools that better suit the needs of health disparities researchers so that they can better serve their target populations. Moreover, it remains difficult to create and sustain online resources either as libraries or repositories in which the translated measures can be stored and made widely accessible to health disparities researchers.

The launch of PubMed Central in 2000 came with much hope that open-access academic publishing would accelerate knowledge production (Holdren, 2013; NIH, 2003), and help to bridge the divide between research communities in the global north and the global south (Cockerill & Knols, 2008). Yet, as the last 15 years have shown, open-access publishing is not the panacea for sharing scientific output that some originally envisioned. Authors’ fees, Internet access, and technical know-how (familiarity with navigating online systems) remain problems for many researchers, particularly those in the developing world (Papin-Ramcharan & Dawe, 2006).

Future Directions

Despite its promise, the El Centro Measures Library is limited in scope, and further work is needed to increase its potential impact. Firstly, it remains to be determined if the measures in the library are psychometrically or conceptually valid for different subgroups of Spanish-speaking people in the United States and other nations. Future work with the library might solicit feedback or data sets from researchers who have used measures in the library to conduct studies on psychometric equivalence across samples from (a) studies done with samples in regions of the United States and abroad, (b) of different national origin, and (c) that differ on socioeconomic and other demographic characteristics. Secondly, the number of measures and the constructs addressed by the measures are mostly limited to those used in El Centro–associated studies. While researchers outside of El Centro have begun to contribute measures that they have used and we intend to continue to expand the library holdings, the work of maintaining and expanding the library requires time and infrastructure support.

We hope through this publication to inform the national and international community of nursing scholars about the availability of the library, and of other resources for translated measures that can be used in research to address health disparities in vulnerable populations. With the paucity in availability and accessibility of translated measures, nursing research geared towards health disparities needs translated measures and will benefit from accessible online resources such as these.

Clinical Resources.

• The Center on Early Adolescence Repository of Measures contains measures pertinent to early adolescent functioning and relevant influential contexts (e.g., family, school, peers, etc.). http://measures.earlyadolescence.org/

• Grid Enabled Measures Database (GEM) is a web-based collaborative tool containing behavioral, social science, and other relevant scientific measures that aims to support and encourage a community of users to drive consensus on best measures and share the resulting data from use of those measures. https://www.gem-beta.org/Public/Home.aspx

• Mapi Research Trust houses 4,000 patient-centered outcome questionnaires as well as their 40,000 translations into some 170 languages. This includes over 250 questionnaires for which Mapi is the official distributor. http://mapi-trust.org/

• The PhenX (consensus measures for phenotypes and exposure) Toolkit is a catalog of recommended, standard measures of phenotypes and environmental exposures for use in biomedical research. It offers well-established, broadly validated measures relevant to investigators in human genomics, epidemiology, and biomedical research. https://www.phenxtoolkit.org/

• The Patient-Reported Outcomes Measurement Information System (PROMIS®) is a set of person-centered measures that evaluates and monitors physical, mental, and social health in adults and children. It can be used with the general population and with individuals living with chronic conditions. http://www.healthmeasures.net/explore-measurement-systems/promis

• Psychological Assessment Resources is a leading publisher of psychological assessment materials offering an array of assessment formats developed to meet the needs of professionals in psychology, mental health, counseling, education, forensics, pharmaceutical product development, and career counseling. http://www4.parinc.com/Company/about_PAR.aspx

• The University of California at San Francisco Center for AIDS Prevention Studies (CAPS) offers surveys and scales designed and tested by CAPS for use by HIV researchers, evaluators, prevention program planners, and designers. https://caps.ucsf.edu/resources/survey-instruments-and-scales/