Skip to main content
Journal of General Internal Medicine logoLink to Journal of General Internal Medicine
. 2002 Jun;17(6):477–486. doi: 10.1046/j.1525-1497.2002.10633.x

Designing and Evaluating Interventions to Eliminate Racial and Ethnic Disparities in Health Care

Lisa A Cooper 1,2, Martha N Hill 1,2,3, Neil R Powe 1,2
PMCID: PMC1495065  PMID: 12133164

Abstract

A large number of factors contribute to racial and ethnic disparities in health status. Health care professionals, researchers, and policymakers have believed for some time that access to care is the centerpiece in the elimination of these health disparities. The Institute of Medicine's (IOM) model of access to health services includes personal, financial, and structural barriers, health service utilization, and mediators of care. This model can be used to describe the interactions among these factors and their impact on health outcomes and equity of services among racial and ethnic groups. We present a modified version of the IOM model that incorporates the features of other access models and highlights barriers and mediators that are relevant for interventions designed to eliminate disparities in U.S. health care. We also suggest that interventions to eliminate disparities and achieve equity in health care services be considered within the broader context of improving quality of care. Some health service intervention studies have shown improvements in the health of disadvantaged groups. If properly designed and implemented, these interventions could be used to reduce health disparities. Successful features of interventions include the use of multifaceted, intense approaches, culturally and linguistically appropriate methods, improved access to care, tailoring, the establishment of partnerships with stakeholders, and community involvement. However, in order to be effective in reducing disparities in health care and health status, important limitations of previous studies need to be addressed, including the lack of control groups, nonrandom assignment of subjects to experimental interventions, and use of health outcome measures that are not validated. Interventions might be improved by targeting high-risk populations, focusing on the most important contributing factors, including measures of appropriateness and quality of care and health outcomes, and prioritizing dissemination efforts.

Keywords: disparities, race, ethnicity, interventions, health care quality, cultural competance


Health care professionals, researchers, and policymakers in the United States have believed for some time that access to care is the centerpiece in the elimination of disparities in health for racial, ethnic, and social class groups.14 After decades of research into the existence and explanation of inequalities in health, there is growing awareness of the importance of designing interventions to reduce or eliminate disparities in health status. Health [status] measures generally reflect the needs and outcomes of health care services for individuals and populations. The World Health Organization defined health in 1948 as “a state of physical, mental, and social well-being, and not merely the absence of disease or infirmity.”5 Health care, in contrast to health status, reflects the total societal effort, whether public or private, to provide, organize, and finance services that promote the health status of individuals and the community.6 This includes care and procedures delivered by health care providers in a variety of settings, including hospitals, outpatient settings, long-term care facilities, and at home, as well as pharmaceutical and other health care devices. Any service paid for by health insurance may be considered a dimension of health care.

The purpose of this paper is to provide a perspective on the literature regarding interventions to reduce ethnic and social class health care disparities and to discuss factors that are potentially important when designing, implementing, and evaluating such interventions. The intended audience for this paper is health services researchers, practicing primary care clinicians, and public health practitioners. A question of great interest to this group of professionals is, what proportion of the observed disparities in health status are attributable to disparities in health care and therefore amenable to improvements in health care delivery? Although not all of the relevant pathways have been completely elucidated, strong evidence suggests that there is room for improvement within the health care sector. Given the substantial resources invested in health care in the United States, not addressing avoidable disparities in health care delivery risks minimizing the returns on investments in health care. Although the fundamental inequalities that exist in society (including social, economic, and environmental factors) are important contextual contributors to disparities in health status, the professionals to whom this paper is addressed have more control over the health care system than they have over other contextual factors. Hence, the focus of this paper is on designing interventions to eliminate disparities in health care. Contextual factors are only discussed to the extent that health services may be changed or restructured to minimize their role in perpetuating racial and ethnic disparities in health.

We also acknowledge the importance of considering interventions to eliminate health care disparities in the broader context of improving health care quality.7 The Institute of Medicine's report, Crossing the Quality Chasm, calls for a redesign of the health care delivery system that includes changes in 4 main areas: 1) applying evidence to health care delivery, 2) using information technology, 3) aligning payment policies with quality improvement, and 4) preparing the workforce. The report also includes the need for health care to be equitable among its 6 aims for improvement in health care delivery.8

CONCEPTUAL FRAMEWORKS AND MODELS FOR DESIGNING INTERVENTIONS

In a report released by the King's Fund in the United Kingdom in 1995, Benzeval et al. proposed a framework for tackling socioeconomic inequalities in health. This framework suggested 4 levels for intervention: 1) improving the physical environment (including the adequacy of housing, working conditions, and pollution levels); 2) addressing social and economic factors (such as income and wealth, unemployment, and social support); 3) improving access to appropriate and effective health and social services; and 4) reducing barriers to adopting healthy lifestyles by changing behavioral risk factors.9 Because racial and ethnic inequalities in health care in the United States are largely, although not completely, mediated by social class differences among patients, a similar framework might be useful for tackling racial and ethnic inequalities in health care in the United States. This paper focuses on interventions in levels 3 and 4 from the U.K. framework that health services can use either alone or in collaboration with other agencies to address health inequalities.

In 1993, the Institute of Medicine's (IOM) Committee on Monitoring Access to Personal Health Services set out to resolve many conceptual problems in the definitions of equitable access to health care. The Committee developed a model and defined access as “the timely use of personal health services to achieve the best possible health outcomes.”10 The IOM model moves beyond using standard approaches that rely mainly on enumerating the presence of health care providers, the number of uninsured, or encounters with health care providers, to detect access problems. Indicators in this model are grouped according to barriers (personal, structural, and financial) that cause underuse of services, and mediators (such as appropriateness or efficacy of treatment received, quality of provider skills, or patient adherence) that impact health outcomes and equity of services. This model provides a useful starting point.

Bierman et al. described an alternative way of understanding access for vulnerable groups. In their model, barriers of access occur on 3 levels: 1) access to the health care system (trouble getting care, delay in care because of cost, and transportation problems); 2) structural barriers within the system (difficulty getting appointments, advice after hours, and completing referrals to specialists); and 3) the ability of the provider to address patients' needs (awareness of patients' conditions and functional limitations, knowledge and clinical skills, cultural competence).11

We have modified the Institute of Medicine's access model and incorporated factors described in the second and third levels of Bierman's access model to provide more specific directions for designing and implementing effective interventions to eliminate health care disparities. Our new model expands the scope of personal and structural barriers; specifies utilization measures to include the type of setting, provider, and procedure; incorporates provider communication skills, cultural competence, and bias or stereotyping behavior as measures of the quality of providers (a mediator in the original IOM model); and includes patient views of care or patient-centeredness (a component of health care quality from Crossing the Quality Chasm) as important outcome measures (Fig. 1).

FIGURE 1.

FIGURE 1

Barriers to and mediators of equitable health care for racial and ethnic groups. Modified from Access to Health Care in America.10

Specifically, we include a number of additional personal barriers in the modified version of the model. First, family structure may impact on individuals' ability and desire to seek health care services. Family members are often involved in medical decision making, especially for children, the elderly, and terminally or chronically ill patients; and a majority of patients prefer direct family involvement in their care.12 Second, patient preferences and expectations of treatment for depression, cardiovascular disease, and renal disease have been shown to differ by race and may impact upon use of health care services.1315 Third, patient involvement in medical decision making has been shown to impact processes such as physician information giving, patient adherence to therapeutic recommendations, and outcomes of care such as glycosylated hemoglobin and health-related quality of life.1619 Fourth, personal health behaviors, including exercise and smoking, that impact upon patients' outcomes are known to differ by race and ethnicity.20,21 Fifth, beliefs about health and disease and benefits of alternative or folk medicine differ across racial and ethnic groups and impact upon health care utilization.22,23 Finally, ethnic minority patients are more likely to have inadequate or marginal health literacy, a factor associated with worse health status and increased risk of hospitalization.2427

We also include in our refined version of the IOM model some barriers identified by Bierman as structural barriers within the system. For example, in addition to the availability of care, how care is organized, and transportation, we include difficulty getting any appointments at all with primary care physicians and specialists and timeliness of appointments in the refined model. Recent work shows that ethnic minority patients seen in primary care settings report more difficulty getting an appointment and waiting longer during appointments, even after adjustment for sociodemographic and health status characteristics.28

The Institute of Medicine's model includes a category for mediators. A mediator is a variable (intermediate, contingent, intervening, causal) that occurs in a causal pathway from an independent to a dependent variable. It causes variation in the dependent variable, and is also caused to vary by the independent variable. Such a variable is statistically associated with both the independent and dependent variables.29 We have expanded the quality of providers (a mediator between barriers and outcomes of care) to include technical skills, interpersonal/communication skills, medical knowledge, and cultural and linguistic competence.

Recent work shows that ethnic minority patients are often cared for by physicians with poorer indicators of technical quality, such as lower procedure volume rates and higher risk-adjusted mortality rates.30,31 Additionally, interpersonal care, or patient–provider communication, has been linked to a host of valued health outcomes as well as to patient satisfaction.16,17,32 Previous work shows ethnic minority patients, patients with poor health status, older patients, and patients with less than high school education rate visits with physicians as less participatory.33,34

Cultural and linguistic competence may be defined as the ability of health care providers and health care organizations to understand and respond effectively to the cultural and linguistic needs brought by patients to the health care encounter.35 At the patient–provider level, it may be defined as the ability of individuals to establish effective interpersonal and working relationships that supercede cultural differences.36 Recent work suggests that race concordance between health care providers and patients is associated with more partnership and higher levels of patient satisfaction, particularly for minority patients.34,37 While racial or ethnic concordance is not to be equated with cultural competence, studies of concordance between patients and providers with regard to race, ethnicity, gender, language, role expectations, and other factors may provide useful insights for researchers who are trying to operationalize the concept of cultural competence.38 Cultural competence is discussed in greater detail in the section of the paper devoted to factors associated with success.

Finally, in addition to health status and equity of services, patient views about health care, including their attitudes toward and experiences with care and satisfaction have emerged as important outcomes that may differ by race, ethnicity, social class, and language.34,37,3943

Interventions to eliminate racial and ethnic disparities might address a number of personal, structural, or financial barriers, mediators, or outcomes from our new model. For example, an intervention to eliminate racial disparities in cardiovascular procedure use might focus on patient preferences, patient–provider communication, and provider knowledge of treatment guidelines. An intervention to eliminate racial disparities in mental health care might target patient attitudes, such as stigma or fear of medications, primary care provider skills in recognition of mental health problems, and structural barriers such as the availability of mental health providers in primary care settings to facilitate collaborative care or treatment facilitators to improve coordination of care between primary care and mental health treatment settings.

Below, we describe different factors that should be considered when designing, implementing, and evaluating interventions to eliminate health disparities among racial and ethnic groups. These include the identification of target groups, appropriate study designs for evaluating interventions, limitations of previous interventions, success factors, summary recommendations, and conclusions, including issues specific to eliminating disparities and those broadly applicable to improving quality.

IDENTIFYING TARGET GROUPS FOR INTERVENTIONS

One of the most important aspects of designing an intervention is defining the target group. Interventions to reduce racial and ethnic disparities could target individuals in the general population, communities served by the health care systems being studied, patients with specific conditions (i.e., conditions with the greatest disease burden or disparity) or patients with specific characteristics (low literacy, poor disease control, low levels of involvement in care). Additionally, interventions could target individual health care providers' attitudes, knowledge, skills, and behaviors, specific settings (i.e., community health centers, rural or urban areas, primary care), or types of organizations (managed care, staff model or network style), or systems (Medicare or Medicaid, referral networks). Indicators of access, as defined in our model, may be used to help determine the appropriate target groups or settings for interventions. For example, if the access indicator of interest is cardiovascular disease care and outcomes, an intervention could target an entire community, individuals at risk (i.e., smokers, diabetics, hypertensives, those aged 40 and older, or those with a family history), or providers or systems serving these populations.

APPROPRIATE STUDY DESIGNS FOR EVALUATING INTERVENTIONS

Experimental research to eliminate ethnic minority health disparities is complex because of the complicated nature of interactions between barriers, service use, and mediators of care, and the additive and interactive impact of these factors on outcomes.44 Intervention studies to reduce or eliminate disparities should assess not only whether the intervention had an effect, but also the importance or magnitude of the changes caused and the relative effects of contributing factors on the outcome variable of interest.

Many researchers believe the best study design is a classic, true experimental design that uses random assignment of study subjects to experimental and control groups and measures all variables of interest pre- and post-intervention. Comparisons between control and experimental groups at baseline assure that randomization is effective, and comparisons between control and experimental groups at follow-up assess the impact of the intervention. In this design, most threats to internal validity (i.e., history and maturation, testing and instrumentation, statistical regression, selection, attrition, and contamination) are minimized.

Strategies such as randomization and using a sufficient sample size decrease the impact of selection bias and contamination. Measurement of key potential confounding variables, such as patient socioeconomic status, education level, health literacy, health insurance status, and availability of race- or language-concordant providers is essential for comparisons across experimental groups and analyses of the effectiveness of interventions in subgroups. The unit of analysis here might be the individual patient, a clinical practice, or an entire organization.

Nonetheless, even studies with true experimental designs are susceptible to threats to external validity. For example, there might still be an interaction effect of testing, an interaction between selection bias and the intervention, or multiple treatment interference. An interaction effect of testing occurs when a pretest increases or decreases the study participants' sensitivity or responsiveness to the experimental variable (intervention), rendering the results unrepresentative of the population that has not received any pretest. Interaction between selection bias and the intervention occurs when subjects selected are not representative of the population and may respond better or worse than the general population to the intervention. Multiple treatment interference occurs in subjects who have had prior exposure to similar interventions and/or who receive several components of an intervention.44

Strategies to address the challenges to external validity include: 1) performing the intervention in the type of health care setting in which such an intervention is likely to be translated into practice; 2) allowing the intervention to be flexible and to vary in level of intensity according to the needs of the individuals or settings being served; and 3) considering several outcomes to be potentially important indicators of success of the intervention.

When true experimental studies are not feasible, field experiments in real-life settings may be used. Field experiments usually have sufficient numbers to be nationally representative. The largest and the most influential health policy experiment to date was the so-called “Rand Study,” begun in 1971, in which over 7,700 participants in 6 sites were randomly assigned to health plans with different levels of cost sharing (deductibles and co-insurance) or to managed care organizations. The coverage levels ranged from free care to an income-related catastrophic plan (95% co-payment). The experiment was designed to test the theory that insurance and cost sharing affected demand for care, and to measure how different income levels and health status would be affected. The study found a clear connection between cost sharing and use, including inpatient use, with relatively little difference due to income categories. Later, some small adverse health effects were found for specific conditions in the lowest income level (hypertension and eye care), but not in any other groups.45

Another alternative to the randomized controlled trial is a natural or social experiment in which naturally occurring events or new social policies or programs are introduced, in which people have different exposures that researchers can document by collecting and analyzing data. An example of a natural experiment is the introduction of The State Children's Insurance Program that enables states to insure children with family incomes too high to qualify for Medicaid but too low to afford private health insurance in one state, the outcomes of which could be compared either before and after the program or to those in a similar state that did not initiate such a program. Field and natural experiments have high external validity.44

LIMITATIONS OF PREVIOUS INTERVENTION STUDIES

Reviews of interventions to eliminate health disparities have identified several types of limitations.46,47 First, in most studies, only a limited number of health determinants have been targeted. Second, many interventions have not been described in sufficient detail to be adapted or repeated in other settings. Third, a relatively small number of studies have been culturally tailored for the ethnic minority groups determined to be at risk for poor outcomes. Whether or not culturally tailored interventions provide benefits above and beyond generic quality improvement interventions is an empirical question for the field of disparities research. Fourth, many interventions, particularly those focused on changing provider behavior, have not used models that account for barriers to changing behavior. For example, recent work shows that physician barriers to using treatment guidelines include lack of awareness, familiarity, or agreement, lack of self-efficacy, lack of outcome expectancy, and inertia of previous practice, as well as external barriers related to environmental guideline–specific, or patient factors.4850 These barriers all prevent providers from changing their behaviors to adhere to practices documented to be effective for patients, and they need to be addressed in interventions to change provider behavior.

Many studies have used small samples and failed to use control groups. Additionally, resource input in these studies has not been measured or linked to outcomes. In several studies, standard and valid evaluation measures were not used. Moreover, the majority of studies have not included measures of health outcomes. Many of these interventions have had follow-up periods that were too short to determine whether “booster” interventions are needed to maintain the results obtained. Finally, few interventions have targeted environmental and social factors. Most have focused on health care services and health behaviors.

Additional barriers to conducting research in ethnic and racial minority populations include the need to address patient knowledge and attitudinal barriers including cultural mistrust, perceptions of exploitation, lack of perceived benefits to the minority community, and misunderstandings about actual research protocols and the purpose of informed consent.51,52 Investigator knowledge and attitudinal barriers regarding research regulations, lack of familiarity with minority communities, and cultural competence must also be addressed. Researchers suggest strategies such as use of community and family members to assist in recruitment, increasing interpersonal trust between primary care providers or researchers and patients, and the use of audiovisual materials in consent procedures to overcome cultural, linguistic, and literacy barriers that complicate the comprehension of written materials.52

FACTORS ASSOCIATED WITH SUCCESS

Systematic reviews identify several studies that have improved the health of disadvantaged groups.46,47 Below, we describe factors associated with success, exploring in greater detail two that we believe are particularly important for working with ethnic minority populations. Successful interventions have used intensive recruitment and follow-up approaches.5355 Yet, even intensive targeting does not always guarantee successful outcomes.56,57 Ensuring community commitment and incorporating input from community leaders and stakeholders have also led to success.58,59 For example, in one study, investigators from an academic institution joined with the leadership of an interdenominational organization of churches in the community to design a church-based smoking intervention in which community leaders and members were integrally involved in every aspect of the study. To be consistent with community activation models, the research staff acted only as facilitators and consultants in the smoking cessation program, offering technical assistance, training, and certification to lay volunteers. Community leaders and members participated in designing and conducting needs assessments, delivering the intervention, and evaluating its impact.58 Multidisciplinary investigator teams and multifaceted approaches are other strategies that have led to improved health for disadvantaged groups.60,61 Conducting a prior needs assessment has helped to define the areas for intervention,62 and using cultural tailoring and assuring the cultural appropriateness and competence of the interventionists have been critical to the success of some interventions.6365 For example, the use of culturally and linguistically appropriate written and audiovisual educational materials is often associated with successful outcomes, such as higher quit rates for African-American smokers and increased used of preventive services such as cervical cancer screening among Asian women in the United Kingdom.63,66 Finally, other successful programs used support materials67 and prompts and reminders to participants,68 and provided system-level support and resources to sustain the intervention after the study period.69,70

Ensuring Community Commitment and Partnership with Stakeholders

We believe it is important to involve stakeholders in the refinement of study design, implementation of interventions, and dissemination of research findings. For example, researchers might convene an advisory committee comprised of leaders from the community, government, advocacy groups, provider organizations, educational organizations, consumer groups, and academic institutions. These groups could convene on a regular basis to provide feedback to study investigators regarding study methods and settings, intervention goals and agents, measures used, and interpretation of findings. The members of these groups could also guide and advise investigators regarding dissemination efforts, and provide important linkages to the institutions and organizations they represent. Community–academic medical center partnerships are a very effective strategy for narrowing the gap in health status between minority and white populations. Many of these partnerships allow effective clinical trials to be incorporated into existing public health programs. They also facilitate the transfer and incorporation of new knowledge and advances in patient care to community health care settings, and promote long-term maintenance of interventions in the community.57,71

The Importance of Cultural Competence in Health Care and in Interventions Research

Increasing the acceptability and effectiveness of health care and health care interventions to individuals in a variety of racial, ethnic, and social class groups involves cultural awareness and competence in a broad sense as well as culture-specific knowledge, skills, and tailoring of interventions. Current efforts to define operational constructs for culturally and linguistically appropriate health care services (CLAS) include the establishment of the CLAS national standards, a collective set of mandates, guidelines, and recommendations issued by the Department of Health and Human Services (DHHS) Office of Minority Health, intended to inform, guide, and facilitate required and recommended practices related to culturally and linguistically appropriate services.35 Additionally, the DHHS has convened a National Advisory Committee to lay out the research agenda for CLAS. The CLAS national standards include suggestions for how the construct may be operationalized and measured.

Brach and Fraser have identified 9 cultural competency techniques: interpreter services, recruitment and retention policies, training, coordinating with traditional healers, use of community health workers, culturally competent health promotion, including family/community members, immersion in another culture, and administrative and organizational accommodations.72 We provide a definition of cultural competence earlier in the paper and group these techniques into broader strategies that are shown in Figure 2.36 While the concept of cultural competence is becoming clearer, there is still a need to operationalize and validate methods of measuring it.

FIGURE 2.

FIGURE 2

Components of cultural competence, adapted from Cooper and Roter, 2002.36

Other definitions of cultural competence have been proposed. For example, the National Center for Cultural Competence, a collaborative project between the Georgetown University Child Development Center and the Health Resources and Services Administration, defines cultural competence as a set of congruent attitudes, behaviors, and policies that come together in a system, agency, or among professionals that enables effective work in cross-cultural situations.73 Campinha-Bacote describes cultural competence as a process in which the health care provider continuously strives to achieve the ability to effectively work within the cultural context of a client, individual, family, or community.74 Lavizzo-Mourey and McKenzie describe cultural competence as the demonstrated awareness, inclusion, and integration of 3 population-specific issues in the delivery of health care: 1) health-related beliefs and cultural values, 2) disease incidence and prevalence, and 3) treatment efficacy.75 Finally, The American Medical Association, in its Cultural Competence Compendium, equates culturally effective care with patient-centered care, another term that has received increasing attention over the last several years.76 Patient-centered care is defined as health care that is closely congruent with and responsive to patients' values, needs, and preferences.77 Currently, we believe there is insufficient evidence to determine whether patient-centeredness is equated with or simply one dimension of cultural competence.78

With respect to the conduct of research, Resnicow et al. describe culturally sensitive interventions as having surface and deep structures.79 Surface structures include matching of intervention materials and messages to observable characteristics of the target population, identifying the channels and settings most appropriate for the delivery of the intervention, and incorporating interpersonal sensitivity and cultural competence. Deep structures, in contrast, involve identifying differences between the minority group being studied and the majority population in core cultural values, and identifying how ethnic, cultural, social, environmental, and historical factors influence the targeted condition or behavior. The authors submit that tailoring the intervention to surface structures increases the acceptability of the intervention; however, tailoring to deep structures is necessary to assure the sustainability of the intervention over time.

Another framework for designing culturally sensitive interventions has been developed by Bernal et al.80 In this model, 8 dimensions of treatment interventions are incorporated, including language, persons, metaphors, content, concepts, goals, methods, and context. “Language” refers to the need for linguistically appropriate materials and services. “Persons” refers to acceptability of the cultural background of the individuals delivering the intervention to the group receiving the intervention. “Metaphors” reflects the need for incorporating symbols or concepts shared by the group receiving the intervention; “content” refers to the need for culture-specific knowledge and handling of information about values, customs, and traditions in designing the intervention. “Concepts” refers to the need for cultural consonance in how constructs are used in the intervention. “Goals” indicates the need for agreement between patients or individuals being studied and the researchers. “Methods” refers to the need to use specific procedures or settings to achieve the research goals. Finally, “context” considers the social, economic, and political context of the health problem being studied across ethnic and cultural groups.

RECOMMENDATIONS

We would like to conclude with specific recommendations for researchers, practicing clinicians and health care managers, and public health administrators designing and evaluating interventions to eliminate health care disparities. We have made additions to recommendations made by Mackenbach and Gunning-Schepers regarding interventions to reduce socioeconomic health disparities in Europe81 (see Fig. 3).

FIGURE 3.

FIGURE 3

Guidelines for designing interventions to eliminate disparities in health. Modified from Mackenbach and Gunning-Schepers, 1997.81

Objectives

First, we recommend that the intervention have specific, measurable objectives. The objectives might clearly state whom the intervention will target, what the intervention will entail, the time frame of the intervention, what the desired outcomes will be, and how they will be defined. Second, our opinion is that the evaluation process should be able to determine the extent to which the intervention reduces differences between ethnic and racial groups. Third, interventions may focus on either health outcomes or intermediate outcomes that have shown clear causal relationships with health (e.g., use of certain preventive or therapeutic procedures, patient–provider communication).

Research Design

With respect to research design, we suggest that studies use experimental designs whenever possible and randomly assign target groups (individuals, patients, providers, systems, or communities) to experimental conditions. If experimental designs are not possible, field experiments and natural or social experiments may be used. Target groups might include those at highest risk for adverse outcomes, and interventions should try to address the most important contributing factors identified for a particular disease condition or population. We suggest that researchers guard against threats to internal and external validity, as previously discussed in the section of this paper devoted to study design.

Interventions

We suggest that interventions might have cultural and linguistic appropriateness as one of their objectives. Assuring the cultural sensitivity of interventions may be facilitated by incorporating patient and community input at all stages of the research design, implementation, and evaluation. Interventions that are multifaceted, intensive, and include face-to-face contact and interaction between interventionists and participants whenever possible are most likely to be successful.

Measurements

When measuring key variables, we recommend that individuals use individual-level data and standard methods (i.e., those used in federal data sources) for recording race, ethnicity, education, occupation, and income. We acknowledge that race-based categorizations may be falling out of favor; nonetheless, racial variables in research have helped to identify subgroups—particularly minority and immigrant groups—to whom additional health care resources need to be directed. Epidemiologists do not specifically endorse race-based categorizations; however, they suggest that racial variables that are used in research have a definite purpose that can be precisely articulated, that they meet the same standards of validity required of any other variables included in the research, and that the potential benefit of using such variables exceeds any potential harm that may result.82

Outcome variables may be derived from the study objectives and use validated methods and instruments. We recommend that the intervention schedule and resources take into account expected time lags between the intervention and desired outcomes as well as the importance of sustaining the effects of the intervention over time. Researchers might consider collecting process variables with respect to the intervention and appropriateness and quality of care in the intervention and usual care groups. For example, measures of cultural competence at the provider and organizational level (such as availability and quality of linguistic services or diversity training for staff) may be incorporated in interventions conducted in health care settings. Collecting information regarding costs of the intervention is highly desirable, so that others planning similar interventions can plan for resource allocation.

Dissemination

Finally, and perhaps most importantly, researchers might consider establishing and maintaining linkages with stakeholders, including payers, policy-makers, provider groups, patient and community organizations, and the media, to increase the likelihood of dissemination of the research findings.83 We suggest that they provide mentoring and training for future health services researchers who belong to underserved groups or who will contribute knowledge to understand and solve ethnic disparities in health.

CONCLUSIONS

Racial and ethnic disparities in health status and health care in the United States have been well documented. For some conditions (mental illness) and populations (children, certain immigrant groups. and ethnic subgroups), barriers to equitable health care have not been well described, and more descriptive and explanatory research is needed. However, for many other disease conditions (cardiovascular disease, cancer, preventive care, asthma, diabetes, HIV) and populations (the elderly, African Americans, and Latinos), research has revealed many potential targets for interventions. Yet, studies conducted to reduce racial and ethnic disparities in health care in the United States have had varying degrees of success. Interpretation of these studies is hampered by problems with study design, measurement of key variables, length of follow-up, clarity of interventions, and determination of resource use. More well-designed interventions with rigorous evaluation are needed.

Certain components of interventions to reduce disparities (collecting relevant and reliable data on race/ethnicity and socioeconomic status, operationalizing cultural competence, ensuring community commitment, achieving equity in care across racial and ethnic groups) are particularly important for racial and ethnic disparities, while other components (use of rigorous study designs and data collection methods, dissemination, incorporation of strategies to increase safety, effectiveness, patient-centeredness, timeliness, and efficiency) are broadly relevant to efforts to translate research into practice to improve quality of care.8 Whether interventions to improve economic and social conditions might play a larger role than health services interventions in reducing inequalities in health is another empirical question.

Health services interventions, whether used alone or in collaboration with social and economic interventions, are likely to play a significant role in reducing racial and ethnic health disparities. We believe these interventions should target high-risk populations, focus on the most important contributing factors for a given community, population, or disease condition, use culturally and linguistically appropriate methods, include measures of quality of care and health outcomes, and prioritize dissemination efforts.

Acknowledgments

The authors would like to thank Carolyn Clancy, MD, Arlene S. Bierman, MD, MS, the editors, and the anonymous reviewers for their valuable suggestions for improving the quality of this manuscript.

This work was supported by grants from The Commonwealth Fund (20010108) and the National Heart, Lung, and Blood Institute (R01HL6943-01) to Dr. Cooper. Dr. Powe is supported by a grant (K24DK02643) from the National Institute of Diabetes and Digestive and Kidney Diseases. Dr. Hill is supported by a grant (R01NR04119) from the National Institute of Nursing Research.

The views presented here are those of the authors and not necessarily those of the Commonwealth Fund, its directors, officers, or staff.

REFERENCES

  • 1.Andrulis DP. Access to care is the centerpiece in the elimination of socioeconomic disparities in health. Ann Intern Med. 1998;129:412–6. doi: 10.7326/0003-4819-129-5-199809010-00012. [DOI] [PubMed] [Google Scholar]
  • 2.Weissman JS, Stern R, Fielding SL, Epstein AM. Delayed access to health care: risk factors, reasons, and consequences. Ann Intern Med. 1991;114:325–31. doi: 10.7326/0003-4819-114-4-325. [DOI] [PubMed] [Google Scholar]
  • 3.Penchansky R, Thomas JW. The concept of access: definition and relationship to consumer satisfaction. Med Care. 1981;19:127–40. doi: 10.1097/00005650-198102000-00001. [DOI] [PubMed] [Google Scholar]
  • 4.Andersen RM, McCutcheon A, Aday LA, Chiu GY, Bell R. Exploring dimensions of access to medical care. Health Serv Res. 1983;18:49–74. [PMC free article] [PubMed] [Google Scholar]
  • 5. Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19–22 June, 1946; signed on 22 July 1946 by the representatives of 61 states (Official Records of the World Health Organization, No. 2, pg 100) and entered into force on 7 April 1948.
  • 6.Shi L. Health Services Research Methods. New York: Delmar Publishers; 1997. [Google Scholar]
  • 7.Fiscella K, Franks P, Gold MR, Clancy CM. Inequality in quality: addressing socioeconomic, racial, and ethnic disparities in health care. JAMA. 2000;283:2579–84. doi: 10.1001/jama.283.19.2579. [DOI] [PubMed] [Google Scholar]
  • 8.Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001. [PubMed] [Google Scholar]
  • 9.Benzeval M, Judge K, Whitehead M. Tackling Inequalities in Health: An Agenda for Action. London: King's Fund; 1995. [Google Scholar]
  • 10.Institute of Medicine. Access to Health Care in America: A Model for Monitoring Access. Washington, DC: National Academy Press; 1993. [Google Scholar]
  • 11.Bierman AS, Magari ES, Jette AM, Splaine M, Wasson JH. Assessing access as a first step toward improving the quality of care for very old adults. J Ambul Care Manage. 1998;21:17–26. doi: 10.1097/00004479-199807000-00005. [DOI] [PubMed] [Google Scholar]
  • 12.Holley JL, Stackiewicz L, Dacko C, Rault R. Factors influencing dialysis patients' completion of advance directives. Am J Kidney Dis. 1997;30:356–60. doi: 10.1016/s0272-6386(97)90279-1. [DOI] [PubMed] [Google Scholar]
  • 13.Cooper-Patrick L, Powe NR, Jenckes MW, Gonzales JJ, Levine DM, Ford DE. Identification of patient attitudes and preferences regarding treatment of depression. J Gen Intern Med. 1997;12:431–8. doi: 10.1046/j.1525-1497.1997.00075.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Whittle J, Conigliaro J, Good C, Joswiak M. Do patient preferences contribute to racial differences in cardiovascular procedure use? J Gen Intern Med. 1997;15:267–73. doi: 10.1046/j.1525-1497.1997.012005267.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Ayanian JZ, Cleary PD, Weissman JS, Epstein AM. The effect of patients' preferences on racial differences in access to renal transplantation. N Engl J Med. 1999;341:1661–9. doi: 10.1056/NEJM199911253412206. [DOI] [PubMed] [Google Scholar]
  • 16.Greenfield S, Kaplan SH, Ware JE, Jr, Yano EM, Frank HJL. Patients' participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med. 1988;3:448–57. doi: 10.1007/BF02595921. [DOI] [PubMed] [Google Scholar]
  • 17.Greenfield S, Kaplan S, Ware JE., Jr Expanding patient involvement in care. Ann Intern Med. 1985;102:520–8. doi: 10.7326/0003-4819-102-4-520. [DOI] [PubMed] [Google Scholar]
  • 18.Brody DS, Miller SM, Lerman CE, Smith DG, Caputo GC. Patient perception of involvement in medical care: relationship to illness attitudes and outcomes. J Gen Intern Med. 1989;4:506–11. doi: 10.1007/BF02599549. [DOI] [PubMed] [Google Scholar]
  • 19.Roter DL. Patient participation in the patient–provider interaction: the effects of patient question asking on the quality of interaction, satisfaction and compliance. Health Educ Monogr. 1977;50:281–315. doi: 10.1177/109019817700500402. [DOI] [PubMed] [Google Scholar]
  • 20.Young DR, Miller KW, Wilder LB, Yanek LR, Becker DM. Physical activity patterns of urban African Americans. J Community Health. 1998;23:99–112. doi: 10.1023/a:1018757307041. [DOI] [PubMed] [Google Scholar]
  • 21.Perez-Stable EJ, Marin G, Posner SF. Ethnic comparison of attitudes and beliefs about cigarette smoking. J Gen Intern Med. 1998;13:167–74. doi: 10.1046/j.1525-1497.1998.00051.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Brown CM, Segal R. The effects of health and treatment perceptions on the use of prescribed medication and home remedies among African American and white American hypertensives. Soc Sci Med. 1996;43:903–17. doi: 10.1016/0277-9536(95)00434-3. [DOI] [PubMed] [Google Scholar]
  • 23.Heurtin-Roberts S, Reisin E. The relation of culturally influenced lay models of hypertension to compliance with treatment. Am J Hypertens. 1992;5:787–92. doi: 10.1093/ajh/5.11.787. [DOI] [PubMed] [Google Scholar]
  • 24.Baker DW, Parker RM, Williams MV, Pitkin K, Parikh NS. The health care experience of patients with low literacy. Arch Fam Med. 1996;5:329–34. doi: 10.1001/archfami.5.6.329. [DOI] [PubMed] [Google Scholar]
  • 25.Baker DW, Parker RM, Williams MV, Clark WS. Health literacy and the risk of hospital admission. J Gen Intern Med. 1998;13:791–8. doi: 10.1046/j.1525-1497.1998.00242.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs AMA. Health literacy. Report of the Council on Scientific Affairs. JAMA. 1999;281:552–7. [PubMed] [Google Scholar]
  • 27.Gazmararian JA, Baker DW, Williams MV, et al. Health literacy among Medicare enrollees in a managed care organization. JAMA. 1999;281:545–51. doi: 10.1001/jama.281.6.545. [DOI] [PubMed] [Google Scholar]
  • 28.Shi L. Experience of primary care by racial and ethnic groups in the United States. Med Care. 1999;37:1068–77. doi: 10.1097/00005650-199910000-00010. [DOI] [PubMed] [Google Scholar]
  • 29.Last JM. A Dictionary of Epidemiology. 2nd ed. New York: Oxford University Press; 1988. [Google Scholar]
  • 30.Mukamel DB, Murthy AS, Weimer DL. Racial differences in access to high-quality cardiac surgeons. Am J Public Health. 2000;90:1774–7. doi: 10.2105/ajph.90.11.1774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Dardik A, Bowman HM, Gordon TA, Hsieh G, Perler BA. Impact of race on the outcome of carotid endarterectomy: a population- based analysis of 9,842 recent elective procedures. Ann Surg. 2000;232:704–9. doi: 10.1097/00000658-200011000-00013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Stewart MA. Effective physician-patient communication and health outcomes: a review. Can Med Assoc J. 1995;152:1423–33. [PMC free article] [PubMed] [Google Scholar]
  • 33.Kaplan SH, Gandek B, Greenfield S, Rogers W, Ware JE. Patient and visit characteristics related to physicians' participatory decision-making style. Results from the Medical Outcomes Study. Med Care. 1995;33:1176–87. doi: 10.1097/00005650-199512000-00002. [DOI] [PubMed] [Google Scholar]
  • 34.Cooper-Patrick L, Gallo JJ, Gonzales JJ, et al. Race, gender, and partnership in the patient-physician relationship. JAMA. 1999;282(6):583–9. doi: 10.1001/jama.282.6.583. [DOI] [PubMed] [Google Scholar]
  • 35. National Standards on Culturally and Linguistically Appropriate Services (CLAS) in Health Care Final Report, March 2001. http://www.omhrc.gov/omh/programs/2pgprgrams/finalreport.pdf. Accessed April 24, 2002.
  • 36.Cooper LA, Roter DL. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. A Report of the Institute of Medicine. Washington, DC: National Academy Press; 2002. Patient-Provider Communication: The Effect of Race and Ethnicity on Process and Outcomes of Health Care; pp. 336–80. [Google Scholar]
  • 37.Saha S, Komaromy M, Koepsell TD, Bindman AB. Patient-physician racial concordance and the perceived quality and use of health care. Arch Intern Med. 1999;159:997–1004. doi: 10.1001/archinte.159.9.997. [DOI] [PubMed] [Google Scholar]
  • 38.Sawyer L, Regev H, Proctor S, et al. Matching versus cultural competence in research: methodological considerations. Res Nurs Health. 1995;18:557–67. doi: 10.1002/nur.4770180611. [DOI] [PubMed] [Google Scholar]
  • 39.Blendon RJ, Scheck AC, Donelan K, et al. How white and African-Americans view their health and social problems. JAMA. 1995;273:341–6. [PubMed] [Google Scholar]
  • 40.Sun BC, Adams J, Orav EJ, Rucker DW, Brennan TA, Burstin HR. Determinants of patient satisfaction and willingness to return with emergency care. Ann Emerg Med. 2000;35:426–34. [PubMed] [Google Scholar]
  • 41.Gross DA, Zyzanski SJ, Borawski EA, Cebul RD, Stange KC. Patient satisfaction with time spent with their physician. J Fam Pract. 1998;47:133–7. [PubMed] [Google Scholar]
  • 42.Murray-Garcia JL, Selby JV, Schmittdiel J, Grumbach K, Quesenberry CP., Jr Racial and ethnic differences in a patient survey: patients' values, ratings, and reports regarding physician primary care performance in a large health maintenance organization. Med Care. 2000;38:300–10. doi: 10.1097/00005650-200003000-00007. [DOI] [PubMed] [Google Scholar]
  • 43.Carrasquillo O, Orav EJ, Brennan TA, Burstin HR. Impact of language barriers on patient satisfaction in an emergency department. J Gen Intern Med. 1999;14:82–7. doi: 10.1046/j.1525-1497.1999.00293.x. [DOI] [PubMed] [Google Scholar]
  • 44.Shi L. Health Services Research Methods. New York: Delmar Publishers; 1997. Experimental Research; pp. 143–63. [Google Scholar]
  • 45.Newhouse JP, Manning WG, Duan N, et al. The findings of the Rand Health Insurance experiment–a response to Welch et al. Med Care. 1987;25:157–79. doi: 10.1097/00005650-198702000-00009. [DOI] [PubMed] [Google Scholar]
  • 46.Gepkens A, Gunning-Schepers LJ. Interventions to reduce socioeconomic health differences. A review of the international literature. Eur J Pub Health. 1996;6:218–26. [Google Scholar]
  • 47.Arblaster L, Lambert M, Entwistle V, et al. A systematic review of the effectiveness of health service interventions aimed at reducing inequalities in health. J Health Serv Res Policy. 1996;1:93–103. doi: 10.1177/135581969600100207. [DOI] [PubMed] [Google Scholar]
  • 48.Tunis SR, Hayward RSA, Wilson MC, et al. Internists' attitudes about clinical practice guidelines. Ann Intern Med. 1994;120:956–63. doi: 10.7326/0003-4819-120-11-199406010-00008. [DOI] [PubMed] [Google Scholar]
  • 49.Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? JAMA. 1999;282:1458–65. doi: 10.1001/jama.282.15.1458. [DOI] [PubMed] [Google Scholar]
  • 50.Cabana MD, Ebel BE, Cooper-Patrick L, Powe NR, Rubin HR, Rand CS. Barriers pediatricians face when using asthma practice guidelines. Arch Pediatr Adolesc Med. 2000;154:685–93. doi: 10.1001/archpedi.154.7.685. [DOI] [PubMed] [Google Scholar]
  • 51.Gamble VN. A legacy of distrust: African Americans and medical research. Am J Prev Med. 1993;9(6 suppl):35–8. [PubMed] [Google Scholar]
  • 52.Corbie-Smith G, Thomas SB, Williams MV, Moody-Ayers S. Attitudes and beliefs of African Americans toward participation in medical research. J Gen Intern Med. 1999;14:537–46. doi: 10.1046/j.1525-1497.1999.07048.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Levine DM, Bone L. The impact of a planned health education approach on the control of hypertension in a high risk population. J Hum Hypertens. 1990;4:317–32. [PubMed] [Google Scholar]
  • 54.Hypertension Dectection and Follow-up Program Cooperative Group. Educational level and 5-year all-cause mortality in the hypertension detection and follow-up program. Hypertension. 1987;9:641–5. [PubMed] [Google Scholar]
  • 55.Tudor Hart J, Thomas C, Gibbons B, et al. Twenty-five years of case finding and audit in a socially deprived community. BMJ. 1991;302:1509–13. doi: 10.1136/bmj.302.6791.1509. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Marsh GN, Channing DM. Narrowing the health gap between a deprived and an endowed community. BMJ. 1988;296:173–6. doi: 10.1136/bmj.296.6616.173. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Hill MN, Bone LR, Hilton SC, Roary MC, Kelen GD, Levine DM. A clinical trial to improve high blood pressure care in young urban black men: recruitment, follow-up, and outcomes. Am J Hypertens. 1999;12:548–54. doi: 10.1016/s0895-7061(99)00007-2. [DOI] [PubMed] [Google Scholar]
  • 58.Stillman FA, Bone LR, Rand C, Levine DM, Becker DM. Heart, body, and soul: a church-based smoking-cessation program for urban African Americans. Prev Med. 1993;22:335–49. doi: 10.1006/pmed.1993.1028. [DOI] [PubMed] [Google Scholar]
  • 59.Voorhees CC, Stillman FA, Swank RT, Heagerty PJ, Levine DM, Becker DM. Heart, body, and soul: impact of church-based smoking cessation interventions on readiness to quit. Prev Med. 1996;25:277–85. doi: 10.1006/pmed.1996.0057. [DOI] [PubMed] [Google Scholar]
  • 60.Davidson LL, Durkin MS, Kuhn L, O'Connor P, Barlow B, Heagarty MC. The impact of the Safe Kids/Healthy Neighborhoods Injury Prevention Program in Harlem, 1988 through 1991. Am J Public Health. 1994;84:580–6. doi: 10.2105/ajph.84.4.580. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Dannenberg AL, Gielen AC, Beilenson PL, Wilson MF, Joffe A. Bicycle helmet laws and educational campaigns: an evaluation of strategies to increase children's helmet use. Am J Public Health. 1993;83:667–74. doi: 10.2105/ajph.83.5.667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Colver AF, Hutchinson PJ, Judson EC. Promoting children's home safety. BMJ. 1982;285:1177–80. doi: 10.1136/bmj.285.6349.1177. (Clin Res Ed) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.McAvoy BR, Raza R. Can health education increase uptake of cervical smear testing among Asian women? BMJ. 1991;302:833–6. doi: 10.1136/bmj.302.6780.833. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Freeborn DK, Mullooly JP, Colombo T, Burnham V. The effect of outreach workers' services on the medical care utilization of a disadvantaged population. J Community Health. 1978;3:306–20. doi: 10.1007/BF01498507. [DOI] [PubMed] [Google Scholar]
  • 65.Resnicow K, Royce J, Vaughn R, Orlandi MA, Smith M. Analysis of a multicomponent smoking cessation project: what worked and why. Prev Med. 1997;26:373–81. doi: 10.1006/pmed.1997.0157. [DOI] [PubMed] [Google Scholar]
  • 66.Orleans CT, Boyd NR, Bingler R, et al. A self-help intervention for African American smokers: tailoring cancer information service counseling for a special population. Prev Med. 1998;27:S61–S70. doi: 10.1006/pmed.1998.0400. [DOI] [PubMed] [Google Scholar]
  • 67.Hibbard BM, Robinson JO, Pearson JF, Rosen M, Taylor A. The effectiveness of antenatal education. Health Educ J. 1979;38:39–46. [Google Scholar]
  • 68.Planos R, Glenwick D. The effects of prompts on a minority children's screening attendance at a community mental health center. Child Fam Behav Therapy. 1986;8:5–13. [Google Scholar]
  • 69.Olds DL, Henderson CR, Kitzman H. Does prenatal and infancy nurse home visitation have enduring effects on qualities of parental caregiving and child health at 25 to 50 months of life? Paediatrics. 1994;1:89–98. [PubMed] [Google Scholar]
  • 70.Paris JMG, Player D. Citizen's advice in general practice. BMJ. 1993;306:1518–20. doi: 10.1136/bmj.306.6891.1518. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Levine DM, Becker DM, Bone LR, Hill MN, Tuggle MBI, Zeger SL. Community-academic health center partnerships for underserved minority populations. JAMA. 1994;272:309–11. [PubMed] [Google Scholar]
  • 72.Brach C, Fraser I. Can cultural competency reduce racial and ethnic health disparities. A review and conceptual model. Med Care Res Rev. 2000;57(suppl 1):181–217. doi: 10.1177/1077558700057001S09. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.National Center for Cultural Competence. Policy Brief 1—Rationale for Cultural Competence in Primary Health Care http://www.georgetown.edu/research/gucdc/nccc/nccc6.html. Accessed April 24, 2002.
  • 74.Campinha-Bacote J. A model and instrument for addressing cultural competence in health care. J Nurs Educ. 1999;38:203–7. doi: 10.3928/0148-4834-19990501-06. [DOI] [PubMed] [Google Scholar]
  • 75.Lavizzo-Mourey R, Mackenzie ER. Cultural competence: essential measurements of quality for managed care organizations. Ann Intern Med. 1996;124:919–21. doi: 10.7326/0003-4819-124-10-199605150-00010. [DOI] [PubMed] [Google Scholar]
  • 76.American Medical Association. Cultural Competence Compendium. Chicago, Ill: American Medical Association; 1999. [Google Scholar]
  • 77.Delbanco TL. Enriching the doctor-patient relationship by inviting the patient's perspective. Ann Intern Med. 1992;116:414–8. doi: 10.7326/0003-4819-116-5-414. [DOI] [PubMed] [Google Scholar]
  • 78.Gerteis M, Edgman-Levitan S, Daley J, Delbanco TL. Understanding and Promoting Patient-Centered Care. San Francisco: Jossey-Bass Publishers; 1993. Through the Patient's Eyes. [Google Scholar]
  • 79.Resnicow K, Baranowski T, Ahluwalia JS, Braithwaite RL. Cultural sensitivity in public health: defined and demystified. Ethn Dis. 1999;9:10–21. [PubMed] [Google Scholar]
  • 80.Bernal G, Bonilla J, Bellido C. Ecological validity and cultural sensitivity for outcome research: issues for the cultural adaptation and development of psychosocial treatments with Hispanics. J Abnorm Child Psychol. 1995;23:67–82. doi: 10.1007/BF01447045. [DOI] [PubMed] [Google Scholar]
  • 81.Mackenbach JP, Gunning-Schepers LJ. How should interventions to reduce inequalities in health be evaluated? J Epidemiol Community Health. 1997;51:359–64. doi: 10.1136/jech.51.4.359. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82.Gordis L. Epidemiology. 2nd ed. Philadelphia: W.B. Saunders; 2000. [Google Scholar]
  • 83.Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317:465–8. doi: 10.1136/bmj.317.7156.465. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of General Internal Medicine are provided here courtesy of Society of General Internal Medicine

RESOURCES