What Should We Include in a Cultural Competence Curriculum? An Emerging Formative Evaluation Process to Foster Curriculum Development

Abstract

Purpose

To identify, prioritize, and organize components of a cultural competence curriculum to address disparities in cardiovascular disease.

Method

In 2006, four separate nominal group technique sessions were conducted with medical students, residents, community physicians, and academic physicians to generate and prioritize a list of concepts (i.e., ideas) to include in a curriculum. Afterward, 45 educators and researchers organized and prioritized the concepts using a card-sorting exercise. Multidimensional scaling (MDS) and hierarchical cluster analysis produced homogeneous groupings of related concepts and generated a cognitive map. The main outcome measures were the number of cultural competence concepts, their relative ranks, and the cognitive map.

Results

Thirty participants generated 61 concepts, 29 were identified by at least 2 participants. The cognitive map organized concepts into four clusters, interpreted as: (1) patient’s cultural background (e.g.,, information on cultures, habits, values); (2) provider and health care (e.g., clinical skills, awareness of one’s bias, patient-centeredness, and professionalism), communication skills (e.g., history, stereotype avoidance, and health disparities epidemiology); (3) cross-culture (e.g., idiomatic expressions, examples of effective communication); and (4) resources to manage cultural diversity (e.g., translator guides, instructions and community resources). The MDS two-dimensional solution demonstrated a good fit (stress=0.07; R²=0.97).

Conclusions

A novel, combined approach allowed stakeholders’ inputs to identify and cognitively organize critical domains used to guide development of a cultural competence curriculum. Educators may use this approach to develop and organize educational content for their target audiences, especially in ill-defined areas like cultural competence.

The Institute of Medicine has suggested that a culturally-sensitive approach to care that avoids bias, prejudice, and stereotyping may help reduce health care disparities.¹ Multiple frameworks have been proposed to improve cultural competence among health care providers.^2–10 Cultural competence training has covered many different topics, has taken any number of different shapes, and has been shown to improve provider knowledge, attitudes, and skills, although patient outcomes are not frequently assessed.^11,12 However, no clear consensus is available for the organization of content that ought to be included in a cultural competence curriculum for physicians or physicians-in-training.

Processes more frequently employed in the social sciences have been useful in gathering and organizing data that can shape educational products. For instance, nominal group technique (NGT) ^13,14 offers advantages over traditional focus group discussions by allowing equal participation, avoiding participant’s inhibition in informal group discussion, avoiding focusing on one particular area, reducing apprehension from public speaking, generating a greater number of ideas, and minimizing inefficiencies and process loss.^15–17 NGT has been used in medical education to assess trainees’ perceptions of successful teaching rounds,¹⁸ to evaluate curricula,¹⁹ and to identify key attributes of an ideal continuing medical education (CME) institution to create a roadmap for excellence,²⁰ among others. In medical practice, NGT has been used to identify intervention strategies to improve osteoporosis treatment in nursing homes, ²¹ identify reasons for lack of medication adjustment in patients with seemingly uncontrolled hypertension, ²² to identify barriers for diabetes self-management, ¹⁷ and to achieve consensus among stakeholders in guidelines development.²³ Importantly, NGT does not limit participants to a pre-determined framework; rather, it forces participants to consider any and all options.

In addition, grouping methods allow the visual representation of mental processes, such as the data gathered during NGT, for instruction. The goal is to reveal the underlying cognitive dimensions, or “cognitive map,” that describes the spatial representation of how the items generated during NGT are related to each other and how ideas are prioritized within each cluster.^24–27 Card-sorting, hierarchical cluster analysis, and multidimensional scaling have been used to organize information and understand group similarities,^28–32 to understand consumer behavior,³³ to design the contextual navigation of Web sites,^28,34,35 to provide insight on how journals are cognitively organized,²⁶ and to understand motivation in schools.³⁶ However, to our knowledge, such methods have not been used in medical education research.^18,37

We set out to inform the development and organization of a cultural competence curriculum that would increase health care professionals’ knowledge of health disparities and fundamental issues of cultural competence as a step in enhancing cardiovascular care for minority groups. We did so by engaging groups of relevant stakeholders (including cultural competence experts, physicians, and physicians-in-training) in a robust, formative assessment to identify, prioritize, and organize the critical components of cultural competence.²² The combined approach we describe is a novel application in medical education research; the qualitative and quantitative approach is specifically designed to guide the process of content development and organization of information.

Method

We report below the development of the Cultural Competence Curriculum program, an Internet-based educational intervention for physicians and physicians in training aimed at reducing health disparities in cardiovascular health.

Information gathering: NGT

First, we used NGT to generate and prioritize a list of concepts deemed important to include in the curriculum. As explained earlier, NGT is a highly structured, small-group-meeting approach used to elicit and organize answers to a specific question.¹⁸ NGT involves the following steps: (1) silent, written generation of responses to a specific question; (2) round-robin recording of ideas; (3) discussion for clarification only; and (4) voting on the rank order of the top three ideas, usually with respect to importance, feasibility, or other characteristics. (These steps are described in much greater detail elsewhere ¹⁸). Figure 1 is a graphical description of the question development, the NGT groups, and how the responses were processed to the question that was developed.

Figure 1.

A graphical representation of the question development, the nominal group technique groups, and how the responses to the question that was developed were processed. The study goal was to identify, prioritize, and organize components of a cultural competence curriculum that could address disparities in care for cardiovascular disease.

Question development

To formulate the central question for our NGT sessions, we convened a panel of clinician-educators, nurses, minority community physicians, minority health researchers, health services researchers, social scientists, and distance-learning experts. The panel reviewed the literature and composed and pilot-tested the question among medical students, faculty members, and a community physician.

NGT sessions

Between February and March 2006, we conducted four separate NGT sessions with medical students (n = 9), internal medicine residents (n = 7), primary care community physicians (n = 7), and disparities or educational researchers (n = 7). The small sample size was by design and is justified, given NGT methodology where the process allows for idea saturation quickly with smaller numbers of participants, provided the demographic composition of the groups broadly represents the target end-user. We tried to have participants who represented a mixture of age, gender, and ethnicity; however, by design, we did not ask or collect information on such characteristics, since such inquiries could have influenced the responses during the NGT sessions.

We recruited two convenience samples of fourth-year medical students and internal medicine house officers from The University of Alabama at Birmingham, Birmingham, Alabama. We recruited community physicians from the Alabama Practice-Based CME Network, a group in regular communication with the academic medical center, for purposes of exploring educational need in the community and easy delivery of educational materials and activities addressing those needs. Participants were recruited via fax and awarded a $100 honorarium. Finally, we recruited investigators funded by the National Heart, Lung, and Blood Institute.³⁸ NGT participants were informed that the purpose of the session was to generate their ideas concerning curricular content, not format, for a cultural competence program focusing on cardiovascular disease risk and treatment; the program would be available to medical students, residents in training, and practicing physicians.

The final question formulated by the panel and posed to NGT participants was “What sorts of things could be included in a curriculum that focuses on cultural competence training for physicians?” Once all responses to the question had been given, each group was asked to identify the three most important ideas and the three most feasible ideas that the responses contained. Experienced facilitators conducted the sessions.

Organizing the ideas: card sort

The ideas from the NGT sessions that received at least 2 votes as “most important” or “most feasible” were then organized by a group consisting of a convenience sample of 45 educators and disparities researchers. These participants physically grouped and ranked the ideas, which were written on index cards. These educators and researchers hailed from 19 cities and 14 states in the United States. There were 29 general internal medicine physicians, one pediatrician, three specialists, four nurses, four PhDs, and four from other professions. Of these participants, 26 were women, 19 were men; there were 24 Caucasians, 8 African Americans, 6 Hispanics, 3 Asians, 3 unknown, and 1 “other.”

Each participant worked independently for 10–20 minutes to

• examine the cards together as a deck,

• sort them into piles based on their own perceptions or criteria for how they should be grouped,

• create at least two but no more than 10 piles,

• place cards that were not in some way related to each other into a pile labeled “cards cannot be grouped,”

• note whether two or more ideas reflected the same concept, and

• make no attempt of assigning a label to each pile.

Last, within each pile, participants ranked the ideas, again based on their own criteria, by placing the most important idea at the top of the pile and the least important idea at the bottom of the pile. We did not provide an incentive or honoraria.

Information analysis: cognitive mapping

To analyze the results of the card sort, we used multidimensional scaling (MDS) and hierarchical cluster analysis (HCA).²² The goal of MDS is to reveal the psychological dimensions in the data that can meaningfully describe the underlying cognitive constructs. The result, a “cognitive map,” is a spatial representation of how the ideas are considered to be similar to or different from each other.^26,39 The coordinates defining each item’s position from the MDS analysis are used as input data in the cluster analysis. The goal of HCA is to facilitate the interpretation of the MDS map by dividing a set of objects into a smaller number of homogeneous groups on the basis of their similarity.⁴⁰ Enclosed within lines, the smaller homogeneous groups represent “families,” and clusters of groups represent “neighborhoods.” ⁴¹ The map in our study reflected how the card sorters cognitively organized the areas to include in the cultural competency training curriculum for physicians.

The data from the card-sorting task were first used to create a matrix for each participant. This co-occurrence matrix indicated the number of times each item was sorted with all other items (29×29 matrix). The individual co-occurrence matrices were then aggregated across all participants to form a group co-occurrence matrix. This matrix indicated the number of times all participants sorted each reason into similar or dissimilar piles. The group co-occurrence matrix was then analyzed with MDS (using the ALSCAL algorithm) and with cluster analysis to derive the cognitive map. This iterative process derived an optimal geometric solution that reflects the proximity of the data and the relative ordering from the distances observed in the matrix.²⁴

The number of dimensions in an MDS solution is determined by its interpretability and analysis. Perhaps the most important is the interpretability. A two-dimension solution is usually recommended because additional dimensions make interpretation more difficult.⁴¹ MDS analysis results in the R-square (RSQ) and stress statistics. The RSQ statistic reflects the extent to which the map corresponds to the actual proximity data, and values of RSQ >.90 suggest high correspondence. The stress statistic reflects the “badness of fit,” with acceptable stress values being >.01 and <.20 (<.01 indicates a degenerate solution, >.20 indicates poor fit). ^40,41

Although MDS and cluster analysis are computationally sophisticated, they are not based on parametric statistics. Consequently, their validity is not sample-size dependent, but dependent on the representativeness of participants in the sorting task. While the location of items on the map is analytically determined, the meaning of the dimensions (i.e., the two axis labels) must be interpreted. We used a standard approach to interpret the derived dimensions by engaging the entire research team and by examining the substantive differences between items at the extremes of the dimensions.

The Institutional Review Board at the University of Alabama at Birmingham approved this study.

Results

The results of the nominal group technique (NGT) and the cognitive mapping are shown in Table 1 and Figure 2.

Table 1.

Organization of 29 Ideas Generated by Participants in Four Independent Nominal Group Technique (NGT) Sessions ^*, ^†

Elements of cognitive map		The NGT sessions: ideas generated, participants, ratings of ideas
Neighborhood	Cognitive map number for the idea	Ideas listed by family	Participant type	Rating	Sumof ratings; (overall sum of ratings)
Patient’s background
Cultural background (religion, customs, moral values)
	3	Identify conflicts between major world religious beliefs and principles of Western medicine (I)	MS4	3,2	5
	7	Include examples of customs that are commonly encountered in different cultures (I)	Res	3,2	5
	24	Include content regarding values held by different cultures (F)	Res	2,2	4
	16	Provide content describing the cultural differences regarding views of marriage, sexuality, and abortions (I)	Comm	2,1	3
Culture and health (religion and cultural impact, behavior and adherence)
	15	Provide information on religious principles of particular cultures and how these might influence medical care (I)	Comm	2,1	3
	10	Include practical examples of cultural differences that have the potential to influence medical care (I)	Res	3,1	4
	13	Describe habits that are unique to the culture and how these habitsmay relate to non-compliance/ adherence (i.e. dietary behaviors) (I)	Comm	3,2,2	7
	27	Include modules indicating cultural variability concerning the nature of the different culture’s views about approaching medicine (F)	Comm	2,1,1	4
Social background (health patterns, CAM, geographic)
	14	Develop social science content, reflective of different cultures views of the roles of patients, caregivers, and physicians (I)	Comm	3,3	6
	12	Present objective data on the cultures of various patient population groups, including different health patterns and the factors that influence the use of health services (I)	Comm	3,3,1	7
	25	Provide information on herbal and folk remedies popular in different cultures (F)	Comm	2,2	4
	26	Include information that addresses regional differences in cultures and ethnicity (F)	Comm	3,1	4 (56)
Provider/health care system
Clinical skills (own bias self-awareness, patient-centered, professionalism)
	17	Include content to help learners become aware of own biases and attitudes and how these influence patient care (I)	CER	3,2,2,2	9
	23	Address issues fundamental to baseline professionalism (F)	MS4	3,2	5
	20	Include basic content emphasizing patient-centered care (I)	CER	3,1	4
Communication skills (history, stereotype avoidance)
	1	Identify critical questions that would permit taking cultural history (i.e. focusing on having physicians develop greater patient-centeredness) (I)	MS4	3,1,1	17
	21	Include exercises to help “stereotype avoidance” without teaching about stereotypes (I)	CER	2, 1	3
Health disparities epidemiology (prevalence, causes, medications)
	4	Provide epidemiological data regarding disease prevalence of different race/ethnic groups (I)	MS4	2,2	4
	19	Provide info re: disparities that exist in CV care --reasons for these disparities (I)	CER	3,3	6
	2	Identify content showing how pharmacological treatments might be tailored to specific ethnic groups(I)	MS4	3,2,1,1	7 (55)
Cross-culture
Cross-cultural communication (idioms, barriers, examples)
	28	Provide information regarding variability in indigenous vernacular, colloquiums, and idioms across ethnicities (F)	Comm	2,1	3
	9	Present information that addresses common barriers that prevent patients from following doctor’s recommendations (I)	Res	2,2,1	5
	18	Present real life examples as case vignettes in the form of interactive video (I)	CER	3,3	6
Cross-cultural resources
	29	Include resources/info about where physicians can go to get information regarding broad issues of cultural competencein medicine (F)	CER	3,2,2,1	8
	11	Provide links to resources that have information regarding the characteristics of different cultures (I)	Res	2,1	3 (25)
Resources		Translator guides, instructions
	5	Develop language translation guides for use in practice (I)	MS4	2,1	3
	8	Provide a list of available translator services (I)	Res	3,2	5
	6	Provide access to resources to ensure that patients and their families comprehend physician and medical instructions (I)	Res	3,3	6
Community resources
	22	Develop a list of community resources relevant to culturally diverse patients (F)	MS4	3,3,2,1	9 (23)

^*Ideas were generated by 30 medical students, residents, and physicians in response to the question: “What sorts of things could be included in a curriculum that focuses on cultural competence training for physicians?” The 29 ideas shown in the table were selected by at least two of 45 educators and researchers. Multidimensional scaling (MDS) and hierarchical cluster analysis produced homogeneous groupings of related concepts and generated a cognitive map. The table presents the ten small groups, or families, of cognitively most-similar ideas, each within four larger groups, or neighborhoods, of cognitive similarity. Ratings indicate the most (3), next-to-most (2), and least (1) important ideas. The higher the sum of the ratings, the greater the idea’s perceived importance. Most of the feasible ideas were not considered the most important ideas, and vice versa. The ideas are listed by the same cognitive map numbers in both this table and Figure 2.

^†I = important, F = feasible, CAM = complimentary and alternative medicine, CER = clinician educators and researchers, Comm = community physicians, Res = internal medicine residents, MS4 = fourth-year medical students.

Figure 2.

Representation of ideas, “families,” and “neighborhoods” to include in a cultural competence curriculum. The x-axis is a dimension of patient centeredness (Patient Centrism) with a neighborhood of issues more intrinsic to the patient (Patient’s Background, on the left side of the figure) and a neighborhood of issues more extrinsic to the patient (Resources, on the right side). Similarly, the y-axis is a dimension of health-care centeredness (Provider and Health- System Centrism) with a neighborhood of issues more intrinsic to health care (Provider/Health Care, on the upper portion of the figure) and a neighborhood of issues more extrinsic to health care (Resources and Cross-Culture, on the lower portion of the figure). “Resources” stands for resources to manage cultural diversity (translator guides, Instructions, and community resources). “Cross-Culture” indicates such things as idiomatic expressions and examples of effective communication. For the statements of the ideas (1–29) see Table 1.

Nominal group technique

During the 4 independent NGT sessions, a total of 61 ideas were generated. Twenty-nine of these ideas were selected by at least 2 participants as most important or feasible; they are all listed in Table 1. The other 32 ideas were discarded and not incorporated into the resulting Web-based program’s content (a list of these discarded ideas is presented in Supplemental Digital Table 1, which may be found at http://----------------------------). [INSERT LINK HERE] As shown in Table 1, the top-rated idea among experts was to “include content to help learners become aware of [their] own biases and attitudes…” (item 17, sum of the ratings = 9); among community physicians the two top-rated ideas were “describe habits that are unique…” (item 13, sum of the ratings = 7), and “present objective data on the cultures …” (item 12, sum of the ratings = 7); among residents it was to “provide access to resources…” (item 6, sum of the ratings = 6); and, among medical students it was to “identify critical questions that would permit taking cultural history …” (item 1, sum of the ratings = 17). Table 1 shows the organization of the 29 ideas selected by at least 2 participants as most important or feasible, classified by family (i.e., small group).

Organization and analysis of ideas: the cognitive map

The 45 clinician-educators and health disparities researchers organized the 29 ideas in a median of 3 piles (range 1–11); each pile had a median of 6 ideas (range 3–10).

The multidimensional scaling (MDS) and hierarchical cluster analysis revealed two possible solutions, on two or three dimensions. Although the three-dimension solution better explained the data than did the two-dimension solution (higher RSQ, 0.97 vs. 0.91; lower stress measure, 0.07 vs. 0.15), we present here the two-dimension solution, as it was the most parsimonious and interpretable.

The cluster analysis revealed ten small groups, or families, of cognitively most-similar items, each within four larger groups, or neighborhoods, of cognitive similarity. The decision to interpret a ten-cluster solution was informed by examining the pattern of the agglomeration coefficients and by visually inspecting the two-dimension plot and the dendrogram to determine the sequence of cluster formation. The cognitive map resulting from the MDS and cluster analysis is shown in Figure 2.

We propose that the four larger groups can be viewed as neighborhoods of related themes: (1) patient’s cultural background (i.e., information on cultures, habits, values); (2) provider and health care (e.g., clinical skills, awareness of one’s bias, patient-centeredness, and professionalism), communication skills (e.g., history, stereotype avoidance, and health disparities epidemiology); (3) cross-culture (e.g., idiomatic expressions, examples of effective communication); and (4) resources to manage cultural diversity. These neighborhoods are shown in Table 1 and Figure 2, which give a fuller picture of them than could be presented in this paragraph. We base our interpretation of the mapping results on the substantive orientation and thematic consistency of the ideas within these clusters of families and among neighborhoods.

The patient’s background neighborhood includes three distinct families: cultural background (e.g., religion, customs, moral values), culture and health (e.g., impact of religion and culture behavior and adherence to cultural standards), and social background (e.g., health patterns, views on complementary and alternative medicine, region of the country).

The “provider and health care neighborhood includes three families: clinical skills (e.g., awareness of one’s bias, patient-centered, professionalism), communication skills (e.g., history, stereotype avoidance), and health disparities epidemiology (e.g., prevalence, causes, medications).

The cross-culture neighborhood includes two families: cross-cultural communication (e.g., idiomatic expressions, barriers, effective communication examples) and cross-cultural resources.

The “resources to manage cultural diversity neighborhood consists of two families: translator guides/instructions and community resources.

In Figure 2, we conceptualize the x-axis as a dimension of patient centeredness (Patient Centrism) with a neighborhood of issues more intrinsic to the patient (Patient’s Background, on the left side of the figure) and a neighborhood of issues more extrinsic to the patient (Resources, on the right side). Similarly, we conceptualize the y-axis as a dimension of health-care centeredness (Provider and Health- System Centrism) with a neighborhood of issues more intrinsic to health care (Provider/Health Care, on the upper portion of the figure) and a neighborhood of issues more extrinsic to health care (Resources and Cross-Culture, on the lower portion of the figure).

We arrived at these interpretations by examining the ideas at the extremes on each axis to determine similarities and differences. For instance, with respect to the dimension of patient centeredness (the Patient Centrism axis), differences between the two items farthest apart were relatively obvious. Item 8 (“Provide a list of available translator services”) is not under the control of the patient or would be the same across all patients. Item 15 (“Provide information on religious principles of particular cultures and how these might influence medical care”) is very individualized to the patient and not the same from patient to patient. However, this approach was more difficult in the case of the dimension of health care centeredness (the Provider/Health-System-Centrism axis), where the variance between extremes was not as great. Consequently, we had to examine other items near the extremes to find meaning. Ultimately, we concluded that the items near the extremes on this axis also related to the level of control of the physician or health care system.

Importantly, the Patient’s Background and the Provider/Health Care neighborhoods seem to be the most important, as illustrated by the higher overall sums of ratings of 56 and 55, respectively, compared with the Cross-Culture or Resources neighborhoods, with sums of ratings of 25 and 23, respectively.

Discussion

As explained earlier, in a formative evaluation of concepts important and feasible for inclusion in a cultural competence curriculum, medical students, residents, community physicians, and academic physicians identified and organized 29 ideas along the two axes of patient centeredness and provider/health-care-system centeredness (illustrated in Figure 2). The resulting cognitive map was interpreted to include four neighborhoods of concepts: patient’s background, provider/health care, cross-culture, and resources to manage cultural diversity; each of them contained a family of ideas related to the neighborhood’s concept. We extended the methods and analytical approach from the social sciences to effectively outline a curriculum for cultural competence training. Our goal with this process was not to identify new areas of content in cultural competence (although in our study, religion emerged as an important dimension not specifically defined in prior frameworks⁴²), but rather to understand in a more structured fashion the ways in which target groups conceptually framed and organized existing cultural competence content.

Implications for curriculum development and organization

The organization and prioritization of information is critical for mental processing, learning, and problem solving, i.e., learning with understanding. The concept of learning with understanding is fully supported by a body of evidence summarized in the National Research Council’s seminal document Brain, Mind, Experience, and School.⁴³ Learners gain a deeper understanding of a theme when they organize ideas, understand facts and ideas in the context of a conceptual framework, and organize knowledge in ways that facilitate retrieval and application.

An approach similar to ours can guide the process of content development, which includes the identification of learning needs as well as organization and prioritization of the content. Our study does not address which modality is best suited for curriculum delivery. However, the product of this formative assessment was, in fact, an online curriculum that our institution created entitled “Cultural Competence Online for Medical Practice (CCOMP)” (www.c-comp.org), which is available in MedEdPortal.⁴⁴ Within the Web site, teaching points are organized according to the dimensions and neighborhoods we discovered through this process. Furthermore, the overall importance of the neighborhoods (expressed as the overall sums of ratings) also informed the Web design. The size of the visual display of the content of the Patient’s Background and the Provider/Health Care neighborhoods was twice as large as the Cross-Culture or Resources neighborhoods. Though we focused on the topic of cultural competence, a similar strategy may work well for others developing curriculum in other topic areas, especially in areas that are ill-defined. Similarly, cognitive mapping can allow the inclusion of perspectives of critical stakeholders such as patients, providers, or health care systems.

We noted significant differences among groups with regard to the ideas rated highest in importance, perhaps stemming from the group members’ own skills at their individual career stages. However, we recognize these differences as potential strengths for curriculum developers. These differences present an opportunity for tailoring content to particular learners or groups of learners based on their professional level. Their unique responses may also provide a foundation from which to review, refocus, or update materials, especially as culture shifts, people change, the health care system evolves, and resources become more or less available.

Uses of the knowledge gained in this study

The use of the NGT in curriculum development is emerging in medical education; however, the use of cognitive mapping methods is a novel application in medical education research. Building from the body of knowledge from this study, our group has used cognitive mapping methods, multidimensional scaling and hierarchical cluster analysis as key processes in medical education.^18,45 The attributes that contribute to successful ward-attending teaching rounds identified by Castiglioni et al ¹⁸ were organized using cognitive mapping methods from the faculty perspective ⁴⁵ and from the medical students’ and residents’ perspectives ³⁷ both published in abstract form. For example, Salanitro et al found that faculty grouped the attributes of successful rounds into six unique clusters, which were learning climate, explaining the clinical rationale, respect, managerial skills, and setting expectations.⁴⁵ Roy et al reported that trainees grouped the same attributes into five unique clusters, which were role modeling, learning environment, teaching process, team management, and expectations.³⁷ The results of those two studies are guiding an ongoing faculty development program. A review of cognitive mapping methods in the social sciences is available elsewhere.^25,27

Limitations

We acknowledge that there may be other, even more meaningful, interpretations of our cognitive map that we did not consider. Though we convened a multidisciplinary group, the sample was relatively small and may not be representative for all U.S. trainees or practicing physicians, and there were certainly perspectives that were not brought to the table or may have been excluded from our expansive discussions. However, our data show great diversity of ideas among groups and sufficient overlap to conclude that we reached some level of saturation. We felt that the final agreed-upon interpretation of the cognitive map best embodied the purpose and outcomes of our formative method.

Conclusion

Curriculum development is difficult, particularly in an area like cultural competence, where multiple frameworks exist. In this study, we set forth one approach for engaging learners, elucidating important topical domains, reaching some level of consensus, and organizing the information in a way that is both logical and helpful in curriculum development in cultural competence. Our approach illustrates one method, technique, or process that may be used by educators to identify content for initial development in any number of clinical or practice areas. It can also be used to revisit or reconsider that content over time for purposes of update or revision in light of new advances or improvements in care.