Tools to Assess Behavioral and Social Science Competencies in Medical Education: A Systematic Review

Abstract

Purpose

Behavioral and social science (BSS) competencies are needed to provide quality health care, but psychometrically validated measures to assess these competencies are difficult to find. Moreover, they have not been mapped to existing frameworks, like those from the Liaison Committee on Medical Education (LCME) and Accreditation Council for Graduate Medical Education (ACGME). This systematic review aimed to identify and evaluate the quality of assessment tools used to measure BSS competencies.

Method

The authors searched the literature published between January 2002 and March 2014 for articles reporting psychometric or other validity/reliability testing, using OVID, CINAHL, PubMed, ERIC, Research and Development Resource Base, SOCIOFILE, and PsycINFO. They reviewed 5,104 potentially relevant titles and abstracts. To guide their review, they mapped BSS competencies to existing LCME and ACGME frameworks. The final, included articles fell into three categories: instrument development, which were of the highest quality; educational research, which were of the second highest quality; and curriculum evaluation, which were of lower quality.

Results

Of the 114 included articles, 33 (29%) yielded strong evidence supporting tools to assess communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence and 19 (17%) weak evidence. Articles mapped to all LCME standards and ACGME core competencies; the most common was communication skills.

Conclusions

These findings serve as a valuable resource for medical educators and researchers. More rigorous measurement validation and testing and more robust study designs are needed to understand how educational strategies contribute to BSS competency development.

In a 2004 report, the Institute of Medicine (IOM) concluded that, although 50% of the causes of premature morbidity and mortality are related to behavioral and social factors, medical school curricula in these areas are insufficient.^1–3 The behavioral and social science (BSS) domains that the IOM deemed critical in their report included: (1) mind-body interactions in health and disease, (2) patient behavior, (3) physician role and behavior, (4) physician-patient interactions, (5) social and cultural issues in health care, and (6) health policy and economics.¹ Within these six domains, the IOM identified 26 high priority topics, such as health risk behaviors, principles of behavior change, ethics, physician well-being, communication skills, socioeconomic inequalities, and health care systems design.¹ The Association of American Medical Colleges (AAMC) similarly identified core BSS content areas and connected them with other educational frameworks, including the Canadian Medical Education Directions for Specialists (CanMEDS) competency framework and the Accreditation Council for Graduate Medical Education (ACGME) core competencies.⁴

In addition, the Liaison Committee on Medical Education (LCME) incorporates, as part of its educational program requirements for accreditation, BSS domains⁵ and requires that schools identify the competencies in these areas that both the profession and the public can expect of a practicing physician. Medical schools must use both content and outcomes-based assessments to demonstrate their learners’ progress toward and achievement of these competencies. To do so, many schools use the broad ACGME core competencies--professionalism, medical knowledge, patient care, interpersonal skills and communication, systems-based practice, and practice-based learning and improvement.⁶ Within these six categories, BSS competencies are nested among other milestones intended to mark learners’ progression toward knowledge and skill acquisition. At present, no fully articulated, standardized list of BSS competencies exists, nor has there been a cross-translation of the LCME standards, the IOM-defined BSS domains, and the ACGME core competencies.

This lack of standardization makes it difficult to pool evaluation data collected across medical schools, which could help evaluate the effectiveness of different training models or instructional designs for BSS curricula. Moreover, determining the levels of achievement of entrustable professional activities or milestones⁷ as well as conducting rigorous educational research require that measures of competency development are validated. However, often this important step is skipped entirely, not fully completed, or lacks the rigor needed to produce reliable results. Given the breadth of the competency assessment literature and the existence of contradictory or incomplete findings, a systematic review of published work will be valuable to educators as well as administrators seeking to satisfy the LCME standards and instruct their learners in the ACGME core competencies.

Thus, we conducted a systematic review to identify and evaluate the quality of the assessment tools used to measure BSS competencies. Studies were classified by article type and quality. The strongest assessment tools were mapped to both the IOM-defined BSS domains and to the BSS-relevant LCME standards and ACGME core competencies. Our findings can guide educators and educational researchers to both validated instruments for assessing BSS competencies in learners and the best evaluation designs and educational strategies to determine what may be needed in future educational efforts.

Method

Guiding principles

We used the Best Evidence Medical and Health Professional Education Guide⁸ in our systematic review. As such, we created two review groups, one to conduct the actual review (P.A.C., R.T.P., M.F.M., E.K.T.) and a second to act as a wider authorship and editorial advisory group (S.E.E., D.K.L., F.E.B., C.R.T., A.L., J.M.S.). We next specified our research question: What valid and reliable instruments have been developed to assess learner (medical student and resident) competencies specifically related to the social and behavioral sciences? We considered instruments that may be applicable to other health professions learners as well. Subsequently, we identified a practical, conceptual framework to identify those competencies specifically related to the social and behavioral sciences that would be of the greatest utility to educators and administrators. To accomplish this step, we analyzed the LCME accreditation requirements,⁵ which are divided into five sections: (1) institutional setting (e.g., governance and organizational environment); (2) educational program for the MD degree (e.g., objectives, learning environment and approach, structure in design and content); (3) medical students (e.g., student demography, admissions, student services); (4) faculty (e.g., qualifications, personnel, organization and governance); and (5) educational resources (e.g., faculty background and time, finances and facilities). As quality assessments of BSS competencies are needed in graduate medical education as well, we also included the ACGME core competencies (professionalism, medical knowledge, patient care, interpersonal skills and communication, systems-based practice, and practice-based learning and improvement) in the development of our conceptual framework.

To focus our review, we selected components from the LCME’s Section II: Educational Program for the MD Degree (ED) and focused specifically on educational content. (The LCME standards provided more detail than the ACGME milestones, and thus we relied heavily on the LCME verbiage as we refined our review.) We reviewed each of the content areas (ED-10 through ED-23), to identify those most relevant to the six IOM-defined BSS domains. Of the 13 possible components, we selected six BSS-relevant curriculum requirements (ED-10, ED-19 through ED-23) and three BSS-relevant integrative program requirements (ED-13 through ED-15), which provided the conceptual framework and core search terms for our literature review (see Table 1). We then weighted each selected LCME standard using a consensus process that included all authors but two (W.J.H., R.T.P.). The weights were assigned to reflect the strength of each standard’s relationship to each IOM-defined BSS domain, with no assigned weight indicating no relationship, + indicating a somewhat relevant relationship, ++ indicating a moderately relevant relationship, and +++ indicating a very relevant relationship.

Table 1.

Conceptual Frameworks, and Assigned Weights, Used in a Systematic Review of the Literature on Tools to Assess Behavioral and Social Science Competencies in Medical Education^a

BSS-relevant LCME standards	IOM-defined BSS domains
	Mind-body interactions in health & disease	Patient behavior	Physician- patient interactions	Physician role & behavior	Social & cultural issues in health care	Health policy & economics
	Curriculum requirements: The curriculum of a medical education program must…
ED-10. Include behavioral and socioeconomic subjects in addition to basic science and clinical disciplines.	+	++			+++	+++
ED-19. Include specific instruction in communication skills as they relate to physician responsibilities, including communication with patients and their families, colleagues, and other health professionals.			+++	+++
ED-20. Prepare medical students for their role in addressing the medical consequences of common societal problems (e.g., provide instruction in the diagnosis, prevention, appropriate reporting, and treatment of violence and abuse).		+++	+++	++	+++	++
ED-21. The faculty and medical students of a medical education program must demonstrate an understanding of the manner in which people of diverse cultures and belief systems perceive health and illness and respond to various symptoms, diseases, and treatments.	++	+++	+++	+++	+++	++
ED-22. The medical students in a medical education program must learn to recognize and appropriately address gender and cultural biases in themselves, in others, and in the process of health care delivery.	+	++	++	+++	+++
ED-23. Include instruction in medical ethics and human values and require its medical students to exhibit scrupulous ethical principles in caring for patients and in relating to patients’ families and to others involved in patient care.			++	+++	+++
Integrative program requirements: The curriculum of a medical education program must…
ED-13. Cover all organ systems, and include the important aspects of preventive, acute, chronic, continuing, rehabilitative, and end-of-life care.	+	+	+		+	+
ED-14. Include clinical experience in primary care.	+	+	+	+	+	+
ED-15. Prepare students to enter any field of graduate medical education and include content and clinical experiences related to each phase of the human life cycle that will prepare students to recognize wellness, determinants of health, and opportunities for health promotion; recognize and interpret symptoms and signs of disease; develop differential diagnoses and treatment plans; and assist patients in addressing health-related issues involving all organ systems.	+	++	++	+++	+++	+

Abbreviations: LCME indicates Liaison Committee on Medical Education; BSS, behavioral and social science; IOM, Institute of Medicine.

^aWeights were assigned to reflect the strength of each LCME standard’s relationship to each IOM domain, with no assigned weight indicating no relationship, + indicating a somewhat relevant relationship, ++ indicating a moderately relevant relationship, and +++ indicating a very relevant relationship.

Search terms

We conducted a preliminary search for articles published between January 1, 2002 and March 1, 2014 using the databases OVID (Medline), CINAHL, PubMed, ERIC, Research and Development Resource Base (RDRB), SOCIOFILE, and PsycINFO. With guidance from a library science expert, terms used in the search included: education, curriculum, course evaluation, students, teaching, competence, and program evaluation. These terms were further combined with the selected BSS-relevant LCME standards and the IOM-defined BSS domain keywords. See Supplemental Digital Appendix 1 (at [LWW INSERT URL]) for a sample search strategy with the limits and quotations used to search the OVID (Medline) database.

Inclusion/Exclusion criteria

We sought to include articles reporting on some form of validity or reliability testing in more than one learning setting for BSS competency assessment measures. When articles were identified, we reviewed their reference lists for additional articles to consider. Two specially trained research assistants independently reviewed all titles and abstracts manually to assess appropriateness for inclusion. The two research assistants and one author (P.A.C.) initiated a consensus process, which continued until agreement among the group was reached on inclusion and exclusion according to title and abstract. Figure 1 outlines the process we undertook to search for and ultimately identify the final articles for detailed review. We excluded articles that did not cover the BSS competencies, that reported solely on learners’ satisfaction or self-reported or self-assessed competency, and that did not describe some form of validation of the assessment instrument.

Figure 1.

Literature search and article selection process for a systematic review of the literature, published between January 2002 and March 2014, on assessment tools used to evaluate behavioral and social science competencies in medical education. ^aPrimary reasons for rejection at title and abstract review included: (1) lack of reporting on psychometric properties or validity or reliability testing in more than one learning setting; (2) measures that did not assess learner competency in one of the selected areas; (3) results that were based solely on learners’ satisfaction or self-reported or self-assessed competency; or (4) the curriculum being tested did not address the behavioral and social sciences (e.g., it focused on anatomy or surgical skills). ^bSome articles were rejected after partial data abstraction for multiple reasons and therefore were counted twice here.

Methods for data abstraction

The review group (P.A.C., R.T.P., M.F.M., E.K.T.) created an abstraction form using the following variables: the type of article, how it was found, if the article described a BSS learner competency and one or more measures of that competency, the quality of the instrument (does the study describe a form of validation of the instrument(s) used), if institutional review board (IRB) review was mentioned, the type of study, the site of the study, learner level of the participants, curriculum specialty, the BSS or competency measurement framework used, the curriculum format tested and for how many hours, how the competency was assessed, what was measured and when, and our classification of the strength of the instrument testing (as described below). The data abstraction form was tested with approximately 30 articles and was iteratively revised and retested to ensure that data capture during the abstraction process was accurate and that only applicable studies would be included. Selected members of the advisory group (F.E.B., A.L., J.M.S.) provided feedback and contributed to the consensus process as needed.

Methods for assessing instrument quality and study design

We focused on both previously validated BSS competency assessment instruments as well as new instruments, validated within the included article. Assessing the evidence derived from the included articles necessarily involved co-mingling assessments of the strength of the instrument itself and of the strength of the study design, as studies rarely focused on just one of these features. For example, a high-quality article was one that applied a validated BSS instrument (either from the published literature or the included article) using a rigorous study design, such as a randomized controlled trial. A low-quality article was one that applied an unvalidated measure of BSS competency and used a weak study design to measure the impact of the educational intervention, such as a post-intervention survey of student satisfaction.

We categorized the level of evidence supporting each BSS competency assessment instrument and study design as weak, moderate, or strong. The weak evidence category included studies containing limited information on the validity and/or reliability of the evaluation instrument or a weak study design, such as a single group pre-post design. The moderate evidence category included studies that provided some information about the reliability of the measures used but were not assessed rigorously, retested in the study sample, or had a moderately strong study design, such as a single group historical cohort assessment. The strong evidence category included studies in which the evaluation instruments were tested rigorously in the study population and used a strong study design, such as a randomized controlled or crossover trial design.

Methods for article categorization, data entry, and analysis

Articles identified for data abstraction were classified into three categories: (1) instrument development with psychometric assessment only, defined as articles devoted to the statistical validation of a new or existing competency tool, such as a measure of physician empathy; (2) educational research, defined as articles that used a specific study design and BSS competency assessment tool to draw conclusions about a defined educational research question; and (3) curriculum evaluation, defined as articles that assessed specific curriculum features.

Three authors (P.A.C., R.T.P., M.F.M.) independently abstracted data from all articles that met the review criteria and then employed a rigorous consensus process for determing the final content of all abstractions during weekly consensus meetings. At these meetings, the variables from each article were discussed until consensus was reached. In one instance, the three authors could not come to a consensus. In this case, the larger advisory group was consulted for a final decision. The final consensus-based abstraction forms were entered into a database designed for this purpose. The data files were then checked and cleaned prior to analysis.

The Web-based system we used for database entry was a free and open-source application (LimeSurvey; Carsten Schmitz, Hamburg, Germany; https://www.limesurvey.org/en/), which was run on a secure and private server hosted at Oregon Health & Science University. Access to the system was limited to team members only, and its use allowed us to easily confirm which team members were reviewing which articles. Descriptive statistics were used to characterize the included articles (SPSS, IBM Corp., Armonk, NY).

Results

Our initial literature review identified 5,104 study titles and abstracts, many of which did not meet our criteria for further review (see Figure 1). Detailed title and abstract review along with searches of reference lists yielded 170 articles that we retrieved for full text review and data abstraction. Of these, we categorized 21 studies as instrument development with psychometric assessment only, 62 as educational research, and 87 as curriculum evaluation (see Supplemental Digital Appendix 2 at [LWW INSERT URL]). A more complete review during data abstraction revealed that 114 met our criteria for full abstraction (see Supplemental Digital Appendix 2).^9–122 At the partial abstraction stage, most article exclusions occurred because instrument validation was absent; this exclusion was most common among articles in the curriculum evaluation category. Other exclusions occurred because the article or assessment tool described did not actually address a BSS competency.

More than 70% of articles (13 of 20 instrument development studies, 35 of 48 educational research studies, and 36 of 46 curricular evaluation studies) mentioned IRB review, with most getting approval or exemption (see Supplemental Digital Appendix 2). Randomized study designs with or without controls were most common for educational research studies (23 of 48, 48%) compared to instrument development studies (1 of 20, 5%) and curricular evaluation studies (0 of 46, 0%), while prospective cohort pre-post designs were most common for curriculum evaluation studies (24 of 46, 52%) compared to educational research studies (6 of 48, 13%) and instrument development studies (1 of 20, 5%) (see Supplemental Digital Appendix 2). Validation using formal psychometric assessment was most common for instrument development (19 of 20, 95%) and educational research studies (25 of 48, 52%) compared to curriculum evaluation studies (17 of 46, 37%). We noted significant variability in the BSS frameworks and competency measures used to guide or evaluate the assessment instruments (see Supplemental Digital Appendix 2).

The most common BSS learner competency assessed across all types of articles was communication skills (see Supplemental Digital Appendix 3 at [LWW INSERT URL]). Cultural competence and behavior change counseling (which included motivational interviewing) also were commonly assessed, especially in educational research and curriculum evaluation studies. Using the ACGME competency language, interpersonal skills and communication (in > 90% of included articles), patient care (> 62% of articles), and medical knowledge (> 43% of articles) were most commonly assessed, with practice-based learning and improvement (≤ 10% of articles) and systems-based practice (≤ 10% of articles) less commonly assessed (see Supplemental Digital Appendix 3).

Validated instruments that assessed knowledge, attitudes, and skills were most commonly used to evaluate BSS competencies (65% – 85%), with standardized patients assessing learners’ performance being the second most common (30% – 44%) (see Supplemental Digital Appendix 3). Very few assessments were based on the direct observation of learners. Articles reporting on psychometric assessments typically provided strong evidence for the validity of the instrument (52% – 90%)--16 articles mentioned testing done without specifying the validation method used. Validation by expert consensus was also reported (15% – 42%), though less often than psychometric assessment.

We ranked 33 articles (29%) as contributing strong evidence to support BSS competency measures of communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Most of these were educational research studies (see Supplemental Digital Appendix 3). In Appendix 1, we present the tools we found to have the strongest evidence for validity and reliability as well as those with strong study or evaluation designs. We also map these tools to both the LCME standards and the ACGME core competencies.

In Supplemental Digital Appendix 4, we provide additional details regarding the included articles. In Supplemental Digital Appendix 5 and 6, we describe the 62 articles (54%) that yielded moderate evidence in support of a BSS assessment tool and the 19 articles (16.7%) that yielded weak evidence, respectively. In Supplemental Digital Appendix 7, we map these articles to the BSS-relevant LCME standards. The majority (n = 65) mapped to the communication skills standard (ED-19), though all LCME accreditation requirements specific to or integrated with BSS competencies are represented in the included articles. Not all articles mapped to the IOM domains, however, with mind-body interactions in health and disease and health policy and economics being represented least often. All supplemental digital content is available at [LWW INSERT URL].

Discussion

This systematic review is the first to identify valid and reliable instruments that have been developed to assess learner competencies specifically related to the social and behavioral sciences. Our aim was to provide the greatest utility to educators and administrators by linking these instruments with the LCME accreditation requirements and the ACGME core competencies. We learned that tools assessing communication skills were supported by the most rigorous validation and study design approaches. These tools included both written tests assessing knowledge, attitudes, and skills as well as assessments conducted with standardized patients. Overall, we found a paucity of assessments that used the direct observation of learners interacting with actual patients. Although such approaches are time and resource intensive, several articles support the value of direct observation in assessing learner competencies.^123–126

Other high-quality assessments evaluated cultural competence, empathy/compassion, behavior change counseling (e.g., motivational interviewing), and professionalism. However, only one high-quality assessment tool, described in a 2008 article, evaluated teamwork. As the national interprofessional education center¹²⁷ has plans to conduct more rigorous instrument development and validation, additional work in this area might be forthcoming. We recommend that educators and educational researchers review the literature for established, validated tools to assess BSS competencies in their learners rather than reinventing the wheel. We found several well-validated tools that were used in only one study.

One of the most significant challenges in completing this review was distinguishing between the strength of the assessment instruments and the strength of the study designs. For example, the tool used might be very strong but the evaluation design was so weak that the strength of the measure could not overcome the weakness in the design in terms of drawing strong conclusions from the study findings. The strongest articles used well validated tools combined with robust evaluation designs, such as randomized designs or historical cohort comparisons. We also included several rigorous qualitative studies in this review. These studies utilized strong qualitative research methodologies and well validated instruments. Alternately, moderate and weak articles used less rigorous approaches to instrument validation, and they had weak study designs that limited the conclusions that could be drawn. Not surprisingly, we found the most rigorous assessments in articles that described robust instrument development and testing. While educational research articles were also likely to apply rigorous study designs, their validation approaches were not always as robust as those described in instrument development articles. This finding is worrisome as readers may draw conclusions from educational research that employs a strong evaluation design, when in reality the design is only as good as the measures used.

Even more concerning is our finding that curriculum assessment studies were the least likely to include validated instruments and frequently used weak research methods. Researchers cannot generate strong evidence for curricular approaches if the evaluation designs or assessment measures they use are sub-optimal. Thus, an important finding from our work is the need for the use of well-validated instruments in quantitative and qualitative studies that represent both educational research and curriculum evaluation. One way to address this issue is to encourage medical school faculty to partner with investigators in either the school of education or public health/community medicine who have more experience with validating and using rigorous instruments and evaluation designs. Efforts to improve the dissemination of validated instruments and study strategies to promote their adoption also could prove beneficial.

The strengths of our study include the rigor with which we approached the consensus process across each phase of the review as well as the detailed information we abstracted from the articles that met our inclusion criteria. This process allowed us to consider not only the strength of the evidence for each included assessment tool but also to map specific studies and instruments to both the LCME accreditation requirements and the ACGME core competencies. By organizing our data in this way, we were able to provide a quick reference for educators who are looking for well-validated instruments to measure medical student competencies in the social and behavioral sciences at their own institutions.

Our systematic review has a number of limitations that arise from the breadth of the topic area, the lack of specificity in describing BSS competencies, and the related but distinctly different frameworks of the IOM, ACGME, and LCME. We identified the quality of the BSS tools and evaluation designs used in studies that were specific to different learner populations, such as undergraduate medical students. Nuances between the IOM, ACGME, and LCME frameworks should be taken into account when applying our findings from one distinct learner population to another. While these nuances do exist, we also feel that the universality of the BSS competencies, as well as the need to assess them rigorously, outweighs any variance in learner level and thus our findings can be of use in all learner populations. In addition, due to the breadth of the topic area and lack of specificity of the BSS competencies, the search terms we used (and their various Boolean operators) were complex and could be difficult to replicate. Although we searched the CINAHL, PsychInfo, and ERIC databases, our use of the IOM, ACGME, and LCME frameworks in data abstraction might have caused us to over-rely on the medical education literature. We did not include the EMBASE database, truncated search terms, or wildcards, which also limited our search. Next, we determined the quality scores by consensus using a subjective approach in assigning articles to strong, moderate, and weak categories. This process was challenging at times as some articles described high-quality instruments but weak study designs that affected our weighting of the evidence, while others described strong study designs but weak instruments that similarly affected our weighting. Finally, with the growth of peer evaluation and an emphasis on critical reflection in medical school curricula, we may have missed an important body of research because we excluded studies of self-reported competencies in the BSS domains; future reviews should consider addressing this gap.

In conclusion, we abstracted data from 114 articles, after reviewing a total of 5,104 identified studies. Of these, 33 (29%) yielded strong evidence to support BSS assessment tools that evaluated communication skills, cultural competence, empathy/compassion, behavioral health counseling, professionalism, and teamwork. Sixty-two (54%) articles yielded moderate evidence, and 19 (17%) yielded weak findings. In the future, more rigorous validation and testing of assessment tools as well as more robust evaluation designs are needed in both educational research and curriculum assessment. At the same time, the conceptual and content domains of BSS pedagogy deserve similar, careful definition and increased specificity so educators can better assess medical student competencies in areas such as population health and social inequalities and their influence on health status, particularly with regards to gender, race/ethnicity, socioeconomic resources, and the social organization of health care.

Appendix 1

Details of the Assessment Tools with “Strong” Evidence of Validity or Reliability Included in a Systematic Review of the Literature on Tools to Assess Behavioral and Social Science Competencies in Medical Education, 2002–2014 (n = 33)

Article first author, year	Assessment tool	Framework	Validation methodology	Type of study	Learner level	LCME standarda	ACGME core competency
Communication skills competencies (n = 17)
Bosse, 20129	Validated KAS assessment tool SPs	Transactional analysis and communication theories of Schulz von Thun Calgary Cambridge Reference Observation Guide	Psychometric assessment	Educational research	Medical students	ED-19, ED-21	Medical knowledge Interpersonal skills/communication
Daeppen, 201210	Validated KAS assessment tool	Transtheoretical model MITI	Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-20	Patient care Medical knowledge Interpersonal skills/communication
Gallagher, 200511	Validated KAS assessment tool	Socioemotional framework	Psychometric assessment	Instrument development	Medical students	ED-10, ED-19	Interpersonal skills/communication
Guiton, 200412	Validated KAS assessment tool SPs	Forrest Lang framework	Psychometric assessment	Instrument development	Medical students	ED-10, ED-19	Patient care Interpersonal skills/communication Professionalism Interpersonal skills/communication
Huntley, 201213	SPs	Liverpool Undergraduate Communication Assessment Scale	Test-retest Psychometric assessment	Instrument development	Medical students	ED-19
Iramaneerat, 200914	Validated KAS assessment tool SPs	Patient-centered communication and interpersonal skills Revised UIC communication/interpersonal skills scale	Pilot testing (no mention of cognitive interviewing) Psychometric assessment	Educational research	Residents	ED-19, ED-20, ED-23	Patient care Interpersonal skills/communication
Jensen, 201015	Validated KAS assessment tool	Four Habits Coding Scheme	Psychometric assessment	Instrument development	Practicing physicians	ED-19	Patient care Medical knowledge Interpersonal skills/communication
Jensen, 201116	Validated KAS assessment tool Actual patients	Four Habits Coding Scheme	Expert consensus Psychometric assessment	Educational research	Practicing physicians	ED-19	Patient care Interpersonal skills/communication
Joshi, 200417	Validated KAS assessment tool	ACGME skills in interpersonal communication	Expert consensus Psychometric assessment	Educational research	Residents	ED-19	Interpersonal skills/communication
Krupat, 200618	Validated KAS assessment tool	Four Habits Model	Psychometric assessment	Instrument development	Residents, practicing physicians	ED-19	Interpersonal skills/communication Patient care Medical knowledge Professionalism
Lim, 201119	Validated KAS assessment tool SPs	Transtheoretical model/motivational interviewing	Test-retest Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-20	Patient care Interpersonal skills/communication
Lurie, 200820	Validated KAS assessment tool	Rochester Communication Rating Scale	Pilot testing (no mention of cognitive interviewing) Test-retest Psychometric assessment	Instrument development	Medical students	ED-19, ED-21, ED-22	Interpersonal skills/communication
Moulton, 200921	Validated KAS assessment tool	None	Psychometric assessment	Educational research	Medical students, residents	ED-19	Patient care Medical knowledge Interpersonal skills/communication
Rees, 200222	Validated KAS assessment tool	Communication Skills Attitude Scale	Test-retest Psychometric assessment	Educational research	Medical students	ED-19	Interpersonal skills/communication Professionalism
Scheffer, 200823	Validated KAS assessment tool SPs	Calgary-Cambridge Observation Guide for Doctor-Patient Communication (translated)	Expert consensus Psychometric assessment	Educational research	Medical students	ED-19	Interpersonal skills/communication
Wouda, 201224	Validated KAS assessment tool	CELI (Control, Explain, Listen, Influence)	Psychometric assessment	Educational research	Residents	ED-19	Interpersonal skills/communication
Yedidia, 200325	SPs	Experimental teaching model for communication	Test-retest Psychometric assessment	Curriculum development	Medical students	ED-10, ED-19 through ED-23, ED-14	Patient care Medical knowledge Interpersonal skills/communication Professionalism
Cultural competence/patient-centered care competencies (n = 3)
Crosson, 200426	Validated KAS assessment tool	Health Belief Attitude Survey	Psychometric assessment	Curriculum development	Medical students	ED-10, ED-19, ED-21, ED-22, ED-14	Patient care Medical knowledge Interpersonal skills/communication Professionalism
Kirby, 201127	Validated KAS assessment tool	SADP Competency WC experience attitudes Bloom’s educational domains Theory of multiple intelligences	Testing mentioned (no specifics) Expert consensus	Educational research	Medical students	ED-19, ED-22	Patient care Medical knowledge Interpersonal skills/communication Professionalism
Wilkerson, 201028	SPs PCC-embedded scale	Mead and Bower: patient-centeredness Kleinmon’s questions of patient experience of illness Shared model for treatment	Pilot testing (no mention of cognitive interviewing) Expert consensus Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-21	Patient care Interpersonal skills/communication Professionalism
Empathy/Compassion (n = 5)
Austin, 200729	Validated KAS assessment tool	Jefferson Scale of Physician Empathy Emotional Intelligence Scale	Psychometric assessment	Educational research	Medical students	ED-19	Patient care Interpersonal skills/communication Professionalism
Fields, 201130	Validated KAS assessment tool	Jefferson Scale of Physician Empathy	Test-retest Psychometric assessment	Instrument development	Nursing students	ED-19	Patient care Interpersonal skills/communication Professionalism
Hojat, 200231	Validated KAS assessment tool	Jefferson Scale of Physician Empathy	Psychometric assessment	Instrument development	Practicing physicians	ED-19, ED-21	Patient care Interpersonal skills/communication Medical knowledge Professionalism
Peterson, 201232	Validated KAS assessment tool	None	Psychometric assessment Pilot testing (no mention of cognitive interviewing)	Instrument development	Medical students	ED-19, ED-23	Patient care Medical knowledge Practice-based learning and improvement Interpersonal skills/communication Professionalism Systems-based practice
Shapiro, 200433	Validated KAS assessment tool	Empathy Construct Rating Scale Balanced Emotional Empathy Scale	Expert consensus Psychometric assessment	Educational research	Medical students	ED-19, ED-21, ED-22	Patient care Interpersonal skills/communication Professionalism
Behavioral health counseling competencies (n = 4)
Mounsey, 200634	Validated KAS assessment tool SPs	Transtheoretical model/motivational interviewing	Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-20	Patient care Interpersonal skills/communication
Prochaska, 201235	Validated KAS assessment tool SPs	5 A’s (Ask, Advise, Assess, Assist, Arrange) Framework SEGUE	Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-20	Patient care Medical knowledge Interpersonal skills/communication
Spollen, 201036	Validated KAS assessment tool SPs	Knowledge and skills in behavior change Behavior change counseling index	Test-retest Expert consensus Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-15	Patient care Medical knowledge Interpersonal skills/communication
Truncali, 201137	Validated KAS assessment tool SPs	Transtheoretical model/motivational interviewing OSCE National Institute on Alcohol Abuse and Alcoholism Modified Saitz pre-post test	Psychometric assessment	Educational research	Medical students	ED-10, ED-19, ED-20	Medical knowledge Interpersonal skills/communication
Professionalism competencies (n = 3)
Crossley, 200938	Validated KAS assessment tool	Professional identity development	Test-retest Psychometric assessment Pilot testing (cognitive interviewing/multiple observers)	Instrument development	Medical students	ED-19, ED-21, ED-22, ED-23	Patient care Interpersonal skills/communication Professionalism
De Haes, 200539	Validated KAS assessment tool Actual patients	Amsterdam Attitudes and Communication Scale	Psychometric assessment	Instrument development	Medical students	ED-19, ED-22	Patient care Medical knowledge Interpersonal skills/communication Professionalism
Noble, 200740	Validated KAS assessment tool Reflective essays (formative)	Doctor-Patient Scale (de Monchy) Confidence in Communicating (Doherty and Marteau)	Psychometric assessment	Curriculum development	Medical students	ED-19, ED-20, ED-23	Interpersonal skills/communication Professionalism
Teamwork competencies (n = 1)
Youngblood, 200841	Validated KAS assessment tool	Kolb’s experiential learning	Psychometric assessment	Educational research	Medical students, residents	ED-19	Interpersonal skills/communication

Abbreviations: LCME indicates Liaison Committee on Medical Education; ACGME, Accreditation Council for Graduate Medical Education; KAS, knowledge, attitudes, and skills; SP, standardized patient; PCC, patient-centered care; MITI, motivational interviewing treatment integrity; UIC, University of Illinois at Chicago; SADP, Scale of Attitudes Toward Disabled Persons; WC, wheelchair; SEGUE, set the stage, elicit information, give information, understand the patients perspective, and end the encounter; OSCE, objective structured clinical exam.

^aSee Table 1 for a list of the LCME standards.