Assessing the adequacy of the HL7/LOINC Document Ontology Role axis

Abstract

The healthcare landscape is changing, driven by innovative care models and the emergence of new roles that are inter-professional in nature. Currently, the HL7/LOINC Document Ontology (DO) aids the use and exchange of clinical documents using a multi-axis structure of document attributes for Kind of Document, Setting, Role, Subject Matter Domain, and Type of Service. In this study, the adequacy of the Role axis for representing the type of author documenting care was assessed. Experts used a master list of 220 values created from seven resources and established mapping guidelines. Baseline certification, licensure, and didactic training were identified as key parameters that define roles and hence often need to be pre-coordinated. DO was inadequate in representing 82% of roles, and this gap was primarily due to lack of granularity in DO. Next steps include refinement of the proposed schema for the Role axis and dissemination within the larger standards community.

BACKGROUND

Federal recommendations on standards as part of the Centers for Medicare & Medicaid Services (CMS) Electronic Health Record (EHR) Incentive Programs,¹ supported by initiatives from the Office of the National Coordinator and coupled with other drivers in healthcare, have led to an ongoing increased momentum in standards adoption. This includes clinical document standards for exchange of select clinical information and for certain public health reporting transactions.² As organizations use clinical documents, customizations to fit local needs are increasing. Evaluating clinical document standards is essential to facilitate adoption and uniformity in implementation.

The attributes of clinical documents can be represented using the HL7/LOINC Document Ontology (DO) that is comprised of five axes for Kind of Document (KOD), Type of Service (TOS), Setting, Subject Matter Domain (SMD), and Role, where each axis consists of a hierarchical set of values. This model was conceptualized by the HL7 Document Ontology Task Force and continued in a joint effort with the LOINC Committee. Development of DO is described by Frazier et al.³ with the current version publicly available as part of the LOINC User's Guide.⁴ The objective of DO is to enable retrieval and use of documents for multiple purposes including research and to facilitate clinical document exchange across systems and organizations.⁴ DO requires values from the KOD axis and at least one other axis for a valid representation. The Role axis is used to describe the type of author involved in the healthcare process documented in the note.

Prior studies on DO have included both evaluation and extensions to various axes. Proposed extensions to the SMD axis by Shapiro et al.⁵ and the TOS axis by the Department of Veterans Affairs (VA) were subsequently incorporated into DO. Shapiro et al.apos;s work extended the SMD axis through a poly-hierarchical structure by adding values from the American Board of Medical Specialties (ABMS). Likewise, extension to the TOS axis involved expansion of the compensation and pension examination value set.⁴ Various studies evaluating DO or LOINC have demonstrated issues with value sets for the different axes and LOINC-related value sets with pre-coordinated codes.6–10 Wang et al.¹¹ performed an in-depth analysis of DO to represent documents in a clinical research data repository from an integrated healthcare delivery system. This work recommended additional values for the Setting and Role axes of DO. Recent work by Rajamani et al.¹² proposed extensions to the Setting axis, which have been made available in the public domain.

A recurring theme in prior studies is the need for iterative and incremental updates to DO to reflect the evolving healthcare delivery landscape, by identifying gaps and proposing extensions, as well as optimal organization. The objective of this study was to address the DO Role axis by evaluating its adequacy for representing healthcare practitioners and emerging professions and to propose extensions based on identified gaps. The DO SMD axis was also used in this evaluation as many roles are integrated with subject matter expertise.

METHODS

This study involved creating a comprehensive list of roles from multiple resources and then primarily evaluating the adequacy of the Role axis in DO for representing each value in this list (figure 1). The methodology consisted of three subcomponents: (1) collecting and integrating values from representative resources to create a master list of roles; (2) mapping each value in the master list to the Role and SMD axes of DO; and (3) summarizing issues and gaps for representing roles using DO.

Figure 1.

Overview of evaluation of the Role axis in HL7/LOINC Document Ontology.

The current DO Role axis and its hierarchical organization of values consists of 14 classes where four of them have subclasses of 16 values for a total of 30 values.⁴ Discussion among subject matter experts, literature searches, and web resources resulted in six other representative resources: (1) a local clinical data repository, (2) the Healthcare Provider Taxonomy, (3) CMS specialty codes, (4) the International Standard Classification of Occupations (ISCO), (5) the Standard Occupational Classification (SOC), and (6) workforce references from the Accountable Care Act (ACA).

Provider values were obtained from a local clinical data repository, created from University of Minnesota-affiliated Fairview Health Services, an integrated healthcare delivery system (79 values).¹¹ The Healthcare Provider Taxonomy¹³ is a Health Insurance Portability and Accountability Act (HIPAA) standard code set that is hierarchically organized and consists of 831 codes, descriptions, and definitions. This code set is currently maintained by the National Uniform Claim Committee (NUCC) and chaired by the American Medical Association (AMA). Only high-level classifications and specializations were included and SMD specific descriptors were excluded (177 values). The CMS specialty codes set¹⁴ is a high-level organization of specialties and is cross-walked to the Healthcare Provider Taxonomy set (18 values). The ISCO¹⁵ has a total of 620 values where only health-related roles were selected for inclusion (49 values). The SOC¹⁶ from the U.S. Department of Labor has 23 major groups of which two groups were chosen: the healthcare practitioners and technical occupations group and the healthcare support occupations group (135 values). Title V of the ACA¹⁷ is focused on the healthcare workforce and roles mentioned were included as part of a representative master list (67 values).

All values obtained from the seven resources including DO were compiled together and organized. Redundancies were eliminated and roles not within the scope of document representation were excluded to create a master list. Initial review underscored the need to involve the DO SMD axis in the evaluation as many roles were closely allied with SMD. Each value from the master list was then mapped to values in the DO Role and SMD axes and mappings were rated as Adequate, Too Broad, Too Specific, Not Covered, or Not Specified (figure 2). Mapping guidelines were developed using a consensus-based process involving five subject matter experts with experience and expertise in clinical care, nursing, public health, standards, and informatics. Guidelines facilitated mapping by defining broader categories of pre-coordinated roles versus post-coordinated role representations using existing or new role and SMD values. If a role had specific certification/licensure requirements, it was considered to be unique enough to be named explicitly and hence represented as a pre-coordinated value. Two experts mapped 10% of the master list values for calculating inter-rater reliability with the kappa statistic and proportion agreement. Remaining mapping was completed by the two experts and reviewed by the group in a series of sessions to achieve consensus. Through this process, issues associated with the Role axis were identified and categorized.

Figure 2.

Example mappings and ratings.

RESULTS

The initial master list created based on seven representative resources consisted of 555 values. After elimination of redundancies and exclusion of roles not within scope of this study, the final evaluation list had 220 values (available at http://www.bmhi.umn.edu/ihi/research/nlpie/). Assessment of the representation of roles in DO demonstrated several issues with role representation, including hierarchical organization, breadth of categories, and omission of some key roles. Of the 30 values currently included in DO, half of them are related to different levels of physicians and nurses. Also, several roles related to assistant are currently in different hierarchies.

Inter-rater reliability between the two experts for role mapping, role rating, SMD mapping, and SMD rating yielded kappa values of 0.89, 0.97, 0.95, and 0.80, respectively, and 96%, 96%, 96%, and 88% proportion agreement, respectively. The DO Role axis was found to provide adequate coverage for 38 values (17%) in the master list and inadequate coverage for 179 (82%) where 162 (74%) of these were rated as Not Covered (table 1). Further analysis of values where the role rating was Not Covered, Too Specific, or Too Broad revealed significant gaps, pointing to the need for pre-coordinated values to capture the specificity of various professions/roles. Issues that emerged were organized based on the Role and SMD axes to assess the adequacy of coverage.

Table 1.

Adequacy of coverage of the Role and SMD axes

Rating	Role axis coverage	SMD axis coverage
Adequate	17% (n = 38)	52% (n = 115)
Too Broad	6% (n = 13)	5% (n = 10)
Too Specific	2% (n = 4)	1% (n = 1)
Not Covered	74% (n = 162)	5% (n = 12)
Not Specified	1% (n = 3)	37% (n = 82)

SMD, Subject Matter Domain.

The mapping results highlighted that certain roles are absent from DO and need to be included (eg, Scribe). Certain role values are intricately associated with SMD (eg, Dentist, Doula, and Nutritionist), disease/pathology (eg, lymphedema therapist), procedure (eg, EchoTech), or care for particular surgical outcomes or conditions (eg, enterostomal therapist/wound ostomy continence nurse). Evaluation suggested a need to differentiate various roles based on training (eg, certified, licensed, or registered). Roles with certification or licensure were considered specific enough to be pre-coordinated. Subspecialty roles with advanced training after basic professional degrees could be post-coordinated with a role value and SMD for specialty (eg, pediatric nurse practitioner and oncologist).

DISCUSSION

Similar to findings with other DO axes,^4,5,11 this study found that DO lacked comprehensiveness in the Role axis. DO was able to represent less than half of the 79 provider values from a clinical data repository from an integrated healthcare delivery system. When compared to a broader schema of healthcare roles in use today as represented by the master list of 220 values, DO Role representation was even lower (17%). Evaluation also revealed that current DO values were either too broad or too specific, as well as ways that the axis might be better organized.

While DO includes ‘Assistant,’ ‘Physician Assistant,’ and ‘Medical Assistant,’ there are no terms to capture ‘Dental Assistant’ or ‘Physical Therapy Assistant,’ except through a combination of ‘Assistant’ role and the corresponding SMD axis (eg, ‘Dentistry’ and ‘Physical Therapy’). This leaves a lacunae for representing subspecialties in those professions (eg, a gap in representation of endodontist, a dental subspecialty). The DO ‘Physician’ role refers to the broad overarching category, which includes allopathic and osteopathic physicians, chiropractors, and dentists. More granularity is needed to represent their services and certification as they are significantly different in scope of practice, which impacts the KODs they would exchange. DO also has no general term to represent ‘Intern’ and ‘Resident’ that are roles in many healthcare professions, but has specific roles such as ‘Physician Intern’ and ‘Physician Resident.’

Another issue is the need to balance the use of pre-coordinated versus post-coordinated values. Guidelines assisted in mapping roles based on which needed new pre-coordinated role terms versus new broader role terms that could be used for post-coordination (table 2). Certain roles seem to be intricately associated with SMD, anatomic location, and pathology, which demonstrates that their representation needs to include both. One of the principles that assisted the guideline development and mapping process was the use of baseline certification, licensure criteria, and didactic training as parameters for defining a particular role. For subspecialties, a role defining the baseline profession needs to be explicit in the Role axis (eg, orthodontists and prosthodontists could be represented using a new role for Dentist and a new SMD for these dental specialties).

Table 2.

Summary of issues with roles and proposed extensions to the DO Role axis

Role Group	Issue	Examples
Pre-coordinated: addition of new values to the DO Role axis
Assistants and Aides	Role+SMD	Anesthesiologist Assistant, Radiology Practitioner Assistant, Psychiatric Aides, Home Health Aides
Coordinators	Not covered	Care Coordinator
Counselors	Role+SMD	Genetic Counselor, Rehabilitation Counselor
Educators and Trainers	Role+SMD	Athletic Trainer, Health Educator, Diabetes Educator
Nurses	Role+Condition	Advanced Practice Midwife
Nutritionists	Role+SMD	Public Health Nutritionist
Other	Not covered	Scribe, Interpreter
Other condition-specific roles	Role+Condition	Lymphedema Therapist
Other location-specific roles	Role+Setting	Emergency Room Specialist, Emergency Room Technician
Other licensed independent practitioners	Role+SMD	Dentist, Optometrist, Chiropractor, Psychologist
Other specialty-specific roles	Role+SMD	Orthotist, Prosthetist
Physicians (Allopathic, Osteopathic)	Role+Post Graduate Training	Physician Attending, Physician Fellow, Physician Resident
Social Workers	Role+SMD	Mental Health Social Worker
Specialists and Technologists	Role+SMD	Hearing Instrument Specialist, Radiologic Technologist
Students	Role+SMD	Physical Therapy Student
Technicians	Role+Training	EchoTech, Pathology Technician
Therapists	Role+SMD	Physical Therapist, Occupational Therapist, Speech Therapist
Post-coordinated: existing DO Role+SMD values (or) new role value+existing SMD value*
Interns, Residents, and Fellows (Non-Physician)	Role+SMD	Pharmacy Intern, Dietetic Intern New DO Role and existing SMD values (eg, Intern+Pharmacy)
Nurses	Role+SMD	Pediatric Nurse Practitioner Existing DO Role and SMD values (eg, Nurse Practitioner+Pediatrics)
Office workers	DO value too specific	Unit Clerk, Dietary Office Worker Existing DO Role and SMD values (eg, Clerk+Nutrition and dietetics)
Other licensed independent practitioners	Role+SMD	Orthodontist New pre-coordinated value of Dentist+new DO SMD
Physicians	Role+SMD	Oncologist, Anesthesiologist, Primary Care Physician Existing DO Role and SMD values (eg, Physician+Oncology)
Physician in training (Interns, Resident and Fellows)	Role+SMD	Oncologist, Anesthesiologist, Primary Care Physician Existing DO Role and SMD values (eg, Physician Resident+Internal Medicine)

*Post-coordination of existing DO values using the Role and SMD axes need additional guidelines/rules.

DO, Document Ontology; SMD, Subject Matter Domain.

Lack of granularity in roles in the current DO leads to less specificity. DO should also have new categories for high-level representation to increase sensitivity. Guidelines and rules on how to represent the multitude of healthcare professions and their varying roles are critical. The complexity and nuances of roles may warrant a poly-hierarchical representation in the DO Role axis.

This study highlights the current breadth of roles in healthcare practice and the dynamic nature of healthcare roles. In creating the master list of values for evaluation, relevant role set resources were included from reputable national organizations. However, many new roles are emerging and not yet captured in code sets (eg, Health Coach), due to their update timelines and the focus of those resources. This study presents a framework of thinking related to ontological representation of the DO Role axis, but additional evaluation is warranted as some roles come with varied didactic training and experience, are defined differently across organizations (eg, Care Coordinator), and certifications for some roles are just being developed.

The findings from this assessment support the need for education, training, and technical assistance related to use of standards such as DO. There may be discrepancies in how a particular standard is implemented by vendors or end-users for a particular application. Any standard must be able to accommodate the needs of the user and the application use case. Studies such as this that evaluate standards being used today are important for providing reasoning and facilitating updates to existing concepts and their relationships by the respective standards development organizations.

CONTRIBUTORS

GBM and ESC conceptualized the overall study design. SR led the development of the master list for evaluation and the writing of the first draft of the paper. GBM, ESC, and MA were instrumental in developing guidelines for evaluation. SR and YW did the initial mapping and YW carried out statistical analysis. All authors were involved with the consensus-based processes used for evaluating the Role axis of DO, as well as reviewing and editing the manuscript.

FUNDING

This work was supported in part by National Library of Medicine 1R01LM011364-01 (EC/GM), Agency for Healthcare Research and Quality 1R01HS022085-01 (GM) and the University of Minnesota Clinical and Translational Science Award 8UL1TR000114-02.

COMPETING INTERESTS

None.

PROVENANCE AND PEER REVIEW

Not commissioned; externally peer reviewed.