Gap Analysis and Refinement Recommendations of Skin Alteration and Pressure Ulcer Enterprise Reference Models against Nursing Flowsheet Data Elements

Abstract

Reference models are an essential instrument to provide structure and guidance in the creation and use of data elements within an organizations’ electronic health record (EHR). Standardization of data elements is imperative to ensure clinical data is consistently and reliably captured for use in clinical documentation, care communication, and a variety of downstream data uses. Ongoing assessment and refinement of reference models and data elements are necessary to ascertain clinical data capture is applicable and inclusive across a variety of caregivers and domains. We performed a gap analysis on current state nursing data elements against two validated interprofessional reference models: skin alteration and pressure ulcer assessments. We present our findings along with recommendations for reference model refinements. We also highlight additional findings of inconsistencies and redundancies within data elements used for nursing documentation and highlight recommendations for improvement.

Introduction

An important aspect of an Electronic Health Record (EHR) is to capture and apply clinical data as a means to support patient care and improve outcomes.^1,2 Utilizing data collected within an EHR is foundational to the National Academies vision of a ‘learning health system’, which supports that a health system can effectively use, learn from, and share health data to guide and improve the overall quality of healthcare.^3,4 Standardization of clinical data is essential to effectually capture, interpret, share and reuse data in a consistent and meaningful way.⁵ The integrity and representation of data collected and documented within an EHR is vital for the reliability and accuracy of downstream usage.^6,7 EHR data are used to support activities at the point of care, such as communicating clinical information and coordinating care within and across organizations. Additionally, secondary use of data (downstream data reuse) includes analytics, safety and quality reporting, regulatory compliance, and clinical decision support algorithms. Conversely, a lack of reliable and complete data elements results in semantic inconsistencies and poor data integrity that may increase the potential for errors when data are used to inform patient care decisions and for secondary uses.^5,8,9,10

Detailed clinical data element reference models, referred to here simply as reference models, support the consistency of data capture by standardizing structured clinical data elements (SCDEs) necessary to represent a specific clinical topic.^9,11 The resultant standardization of data capture ultimately promotes increased data integrity and reliability in downstream data uses. For a reference model to be most effective a detailed gap analyses should ensure that it is clinically relevant across interprofessional clinical end-users and domains and applicable to a variety of EHR workflows and functionality. Importantly, these detailed gap analyses should include refinement on an ongoing basis to incorporate new clinical knowledge or EHR functionality, even after initial validation by clinical subject matter experts.

Skin alteration assessment documentation, including assessment of wounds and pressure ulcers, is an important multi-disciplinary clinical topic that has clinical, regulatory compliance, and financial impacts to organizations. Pressure ulcers are one of the “Present on Admission (POA)” indicators hospitals are required to document for each inpatient admission.¹² Moreover, based on the Hospital-Acquired Condition (HAC) Reduction Program, Stage III and IV pressure ulcers acquired during a patient’s hospitalization are not reimbursable.¹² Accurate documentation of pressure ulcers upon admission, pressure ulcer risk identification, managing pressure ulcer prevention measures and appropriate documentation of pressure ulcer staging throughout a hospital stay therefore have become increasingly important to prevent erroneous incidence of non-payment of services. Furthermore, documenting wound assessment in a structured and consistent manner is paramount to proper assessment, treatment, and optimal wound management and outcomes.^13,14

Background

Partners HealthCare System, a large Boston-based non-profit hospital and physician’s network, implemented an enterprise-wide Electronic Health Record (EHR) in May 2015. An enterprise Structured Clinical Data Element (SCDE) workgroup was formed and tasked with the principle responsibility of identifying high priority clinical topics and cataloging, evaluating, and standardizing enterprise SCDEs for those clinical topics. The group is comprised of clinical, informatics, and EHR analyst experts from across the organization.

Previous publications outlining work done within our organization describe the process used to guide the governance of reference models for use in our EHR, including development, validation, implementation, and evaluation. In one publication, a 10-step approach for reference model governance is outlined: 1) identify clinical topics, 2) create draft reference models for clinical topics, 3) identify downstream data needs for clinical topics, 4) prioritize clinical topics, 5) validate reference models for clinical topics, 6) perform gap analysis of EHR SCDEs compared against reference model, 7) communicate validated reference models across project members, 8) request revisions to EHR SCDEs based on gap analysis, 9) evaluate usage of reference models across project, and 10) monitor for new evidence requiring revisions to reference model.¹⁵ A subsequent publication specifies detailed methods used to identify priorities for the definition of clinical data reference models and to identify and resolve gaps between existing EHR data collection tools and validated reference models.¹¹ Based on that prior work, skin alteration and pressure ulcer were identified as high priority clinical topics.¹¹

The skin and pressure ulcer reference models were created with an interprofessional group of SMEs across our organization’s various sites and with diverse stakeholder participation. The resultant skin reference models include 48 data elements within the 10 distinct categories: 1) skin Inspection, 2) skin alteration type, location and condition, 3) skin alteration size, 4) wound bed, 5) wound tunneling, 6) wound undermining, 7) wound exudate, 8) wound drain 9) wound dressing, and 10) Skin Alteration Sub-Types. The pressure ulcer reference model includes a total of 22 data elements without additional categorization. The intent of the models is to represent basic skin and pressure ulcer documentation and does not include pediatrics or clinical specialties. The reference model was then compared to and validated against existing “form-based” enterprise data elements being used in the Though the content from legacy systems may have been leveraged in the creation of the reference models, the gap analysis was based solely on the live enterprise EHR. The form-based format of capturing clinical documentation is typically used by physicians and other care team members in the inpatient and outpatient setting. However, nursing documentation in the inpatient setting requires a somewhat different approach due to differences in workflow and clinical documentation requirements. Instead of a form-based format, a flowsheet format is used for most inpatient nursing data collection. Though nursing representation was included in the reference model creation and validation, the uniqueness of the nursing workflow, the differences in data capture needs, and the EHR format limitations required an additional detailed gap analysis (comparison of skin alteration and pressure ulcer reference models with current state flowsheet data elements used to capture skin and pressure ulcer documentation) to ensure reference model comprehensiveness and to guide refinement recommendations for implementation in nursing flowsheet documentation.

Methods

The analysis was conducted over a 10 week period of time totaling approximately 100 hours of work. An initial preparatory step in this analysis was to identify all current state flowsheet data elements used to capture skin and pressure ulcer documentation across the enterprise. Our SCDE workgroup identified all unique skin and pressure ulcer flowsheet data element sets used in our EHR for analysis. An EHR data element extraction was completed based on the identified data element sets and queried for all associated data elements and the respective value-sets. Flowsheet data elements are nested within data element sets as structures that organize small groups (e.g., typically less than 10) of clinically-related data elements. A data element may belong to multiple sets. These sets, and the data elements that comprise them, are further stratified based on their intended use and EHR functionality. In the use case of skin alteration and pressure ulcer assessments the stratifications are: 1) documentation of simple skin alteration assessment at one point in time (e.g., skin color), 2) documentation of static data for a complex skin alteration that allows for comparative assessments overtime, such as the location of a pressure ulcer, and 3) documentation of dynamic data for a complex skin alteration that allows for comparative assessments overtime, such as pressure ulcer drainage. The different types of data element sets described above assist in organizing the flowsheet structure and functionality; however, the gap-analysis performed in this study was completed at the data element level since the value-set list associated to that data element was a significant feature of the comparison and any required alignment. The same process for comparison was used across all data elements.

For each unique flowsheet data element a gap analysis was performed using predefined metrics (See Table 1) against both the skin alteration and the pressure ulcer reference models using the following fields: 1) data element name, 2) data element set associations, 3) data element type (i.e. boolean, value list, numeric, and string), 4) value-set (where applicable). Value-sets for each category list data element were considered when determining whether a similar documentation artifact was an exact match or a partial match. This was an important designation since the aim of the comparison was to align both the data elements and any associated value-sets.

Table 1.

Gap Analysis Mapping Codes and Descriptors

Mapping Code*	Descriptor
Exact Match	Validated reference models and current state flowsheet data elements and values match exactly
Partial Match	Validated reference model and current state flowsheet data elements and value-sets partially match
Extra	Validated reference models do not include data elements from current state skin and pressure ulcer flowsheets
Missing	Validated reference model data element not included in current state skin and pressure ulcer flowsheets
Not in Scope: Pediatric/Newborn	Out of scope for analysis due to Pediatric/Newborn based sets only
Not in scope: Not Skin/Wound	Out of scope for analysis due to data element not a true skin or pressure ulcer based data element
Adaptation of MUC-5 Evaluation Metrics16

Our team has previously reported on methods used for gap analyses of EHR data elements compared against reference models.¹⁵ Here we use an adaptation of the MUC-5 (Fifth Message Understanding Conference) Evaluation Metrics.¹⁶ We used the MUC-5 Evaluation metrics to compare the 2 reference models: skin alteration and pressure ulcer against current state flowsheet data elements. These metrics identify if flowsheet data elements are a match, a partial match, extra, missing, or out of scope. Data elements used to assess skin concepts unrelated to a skin alteration were out of scope for the reference models. Additionally, because the reference models were not initially validated against the pediatric or newborn population, data elements used exclusively in pediatric or newborn sets were deemed out of scope for reference model inclusion, pending further expansion of the reference model for those populations.

Once the initial data element level mappings were complete, the extra and partial match data elements underwent a further analysis to determine recommended inclusion in the reference model using pre-determined recommendation metrics (See Table 2). The value sets of the partial match category list data elements were also cross mapped against the reference model value-sets and evaluated for value-set alignment. Recommendations were considered based on scope and breath of the reference model and clinical alignment to skin alteration and pressure ulcer concepts and integrity of clinical data capture (See Table 3). Once the author performed the analysis, the methods and results were checked by a second author to ensure accuracy and completeness. Additionally, the analysis and recommendations for inclusion were reviewed with the SCDE workgroup for feedback and consensus.

Table 2.

Data Element Recommendation Metrics

Recommendation	Definition
1. Add to Reference Model	Recommendation that this data element be added to the enterprise skin or pressure ulcer reference model
2. Do Not Add-No evidence	Recommendation that this data element NOT be added to the enterprise skin or pressure ulcer reference model due to absent or limited evidence
3. Do Not Add-Not in Scope	Recommendation that this data element NOT be added to the enterprise skin or pressure ulcer reference model due to being out of scope
4. Do Not Add-Inherent in existing data element	Recommendation that this data element NOT be added to the enterprise skin or pressure ulcer reference model due to being included within another data element or value set of a data element
5. Align EHR to Reference Model	Recommendation that this data element be aligned with enterprise skin or pressure ulcer reference model

Table 3.

Data Element Value-set Recommendation Metrics

Recommendation	Definition
1. Recommend Value Inclusion in Reference Model	Recommendation that a flowsheet value be added to the enterprise skin or pressure ulcer reference model
2. Do Not Include-No evidence	Recommendation that this value-set NOT be added to the enterprise skin or pressure ulcer reference model due to absent or limited evidence
3. Do Not Include-Not in Scope	Recommendation that this value-set NOT be added to the enterprise skin or pressure ulcer reference model due to being out of scope
4. Do Not Include-Inherent in existing data element	Recommendation that this value-set NOT be added to the enterprise skin or pressure ulcer reference model due to being included within another data element or value set
5. Recommend Build Alignment to Reference Model	Recommendation that this data element type or value-set be aligned with enterprise skin or pressure ulcer reference model

Results

Data extraction results

We identified and extracted 73 unique skin and pressure ulcer flowsheet data element sets used in our EHR. 465 unique flowsheet data elements were nested within the 73 identified sets. The data elements and data element sets were broken out based on their intended functionality for documentation: 1) simple skin alteration, 2) static data for a complex skin alteration, and 3) dynamic data for a complex skin alteration (See Table 3).

Data element gap analysis results

Figure 1 shows the results of the 465 unique current state flowsheet data elements against both the skin and the pressure ulcer reference models based on the predefined metrics. Of particular importance to the intent of this analysis are the 46% of current state flowsheet data elements mapped as “extra”- not included in the reference model, and the 29% qualified as a “partial match”- indicating that the general theme of the data elements matched, although may contain a value-set disparity or the concept was defined at a broader or narrower level in another data element. The 15 data elements (3%) that qualified as an exact match were, not surprisingly, those used within validated assessment tools (e.g. Braden scale assessment concepts) or those with a concrete definition such as with the measurement concepts (e.g. length and width), or a Boolean concept with a very specific and unambiguous meaning (e.g. “Is wound present on admission”). Interestingly, we identified 3 data elements that exist in the reference models though are missing from current state flowsheet data elements.

Figure 1.

Frequency of Mapping Code for EHR data elements Compared to Reference Model Frequencies

Out of scope data elements were included in the list of 465 extracted data elements due to their inclusion in some of the 73 sets (i.e. neurovascular wound assessment set) and general proximity in documentation due to the sequence of clinical assessments. In this example, though capillary refill is an important clinical assessment of overall skin health and potential risk of skin impairment, the intent of the reference model does not capture that breadth or specificity. These combined data elements that were not in scope totaled 21% (n=98), with 5% (n=21) defined as pediatric/newborn related and reserved for future reference model scope extension.

Recommendation results for data element inclusion in reference model

Of the 215 flowsheet data elements identified as extra (i.e., included in the flowsheet data element sets though not in the skin alteration or pressure ulcer reference models) 1 data element, “wound shape”, was determined to be a candidate for inclusion (See Figure 2). Wound shape is considered an important part of the nursing wound assessment documentation and may be a clinically essential indicator when considering the wound location, appropriate wound measurement, and as a factor for overall wound management.^14,13 The recommendation to include the “wound shape” data element was presented at the SCDE workgroup and attained group agreement with the caveat that future work include an analysis of data usage for this data element and validation with SMEs to confirm clinical relevance.

Figure 2.

Data Element Recommendations for Inclusion in Reference Models

The reference models were created to comprise a broad level of skin alteration assessment and pressure ulcer assessment documentation. The scope did not include specialty areas such as NICU, pediatrics, or dermatology. Whereas the vast majority of “extra” flowsheet data elements (99%) were specialty focused, this content was notrelevant for inclusion in the models. Future work could expand the scope of our reference models to include these specialty domains.

No flowsheet data elements were coded as either: 1) Do Not Include-No evidence, or 2) Do Not Include-Inherent in existing data element. Whereas the majority of the extra flowsheet data elements were more specific than the reference models there was not a recommendation of any extra flowsheet data elements to align with the reference models (See Figure 2).

Recommendation results for “partial match” value-set reference models/flowsheet alignment

Of the 138 flowsheet data elements identified as a partial match (i.e. flowsheet data element similar to reference model data element), 33 flowsheet data elements had value-sets that were recommended for extension to existing reference models value-sets. Those 33 flowsheet data elements were then mapped to 9 reference model data elements (See Table 4). The skin alteration reference model includes 7 data element value-set adjustments whereas pressure ulcer includes 5 data element value-set adjustments. Three data elements overlap across both models.

Table 4.

Terms and definitions related to camps and roles played by participant

Reference Model	Reference Model Data Element	Value-set Inclusion
Skin alteration	Skin alteration type	Abscess, Skin Graft Donor Site, Extravasation, Fasciotomy, Flap, Frostbite, Excoriation, Hives, Lesion, Tear
Skin alteration	Wound bed appearance	(Appearance) Erythema, Hematoma, Moist (Color) Black, Yellow, Red
Skin alteration	Primary wound dressing types	Button, Eye shield, Gauze-Iodoform, Hydrofiber wsilver, Hydrogel-impregnated gauze, Impregnated foam, Lap Pad, Negative Pressure Wound Therapy, Packing strip, Packing strip/iodine compound, Petroleum-impregnated gauze, Silicone, Silver, Tourney, Towel-Radiopaque, Vaginal Packing, VesselLoop, Xeroform
Skin alteration	Secondary wound dressing types	Button, Eye shield, Gauze-Iodoform, Hydrofiber wsilver, Hydrogel-impregnated gauze, Impregnated foam, Lap Pad, Negative Pressure Wound Therapy, Packing strip, Packing strip/iodine compound, Petroleum-impregnated gauze, Silicone, Silver, Tourney, Towel-Radiopaque, Vaginal Packing, Vessel Loop, Xeroform
Skin alteration	Associated device type	Prophylactic device (e.g., intermittent pneumatic compression)
Skin alteration	Orthopedic device type	Cervical collar, Prosthetic, Splint
Skin alteration and pressure ulcer	Periwound condition	Intact skin, Crystals, Denuded, Eroded, Hematoma, Hypergranulation Hyperplasia, Ulcerated
Skin alteration and pressure ulcer	Skin/pressure ulcer alteration anatomical location	Cheek, Spine, Generalized
Skinalteration and pressure ulcer	Skin/pressure ulcer alteration anatomical location qualifier	Bilateral, Dorsal, Palmar, Plantar

Twenty-two partial match flowsheet data elements were not recommended for inclusion because the concept was inherent at a different level of specificity within another data element or value set. Two of the partial match flowsheet data elements were determined to be out of scope and no flowsheet data elements were excluded due to lack of evidence. Furthermore, it was recommended that all of those same 33 data elements and an additional 81 flowsheet data element value sets (a total of 114) be aligned with the reference models data element value-sets (See Figure 3).

Figure 3.

Data Element Value-set Recommendations for Inclusion in Reference Models

Additional findings

Throughout the gap analysis there were many instances identified of inconsistency within and across flowsheet data elements. Wound drainage color, wound description, and wound odor for example are discrete clinical wound assessment concepts, however, in some instances the concepts were grouped together into one data element with a combined value-set. Additionally there were multiple data elements describing the same concepts in differing ways. There were also unique data elements describing a concept with different value-sets.

The concept of “wound location” demonstrated a key challenge of EHR flowsheet implementation. In our reference models only two clinical concepts are needed to capture a generic wound location: 1) anatomical location (i.e. abdomen, foot) and 2) anatomical location qualifier (e.g. right, left, and medial, proximal). However, EHR flowsheets constrain the ability to reuse generic location data elements due to ambiguity in associating the correct location data with the correct skin alteration for a patient with multiple skin alterations. There was also notable variation in the value-sets for these anatomical location data elements that was not related to flowsheet constraints. In some instances, the value-sets included both the location and the location qualifier (i.e. abdomen, foot, right, left, medial, proximal) (See Table 5). In addition to the redundancy and overlap amongst wound location data elements, there were yet another group of data elements for location that were free text and were used for simple skin alteration assessments.

Table 5.

Inconsistencies of Value-sets Within and Across Data Elements

Data Element	Value-sets
Site Location	Right, Left, Lateral, Midline, Upper, Lower, Abdomen, Ankle, Arm, Chest, Elbow, Flank, Hand, Hip, Knee, Leg, Shoulder, Umbilicus, Wrist, Other (comment)
Location	Abdomen, Ankle, Anus, Arm, Axilla, Back, Breast, Bridge of nose, Buttocks, Chest, Coccyx, Ear, Elbow, Eye, Face, Finger (Comment which digit), Foot, Forehead, Generalized, Groin, Hand, Head, Heel, Hip, Ischial tuberosity, Jaw, Knee, Labia, Leg, Lip, Malleolus, Meatus, urinary, Mouth, Nares, Nasal Septum,, Neck, Nose, Pelvis, Penis, Perineum, Rib Cage, Sacrum, Scapula, Sclera, Scrotum, Shoulder, Spine, Sternum, Thigh, Throat, Tibia, Toe (Comment which digit), Trach site, Trocanter, Umbilicus, Vagina, Wrist, Other (Comment)

Discussion

Based on the gap analysis it is our recommendation that the skin alteration and pressure ulcer reference models are refined to include 1 additional data element, “wound shape”, and 9 value-set adjustments. We also recommend that 114 of the current state flowsheets data elements be modified to align with the validated reference models to minimize redundancy and inconsistency across data elements and value-sets. Moreover, data type alignment to minimize free text documentation is an important part of flowsheet adjustments. We also recommend future analysis to identify additional inconsistencies in flowsheet data elements and value-sets and usage data analysis to help determine clinical need and guide further refinement.

Overall, the results of the analysis provide evidence that the validated skin alteration and pressure ulcer reference models sufficiently comprise a broad level of skin and wound clinical concepts required to accurately capture nursing domain clinical concepts within flowsheet documentation. The findings also provided useful insight into inconsistent and overlapping data elements, as well as free text data elements, that require adjustment to align to the reference models and the need for future analysis to identify inconsistencies.

Pragmatic reference models are an important mechanism to guide the end user in collecting clinical data in a consistent and meaningful manner. Without a means for relevant and complete data capture, downstream data usage and patient care are negatively impacted. Continuous refinement of reference models used within an EHR is an important activity to ensure relevant and complete clinical data capture. Similarly, continuous refinement of the EHR over time is also an ongoing process that requires appropriate resources. Though the skin alteration and pressure ulcer reference models were initially validated with SMEs and for implementation in the form-based documentation in our EHR, the nursing flowsheet data elements required additional review and validation. Importantly, reference models should remain pragmatic, reflecting real use needs and any constraints imposed by EHR systems.

Next steps

The data element usage data for the recommended additions to the reference models will provide insight into how many times the data element “wound shape” is being used as an indicator of clinical relevance and rationale for exclusion. Future work evaluating skin alteration and pressure ulcer assessment concepts, and other clinical reference model topics, should include evaluation of usage data as a tool to identify relevant and important documentation fields. Similarly, usage data can provide information about nurses’ interactions with the data element value-sets for optimization of values. Additionally, validation of the skin and pressure ulcer reference models by comparing against the EHR content of other organizations may help to reduce any unintended institutional bias and to assure external validity of our findings.

Limitations

Data was limited to 1 EHR implementation and 2 reference models developed and validated at our organization. Process should be reusable at other size and types of healthcare organizations, even if models and flowsheets are different. Findings may vary however at other sites and gap analyses should be conducted at other sites to identify generalizability of findings for reference model refinement recommendations.

Conclusion

Refinement of existing reference models that guide SCDE build and resultant data collection tools across an enterprise EHR system is an important activity. Skin alteration and pressure ulcer documentation is considered a high priority model due to regulatory, billing, and downstream CDS impact.¹¹ Through a gap analysis of current state data elements against our skin alteration and pressure ulcer enterprise reference models we confirmed the comprehensiveness and relevance of the models to accurately capture nursing domain clinical concepts for flowsheet documentation. Based on the analysis, one data element, “wound shape”, was identified as a potential addition to the pressure ulcer model. Additionally, nine data elements are recommended for value-set expansion. We also identify many instances of current state data elements and associated value-sets that are not aligned with the models and have made recommendations for EHR modifications.

Table 3.

Count of Skin/Pressure Ulcer Flowsheet Data Elements per Type

FlowsheetSetType	SetCounts	Data ElementCounts
Set A: Simple Skin Alterations	29	253
Set B: Static Data for Complex Skin Alteration	16	54
Set C: Dynamic Data for Complex Skin Alteration	28	158
Total	73	465