Building an emergency general surgery registry to track and benchmark outcomes

Abstract

Emergency general surgery (EGS) represents a significant portion of admissions to hospitals and overall surgical care delivery, yet comprehensive outcome tracking and quality improvement efforts have lagged behind elective surgery and trauma care. This article outlines the gap in data and provides three practical solutions to registry development: third-party registry software with dedicated registrars, leveraging existing National Surgical Quality Improvement Program (NSQIP) infrastructure through the EGS-NSQIP module, and automated data extraction from electronic medical records. Each approach offers unique advantages in addressing the dual challenges of case identification and risk-adjusted outcome measurement for both local quality improvement and national benchmarking. The aim of this white paper is to provide pragmatic, scalable data solutions for EGS leaders seeking to implement contemporary quality improvement and verification efforts.

Introduction

Emergency general surgery (EGS) encompasses a diverse spectrum of urgent and emergent surgical conditions, from appendicitis and cholecystitis to perforated viscus and necrotizing soft tissue infections.^{1 2} Despite accounting for approximately 8–10% of all hospital admissions and generating significant healthcare expenditures,^{3 4} EGS has historically operated without the systematic data collection and outcome monitoring that has transformed trauma care and elective surgery.⁵ EGS now occupies a similar developmental stage to trauma care prior to the widespread adoption of trauma registries and verification. Recently, the American College of Surgeons (ACS) has introduced the EGS Verification Program, a landmark program aimed to bridge the gap in quality of care.^6–8

The heterogeneous nature of EGS creates substantial barriers to consistent case identification, risk-adjustment/benchmarking and longitudinal outcome measurement.^{1 2 9–11} First, unlike trauma or elective surgery, EGS classically has lacked universally accepted case definitions and inclusion criteria, making it difficult to consistently compare hospitals with distinct case-mixes. Second, the heterogeneous nature of EGS conditions, ranging from straightforward appendectomies to complex cases of necrotizing soft tissue infections in frail, elderly patients, for example, requires sophisticated risk adjustment to enable meaningful comparisons. Third, hospitals and surgeons need both local data for quality improvement (QI) initiatives and national benchmarks to contextualize their performance, yet most institutions lack the infrastructure to capture, analyze, and report EGS outcomes systematically. Addressing this final challenge, EGS lacks a dedicated International Classification of Diseases, 9th or 10th Revision (ICD-9/10) diagnostic category (a critical distinction from trauma) thereby complicating both retrospective and prospective patient identification through ICD-9/10 and Current Procedural Terminology (CPT) coding systems. Although the AAST and subsequent researchers have developed several codesets to capture EGS patients by ICD-9 and ICD-10 systems, strategies which use this approach may “over-capture” patients with incidental and common diagnoses such as biliary disease, gastrointestinal bleed, and sepsis.^{2 12–14} Right-sizing data capture to the actual EGS service and operations is paramount to creating an accurate and manageable EGS registry.

The imperative for EGS registries stems from multiple stakeholders. Surgeons require data to identify opportunities for improvement, justify resource allocation, and demonstrate value-based care. Hospital administrators need metrics to monitor surgical quality, manage capacity, understand reimbursement, and meet regulatory requirements. Payers increasingly demand risk-adjusted outcome data to inform reimbursement models. Perhaps most importantly, patients deserve the assurance that their emergency surgical care is examined with the same rigor as other surgical disciplines. Implementation requires a planned and collaborative approach, figure 1.

Figure 1. Implementation of a Home-Grown EGS Registry: The process begins with defining EGS locally—which patients can you capture and which do you want to capture? By leveraging existing EMR capabilities to create daily patient lists, the registry moves from retrospective to prospective identification. Data collection uses a hybrid model: automated extraction for objective data and trained registrars for complex clinical validation. This data feeds an internal dashboard used by leadership to drive local quality improvement and justify resource allocation. EGS, emergency general surgery; EMR, electronic medical record; OR, operating room; PI, performance improvement; QI, quality improvement.

This article examines three viable approaches to building an EGS registry, each with distinct methodologies, resource requirements, and capabilities. These solutions share overlapping goals: to consistently identify EGS cases, enable risk-adjusted outcome measurement, facilitate local QI, and provide data for national benchmarking. The choice among these approaches depends on institutional resources, existing infrastructure, data quality requirements, and strategic priorities (table 1).

Table 1. Comparison of three proposed solutions for EGS registries.

Characteristic	Solution 1 registry software	Solution 2 NSQIP Model	Solution 3 Automated EMR extraction
Cost structure	Initial: High Ongoing: High	Initial: Minimal if already NSQIP participant. Ongoing: Minimal/moderate.	Initial: High Ongoing: Minimal
Data quality	High (90–95%+accuracy) Manual abstraction by trained registrars. Human judgment for ambiguous cases. Inter-rater reliability validation.	Very high (95%+ accuracy) Gold standard for surgical registries. External ACS audits. 30-day follow-up protocols.	Variable (70–90% accuracy) Depends on documentation quality. “Garbage in, garbage out”. May require extensive validation.
Time to data	Prospective 4–12 weeks lag postdischarge. Batch reporting (monthly/quarterly). Some real-time capability.	Retrospective 6–12 weeks lag postdischarge. Semiannual reporting cycles. Strong for benchmarking.	Near real-time potential 24–48 hours lag possible. Real-time dashboards feasible. Clinical decision support integration.
Case capture	Comprehensive Can capture 100% of cases. Limited by registrar capacity. Manual review ensures accuracy.	Sampling or comprehensive Traditional: 20–40% sample. Can expand to 100% with resources. Validated methodology.	Comprehensive Scalable to 100% of cases. No volume limitations. False negatives possible.
Flexibility	Moderate-high Can add custom data elements. Software may limit some fields. Standardized approach.	Low-moderate Standardized NSQIP dictionary. Custom variables. Focused on validated elements.	Very high Fully customizable queries. Can tailor to local priorities. Requires technical expertise.
Risk adjustment	Good Requires statistical modeling. Custom models needed. Can join collaboratives.	Excellent Gold-standard validated models. Preoperative risk adjustment. National benchmarking built-in.	Variable Depends on variables captured. Requires statistical expertise. Limited external validation.
External benchmarking	Indeterminate Unclear how to include “home-grown” solutions nationally.	Excellent National ACS NSQIP benchmarking. Semiannual reports. Credible external validation.	Indeterminate Unclear how to include “home-grown” solutions nationally.
Ideal institutional profile	High volume EGS to justify cost. Existing trauma registry infrastructure. Institutional goal to maximize local data granularity and real-time QI capability. No existing NSQIP program.	Already participating in NSQIP with trained SCRs in place. Prioritizing national/external benchmarking with existing QI/PI processes for other NSQIP programs.	Large academic or integrated health systems with dedicated informatics and accessible EMR backends. High EGS volume with desire for high-variable density and manual extraction is cost-prohibitive. Surgeon-champion to partner with IT.

ACS, American College of Surgeons; EGS, emergency general surgery; EMR, electronic medical record; IT, information technology; NSQIP, National Surgical Quality Improvement Program; PI, performance improvement; QI, quality improvement; SCRs, surgical clinical reviewers.

Patient identification: leveraging the electronic medical record

Common to each solution is how to best identify EGS patients whose care can be impacted by EGS teams. Although diagnosis codes and procedure codes provide one mechanism for retrospective case identification, they often miss patients and may include patients undergoing elective surgery or who never received an EGS consult. Similarly, relying on CPT codes will miss non-operative EGS patient identification, up to 50% of the EGS population, which is particularly challenging.¹⁵

Multiple electronic health record systems exist, with the most frequently used being Epic^® and Cerner’s PowerChart. Although not all hospitals use Epic and Cerner PowerChart, most electronic medical records (EMRs) will offer electronic solutions to identify EGS patients at the point of care. These systems can save the daily patient care lists kept by most institutions to analyze who is or has been under their care (either admitted to the EGS service or as a consult). Rather than waiting for discharge coding to identify cases weeks or months later, this prospective identification enables contemporaneous quality monitoring and opportunities for real-time clinical intervention. The list can be reviewed and validated through either manual verification or through automated validation using additional EMR data points to confirm appropriateness. This technique has been described before in the EGS literature and produced a high-fidelity method for identifying EGS patients (0% false positive rate, 85% sensitive).¹⁶ For hospitals that are not on one of these EMRs or do not have hospital support for building these processes, identification of patients can happen via a manual process as well, though this is often limited by the amount of time and resources it requires. Focusing on two to four diagnoses may allow a starting point to begin this journey.

From a registry perspective, prospective identification allows data abstraction to begin during hospitalization rather than weeks after discharge, improving data quality and enabling rapid-cycle QI. This approach to patient identification represents a critical foundation, regardless of which registry solution an institution ultimately implements. Whether identifying cases for manual abstraction by dedicated registrars, identifying patients for National Surgical Quality Improvement Program (NSQIP) sampling, or serving as the starting point for automated data extraction, EMR-based prospective patient identification ensures that institutions are tracking the patients whose care they can actually impact. Prospective identification is also critical for prompt quality and performance improvement (PI), and can even be used to imbed tailored risk scores to guide patients, families, and surgeons on operative risk.¹⁷

Solution 1: registry software with dedicated registrars

The first approach to EGS registry creation leverages the proven model of trauma registries by adapting commercially available third-party software platforms designed for trauma care, or existing software such as Microsoft Excel. This methodology employs dedicated, trained surgical registrars who manually abstract data from medical records according to a standardized data dictionary. Major vendors in this space have developed or are developing EGS-specific modules, for example, ESO¹⁸ and National Quality Systems.¹⁹

The operational model mirrors the highly successful trauma registry framework. Trained registrars review cases that meet pre-defined EGS inclusion criteria, typically identified through combinations of diagnosis codes (ICD-10), procedure codes (CPT), and admission source indicators (such as an EGS patient list in the EMR). The registrar then abstracts data elements per case, including patient demographics, comorbidities, presenting physiology, operative details, complications, and outcomes. This information populates a relational database that enables both internal reporting and submission to collaborative QI networks.

Advantages

The primary advantage of this approach lies in data quality and comprehensiveness. Human abstraction, when performed by trained registrars using explicit definitions, achieves high accuracy and completeness. Registrars can exercise clinical judgment in ambiguous cases, identify complications documented across disparate parts of the medical record, and capture nuanced information that automated systems might miss. The structured data dictionary ensures consistency over time and across institutions, facilitating meaningful comparisons. This strategy allows for a registrar-generated list for EGS leadership at a hospital to review for validation, QI, and PI.

Furthermore, this model enables participation in multi-institutional collaboratives, allowing hospitals to benchmark against peers and contributing to a larger evidence base. The software platforms typically include robust analytics tools, visualization dashboards, and reporting capabilities that support both clinical QI and administrative needs.

Limitations

The resource intensity represents the most substantial barrier, as employing full-time equivalent (FTE) registrars costs between US$60,000–80,000 annually per FTE. A busy EGS service may generate 3,000–4,000 cases annually, potentially requiring multiple FTEs depending on case complexity and data element requirements. Dedicated registrars are an ongoing operational expense that many institutions, particularly smaller hospitals or those with limited surgical volumes, may struggle to justify. Combining human review for ambiguous cases with automated extraction of highly reliable EMR data elements such as age, vital signs, and laboratory values may be the most efficient approach. Additionally, using procedure codes may miss non-operative cases, so the system needs to include a strategy to capture non-operative EGS diagnoses and filtering elective cases. This can be problematic in systems where EGS surgeons also do elective cases, limiting the ability to include by provider/group. A dedicated list with manual review for exclusion of elective surgeries can overcome this limitation.

Solution 2: Emergency General Surgery NSQIP Module

The second approach leverages existing institutional investments in the ACS NSQIP. Recognizing that many hospitals already participate in NSQIP and employ trained surgical clinical reviewers (SCRs), the EGS NSQIP model extends this infrastructure to capture EGS cases systematically. This method was extensively piloted by the ACS and a process for capturing non-operative patients (not typically captured in NSQIP) was validated.²⁰

Traditional NSQIP uses a sampling methodology, capturing detailed data on a subset of surgical cases (typically 20–40% depending on hospital volume) to generate risk-adjusted outcome metrics. Since 2012, NSQIP has offered “procedure targeted” data collection, to allow more detailed exploration of operative details and outcomes unique to specific procedures. Procedure-targeted data collection can continue with a sampling methodology or capture 100% of cases at the discretion of the hospital. The EGS-NSQIP model adapts this framework specifically for EGS by employing EGS-specific case selection criteria, data elements, and risk adjustment models for 11 EGS diagnoses. Notably, non-operative cases can be included. Participating hospitals may capture all operative and non-operative EGS cases, or may designate certain CPT codes and clinical scenarios of interest to their hospital and ensure adequate sampling of these cases within their NSQIP data collection.

Advantages

The infrastructure advantages are substantial. Many hospitals already employ NSQIP SCRs who are trained in data abstraction, validation, and quality assurance. These reviewers understand surgical complications, follow standardized definitions, and participate in inter-rater reliability audits. Additionally, there is the backing and support of the ACS behind this method, with EGS-specific SCR training, patient risk adjustment, and logistical support. The additional cost of expanding NSQIP to capture EGS cases may be incremental rather than de novo, often requiring minimal to modest increases in SCR time rather than entirely new positions. Some sites transitioned procedure-targeted case capture from other procedures to incorporate EGS cases without any additional SCR needs. Certainly, due to the inclusion of non-operative cases, there is some initial investment in process required to identify and abstract non-operative cases compared with operative cases which are more readily available.

NSQIP provides risk adjustment through validated statistical models that account for preoperative patient factors, enabling comparisons across institutions with different case mixes. The program’s 30-day follow-up, including phone calls to patients if information is missing in the EMR, is designed to achieve complete capture of complications and readmissions. This addresses a major weakness of administrative data sources that may miss post-discharge events. Participation provides access to analytics platforms, semiannual reports comparing institutional performance to national benchmarks, and opportunities to engage with ACS QI resources. Benchmarked data is just beginning to be populated and will further evolve to include procedure and volume-specific adjustment and metrics.

Limitations

The NSQIP infrastructure, although less expensive than building a completely independent registry, still requires significant investment—annual NSQIP participation costs depend on hospital size, plus SCR salaries. Although leveraging existing SCRs may be an option, training to capture EGS-variables, and building an infrastructure to capture non-operative cases, both require time. For busy EGS practices, capturing all EGS cases rather than a sample may require additional SCR FTEs. Additionally, the list of EGS-specific CPT codes is not necessarily inclusive of all EGS cases—for example, many incision & debridement cases may not be included, nor will EGS complications of elective surgical procedures (such as postoperative small bowel obstruction), even if managed by the EGS team. Finally, repetitive sequela and intensive care unit (ICU) procedures performed by the EGS team, like tracheostomy and percutaneous endoscopic gastrostomy (PEG) tube placement for medical ICU patients, are too numerous to be abstracted and outside the EGS targeted diagnoses.

The standardized NSQIP data dictionary, although comprehensive for many surgical outcomes, may not capture all EGS-specific variables that clinicians desire for QI. For example, clinically used disease-specific severity scores (Parkland²¹/Hinchey²² may require supplementary data collection). Although AAST grading scores are tracked, these scales are unfamiliar and take a great deal of clinical insight and input from surgeons to help SCR abstract correctly. The 30-day outcome window, although appropriate for many outcomes, may miss longer-term complications relevant to certain EGS conditions.

Solution 3: Automated EMR data extraction

The third approach represents the most technologically ambitious but feasible solution: leveraging EMR systems, particularly widely deployed platforms such as Epic and Cerner PowerChart, to automatically capture and extract registry data with minimal human intervention. This methodology employs a combination of structured data queries and collection of potentially more robust and detailed information of unstructured data using natural language processing. It is also possible to integrate clinical decision support. Several automated models across regional health systems have been built to examine EGS patients specific procedures and outcomes, and then used to evaluate optimizing the system of care delivery.^{14 16}

Modern EMR systems contain vast quantities of clinical data: structured fields (vital signs, laboratory values, medications, diagnosis codes), semistructured data (operative reports, pathological finding results), and unstructured narrative documentation. Automated extraction strategies query these data sources using defined logic to identify EGS cases and populate registry variables. For example, the system might flag cases combining emergency admission status, specific diagnosis codes, and surgical procedure codes occurring within defined time frames.

An example workflow is to use existing clinical infrastructure, such as the EGS service patient list, to, on a daily basis, generate a cohort of potential registry cases. Each patient’s unique identifier is used to automatically pull both structured and unstructured data from the EMR backend.

Data reliability analyzes the subsequent handling. High-reliability structured inputs such as age, sex, admission source, vital signs, and laboratory values are accepted directly into the registry with minimal review. Lower-reliability or unstructured data such as comorbidities, disease severity, and postoperative complications documented in narrative notes undergo a secondary validation step. Here, a trained large language model (LLM) can assign probability scores reflecting confidence in each extracted variable, flagging uncertain entries for human review. Critically, rather than requiring a registrar to re-read the entire medical record, the LLM can surface only the specific EMR data elements that informed each decision, focusing human effort where it is most needed. This tiered approach can substantially reduce the manual abstraction burden even for complex, unstructured data, making comprehensive registry capture feasible without proportional increases in staffing.

Advantages

The potential human resource advantages are immense. Automated extraction dramatically reduces or eliminates manual abstraction costs. After initial development and validation, the marginal cost per case approaches zero, making comprehensive capture of all EGS cases economically feasible regardless of volume. Real-time or near-real-time data availability enables rapid identification of complications, timely QI interventions, and contemporaneous feedback to clinical teams. Moreover, it will be possible to evaluate compliance with clinical pathways, supporting not only feedback of outcomes, but processes of care as well.²³

EMR-based registries can capture longitudinal data extending beyond the index hospitalization, tracking readmissions, emergency department visits, and ambulatory follow-up care documented within the system. For integrated health systems, this provides insight into the complete episode of care. The flexibility of custom queries allows institutions to define registry variables aligned with local QI priorities and still maintaining standardized elements for external benchmarking. This data can then be merged, combined, and queried for other ad hoc uses such as evaluating operating room utilization, future surgeon and service planning, PI identification, charge capture and cost containment, population growth and community needs.

Limitations

Substantial challenges temper enthusiasm for fully automated solutions. EMR data quality depends entirely on clinical documentation, the “garbage in, garbage out” principle applies with full force. Incomplete documentation, inconsistent terminology, and variable coding practices create data integrity issues that manual abstraction can overcome but automated systems cannot. Validation studies are lacking to ascertain sensitivity and specificity compared with trained human reviewers for many clinical variables. This often leads to significant upfront costs to the EGS clinical leads to pair with information technology (IT) and data management support personnel to ensure the correct logic, data capture, data categorization, and variable definitions are correct.

The technical complexity and resource requirements for development are significant. Building robust extraction logic requires expertise in health informatics, database architecture, clinical knowledge, and often programming skills. Institutions need dedicated analytic support, IT infrastructure, and clinical champions to design, implement, validate, and maintain automated registry systems. Initial development costs can be substantial, though operational costs decrease over time. Untoward consequences of seemingly small IT shifts, such as coding definitions, billing practices, and outdated surgeon lists can break models that have taken hundreds of hours to construct. Large IT shifts such as new EMR systems can fundamentally break this type of EGS registry, effectively starting from scratch. However, despite this, advances in artificial intelligence may make this type of registry both cost- and time-effective in the near future.

Comparative analysis and institutional considerations

These three approaches represent points along a spectrum trading off data quality, resource requirements, flexibility, and scalability, table 1. Third-party registry software with dedicated registrars offers the most relevant local data to impact change and established collaborative infrastructure but at the highest ongoing cost and with the longest time lag. NSQIP leverages existing infrastructure for credible, benchmarked, risk-adjusted data with moderate incremental cost but may be limited in the real-time impact of local EGS practices. Automated EMR extraction promises scalability and real-time data at low marginal cost but requires high upfront investment and expertise, raising data quality concerns. Importantly, as EGS is in the early stages of development for widespread data analytics, options for risk adjustment are limited in all approaches. However, this is an active area of development for EGS NSQIP.

Institutional characteristics should guide selection among these approaches, table 1. Large academic medical centers with high EGS volumes, robust IT infrastructure, and employed informaticists may find automated EMR extraction attractive, particularly for real-time clinical decision support and comprehensive capture. Community hospitals already participating in NSQIP may find the incremental investment to comprehensively capture EGS cases most efficient. For community hospitals without NSQIP participation or dedicated informatics support, focused manual abstraction of a limited EGS cohort offers a feasible entry point. An important consideration is also national benchmarking, of which the only current approach is NSQIP.

Hybrid approaches merit consideration. Institutions might employ automated EMR screening to identify potential EGS cases, which trained registrars then review for detailed abstraction—combining the scalability of automation with the quality assurance of human review. They may use these data to also contribute to EQIP to take advantage of national benchmarking. This would create a comprehensive dataset leveraging the strengths of both approaches. Notably, at the 2025 Quality and Safety Conference, the ACS announced their data strategy. In collaboration with EPIC over the coming years, many of the tools described in automated EMR extraction, including NLP, will be used to populate the NSQIP database—increasing the ability to capture more cases and minimizing the need for human resources.

Regardless of approach, registries enable rapid PI, pathway adherence monitoring, verification preparedness, and financial analysis.

Conclusion

The need to systematically track and improve EGS outcomes has never been clearer. As healthcare systems face increasing pressure to demonstrate value, reduce complications, and optimize resource utilization, the ad hoc approach to EGS quality measurement that has prevailed for decades must give way to rigorous, data-driven improvement. Beyond the clear patient benefits, EGS patients contribute significantly to morbidity and mortality metrics and are an obvious target for QI.

Each of the three approaches examined, dedicated registry software, NSQIP expansion, and automated EMR extraction, offers viable pathways to building an EGS registry capable of identifying cases, risk-adjusting outcomes, supporting local QI, and enabling national benchmarking. The optimal choice depends on institutional context, resources, strategic priorities, and existing infrastructure. Forward-thinking institutions may employ hybrid models that combine the strengths of multiple approaches. Important future work will provide valuable quantitative comparisons of each approach including patient capture rates, sensitivity/specificity and missed/misclassified patients.

The patients presenting to our emergency departments with surgical emergencies deserve no less than the assurance that their outcomes are monitored, analyzed, and continuously improved through rigorous registry-based QI. Finding our patients and tracking their outcomes with the same diligence we apply to other surgical populations, represents both a professional obligation and an opportunity to transform emergency surgical care. EGS registries are no longer aspirational; they are achievable using existing tools. Programs should select an approach aligned with their resources and begin systematic outcome tracking now.