Introduction:
The objectives of this study are (1) to develop a provider-friendly, evidence-based data capture system for lower-limb orthopaedic surgeries and (2) to assess the performance of the data capture system on the dimensions of agreement with operative note and implant log, consistency of data, and speed of provider input.
Methods:
A multidisciplinary team developed a database and user interfaces for Android and iOS operating systems. Branching logic and discrete fields were created to streamline provider data input. One hundred patients were randomly selected from the first four months of data collection (February to June 2015). Patients were limited to those undergoing anterior cruciate ligament reconstruction, meniscal repair, partial meniscectomy, or a combination of these procedures. Duplicate data on these 100 patients were collected through chart review. These two data sets were compared. Cohen's kappa statistic was used to assess agreement.
Results:
The database and smartphone data capture tool show almost perfect agreement (kappa > 0.81) for all data tested. In addition, data are more comprehensive with near-perfect provider completion (100% for all data tested). Furthermore, provider data entry is extremely efficient (median 151-second completion time).
Conclusion:
A well-designed database and user-friendly interface have greater potential for research utility, clinical efficiency, and, thus, cost-effectiveness when compared with standard voice-dictated operative notes. Widespread utilization of such tools can accelerate the pace and improve the quality of orthopaedic clinical research.
Level of Evidence:
Level IV
As health care becomes increasingly value driven, the ability to justify treatment is enhanced by prospective, high-quality, standardized databases.1 Examples of large anterior cruciate ligament reconstruction (ACLR) databases include the Swedish National Anterior Cruciate Ligament (ACL) Register (SNKRL), Danish Cruciate Ligament Registry (DKKR), Kaiser Permanente Anterior Cruciate Ligament Reconstruction Registry (KPACLRR), and the Multicenter Orthopaedic Outcomes Network and Multicenter ACL Revision Study cohort.2,3,4,5,6 The uses of high-quality surgical data include, but are not limited to, internal quality improvement initiatives, large-scale comparative effectiveness research, outcomes research, cost-effectiveness research, and clinical trials.7-9 Such data have greatly contributed to our understanding of surgical practice.10 The availability of prospective, standardized, high-quality data is a foundational component for the advancement of the field of orthopaedic surgery.
Several practical challenges exist in constructing large high-quality data sets. Much of the important data that are captured in the electronic medical record (EMR) cannot be accessed easily for statistical analysis.11 Error rates in the manual extraction of data from EMRs have been reported from 8% to 23%, with data varying by site, clinical area, and surgical specialty.12 Increasing the accuracy of such data through quality assurance and data review increases cost.12 Operative reports, a key data source for surgical research, tend to infrequently report quantitative data, markedly limiting the precision of research and quality measurement.13 Even best-in-class registry data are collected through a secured web-based or paper form, which have obvious limitations in workflow (both), data accessibility (paper), and an exhaustivity of 85% to 90% for surgeon-reported data.10
To further advance orthopaedic outcomes measurement nationally, investigators developed the OrthoMiDaS (Orthopaedic Minimal Data Set) Episode of Care (OME) database and data collection methodology. OME's goal is to accurately and consistently collect patient-reported outcome measures (PROMs) immediately before and at the time of peak function after high-volume elective orthopaedic surgeries, as well as to accurately and consistently collect information about the actual surgical intervention in a manner that is both faster and more detailed than previously designed outcomes database systems.
This study describes the development of the OME database and provider-friendly, evidence-based smartphone data collection methodology and assesses the performance of the OME data capture system on the dimensions of agreement with operative note and implant log, consistency of data, and speed of provider input. We hypothesized that the use of the unique, provider-friendly, smartphone-based OME data capture system would increase the quality of orthopaedic procedure data in the context of ACLR and meniscal repair arthroscopic procedures.
Methods
OrthoMiDaS Episode of Care Database Design
A multidisciplinary team including administrators, orthopaedic surgeons, and software developers was assembled for design purposes. The OME database collects the following two distinct classes of data: PROMs and procedural data about the orthopaedic surgery itself. PROMs collected for arthroscopic knee surgeries include the Knee injury and Osteoarthritis Score, The Hospital for Special Surgery Pediatric Functional Activity Brief Scale, and the Veterans Rand 12-Item Health Survey.14,15,16,17 Upon check-in at the surgery center, patients receive an iPad with their PROM form and complete it before going back into the preoperative holding area. This process is designed to be built into the standard clinical workflow so as not to slow down the operating rooms or require any additional staffing to execute.
The second class of data, the procedural details about surgery, are entered into OME by the surgeons themselves immediately after surgery using their hospital-issued smartphones or desktop/laptop computers (Figure 1). Demographics such as height, weight examination under anesthesia findings, commonly cited operative report parameters, and key predictors of surgical outcomes for ACLR, meniscal repair, and cartilage repair as identified in the literature are collected in the form of discrete data entry fields.6,18-20 In total, OME currently provides 449 fields in the knee arthroscopy surgery data set. Branching logic is used to streamline provider data input. For primary meniscal repair and ACLR procedures, the branching logic system will require the surgeon to input approximately 13 fields for each meniscal repair and approximately 16 fields for the ACLR. However, the number of fields will expand as complexity increases. The basic set of meniscal pathology and repair data are shown in Appendix 1. Within the user interface, the branching logic displays fields only relevant to the individual procedure and pathology, greatly expediting the data collection process and decreasing the cognitive load on the surgeon. For example, selecting meniscal repair presents the surgeon with repair types, followed by manufacturers of implants/devices, and then specific device/implant offerings by that company, guiding the surgeon to procedure-specific details rather than requiring unnecessary data input as is often the case in EMR template usage. On the morning of surgery, surgeons receive reminder e-mails for each surgery they are about to perform that day with links to the surgeries' corresponding OME surgeon forms; like the PROMs-collection workflow, surgical data collection is built into surgeons' workflows to minimize delays in the daily routine.
Figure 1.
OrthoMiDaS Episode of Care iPhone interface demonstrating detailed data capture and speed dial provider–specific templates. ACL = anterior cruciate ligament, ACLR = anterior cruciate ligament reconstruction
Architecturally, OME exists as a collection of research electronic data capture (REDCap) databases managed by the custom-built software that manages the multiplatform data entry and distribution.21,22 All software is hosted locally and was approved by the local institutional review board (IRB) and information security. This study was also approved by the local IRB (IRB# 06-196).
Patient Selection
The OME database was launched on February 18, 2015. One hundred patients undergoing ACLR, meniscal repair, partial meniscectomy, or a combination of these procedures were included in the data set from the first four months of data collection (February through June 2015). All 12 surgeons at the sports health center who performed these procedures were included in the data set.
Data Collection and Validation
OME data used in this study were prospectively captured by the surgeons directly following their surgical procedures and were exported into a study database in an automated fashion. To evaluate the agreement with surgeons' operative dictation and/or implant logs, an independent REDCap database was established. Independent chart review data for comparison were collected from the operative report and the implant log in the Epic EMR system (Epic Systems). Reviewers of the operative report and the implant log were blinded to the OME REDCap results. Before analysis, the two data sets were reviewed for discrepancies, and all unmatching data were rechecked.
Assumptions in Chart Review Data Collection
The following assumptions were made in obtaining chart-reviewed operative dictation data: (1) If only one tunnel or reamer size was specified, both tibial and femoral tunnels were assumed to be the same size. (2) The operative report was used to determine the implant number and type; when a discrepancy or lack of clarity arose, the reviewer deferred to the implant log. (3) If the bone reamer or tunnel sizes were not specified, the graft size was used to approximate the tunnel size. (4) Regarding ACL status, normal status and no status were considered equivalent.
Statistical Analyses
All analyses were performed using R software (R version 3.2.3 [2015-12-10]). Agreement on nominal variables was measured by Cohen's (unweighted) kappa. 0.81 to 1.00 was considered almost perfect agreement; 0.61 to 0.80 was considered substantial agreement; and 0.41 to 0.60 was considered moderate agreement.23,24 Agreement on numeric variables was measured by the concordance correlation coefficient (CCC). 0.99 to 1.00 was considered almost perfect agreement; 0.95 to 0.99 was considered substantial agreement; 0.90 to 0.95 was considered moderate agreement; and <0.90 was considered poor agreement.25 Operative report and OME completion rates are presented as percent completion and are compared using the McNemar test (with continuity correction).
Results
Basic Demographics
The sample included 100 patients undergoing arthroscopic ACLR, arthroscopic meniscal repair, or both. The median age of the patients was 23 years (95% confidence interval [CI] [24.2, 28.7]), and 48% (n = 48) were women. This sample included 94 ACLRs, 58 partial meniscectomies, and 26 meniscal repairs as described in Table 1.
Table 1.
Knee Arthroscopy Patient Characteristics
| Factor | N (%) | N |
| Characteristic | ||
| N | 100 | |
| Female | 48 (48.0%) | |
| Median age (95% CI) | 23 (24.2, 28.7) | |
| Operative limb | 100 | |
| Both | 1 (1.00%) | |
| Left | 45 (45.0%) | |
| Right | 54 (54.0%) | |
| Medial meniscus | ||
| Meniscus status | 100 | |
| Complete tear | 28 (28.0%) | |
| Normal | 55 (55.0%) | |
| Partial tear | 17 (17.0%) | |
| Main tear type | 45 | |
| Bucket-handle | 9 (20.0%) | |
| Complex | 7 (15.6%) | |
| Horizontal | 2 (4.44%) | |
| Longitudinal | 27 (60.0%) | |
| Location to horns | 45 | |
| Both | 8 (17.8%) | |
| Posterior only | 37 (82.2%) | |
| Location to blood supply | 36 | |
| Red-red | 14 (38.9%) | |
| Red-white | 16 (44.4%) | |
| White-white | 6 (16.7%) | |
| Main tear length | 45 | |
| <6 mm | 5 (11.1%) | |
| 6–10 mm | 4 (8.89%) | |
| 11–15 mm | 11 (24.4%) | |
| 16–20 mm | 18 (40.0%) | |
| >20 mm | 7 (15.6%) | |
| Treatment | 45 | |
| Abrade + trephine | 1 (2.22%) | |
| No treatment for tear | 9 (20.0%) | |
| Partial excision | 15 (33.3%) | |
| Repair | 20 (44.4%) | |
| Repair technique | 20 | |
| All inside | 19 (95.0%) | |
| Both inside-out and all-in | 1 (5.00%) | |
| Implant manufacturer | 20 | |
| Arthrex | 3 (15.0%) | |
| Smith & Nephew | 17 (85.0%) | |
| Implant system | 20 | |
| DartStick | 3 (15.0%) | |
| FAST-FIX 360 | 17 (85.0%) | |
| Number of implants Med [Q1, Q3] | 2.00 [2.00; 3.00] | 20 |
| Lateral meniscus | ||
| Meniscus status | 100 | |
| Complete tear | 48 (48.0%) | |
| Normal | 43 (43.0%) | |
| Partial tear | 9 (9.00%) | |
| Main tear type | 57 | |
| Bucket-handle | 5 (8.77%) | |
| Complex | 12 (21.1%) | |
| Horizontal | 3 (5.26%) | |
| Longitudinal | 12 (21.1%) | |
| Oblique/flap | 9 (15.8%) | |
| Radial | 14 (24.6%) | |
| Root | 2 (3.51%) | |
| Location to horns | 57 | |
| Anterior only | 4 (7.02%) | |
| Both | 13 (22.8%) | |
| Posterior only | 40 (70.2%) | |
| Location to blood supply | 17 | |
| Red-red | 3 (17.6%) | |
| Red-white | 7 (41.2%) | |
| White-white | 7 (41.2%) | |
| Main tear extent | 16 | |
| Complete to periphery | 4 (25.0%) | |
| Partial, periphery intact | 12 (75.0%) | |
| Main tear length | 57 | |
| <6 mm | 10 (17.5%) | |
| 6–10 mm | 14 (24.6%) | |
| >20 mm | 11 (19.3%) | |
| 11–15 mm | 19 (33.3%) | |
| 16–20 mm | 3 (5.26%) | |
| >20 mm | 11 (19.3%) | |
| Treatment | 57 | |
| No treatment for tear | 8 (14.0%) | |
| Partial excision | 43 (75.4%) | |
| Repair | 6 (10.5%) | |
| Repair technique | 6 | |
| All inside | 5 (83.3%) | |
| Inside-out | 1 (16.7%) | |
| Implant system | 5 | |
| DartStick | 1 (20.0%) | |
| FAST-FIX 360 | 4 (80.0%) | |
| Number of implants, Med [Q1, Q3] | 3.00 [2.00; 4.00] | 5 |
| Anterior cruciate ligament | ||
| ACL status | 100 | |
| Complete tear | 91 (91.0%) | |
| Normal | 6 (6.00%) | |
| Partial tear | 3 (3.00%) | |
| ACLR performed | 94 | |
| Primary | 93 (98.9%) | |
| Revision | 1 (1.06%) | |
| Graft | 94 | |
| ALLO Achilles | 5 (5.32%) | |
| ALLO other (HG, TA, PT, and ITB) | 1 (1.06%) | |
| Auto BTB | 32 (34.0%) | |
| Auto HG | 56 (59.6%) | |
| Number of strands, Med [Q1, Q3] | 4.00 [1.00; 5.00] | 94 |
| Femur tunnel (mm), Med [Q1, Q3] | 9.00 [8.50; 10.0] | 94 |
| Tibia tunnel (mm), Med [Q1, Q3] | 9.00 [8.62; 10.0] | 94 |
| Primary femoral fixation | 94 | |
| Cross-pin | 8 (8.51%) | |
| Interference screw | 32 (34.0%) | |
| Suspensory | 54 (57.4%) | |
| Primary femoral screw type | 32 | |
| Bioabsorbable | 14 (43.8%) | |
| Metal | 18 (56.2%) | |
| Primary tibial fixation | 94 | |
| Interference screw | 77 (81.9%) | |
| Suspensory | 17 (18.1%) | |
| Primary tibial screw type | 77 | |
| Bioabsorbable | 41 (53.2%) | |
| Metal | 26 (33.8%) | |
| PEEK | 10 (13.0%) |
ACL = anterior cruciate ligament, ACLR = anterior cruciate ligament reconstruction, ALLO = allograft, BTB = bone-patellar tendon-bone, CI 2= confidence interval, HG = hamstring graft, ITB = iliotibial band, PEEK = polyetheretherketone, PT = patellar tendon, TA = tibialis anterior
Agreement
OME and chart-reviewed data comparison showed “near perfect” (kappa ≥ 0.81) or “substantial” to “almost perfect” agreement (CCC > 0.95) for all data tested (Table 2). The highest agreement level among nominal variables occurred in the reporting of graft type (kappa = 1.000, 95% CI [1.000, 1.000]), medial meniscus implant system (kappa = 1.000, 95% CI [1.000, 1.000]), lateral meniscus implant system (kappa = 1.000, 95% CI [1.000, 1.000]), and lateral meniscus treatment (kappa = 1.000, 95% CI [1.000, 1.000]). The lowest degree of agreement occurred in reporting of lateral meniscus status (kappa = 0.859, 95% CI [0.759, 0.960]). For the three numeric variables, the CCC was lowest for femur tunnel size (CCC = 0.977, 95% CI [0.964, 0.985]) and highest for graft strand number (CCC = 0.998, 95% CI [0.997, 0.999]).
Table 2.
Agreement Between Chart-Reviewed and OrthoMiDaS Episode of Care Data
| Measure | n | Agreement Statistic | 95% Confidence Interval |
| Operative limb | 100 | 0.940 | 0.873, 1.000 |
| ACL | |||
| Status | 100 | 0.904 | 0.716, 1.000 |
| Graft type | 93 | 1.000 | 1.000, 1.000 |
| Graft strand number | 75 | 0.998a | 0.997, 0.999 |
| Femur tunnel size | 79 | 0.977a | 0.964, 0.985 |
| Primary femoral fixation type | 94 | 0.961 | 0.908, 1.000 |
| Femoral screw type | 31 | 0.871 | 0.699, 1.000 |
| Tibial tunnel size | 79 | 0.990a | 0.984, 0.993 |
| Primary tibial fixation type | 93 | 0.925 | 0.821, 1.000 |
| Tibial screw type | 76 | 0.934 | 0.860, 1.000 |
| Medial meniscus | |||
| Status | 100 | 0.898 | 0.812, 0.985 |
| Treatment | 37 | 0.952 | 0.860, 1.000 |
| Implant system | 18 | 1.000 | 1.000, 1.000 |
| Lateral meniscus | |||
| Status | 100 | 0.859 | 0.759, 0.960 |
| Treatment | 46 | 1.000 | 1.000, 1.000 |
| Implant system | 5 | 1.000 | 1.000, 1.000 |
ACL = anterior cruciate ligament
Agreement between numeric variables is measured through the concordance correlation coefficient.
Provider Completion Rate
Graft strand number was reported in OME in 100% of the ACLR patients, but only in 80% of the ACLR patients in the operative reports. Femur tunnel size was reported in OME in 100% of the ACLR patients, but only in 85% of the ACLR patients in the operative reports. Tibial tunnel size was reported in OME in 100% of the ACLR patients, but only in 85% of the ACLR patients in the operative reports (Table 3).
Table 3.
Provider Data Completion Rates
| Measure | Operative Report | OrthoMiDaS Episode of Care Database | P Value | ||
| n | % Completion | n | % Completion | ||
| ACL | |||||
| Graft strand number | 75 | 80 | 94 | 100 | P < 0.001 |
| Femur tunnel size | 80 | 85 | 94 | 100 | P < 0.001 |
| Tibial tunnel size | 80 | 85 | 94 | 100 | P < 0.001 |
ACL = anterior cruciate ligament
Time
The median provider time to complete the data entry for a single patient was approximately 2 minutes.
Discussion
Agreement
Although all data tested showed near-perfect agreement, ACL status (kappa = 0.904), lateral meniscus status (kappa = 0.898), femoral screw type (kappa = 0.871), and medial meniscus status (kappa = 0.859) displayed the least agreement of all data tested. Despite the obvious need for accuracy in determining these parameters, a high degree of variability exists in surgeons' operative dictation description of these lesions and implants. Some surgeons provided detailed, quantitative descriptions (eg, 5-mm, full-thickness, radial tear in the posterior horn), whereas other surgeons gave very brief descriptions (eg, small tear in the lateral meniscus), leading to the inability to extract accurate data from the operative report.
Improved Completion Rate
Historically, the operative report is used as a key data source for most orthopaedic retrospective research. Despite widespread use, the quality of these data is limited.26 Operative reports tend to infrequently report quantitative data, markedly limiting the precision of research and quality measurement.13 Moreover, late dictation of reports, or dictation by residents, may also increase error rates.27,28 This error is compounded by errors in data extraction from the medical record itself. This limited quality of data contributes to the large gap, both in validity and reliability, between prospective and retrospective research.29 Together, these limitations of operative report data delay the progress of the field and cause increased cost burden.
Some improvements in the quality of capturing surgical data have been previously made. Customized, computerized, templated operative reporting systems are available and can improve the consistency of reporting of key operative parameters.30 The branching logic of OME not only improves the completeness of reporting operative parameter details but also maximizes efficiency in data input. Such systems can dramatically decrease completion delays and reduce the cost of surgical documentation.31
Before this study, templates and computer assistance are effective in increasing the completeness of operative report data.30 This finding was consistent with our findings that graft strand number, femur tunnel size, and tibial tunnel size were all more frequently collected in the OME data capture system. The OME user interface incorporates dropdown menus that likely function as memory aids for key surgical parameters.30 The OME computer defaults and branching logic will not accept missing key risk factors identified in the most recent ACLR and meniscal outcomes research.6,18-20 Branching logic and the smartphone interface speed the data entry process. Finally, an automated e-mail reminder system is used to ensure the high data capture rate.
Limitations
Currently, although the operative report is widely used, no true benchmark exists for operative reporting. Thus, the absolute accuracy of the OME database (ie, OME data compared directly with true occurrences in the operating room) is difficult to assess. However, the present study indicates OME data are consistent with the current most pervasive methodology and demonstrate less information loss.
Conclusion
The OME data capture system demonstrated “almost perfect” agreement (kappa ≥ 0.81) on all 13 nominal variables and “substantial” agreement (CCC > 0.95) to “almost perfect” agreement (CCC > 0.099) on all three numeric variables tested. In addition, the OME data capture system improved the reporting of key operative parameters necessary for outcomes research and internal quality improvement (100% for all data tested). Furthermore, the developers of this system continue to develop branching logic and data capture tools for additional high-volume orthopaedic surgical procedures. Moreover, widespread use of this system can bring the highest level of prospective quality data to everyday surgical practice. Finally, this technology could potentially replace standard narrative/dictation-based operative reporting, and transform observational and retrospective orthopaedic clinical research, to a prospective cohort model.
Appendix 1. OrthoMiDaS Episode of Care Basic Meniscal Data Fields
| Data Field | Dropdown Selections |
| Meniscus status | Normal |
| Complete tear | |
| Partial tear | |
| Previous partial excision | |
| Previous repair | |
| New tear | None |
| Partial | |
| Complete | |
| Main tear type | Oblique/flap |
| Longitudinal | |
| Bucket-handle | |
| Radial | |
| Root | |
| Horizontal | |
| Complex | |
| Location to horns | Anterior only |
| Posterior only | |
| Both | |
| Location to blood supply | Red-red |
| Red-white | |
| White-white | |
| Main tear extent | Partial, periphery intact |
| Complete to periphery | |
| Main tear length (mm) | <6 mm |
| 6–10 mm | |
| 11–15 mm | |
| 16–20 mm | |
| >20 mm | |
| Treatment | No treatment for tear |
| Partial excision | |
| Repair | |
| Abrade + trephine | |
| Meniscal transplant | |
| Repair technique | All inside |
| Inside-out | |
| Both inside-out and all in | |
| Outside-in | |
| Number of inside-out and/or outside-in sutures | Arthrex |
| Biomet | |
| Covidien | |
| CONMED | |
| Cayenne | |
| DePuy Mitek | |
| Smith & Nephew | |
| Others | |
| Implant system | Select from preprogrammed implant system library specific to manufacturer selection |
| Number of implants | Select integer value between 0 and 10 |
| Discoid meniscus | No |
| Yes, partial | |
| Yes, complete |
Footnotes
Research reported in this publication was partially supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number R01AR053684 (Dr. Spindler) and under award number K23AR066133, which supported a portion of Morgan H. Jones's professional effort. The mentioned grant did not support this specific publication.
Mr. Strnad reports other from nPhase during the conduct of the study. Dr. Jones reports grants from NIH, personal fees from Samumed, and personal fees from Journal of Bone and Joint Surgery, outside the submitted work. Dr. Miniaci reports personal fees from Arthrosurface, other from Zimmer Biomet, other from Wolters Kluwer, other from Amniox Medical, other from Rock Medical, other from Linvatec Corporation, other from Stryker, other from Trice, outside the submitted work; and Board of ASES Foundation, Arthrosurface, Trice. Dr. Rosneck reports personal fees from Smith & Nephew, outside the submitted work. Dr. Spindler reports other from nPhase, during the conduct of the study; grants from NIH/NIAMS R01AR053684, other from Smith & Nephew Endoscopy, other from DonJoy Orthopaedics, other from NFL, other from Cytori, other from Mitek, other from Samumed, and other from Flexion Therapeutics, outside the submitted work. None of the following authors or any immediate family member has received anything of value from or has stock or stock options held in a commercial company or institution related directly or indirectly to the subject of this article: Mr. Featherall, Dr. Oak, Dr. Farrow, Dr. Parker, and Dr. Saluan.
OME development: Michael Kattan and Alex Milinovich. Editorial assistance: Elizabeth Sosic and Brittany Stojsavljevic. Orthopaedic surgeons/data contributors: Thomas Anderson, Alan Davis, Lutul Farrow, Timothy Gajewski, Ryan Goodwin, Morgan Jones, Anthony Miniaci, Richard Parker, James Rosneck, Paul Saluan, and Joseph Scarcella. Statistician: William Messner.
The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health. Thanks to the Cleveland Clinic orthopaedic patients, staff, and research personnel whose efforts related to regulatory, data collection, subject follow-up, data quality control, analyses, and manuscript preparation have made this consortium successful.
References
References printed in bold type are those published within the past 5 years.
- 1.Porter ME, Larsson S, Lee TH: Standardizing patient outcomes measurement. N Engl J Med 2016;374:504-506. [DOI] [PubMed] [Google Scholar]
- 2.Ahlden M, Samuelsson K, Sernert N, Forssblad M, Karlsson J, Kartus J: The Swedish National Anterior Cruciate Ligament Register: A report on baseline variables and outcomes of surgery for almost 18,000 patients. Am J Sports Med 2012;40:2230-2235. [DOI] [PubMed] [Google Scholar]
- 3.Lind M, Menhert F, Pedersen AB: Incidence and outcome after revision anterior cruciate ligament reconstruction: Results from the Danish registry for knee ligament reconstructions. Am J Sports Med 2012;40:1551-1557. [DOI] [PubMed] [Google Scholar]
- 4.Paxton EW, Namba RS, Maletis GB, et al. : A prospective study of 80,000 total joint and 5000 anterior cruciate ligament reconstruction procedures in a community-based registry in the United States. J Bone Joint Surg Am 2010;92(Suppl 2):117-132. [DOI] [PubMed] [Google Scholar]
- 5.Lynch TS, Parker RD, Patel RM, et al. : The impact of the Multicenter Orthopaedic Outcomes Network (MOON) research on anterior cruciate ligament reconstruction and orthopaedic practice. J Am Acad Orthop Surg 2015;23:154-163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wright RW, Huston LJ, Haas AK, et al. : Effect of graft choice on the outcome of revision anterior cruciate ligament reconstruction in the Multicenter ACL Revision Study (MARS) cohort. Am J Sports Med 2014;42:2301-2310. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pugely AJ, Martin CT, Harwood J, Ong KL, Bozic KJ, Callaghan JJ: Database and registry research in orthopaedic surgery: Part I: Claims-based data. J Bone Joint Surg Am 2015;97:1278-1287. [DOI] [PubMed] [Google Scholar]
- 8.Pugely AJ, Martin CT, Harwood J, et al. : Database and registry research in orthopaedic surgery: Part 2: Clinical registry data. J Bone Joint Surg Am 2015;97:1799-1808. [DOI] [PubMed] [Google Scholar]
- 9.Mather RC, Hettrich CM, Dunn WR, et al. : Cost-effectiveness analysis of early reconstruction versus rehabilitation and delayed reconstruction for anterior cruciate ligament tears. Am J Sports Med 2014;42:1583-1591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Boyer P, Villain B, Pelissier A, et al. : Current state of anterior cruciate ligament registers. Orthop Traumatol Surg Res 2014;100:879-883. [DOI] [PubMed] [Google Scholar]
- 11.Capurro D, Yetisgen M, van Eaton E, Black R, Tarczy-Hornoch P: Availability of structured and unstructured clinical data for comparative effectiveness research and quality improvement: A multi-site assessment. EGEMS (Wash DC) 2014;2:1079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Pan L, Pan L, Fergusson D, Schweitzer I, Hebert PC: Ensuring high accuracy of data abstracted from patient charts: The use of a standardized medical record as a training tool. J Clin Epidemiol 2005;58:918-923. http://journals.ohiolink.edu/ejc/article.cgi?issn=08954356&issue=v58i0009&article=918_ehaodamraatt. Accessed July 15, 2016. [DOI] [PubMed] [Google Scholar]
- 13.Scherer R, Zhu Q, Langenberg P, Feldon S, Kelman S, Dickersin K: Comparison of information obtained by operative note abstraction with that recorded on a standardized data collection form. Surgery 2003;133:324-330. [DOI] [PubMed] [Google Scholar]
- 14.Spindler KP, Huston LJ, Wright RW, et al. : The prognosis and predictors of sports function and activity at minimum 6 years after anterior cruciate ligament reconstruction: A population cohort study. Am J Sports Med 2011;39:348-359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Fabricant PD, Robles A, Downey-Zayas T, et al. : Development and validation of a pediatric sports activity rating scale: The Hospital for Special Surgery Pediatric Functional Activity Brief Scale (HSS Pedi-FABS). Am J Sports Med 2013;41:2421-2429. [DOI] [PubMed] [Google Scholar]
- 16.Selim AJ, Rogers W, Fleishman JA, et al. : Updated U.S. population standard for the Veterans RAND 12-Item Health Survey (VR-12). Qual Life Res 2009;18:43-52. [DOI] [PubMed] [Google Scholar]
- 17.Roos EM, Lohmander LS: The Knee injury and Osteoarthritis Outcome Score (KOOS): From joint injury to osteoarthritis. Health Qual Life Outcomes 2003;1:64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kaeding CC, Pedroza AD, Reinke EK, Huston LJ, Spindler KP: Risk factors and predictors of subsequent ACL injury in either knee after ACL reconstruction: Prospective analysis of 2488 primary ACL reconstructions from the MOON cohort. Am J Sports Med 2015;43:1583-1590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Katz JN, Brophy RH, Chaisson CE, et al. : Surgery versus physical therapy for a meniscal tear and osteoarthritis. N Engl J Med 2013;368:1675-1684. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Cox CL, Huston LJ, Dunn WR, et al. : Are articular cartilage lesions and meniscus tears predictive of IKDC, KOOS, and Marx activity level outcomes after anterior cruciate ligament reconstruction?: A 6-year multicenter cohort study. Am J Sports Med 2014;42:1058-1067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Shah J, Rajgor D, Pradhan S, McCready M, Zaveri A, Pietrobon R: Electronic data capture for registries and clinical trials in orthopaedic surgery: Open source versus commercial systems. Clin Orthop Relat Res 2010;468:2664-2671. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377-381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Viera AJ, Garrett JM: Understanding interobserver agreement: The kappa statistic. Fam Med 2005;37:360-363. [PubMed] [Google Scholar]
- 24.Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174. [PubMed] [Google Scholar]
- 25.McBride GB: A proposal of strength of agreement criteria for Lin's concordance correlation coefficient: NINA Client Report 2005, New Zeal, Natl Inst Water Atmos Res Ltd Hamilton, 2005, pp 1-10. [Google Scholar]
- 26.Severn Audit and Research Collaborative in Orthopaedics (SARCO), Blackburn J: Assessing the quality of operation notes: A review of 1092 operation notes in 9 UK hospitals. Patient Saf Surg 2016;10:5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Novitsky YW, Sing RF, Kercher KW, Griffo ML, Matthews BD, Heniford BT: Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents. Am Surg 2005;71:627-631. [PubMed] [Google Scholar]
- 28.ten Broek RPG, van den Beukel BAW, van Goor H: Comparison of operative notes with real-time observation of adhesiolysis-related complications during surgery. Br J Surg 2013;100:426-432. [DOI] [PubMed] [Google Scholar]
- 29.Nagurney JT, Brown DFM, Sane S, Weiner JB, Wang AC, Chang Y: The accuracy and completeness of data collected by prospective and retrospective methods. Acad Emerg Med 2005;12:884-895. [DOI] [PubMed] [Google Scholar]
- 30.Gur I, Gur D, Recabaren JA, et al. : The computerized synoptic operative report. Arch Surg 2012;147:71. [DOI] [PubMed] [Google Scholar]
- 31.Laflamme MR, Dexter PR, Graham MF, Hui SL, McDonald CJ: Efficiency, comprehensiveness and cost-effectiveness when comparing dictation and electronic templates for operative reports. AMIA Annu Symp Proc 2005;2005:425-429. [PMC free article] [PubMed] [Google Scholar]