Abstract
Introduction
The aim of this study was to validate the registration in the Danish Knee Ligament Reconstruction Register (DKRR) by assessing the registration completeness of the anterior cruciate ligament (ACL) reconstruction code and detecting the validity of important key variables. Furthermore, we assessed data quality of patient-related outcome scores.
Material and methods
All operation codes for ACL reconstruction from 2005–2011 were identified in the Danish National Registry of Patients and were compared with the cases registered in the DKRR to compute the completeness of registration. We also assessed the validity of key variables in the DKRR using medical records as a reference standard to compute the positive predictive value. Finally, we assessed potential differences between responders and nonresponders to subjective patient-related outcome scores (Knee Injury and Osteoarthritis Outcome Score [KOOS] and Tegner scores) 1 year after surgery.
Results
The completeness of the registration of patients in the DKRR increased from 60% (2005) to 86% (2011). Large-volume hospitals had a higher completeness than small-volume hospitals. With a positive predictive value between 85%–100%, the validity of key variables was good. KOOS scores versus Tegner scores for responders and nonresponders were comparable.
Conclusion
The results show a good registration of ACL reconstruction procedures in the DKRR, but there is room for improvement mainly at small-volume hospitals. Overall, the validity of the key variables in the DKRR was good and no difference was found in KOOS and Tegner scores for responders versus nonresponders. Therefore, we conclude that the DKRR is a valid source for future research.
Keywords: ACL, anterior cruciate ligament registry, predictive value
Introduction
Anterior cruciate ligament (ACL) injury is a common and serious injury seen in the young and active population. It is important to evaluate which operation method ensures the optimal result for the individual patient. The surgeon and the patient are faced with the difficult task of deciding in favor of either conservative rehabilitation or surgery to achieve the best outcome and, if the choice falls on surgery, which kind of surgery is best for the patient. In a recent descriptive epidemiological study, Nordenvall et al1 described the incidence of 78 ACL injuries per 100,000 inhabitants in Sweden. A total of 38% of these patients underwent ACL reconstruction. On the basis of this Swedish ACL registry, they found that patients who underwent surgery were younger than those who were offered initial rehabilitation.1 The study by Nordenvall et al1 shows the importance of reporting baseline epidemiological data to facilitate validation and assessment of the generalizability of the results from registers and clinical studies.
Detailed knowledge on ACL lesions and treatment modalities is of critical importance for the study of postreconstruction graft survival and its long-term clinical outcome. Clinical studies on the basis of primary data collection have been performed to evaluate the outcome after ACL reconstruction. However, these studies are often small and are based on single-center data because data collection is time-consuming and costly, and their design is often open to recall and selection bias.2,3 Large, population-based studies are therefore desirable.
Recent years have seen advances in information technology that have facilitated access to large clinical databases. This gives clinicians the possibility to assess disease outcomes on a large scale. Clinical databases are an attractive source for epidemiological research for many reasons: they are readily available, they contain large amounts of data that otherwise could not be obtained, they carry little risk of bias, and they afford low-cost data access.4,5 In addition, use of clinical databases for research fosters timely and early dissemination of information on specific clinical issues. The Danish Knee Ligament Reconstruction Register (DKRR) is one such population-based database, which has been monitoring the quality and developments in ACL reconstructions since 2005.6
It is of crucial importance to evaluate the registration completeness and to validate the data quality of key variables in the database to be able to draw valid and reliable conclusions. Furthermore, continuous improvement of the validity and reliability of data is important to the future use of national registries for clinical and research purposes.5 In Denmark, national clinical registries for joint replacement have existed for several decades; they have been validated and are deemed highly reliable.7 However, the validity of the DKRR data is not known.
Therefore, the objective of our study was to validate the data in the DKRR. Specifically, we aimed to (1) assess the registration completeness of the ACL reconstruction; (2) validate the data quality of key variables in the DKRR; and (3) validate the quality of patient-related outcome scores by tracing differences between responders and nonresponders, and differences in the Knee Injury and Osteoarthritis Outcome Scores (KOOS) and Tegner scores registered for these groups.
Methods and materials
Data sources
We conducted a validation study of the population-based national DKRR. Denmark has a population of 5.5 million people with free health care to all citizens. Patients with acute medical conditions are treated by specialists at public hospitals. Private hospitals are also accessible in Denmark, and they also have reimbursement agreements with the Danish state, as well as private insurance patients and self-paying patients. Danish citizens are registered in various administrative and medical registers with a unique personal security number. Because this personal identification number is consistently used in all Danish registries, it is possible to obtain precise individual-level data through data linkage among the Danish registries.
The Danish Knee Ligament Reconstruction Register (DKRR)
The DKRR is a nationwide clinical database that was established on July 1, 2005. The purpose of this registry was to improve the monitoring and quality of both primary and revision knee ligament surgery in Denmark.6 According to Declaration number 459 of June 15, 2006, registration is compulsory, and all public (n = 24) and private (n = 27) hospitals report to this registry.8 Using a standardized form and a secured internet portal, detailed preoperative, intraoperative, and 1-year follow-up data are recorded by the operating surgeon.6 Furthermore, the patients report outcome scores on the functioning of their knee using the following self-assessment scores: the KOOS and the Tegner functional score.9,10 These data are recorded online by the patient before surgery and 1 year after surgery.
The KOOS range from 0–100 with higher scores representing better results. KOOS is a patient-reported outcome measure used in the evaluation of knee function.9 The KOOS was developed for younger, physically active patients with knee injuries and osteoarthritis and has proven to be a very responsive instrument. The KOOS consists of five subscores: sports, pain, quality of life, activities of daily living, and symptoms. The five subscores should be evaluated separately.
The Tegner scores range from 1–10, with higher scores representing better results. The Tegner score represents specific activities. Hence, a patient participating in competitive sports at the elite level is considered to have a Tegner score of 10, and an individual with sports activities at a recreational level is considered to have a Tegner score of 6. Patients on sick leave or those who are receiving a disability pension because of knee problems are considered to have a Tegner activity score of 0.11
Danish National Registry of Patients
The Danish National Registry of Patients (DNRP) includes data on 99.4% of all discharges from Danish nonpsychiatric hospitals since 1977, and outpatient visits since 1995.12 Data in the DNRP include each individual’s personal security number, admission and discharge date, discharge diagnosis, operations, and so on.13 In Denmark, hospital discharge codes are registered by the International Classification of Diseases 10th Revision of 1994,14 and operation codes are registered according to the Nordic Medico-Statistical Committee (NOMESCO), which was established in 1966.15 These discharge and operation codes are recorded by the physicians, and without these registrations there will be no financial reimbursement from the Danish state to the hospitals.
In this study, we used the DNRP to identify patients with ACL reconstruction using the following NOMESCO codes: NGE45 (NGE45B, NGE45C, NGE45D, NGE45E): “Primary arthroscopic plastic repair of ACL of knee not using prosthetic material;”15 and NGE55C: “Primary arthroscopic plastic repair of ACL of knee using prosthetic material.”15
Medical records
For the validation of the key variables, we aimed to review medical records from a random sample of approximately 5% of all primary ACL reconstruction surgeries registered in the DKRR from January 1, 2008, until December 31, 2009 (n = 240). A computer-generated random sample was obtained from six different hospitals, both private and public, throughout Denmark. A 5% random sample was chosen to ensure a reasonable statistical precision of the estimated positive predictive values (PPVs), relying on the sample size used in the validation studies of registry data previously published in a similar area of research.16 The medical records were systematically reviewed and information was retrieved on these variables:
Cartilage injury
Meniscal injury treatment
Activity leading to ACL rupture
Diagnosis registered as ACL rupture
Choice of graft
Choice of femoral tunnel placement (anteromedial or transtibial technique)
Number of femoral tunnels
Date of operation
Choice of technique for femoral and tibial fixation.
The medical record review was performed by a single independent researcher (LR-W), who was not involved in the treatment and who used a standardized form and the EpiData (EpiData Association, Odense, Denmark) program.
EpiData is a free data entry and data documentation program accessable on the Internet (http://www.epidata.dk). This program facilitates secure data entry owing to its different features of error detection. To optimize security in data entry in this study, preinstalled “checks” were made in the EpiData, which made it impossible to enter invalid numbers not previously defined by the reviewer. Furthermore, data entry and review were done twice.
Study population
Completeness of patient registration
In the first analysis, all operation codes for primary ACL reconstructions performed between July 1, 2005, and December 31, 2011 were identified in the DNRP and DKRR. The registration in the DNRP and DKRR was compared to compute the registration completeness of the ACL reconstruction surgeries in the DKRR.
We identified 14,943 primary ACL procedures in 14,721 patients from the DKRR. In a likewise fashion, we identified 17,276 primary ACL procedures in 16,734 patients from the DNRP. We performed matching between the DKRR and DNRP at the individual level using the unique personal security numbers. Records of which knee had been operated on were not available in 33% of the ACL reconstructions registered in the DNRP. Therefore, we decided to include only the first operation for all patients in the registry. This method implied exclusion of 222 operated knees from the DKRR and 542 operated knees from the DNRP. For 221 knees, the year of operation registered was not the same in the DNRP and DKRR. These knees were therefore excluded. This left us with 14,500 operated knees registered in the DKRR and 16,513 operated knees registered in the DNRP.
In accordance with Ytterstad et al,17 large-volume hospitals were defined as hospitals performing more than 30 operations a year and small-volume hospitals as hospitals performing 30 or less operations per year.
Completeness was stratified on age because previous studies have shown that the risk for revision surgery seems to be higher in the young-age group than in the older-age group.8,18
Data quality of key variables
In the second analysis, we validated the data quality of key variables including cartilage damage, meniscal treatment, activity leading to ACL rupture, diagnosis registered as ACL rupture, choice of graft, choice of femoral tunnel placement (anteromedial or transtibial technique), number of femoral tunnels, date of operation, and choice of femoral and tibial fixation. We randomly selected 240 patients registered in the DKRR with primary ACL reconstruction surgery from January 1, 2008 until December 31, 2009. This period was chosen to obtain an interval in which the database had been running for a certain amount of time so that the surgeons could familiarize themselves with the registration task.
Data quality of patient-related outcome scores
In the third analysis, we assessed the data quality of the recorded patient-related outcome scores (KOOS and Tegner scores). We performed the assessment by comparing the patients who gave their subjective scores 1 year after the operation (responders) with the patients who did not give their subjective scores 1 year after the operation (nonresponders). The aim was to evaluate if there were any differences in registered patient-related outcome data in the DKRR between the two groups.
We therefore conducted a study in which new KOOS and Tegner questionnaires were sent to 100 responders and to 100 nonresponders. To achieve 95% statistical power and a 5% probability that the null hypothesis is false, we made power calculations and randomly selected 100 patients for each group. The questionnaires were sent out in the spring of 2010 (ie, approximately 2 years after the primary ACL reconstruction surgery had been performed). After 2 months, the reminder was sent to the nonresponders. Sixty-two (62%) of the responders and 39 (39%) of the nonresponders answered this questionnaire, and an estimate of their mean scores was calculated.
Statistical analysis
Completeness of patient registration
The DNRP was used as a reference standard to calculate the registration completeness of the ACL reconstructions in the DKRR. Data from the DNRP and DKRR was merged on an individual level. The registration completeness was defined as the number of patients registered in both the DKRR and DNRP with ACL reconstruction, divided by the number of patients registered in the DNRP with ACL reconstruction in the same period. Analyses were stratified according to age, sex, and hospital volume to evaluate for any association between these variables and the completeness of the data registration.
Data quality of key variables
To validate data quality of selected key variables, we assessed the PPV using medical records as a reference standard. For each of the selected registered variables, we defined the PPV as the number of patients with a given variable registered in both the DKRR and medical records, divided by the total number of patients with a given variable registered in the DKRR.
Data quality of patient-related outcome scores
The percentage of registered KOOS and Tegner scores was calculated as the number of registered KOOS and Tegner scores divided by the total number of operations registered in the DKRR preoperatively and postoperatively. To evaluate if there was any difference in subjective scores (KOOS and Tegner scores) between the responders and nonresponders in the 1-year postoperative evaluation questionnaire, we calculated the mean score for each group; we tested data differences using Student’s t-test.
For all estimates, a 95% confidence interval (CI) was calculated. We analyzed data using STATA version 12 (StataCorp LP, College Station, TX, USA). EpiData was used for data entry.
This study was approved by The National Board of Health and The Danish Data Protection Agency, journal number 2011-41-6320.
Results
In total, 18,050 patients were identified in the DKRR and DNRP from July 2005 to December 2011. A total of 12,963 patients (71.8%) were registered in both registries; 3550 (19.7%) were only registered in the DNRP, and 1537 (8.5%) were only registered in the DKRR. Patient demographics for these three groups are shown in Table 1.
Table 1.
Patient characteristics
| Patient characteristics | Type of register
|
||
|---|---|---|---|
| Both DNRP and DKRR, n (%) n = 12,963 | DNRP only, n (%) n = 3550 | DKRR only, n (%) n = 1537 | |
| Sex | |||
| Male | 7783 (60.0) | 2208 (62.2) | 937 (61.0) |
| Female | 5180 (40.0) | 1342 (37.8) | 600 (39.0) |
| Mean age at surgery | 29.7 (95% CI: 29.5–29.9) | 31.0 (95% CI: 30.6–31.3) | 31.9 (95% CI: 31.4–32.5) |
| Age at surgery (years) | |||
| ≤20 | 3449 (26.6) | 773 (21.8) | 299 (19.5) |
| >20 | 9514 (73.4) | 2777 (78.2) | 1238 (80.5) |
Abbreviations: DNRP, Danish National Registry of Patients; DKRR, Danish Knee Ligament Reconstruction Register; n, number; CI, confidence interval.
Completeness of patient registration
The overall registration completeness of the ACL reconstruction in the DKRR was 78.5% (95% CI: 77.9%–79.1%) from 2005–2011 (Table 2). The completeness increased from 60.3% (95% CI: 57.4%–63.1%) in 2005 to 86.3% (95% CI: 84.9%–87.7%) in 2011 (Figure 1).
Table 2.
Completeness of registration of ACL reconstruction surgery in the DKRR compared with the DNRP
| Type of register
|
Total (n) | Degree of completeness % (95% CI) | |||
|---|---|---|---|---|---|
| Both DNRP and DKRR n (%) | DNRP only n (%) | DKRR only n (%) | |||
| 2005–2011 | 12,963 (71.8) | 3550 (19.7) | 1537 (8.5) | 18,050 | 78.5 (77.9–79.1) |
| ≤20 years | 3449 (76.3) | 773 (17.0) | 299 (6.6) | 4521 | 81.8 (80.6–82.9) |
| >20 years | 9514 (70.3) | 2777 (20.5) | 1238 (9.2) | 13,529 | 77.4 (76.7–78.1) |
| Male | 7783 (71.3) | 2208 (20.2) | 937 (8.6) | 10,928 | 77.9 (77.1–78.7) |
| Female | 5180 (72.7) | 1342 (18.8) | 600 (8.5) | 7122 | 79.4 (78.4–80.4) |
| Small volumea | 1317 (56.0) | 726 (30.8) | 311 (13.2) | 2354 | 64.5 (62.3–66.5) |
| Large volumea | 11,646 (75.1) | 2628 (17.0) | 1226 (7.9) | 15,500 | 81.6 (80.9–82.2) |
| 2005 | 690 (52.2) | 455 (34.5) | 176 (13.3) | 1321 | 60.3 (57.4–63.1) |
| 2006 | 1870 (65.6) | 758 (26.6) | 224 (7.8) | 2852 | 71.2 (69.4–72.9) |
| 2007 | 1828 (67.5) | 660 (24.4) | 219 (8.1) | 2707 | 73.5 (71.7–75.2) |
| 2008 | 1737 (70.7) | 480 (19.9) | 231 (9.4) | 2458 | 78.0 (76.2–79.7) |
| 2009 | 2335 (77.3) | 407 (13.5) | 278 (9.2) | 3020 | 85.2 (83.8–86.5) |
| 2010 | 2323 (77.6) | 433 (14.5) | 238 (7.9) | 2994 | 84.3 (82.9–85.7) |
| 2011 | 2180 (80.8) | 347 (12.9) | 171 (6.3) | 2698 | 86.3 (84.9–87.7) |
Notes: Results were stratified according to patient sex, age, and small-volume versus large-volume hospitals.
196 ACL-reconstructed knees that could not be placed in either the small-volume or the large-volume group because of imprecise definition in the DNRP.
Abbreviations: ACL, anterior cruciate ligament; DKRR, Danish Knee Reconstruction Register; DNRP, Danish National Registry of Patients; n, number; CI, confidence interval.
Figure 1.
Development of data completeness in the DKRR.
Abbreviation: DKRR, Danish Knee Ligament Reconstruction Register.
There was no difference in completeness when data were stratified according to patient sex and age. However, a considerable difference was seen when data were stratified according to hospital volume: large-volume hospitals (81.6% [95% CI: 80.9%–82.2%]) had a higher degree of registration completeness than small-volume hospitals (64.5% [95% CI: 62.3–66.5]) (Table 2).
Accuracy of key variables
In general, the data quality of the key variables was good. The PPV ranged from 85%–100% (Table 3). The PPV was 96% for the use of transtibial placement of the femoral canal, 100% for “choice of graft,” 85% for “cartilage damage,” and 96% for “treatment of meniscal damage.”
Table 3.
Validity of key variables registered in DKRR
| DKRR database | Medical record
|
Missing medical record | PPV% (95% CI) | ||
|---|---|---|---|---|---|
| Yes | No | ||||
| Cartilage lesion | Yes | 29 | 5 | 0 | 29/34 = 85 (69–95) |
| No | 27 | 127 | 12 | ||
| Missing DKRR | 13 | 26 | 1 | ||
| Treated meniscal lesionsa | Yes | 103 | 4 | 0 | 103/107 = 96 (90–99) |
| No | 26 | 107 | 0 | ||
| Sport as activity reason for the ACL rupture | Yes | 172 | 29 | 1 | 172/201 = 86 (80–90) |
| No | 4 | 33 | 0 | ||
| Missing DKRR | 0 | 1 | 0 | ||
| Diagnosis of ACL lesion | Yes | 240 | 0 | 240/240 = 100 | |
| No | 0 | 0 | |||
| Hamstring as graft choiceb | Yes | 208 | 0 | 2 | 208/208 = 100 |
| No | 0 | 24 | 0 | ||
| Missing DKRR | 4 | 1 | 1 | ||
| Transtibial placement of femoral canal | Yes | 136 | 6 | 60 | 136/142 = 96 (91–98) |
| No | 3 | 18 | 16 | ||
| Missing DKRR | 1 | 0 | 0 | ||
| One femoral canal (versus two) | Yes | 237 | 0 | 2 | 237/237 = 100 |
| No | 0 | 0 | 0 | ||
| Missing DKRR | 1 | 0 | 0 | ||
| OP date | Identical | 238 | 0 | 1 | 238/239 = 99 (93–100) |
| Not identical | 0 | 1 | |||
| Method for fixation of graft in femur | Identical | 217 | 7 | 217/233 = 93 (89–96)c | |
| Method for fixation of graft in tibia | Identical | 220 | 6 | 220/234 = 94 (90–97)c | |
Notes: Medical records were used as a reference standard, and 240 medical records were used. “Identical” indicates that the registration in the DKRR is identical to the registration in the medical records.
The registration of this variable in DKRR is to choose a treatment of the meniscal lesion. Hence, if no registration is made in this variable it can be etiher: no treatment or a missing value
comparing if hamstring or bone–patellar tendon–bone graft is used. Other graft choices were deleted because they account for less than 2%
About 20 different modes of fixation are possible in this variable. Therefore, calculation of completeness was made by inputing the number of variables registered identical in the DKRR and in the medical records into the numerator, and dividing them by all the cases registered in the denominator.
Abbreviations: DKRR, Danish Knee Ligament Reconstruction Register; PPV, positive predictive value; CI, confidence interval; ACL, anterior cruciate ligament; OP, operation.
Data quality of patient-related outcome scores
Subjective outcome scores for both KOOS and Tegner were available for 4799 (33.1%) of 14,500 patients preoperatively; 1 year after surgery, these outcome scores were available for 3852 (26.6%) of 14,500 patients. No difference was found in either the five KOOS subscores or in the Tegner scores between responders and nonresponders in the 1-year postoperative questionnaire (Table 4).
Table 4.
Results from questionnaires on KOOS and Tegner scores
| Previously answered subjective measures | N | Pain mean (SD) | Symptom mean (SD) | Activity of daily living mean (SD) | Sport mean (SD) | Qol mean (SD) | Tegner mean (SD) |
|---|---|---|---|---|---|---|---|
| Yes (responders) | 62 | 83.1 (15.0) | 60.4 (13.8) | 88.0 (15.7) | 64.5 (26.9) | 64.5 (25.4) | 4.87 (2.0) |
| No (nonresponders) | 39 | 78.5 (19.2) | 59.8 (12.1) | 83.7 (19.7) | 56.9 (26.8) | 59.3 (24.5) | 4.56 (1.9) |
| P-value | 0.09 | 0.38 | 0.12 | 0.97 | 0.82 | 0.72 |
Notes: Mean value of the different subjective scores in the group of patients who previously recorded subjective scores (responders) compared with the group of patients who did not previously record subjective scores (nonresponders).
Abbreviations: KOOS, Knee injury and Osteoarthritis Outcome Score; N, number; SD, standard deviation; Qol, quality of life.
Patient characteristics in the DKRR are outlined in Table 5. Nonresponders to the 1-year questionnaire were younger than the responders, and there were more males than females in the nonresponders. The amount of meniscal and cartilage lesions was comparable in the two groups. A small difference in graft choice was seen, yet more than 80% of patients underwent reconstruction with a hamstring graft in both groups. The percentage of revisions was low in both groups, but there were more revisions among responders (Table 5).
Table 5.
Patient characteristics from the Danish Knee Reconstruction Registry with (responders) or without (nonresponders) subjective scores registered 1 year postoperatively
| Patient characteristics | Subjective scores registered 1 year after operation
|
|
|---|---|---|
| Yes, n (%) | No, n (%) | |
| Sex | ||
| Male | 2240 (58.1) | 6704 (62.9) |
| Mean age at time of surgery (years) | 30.5 (95% CI: 30.1–30.8) | 29.7 (95% CI: 29.5–29.9) |
| Age at time of surgery (years) | ||
| ≤20 | 965 (25.1) | 2928 (27.5) |
| ACL revision | ||
| Yes | 144 (3.7) | 255 (2.4) |
| Meniscal treatment | ||
| Yes | 1444 (37.5) | 4227 (39.6) |
| Cartilage damage | ||
| Yes | 713 (18.5) | 2369 (22.2) |
| Prior knee surgery | ||
| Yes | 1086 (28.2) | 2935 (27.6) |
| Sport activity leading to tear | ||
| Yes | 3328 (86.4) | 8761 (82.2) |
| Graft choice (hamstring) | ||
| Yes | 2974 (77.2) | 8767 (82.3) |
Abbreviations: n, number; ACL, anterior cruciate ligament; CI, confidence interval.
Discussion
Completeness of patient registration
The overall registration of ACL reconstruction procedures in the DKRR was good and has improved over time, as the surgeons have become familiar with the registration procedure. Sensitivity and specificity must generally be calculated to estimate the quality of the classification of hospital discharge and operative procedures. In accordance with Sorensen et al,4 completeness is an estimate of the sensitivity, which we calculated in this study. The comparison between registries (the DKRR and DNRP) performed in this study does not give us the opportunity to measure specificity, but we assume that the specificity is close to one because our background population is large and the ACL reconstruction procedure is rare.4
The completeness of other databases has been evaluated in previous studies.7,16,22 A study from the Norwegian Registry of Knee Ligament Reconstruction reported a very high completeness (86%) when using hospital protocols as a reference standard and 84% when using the Norwegian National Patient Registry as a reference standard.19 The higher completeness in the Norwegian study versus our study may be explained by several factors. First, the Norwegian study used another method for calculation of completeness. We defined completeness as a measure of the sensitivity, calculated as the number of cases registered in both registries (DKRR and DNRP) divided by the number of cases registered in the DNRP within the same period. The Norwegian study calculated completeness as the number of operations registered in the Norwegian National Knee Ligament Registry divided by the number of cases registered in the hospital protocols (or in the Norwegian National Patient Registry). Our completeness changed from 78.5% to 87.8% when we calculated our completeness from 2005–2011 using the same method as that used in the Norwegian study. Second, the Norwegian estimates were based on a selected population because the authors in the Norwegian study only used ten of 46 hospitals reporting to the registry. Our results are based on all hospitals in Denmark performing ACL reconstructions. Pedersen et al7 showed a very good completeness of 94% of hip arthroplasty registration from 1995–2000 in the Danish Hip Arthroplasty Register. Again, this study differs from our study because they excluded patients operated on at private hospitals, which were mostly the small-volume hospitals with low registration completeness in our study. Likewise, the Norwegian Arthroplasty Register showed a high completeness of 98% for hip replacements, and 109% for hip revision surgery.20 Again, the Norwegian Arthroplasty Register’s calculation of completeness is different from ours. Also, we calculated the completeness at an individual level (because of the unique personal security number), which may not be the case in previous studies on completeness.20 This trend may also explain some of the differences in results.
The differences in the design of these validation studies hamper their comparison. However, the abovementioned studies do indicate an overall high degree of registration in the Scandinavian registries, which makes them reliable for future research. Our review of the literature highlights the importance of reporting the completeness estimates and a thorough description of the calculation methods to be able to compare the results across countries.
In our study, registration completeness improved with time. This may be explained by the surgeons becoming more familiar with the registration task.20,21 Although a declaration was issued by the authorities requiring all surgeons to register surgical procedures performed, no penalty is invoked by failure to register. Hence, the surgeon must become familiar with the registration procedure over time to achieve the overall goal of registration completeness. The goal for completeness in the DKRR is defined as more than 90%,8 which has been reached by other registries in Scandinavia.7,22 We stratified completeness according to small-volume versus large-volume hospitals and found that large-volume hospitals performed better than small-volume hospitals, which is in accordance with a recent study from the Norwegian National Knee Ligament Registry.17 In the future, intensive feedback – primarily to small-volume departments – is necessary to improve completeness.
We regarded the DNRP as the reference standard in the study of completeness. However, we identified 1537 procedures in the DKRR that had not been registered in the DNRP, which shows that the DNRP is not a perfect reference. Approximately 79% of these 1537 missing patients in the DNRP underwent knee surgery at private hospitals. Private hospitals receive no financial reimbursement when treating insurance patients and self-paying patients. Hence, there is no financial incentive to register these procedures in the DNRP, which may explain why so many surgical procedures had not been registered.
Data quality of key variables
A high PPV (85%–100%) for key variables was shown in this study. In the DKRR, many data on “cartilage lesions” and “different sports activities leading to ACL rupture” were missing. This discovery has prompted an evaluation of the registration system, which found that registration of cartilage damage was imprecise. Therefore, registration of cartilage lesions has now been simplified, which will most likely improve registration of this variable.
The medical records showed a high percentage of missing data on the method used for femoral tunnel drilling (transtibial or anteromedial). The femoral tunnel was previously almost always drilled transtibially,23 and, because the method is customarily used, surgeons may have deselected registration of this method for femoral tunnel drilling in the medical records. When only one method is available, the surgeon probably does not believe that it is essential to record this action. Therefore, improvement in registration completeness of the method of femoral tunnel drilling probably will not be achieved until the surgeons become fully accustomed to the existence of other methods and are faced with a choice between relevant alternatives.
Data quality of patient-related outcome scores
As expected, registration of subjective outcome scores was low, which has also been shown in other studies.23,24 Therefore, it was important to evaluate if there was any significant and clinically important difference between the responders and the nonresponders in their subjective scores at the 1-year follow-up. No significant difference was noted in the mean estimates of the KOOS and Tegner scores between responders and nonresponders. However, these estimates are imprecise because of a low number of participants. Furthermore, these estimates tell us nothing about the difference between the second-time responders and “never-responders.” Although the percentage of revision was low in both groups, there were fewer revisions among responders. The time at risk was not taken into account. We did not include survival analysis to compare the cumulative revision rates in this study because we found that it was out of the scope of our present study.
Study strengths and weaknesses
Our study was based on data from a national clinical registry; therefore, it had several strengths and weaknesses. That the DKRR and DNRP are large national databases is an obvious strength from a data quality perspective. Also, owing to the unrestricted and free access to health care in Denmark, the DKRR provides an unselected study population. Furthermore, the DKRR has the potential for extensive linkage to other important databases at the level of the individual, owing to the unique personal security numbers given to all Danish citizens. This linkage affords the possibility of individual measurements. Moreover, these nationwide population-based databases provide an excellent data source at low cost and with little risk of bias. Another strength of our validation study was the thorough evaluation of medical records, which allowed for validation of key variables in the DKRR and made the DKRR a valid tool for future research. Furthermore, the medical records were randomly picked by a computer program. The risk of information bias was low because previously designed forms were made in EpiData, and data entry was made twice. In addition, the reviewer of the medical records was blinded to the results in the DKRR and, hence, did not know the optimal result.
Through the years, the DNRP has been considered a valuable source of high-quality data information.25 However, no validation of the ACL reconstruction code has been undertaken. Thus, a limitation in our study was the deviation in the quality of the data retrieved from the DNRP. Medical records are a valuable data source when validation studies are conducted.5 Yet, they are not a perfect source of information, because it is up to the operating surgeon to decide which information he or she wishes to enter into the medical record. Some information is not considered to be of importance for the specific physician, and some variables will therefore be missing in the medical records. Although a number of missing variables were found in some of the key variables in our study, the high values of the PPV ensure satisfactory data quality.
As to the evaluation of responders versus nonresponders to the questionnaire at 1 year after surgery, we know nothing about the “never-responders,” and the numbers of observations are low in this substudy compared with the entire study cohort.
Conclusion
In conclusion, this validation study showed good completeness of registration of the ACL reconstructions in the DKRR. We found a high PPV for most key variables in the DKRR. Thus, the DKRR is a valid and substantial resource for future epidemiological studies. However, a future effort to improve registration completeness from small-volume clinical departments is needed.
The KOOS and Tegner scores at the 1-year follow-up were comparable for responders and nonresponders. We therefore conclude that the DKRR data are a valid source for future clinical and epidemiological research.
Acknowledgments
The authors acknowledge the help and support from Frank Mehnert at the Department of Clinical Epidemiology and the kind secretarial help provided by Anne Haagen Hjelm at the Department of Clinical Epidemiology. This study received financial support from the Danish Rheumatism Association and Aase and Ejnar Danielsens Foundation.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.Nordenvall R, Bahmanyar S, Adami J, Stenros C, Wredmark T, Felländer-Tsai L. A population-based nationwide study of cruciate ligament injury in Sweden, 2001–2009: incidence, treatment, and sex differences. Am J Sports Med. 2012;40(8):1808–1813. doi: 10.1177/0363546512449306. [DOI] [PubMed] [Google Scholar]
- 2.Laxdal G, Kartus J, Hansson L, Heidvall M, Ejerhed L, Karlsson J. A prospective randomized comparison of bone-patellar tendon-bone and hamstring grafts for anterior cruciate ligament reconstruction. Arthroscopy. 2005;21(1):34–42. doi: 10.1016/j.arthro.2004.09.014. [DOI] [PubMed] [Google Scholar]
- 3.Lidén M, Ejerhed L, Sernert N, Laxdal G, Kartus J. Patellar tendon or semitendinosus tendon autografts for anterior cruciate ligament reconstruction: a prospective, randomized study with a 7-year follow-up. Am J Sports Med. 2007;35(5):740–748. doi: 10.1177/0363546506298275. [DOI] [PubMed] [Google Scholar]
- 4.Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol. 1996;25(2):435–442. doi: 10.1093/ije/25.2.435. [DOI] [PubMed] [Google Scholar]
- 5.Goldberg J, Gelfand HM, Levy PS. Registry evaluation methods: a review and case study. Epidemiol Rev. 1980;2:210–220. doi: 10.1093/oxfordjournals.epirev.a036224. [DOI] [PubMed] [Google Scholar]
- 6.Lind M, Menhert F, Pedersen AB. The first results from the Danish ACL reconstruction registry: epidemiologic and 2 year follow-up results from 5,818 knee ligament reconstructions. Knee Surg Sports Traumatol Arthrosc. 2009;17(2):117–124. doi: 10.1007/s00167-008-0654-3. [DOI] [PubMed] [Google Scholar]
- 7.Pedersen A, Johnsen S, Overgaard S, Søballe K, Sørensen HT, Lucht U. Registration in the Danish hip arthroplasty registry: completeness of total hip arthroplasties and positive predictive value of registered diagnosis and postoperative complications. Acta Orthop Scand. 2004;75(4):434–441. doi: 10.1080/00016470410001213-1. [DOI] [PubMed] [Google Scholar]
- 8.Lind M, Menhert F, Pedersen AB. Incidence and outcome after revision anterior cruciate ligament reconstruction: results from the Danish registry for knee ligament reconstructions. Am J Sports Med. 2012;40(7):1551–1557. doi: 10.1177/0363546512446000. [DOI] [PubMed] [Google Scholar]
- 9.Roos EM, Toksvig-Larsen S. Knee Injury and Osteoarthritis Outcome Score (KOOS) – validation and comparison to the WOMAC in total knee replacement. Health Qual Life Outcomes. 2003;1:17. doi: 10.1186/1477-7525-1-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tegner Y, Lysholm J. Rating systems in the evaluation of knee ligament injuries. Clin Orthop Relat Res. 1985;(198):43–49. [PubMed] [Google Scholar]
- 11.Briggs KK, Steadman JR, Hay CJ, Hines SL. Lysholm score and Tegner activity level in individuals with normal knees. Am J Sports Med. 2009;37(5):898–901. doi: 10.1177/0363546508330149. [DOI] [PubMed] [Google Scholar]
- 12.Nørgaard M, Skriver MV, Gregersen H, Pedersen G, Schønheyder HC, Sørensen HT. The data quality of haematological malignancy ICD-10 diagnoses in a population-based hospital discharge registry. Eur J Cancer Prev. 2005;14(3):201–206. doi: 10.1097/00008469-200506000-00002. [DOI] [PubMed] [Google Scholar]
- 13.Lynge E, Sandegaard JL, Rebolj M. The Danish National Patient Register. Scand J Public Health. 2011;39(Suppl 7):30–33. doi: 10.1177/1403494811401482. [DOI] [PubMed] [Google Scholar]
- 14.World Health Organization . International Statistical Classification of Diseases and Related Health Problems. 10th ed. Geneva, Switzerland: World Health Organization; 1994. [Google Scholar]
- 15.Nordic Medico-Statistical Committee . NOMESCO Classification of Surgical Procedures, version 1.11. Copenhagen: NOMESCO; 2006. [Google Scholar]
- 16.Ingeman A, Andersen G, Hundborg HH, Johnsen SP. Medical complications in patients with stroke: data validity in a stroke registry and a hospital discharge registry. Clin Epidemiol. 2010;2:5–13. doi: 10.2147/clep.s8908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ytterstad K, Granan LP, Ytterstad B, et al. Registration rate in the Norwegian Cruciate Ligament Register: large-volume hospitals perform better. Acta Orthop. 2012;83(2):174–178. doi: 10.3109/17453674.2012.678800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Magnussen RA, Lawrence JT, West RL, Toth AP, Taylor DC, Garrett WE. Graft size and patient age are predictors of early revision after anterior cruciate ligament reconstruction with hamstring autograft. Arthroscopy. 2012;28(4):526–531. doi: 10.1016/j.arthro.2011.11.024. [DOI] [PubMed] [Google Scholar]
- 19.Solumshengslet K, Granan L-P, Steindal K, Furnes ON, Engebretsen L. Registreringsgrad i Nasjonalt Korsbåndsregister [Registration completeness of the Norweigian National Cruciate Ligament Register] Idrettsmedisin. 2007;2:7–11. Norwegian. [Google Scholar]
- 20.Espehaug B, Furnes O, Havelin LI, Engesaeter LB, Vollset SE, Kindseth O. Registration completeness in the Norwegian Arthroplasty Register. Acta Orthop. 2006;77(1):49–56. doi: 10.1080/17453670610045696. [DOI] [PubMed] [Google Scholar]
- 21.Puolakka TJ, Pajamäki KJ, Halonen PJ, Pulkkinen PO, Paavolainen P, Nevalainen JK. The Finnish Arthroplasty Register: report of the hip register. Acta Orthop Scand. 2001;72(5):433–441. doi: 10.1080/000164701753532745. [DOI] [PubMed] [Google Scholar]
- 22.Ytterstad K, Granan LP, Engebretsen L. God rapportering til korsbåndsregisteret [The Norwegian Cruciate Ligament Registry has a high degree of completeness] Tidsskr Nor Laegeforen. 2011;131(3):248–250. doi: 10.4045/tidsskr.10.0536. Norwegian. [DOI] [PubMed] [Google Scholar]
- 23.Rahr-Wagner L, Thillemann TM, Pedersen AB, Lind MC. Increased risk of revision after anteromedial compared with transtibial drilling of the femoral tunnel during primary anterior cruciate ligament reconstruction: results from the Danish Knee Ligament Reconstruction Register. Arthroscopy. 2013;29(1):98–105. doi: 10.1016/j.arthro.2012.09.009. [DOI] [PubMed] [Google Scholar]
- 24.Ahldén M, Samuelsson K, Sernert N, Forssblad M, Karlsson J, Kartus J. The Swedish National Anterior Cruciate Ligament Register: a report on baseline variables and outcomes of surgery for almost 18,000 patients. Am J Sports Med. 2012;40(10):2230–2235. doi: 10.1177/0363546512457348. [DOI] [PubMed] [Google Scholar]
- 25.Nickelsen TN. Datavaliditet og dækningsgrad i Landspatientregisteret [Data validity and coverage in the Danish National Health Registry. A literature review] Ugeskr Laeger. 2001;164(1):33–37. Danish. [PubMed] [Google Scholar]