Do administrative databases accurately measure waiting times for medical care? Evidence from general surgery

Abstract

We abstracted the records of patients from general surgeons' offices in Winnipeg to compare waiting times from charts (i.e., the gold standard) with waiting times using administrative data. The administrative data method relies on physicians' billing data to identify a visit to the surgeon preoperatively to mark the start of the waiting time. There was no difference between waiting times using patient records versus administrative data. The study supported the use of administrative data to monitor waiting times.

In this study, we abstracted patients' records from general surgeons' offices to compare waiting times from charts (i.e., the gold standard) with waiting times using administrative data. Administrative data arise from the administration of publicly insured hospital and medical services. To estimate waits, hospital abstracts are used to identify surgical procedures and the surgeon who performed the procedure. A search is then made to find a preoperative visit to the surgeon, which is considered the marker of the beginning of the waiting time.^1,2 If there is more than 1 preoperative visit, the visit closest to surgery is used.

An advantage of the administrative data method is that it is relatively inexpensive, because the data are already collected as part of the health care system; this makes it an attractive alternative for high-volume surgical procedures, such as general surgery. The main disadvantage is that it uses a proxy measure — a preoperative visit to the surgeon — as the beginning of the wait. If this method is to be used as a means of monitoring waiting times, its validity needs to be assessed.

Methods

Data were abstracted from the records of a random sample of patients in 1 general surgery clinic in Winnipeg, Manitoba. The sample was limited to patients who had undergone 1 of 5 elective general surgical procedures between Jan. 1, 1998 and Aug. 31, 1999: cholecystectomy, breast tumour surgery, colectomy, hernia repair or varicose vein surgery. The 5 surgeons who participated operated in 3 urban hospitals. Prior to collecting the data, we estimated that a sample size of 140 was required to find a 90% agreement, with a standard deviation (SD) of 5% between chart and administrative data. We obtained permission to use surgeons' clinical data and obtained ethics approval from the University of Manitoba Research Ethics Board.

Data abstracted from the medical records included dates of the first visit and the decision to operate (if these were different), surgical date and procedure performed. No names, addresses or other personal identifying data were abstracted. The decision-to-operate date was either the date of a relevant physician note in the record or the date of the surgical request form. Date of surgery was confirmed on the basis of the operative dictation. The surgical waiting time was defined as the time between the decision-to-operate date and the surgery date.

We merged data from the chart audit with data from Manitoba's Population Health Research Data Repository. The repository contains anonymous encounter-based records of individuals' interactions with the provincial health care system. Probabilistic linkage was used based on the date of surgery and the procedure and diagnosis codes. Previous research has shown that the date of surgery is recorded accurately in hospital abstracts.³ Once the link was made with the hospital file medical claims were searched for the preceding 36 months for a preoperative visit to the surgeon. We compared the beginning of the waiting time as recorded from the medical record with the date of the presurgery visit. After we log-transformed the data to overcome the skewness,⁴ we used paired t tests to compare the 2 waiting times.

Results

We abstracted data for 170 procedures in 163 patients who underwent surgery between Jan. 1, 1998 and Aug. 31, 1999. The mean surgical waiting time for the 170 procedures was 47.8 days (SD 63.6 d). The median was 30 days (interquartile range [IQR] 18–55 d).

Seven patients had 2 or more procedures during different hospital stays, and only their first procedure was included when we merged chart, audit and administrative data. We merged the records for 148 (90.8%) of the remaining 163 cases with hospital administrative data; 90% record linkage can be considered very good to excellent.⁵ Reasons for the inability to link included the absence of a surgical procedure code in the hospital file (n = 3), mismatch between the procedure abstracted and the procedure coded (n = 1), no claim found (n = 1) and no match, using the limited fields available (n = 10).

Of these 148 patients, a claim for a presurgical visit to the surgeon was found for 141 (95.3%) patients. Therefore, 22 (13.5%) either did not match based on the date of surgery and the procedure codes or did not have a claim for a preoperative visit to the surgeon who performed the surgery. We compared the matched and unmatched records. The mean and median waiting times (based on the chart data) between the matched and unmatched records were not significantly different, nor were the distributions among procedures or among surgeons different between matched and unmatched cases.

Of the 141 cases that were linked with administrative data and for which a preoperative visit to the surgeon was found, for 123 (87.2%), the date of the visit matched the decision-to-operate date recorded from the patient's medical record. Of the 18 unmatched records, there were very large discrepancies for 2 patients (215 d and 305 d), and the rest were smaller (5–70 d). Over all of the matched records (n = 141), the average absolute difference in waiting times between the 2 methods was 5.0 days (SD 21.4 d).

The mean and median waiting times for the 2 methods are shown in Table 1. Mean waiting times were virtually the same with patient records (49.0 d) as with administrative data (49.4 d). Median waiting times were 30 days (IQR 19–56 d) and 33 days (IQR 20–57 d) using patient records and administrative data, respectively. Paired t tests using log-transformed waiting times confirmed that there was no statistically significant difference between the 2 methods.

Table 1

Discussion

This study supports the validity of using administrative data to estimate waiting times. When medical record data were merged with administrative data, the decision-to-operate dates were identical for 87% of cases. Further, the mean and median waiting times estimated with the 2 techniques were not different.

This is the first published report of a direct comparison between chart and administrative data, although 2 other studies support this method.^4,6 Sanmartin compared data from the hospital booking system with administrative data in British Columbia for 4 procedures: knee replacement, hip replacement, coronary artery bypass surgery (CABS) and cataract surgery. About 70% of procedures were booked after the last visit before surgery, except for cataracts, at only 55%.⁶ Shortt and colleagues analyzed chart data for over 30 000 surgeries that took place from July 1 1992 to June 30, 1996 in Kingston.⁴ They compared the date the patient was placed on a waiting list with the date of the patient's last visit to the surgeon. For general, neurological, ophthalmic, thoracic, vascular and urological surgery, the difference between the decision date and the last visit was negligible, from 0.1 day for ophthalmology to 1.5 days for neurosurgery.

The current study further bolsters the use of administrative data to estimate surgical waiting times. However, there are important limitations. This was a small sample of patients from 1 general surgery group in 1 city and may not be generalizable. Also, this method is only valid for patients who actually had surgery, because there is only a hospital abstract after the procedure has been completed. We have no estimate of waits for patients who are still in the queue or for patients who had to cancel for some other reason. Retrospective designs have been found to underestimate the wait somewhat.⁷ There is also some indication that this method works best for procedures requiring a relatively short wait. In a study that compared the administrative and waiting list registry data for cataract surgery for which average waits were 6 months, the administrative method underestimated the actual wait as recorded in the registry.³

In this study, we analyzed only 86.5% of the cohort. The ability to link between abstracted data and administrative data was somewhat hampered because there were few fields available for linkage. Data abstraction was limited to protect patient confidentiality; thus the minimum number of data elements were abstracted for the study. Had additional personal identifiers been available, the linkage rate would likely have been higher.⁸

Methods of estimating waits that rely on administrative data have been questioned by some health care providers. The results of this study suggest that using administrative data to estimate waiting times for patients who have undergone elective general surgical procedures is accurate. This finding is important because setting up and managing a comprehensive patient registry is time consuming and expensive, whereas administrative data already exist as part of the publicly insured health care system and can be analyzed more readily. Administrative databases can provide a fast, generally reliable and inexpensive way of assessing waiting times for surgical procedures.

Competing interests: None declared.

Correspondence to: Dr. Carolyn De Coster, Quality, Safety and Health Information, 4520-16th Ave. N.W., Calgary AB T3B 0M6; fax 403 944-8950; Carolyn.Decoster@calgaryhealthregion.ca