Abstract
PURPOSE We wanted to explore test results management systems in family medicine offices and to delineate the components of quality in results management.
METHODS Using a multimethod protocol, we intensively studied 4 purposefully chosen family medicine offices using observations, interviews, and surveys. Data analysis consisted of iterative qualitative analysis, descriptive frequencies, and individual case studies, followed by a comparative case analysis. We assessed the quality of results management at each practice by both the presence of and adherence to systemwide practices for each results management step, as well as outcomes from chart reviews, patient surveys, and interview and observation notes.
RESULTS We found variability between offices in how they performed the tasks for each of the specific steps of results management. No office consistently had or adhered to office-wide results management practices, and only 2 offices had written protocols or procedures for any results management steps. Whereas most patients surveyed acknowledged receiving their test results (87% to 100%), a far smaller proportion of patient charts documented patient notification (58% to 85%), clinician response to the result (47% to 84%), and follow-up for abnormal results (28% to 55%). We found 2 themes that emerged as factors of importance in assessing test results management quality: safety awareness—a leadership focus and communication that occurs around quality and safety, teamwork in the office, and the presence of appropriate policies and procedures; and technological adoption—the presence of an electronic health record, digital connections between the office and testing facilities, use of technology to facilitate patient communication, and the presence of forcing functions (built-in safeguards and requirements).
CONCLUSION Understanding the components of safety awareness and technological adoption can assist family medicine offices in evaluating their own results management processes and help them design systems that can lead to higher quality care.
Keywords: Delivery of health care; quality of care; patient safety; medical errors, office management, qualitative research
INTRODUCTION
Primary care physicians order tests for a considerable number of patients. Recent estimates are that family physicians and general internists order laboratory tests in 29% and 38% of patient visits and imaging studies in 10% and 12%, respectively.1 These tests serve multiple purposes—some are for screening, others are diagnostic, and still others are used to manage and monitor medications and chronic health problems. Although some tests are performed in physicians’ offices while patients wait, most are sent to reference laboratories, hospitals, and outside facilities.
For more than a decade, research has documented that primary care physicians are concerned that their systems for managing test results are unsatisfactory.2–5 In 3 primary care patient safety studies, 15% to 54% of all errors reported by physicians were related to the testing process.6–8 There are multiple steps involved in the management of test results.1,9 These steps begin when offices track orders and the return of results to the physician’s office from the outside testing facility. If this step is not done automatically by means of an electronic health record (EHR), the results must then be given to the clinician who responds to the result. The patient is notified and followed up, and documentation is made in the medical record. A recent study of testing process errors reported by family physicians and their staff found that errors occurred in all the multiple results management steps.9 Serious harm can and has befallen patients by errors in results management.9–11 Although all results management steps are important and interrelated, errors in patient notification were predictive of more patient adverse events.9
Previous attempts to study office management of test results have depended heavily on surveys2,5,12 and reports of errors by physicians and staff,6–9 whereas chart reviews4 and interviews3,4 are used less often. Even though each of these techniques gives some insight into the problems faced in performing these tasks, none fully describes the complexity of the results management process. Fewer studies actually report on what works well.4 Using a multimethod protocol of observations, interviews, and surveys, as well as a comparative case design analysis, we intensively studied 4 purposefully chosen family medicine offices to determine and describe components of results management qualilty.
METHODS
To gather the maximum amount of data with minimal interference to patient care and productivity, a family physician (N.C.E.) and a human factors graduate student (T.R.M.) performed site visits at each office for 2 to 4 days and collected other data before and after each visit. This study received approval from the University of Cincinnati and Wright State University institutional review boards.
Participant Selection
We purposefully selected physcians’ offices within the southwest Ohio region to provide variation in geographic location (rural, suburban, urban), practice size, patient insurance status, technology level (EHR, no EHR), and residency program (program, no program). Physicians at 6 offices initially expressed interest, and 4 ultimately agreed to participate. All offices had some degree of affiliation with a larger health care system or a hospital, but each functioned independently. As our research was an exploratory study to describe potential results management quality determinants, we had no means of knowing an office’s results management level of quality before the study.
All office staff were observed and participated in think-aloud interviews. Key informants were identified from staff observations at each office and selected for individual interviews. Individuals interviewed included those who had interest or additional roles in test results management or leadership in the office or who were opinion leaders. There were a total of 17 key informant interviews at the 4 offices.
All adult patients (and parents of child patients) for whom laboratory tests were ordered during the site visit and for a 1- to 4-week period afterward (depending on office size) were asked to participate in a future mail-based survey. Medical assistants handed these patients a 1-page form describing the survey, and those who wished to participate supplied their name and address.
Data Collection
An office information form and Testing Process Survey questionnaire (adapted from the American Academy of Family Physicians National Research Network) 3,9 were completed by the office manager at each office. These forms elicited demographic data about the office and how test results are officially managed. A brief, paper, networking questionnaire was distributed to all staff and physicians, which asked to whom they turned for advice and assistance about results management issues.
At each office, we reviewed 25 random charts that contained laboratory or imaging results in the last 12 months. This review assessed documentation of test orders, results, clinician response, patient notification, and follow-up of abnormal results. Participating patients for whom laboratory testing was ordered during and after the site visits received a mailed, written questionnaire (3 mailings done) about their experience getting the test taken and the results back.
We observed staff and physicians performing their tasks related to the testing process in hallways, nursing stations, the laboratory area, medical records, and other office areas. Occasionally we asked participants to “tell us what you are doing” or to give opinions, ideas, and concerns about the testing process. We also collected written documents from each office, including billing sheets, order forms, interoffice communication forms, patient communication forms, and testing process protocols and procedures. During semistructured interviews with key informants in the offices, we inquired about the testing process, participants’ roles in it, and experiences of errors and safety. Using field notes, we documented how each office responded to an individual written report prepared for them.
Preliminary Data Analysis
During each site visit we met several times each day to discuss findings and to develop process flow diagrams that graphically displayed the detailed tasks each office member performed to manage test results. At the end of each day, we updated the process flow diagrams, which then guided observations the next day. At the end of the each site visit, we put the flow diagrams into a final form and summarized the observation notes for each office.
Quantitative data were entered into a database, and simple descriptive frequencies were summarized for each office.
Each interview was audio-recorded and reviewed by the interviewer (T.R.M.), who made extensive notes on each interview and transcribed selected sections verbatim. The other investigators then reviewed and discussed the notes, looking for important themes and stories about test results management.
Initially we analyzed each site as an independent case study. All data sources were read, reviewed, and discussed by all the researchers. From this discussion, each step in the testing process was described in detail, and safety threats and strengths for each office were delineated and summarized in individual reports, which we then presented to each office.
Data Analysis
The notes and selected quotes from the interviews, the observation notes, and the individual office case studies were entered into the qualitative software program NVivo 2.0 (QSR International Pty Ltc, Doncaster, Victoria, Australia). We focused the analysis on finding examples, descriptions, and comments about each step in results management, as well as safety threats and strengths. Coding was performed by the lead author (N.C.E.) and reviewed by the interviewer (T.R.M.). After discussion and agreement on coding categories, all data were reviewed and coded a second time. Then all the data sources described above, including the quantitative data, were reviewed, along with the summary categories from the qualitative coding. During this review, specific details about the management of each step in results management were collated, and factors of importance for test results management safety and quality for these offices were determined. To assist with thematic formation, existing recommendations from the literature1,4,5,9,12–29 about test results management were compared with the findings from the offices.
RESULTS
Characteristics of the participating family medicine offices are displayed in Table 1 ▶.
Table 1.
Characteristics of Participant Family Medicine Offices
| Characteristic | Office A | Office B | Office C | Office D |
| Location | Rural | Suburban | Urban | Suburban |
| Clinicians, n | ||||
| Full-time | 4 | 1 | 2 | 7 |
| Resident | 12 | 0 | 0 | 0 |
| Part-Time | 0 | 3 | 2 | 6 |
| Total | 16 | 4 | 4 | 13 |
| Women clinicians, n | 8 | 2 | 3 | 7 |
| African-American clinicians, n | 1 | 0 | 1 | 0 |
| Staff, n | ||||
| Full-time | 16 | 1 | 9 | 23 |
| Part-time | 4 | 0 | 0 | 2 |
| Total | 20 | 1a | 9 | 25 |
| Patient-payer mix, % | ||||
| Ensured | 35 | 47 | 24 | 50 |
| Medicare | 30 | 47 | 41 | 45 |
| Medicaid | 25 | 1 | 17 | 0 |
| Self-pay | 10 | 1 | 18 | 5 |
| Residency practice | Yes | No | No | No |
| Electronic health record | No | Yes | No | No |
| Outside laboratories used, n | 2 | 2 | 1 | 2 |
a Contracts with outside phlebotomy, receptionist, and health system billing.
Performance of Test Results Management Steps
We identified multiple ways the main steps of test results management are performed, beginning with tracking orders through follow-up of test results needing further care after the patient is initially notified. In Figure 1 ▶, we list all the different ways these tasks are actually performed at the 4 family medicine offices. For no step did every office perform the results management process the same way. This variability was occasionally minimal (office A had physicians telephone patients with critical results, whereas office B had nurses make the telephone call), but for some steps, like tracking orders, completely different systems existed at each office (or there were no systems at all).
Figure 1.
Results management steps and specific tasks.
EHR=electronic health record; lab=laboratory; Pap=Papanicolaou.
Test Results Management Quality Data
Because there are no reference standards in the medical literature for how best to manage test results, we used multiple sources of data to assess quality. Table 2 ▶ outlines some of the specific data indicators for each office. We looked for written protocols for results management steps, which are an indication of attempts to standardize office practices. We also assessed the standardization of results management steps actually practiced office-wide. Office D had only 1 such practice that was routinely followed (the return of results to the office); for the other 3 offices, standardized practices existed for getting the results to the office and to the physician. Only office B, which had an EHR, had an office-wide system for getting physician signatures on all results. Offices A and C had standardized tracking systems. For example, tracking system at office A consisted of attaching copies of order forms to charts, which were then placed in a separate holding area in the office. Tracking review was performed by a medical assistant assigned to a support position for 1 week at a time. Office B, on the other hand, had no tracking system. Orders entered in the EHR were not automatically linked to results, nor did any staff routinely check for pending orders without results. None of the offices had a standardized way to notify patients of normal test results. Depending on the physicians or medical assistants, they mailed letters or copies of results, made telephone calls, had staff make telephone calls, waited for patients to make follow-up visits, or did not notify patients at all.
Table 2.
Summary of Results From Study Visits, Patient Survey, and Chart Audit
| Description | Office A | Office B | Office C | Office D |
| Number of results management steps performed in an office that are regularly adhered to, No. (%) | 3/8 (37.5)a | 3/8 (37.5)b | 3/8 (37.5)a | 1/8 (12.5)c |
| Number of result management steps for which written protocols exist, No. (%) | 2/8 (25) | 0/8 (0) | 3/8 (37.5) | 0/8 (0) |
| Patient survey to assess laboratory testing notification and understanding | ||||
| Participants | ||||
| Survey response rate, No. (%) | 27/41 (66) | 8/17 (47) | 9/19 (47) | 19/31 (61) |
| Women, % | 81.3 | 62.5 | 67.0 | 89.0 |
| Mean age, y | 58.8 | 50.8 | 50.4 | 48.0 |
| Were you told what laboratory tests were being ordered? Yes/total returned, No. (%) | 26/27 (96) | 8/8 (100) | 9/9 (100) | 18/19 (95) |
| Have you received the results of the laboratory test? Yes/total returned, No. (%) | 27/27 (100) | 7/8 (87.5) | 9/9 (100) | 17/19 (89) |
| Were you given any instructions, advice or information about your test results? Yes/total returned, No. (%) | 18/27 (66) | 5/8 (62.5) | 8/9 (89) | 14/19 (74) |
| Chart audit to assess documentation of orders, results, and documentation | ||||
| Is there a test result in the chart for every order? Yes/number of tests ordered, No. (%) | 26/30 (87) | 55/57 (96) | 29/36 (81) | 37/41 (90) |
| Are the results located in the appropriate place in the chart yes/number of results, No. (%) | 29/31 (81) | 55/55 (100) | 29/30 (94) | 44/44 (100) |
| Is there a clinician signature on each result? Yes/number of results, No. (%) | 28/31 (90) | 55/55 (100) | 26/30 (87) | 34/44 (77) |
| Is there documentation in the chart of the clinician’s response to the result? Yes/number of results. No. (%) | 26/31 (84) | 45/55 (82) | 14/30 (47) | 33/44 (75) |
| Is there documentation that the patient was notified? Yes/number of results, No. (%) | 18/31 (58) | 47/55 (85) | 23/30 (77) | 35/44 (80) |
| Is there documentation that advice, recommendations or information were given to the patient about abnormal results? Yes/number of abnormal results, No. (%) | 5/15 (30) | 23/42 (55) | 5/18 (28) | 11/27 (41) |
a Tracking, results return to office, results to physician.
b Results return to office, results to physician, physician signature.
c Results return to office.
The patient questionnaire elicited information about notification of test results. Generally speaking, all the patient responses were relatively high, with 87.5% to 100% of respondents acknowledging they had received their test results. A smaller percentage actually received information, advice, or instructions about the results. The chart review assessed documentation of clinician response to results, patient notification, and follow-up instructions. As with the patient survey, results management steps were inconsistent across the offices, with all offices having some areas of strengths and weaknesses.
When the individual office findings were presented to the offices, offices B and C met with the study team to review the findings, office A acknowledged the findings in writing, and office D never commented on the report.
Assessment of Test Results Management Quality
No single office emerged as having notably higher quality test results management indicators, although office D had fewer standardized practices and less interest in learning about their results management assessment. We found a similar story from our interview and observation data. As we analyzed these qualitative data, we found that 2 themes, safety awareness and technological adoption, emerged as important for test results management safety. Table 3 ▶ displays our observations of how the 4 offices used the 8 factors associated with these themes in test results management.
Table 3.
Office Performance of Tasks in the 2 Test Results Management Safety Themes, Safety Awareness and Technological Adoption
| Themes and Safety Factors | Observed Behaviors in Office Sites for Each Factor | Office Sites Where Behaviors Were Observed |
| Safety awareness | ||
| Leadership | Leadership often speaks out on safety and quality; backs up words with actions | A |
| Leadership occasionally speaks out on safety and quality; occasionally backs up words with actions | B, C | |
| Leadership rarely speaks out on safety and quality; rarely backs up words with actions | D | |
| Communication | Communication between staff, physicians, and management sometimes occurs often around patient safety and quality, but communication occasionally lacks respect or timeliness; it uses only written and verbal strategies | A, B, C |
| Communication between staff, physicians, management, and patients rarely occurs around patient safety and quality and is at times disrespectful and untimely; it uses only written and verbal strategies | D | |
| Teamwork | Teamwork between staff, physicians, and leadership is present in certain areas and tasks | A, B, C |
| Teamwork between staff, physicians, and leadership is spotty, occurring only occasionally in certain areas and tasks | D | |
| Protocols and procedures | Procedures and protocols exist for some results management steps, are occasionally evaluated and revised as needed | A, C |
| Procedures and protocols do not exist for results management steps | B, D | |
| Adoption of technology | ||
| EHR/computer technology | No EHR; office computer systems exist for billing, scheduling, and for additional tasks in results management | A, C |
| EHR exists and incorporates billing, scheduling, and additional tasks in office management or care of patients | B | |
| No EHR; computer systems exist for billing and scheduling only | D | |
| Digital connections | Digital connection between 1 office computer to the hospital laboratory for order entry and to a different computer for radiology result retrieval | A |
| EHR is digitally connected to major laboratory and radiology center for results retrieval only | B | |
| Digital connection between 1 office computer to the laboratory for order entry only and between 1 office computer and hospital for radiology results only | C | |
| Digital connection between 1 office computer to the laboratory for order entry only | D | |
| Patient communication | No electronic communication with patients and no electronic generation of patient communication materials | A, C, D |
| EHR generates paper communication for patients with test results | B | |
| Forcing functionsa | No computer technology automatically performs or monitors results management steps | A, C, D |
| Return of results is only automatic/forced step in EHR; others (tracking, signatures, patient notification), but must be regularly performed or monitored by staff or physicians | B | |
EHR=electronic health record.
a Technology automatically performs a task or requires user to perform a task to move forward.
Safety awareness is the observed behaviors and expressed beliefs of office staff that show a commitment to safety and include a leadership focus on quality and safety, communication that occurs around quality and safety, teamwork in the office, and the presence of appropriate policies and procedures. For example, in office A, the leadership (medical director, office manager) frequently talked about safety and quality, the staff remarked on their commitment, and quality initiatives were discussed at team meetings. We observed functional teamwork in several areas of the office, with polite and appropriate staff and physician interactions. There were written policies and procedures for several testing process steps. By comparison, in office B, leadership only occasionally mentioned safety and quality, and staff and other physicians did not remark on their commitment. There were no team meetings on quality, but we did observe functional teamwork in some areas of the office. The office was small and staff showed minimal support for each other. There were no written protocols or procedures related to the testing process.
Adoption of technology is the incorporation and appropriate use of technology in an office to advance results management, including the presence of an EHR, digital connections between the office and testing facilities, use of technology to facilitate patient communication, and the presence of forcing functions (built-in safeguards and requirements) in the technology. Using the same offices for examples, office A had no EHR, but they did connect on 1 computer terminal to the main referral hospital laboratory (for order entry) and on another terminal for radiology results retrieval. Laboratory results were returned by means of a dedicated printer digitally connected to the laboratory. They did not electronically communicate with patients or generate patient communications, and there was no electronic management of results management steps. Office B did have an EHR, and although it automatically received test results from the main laboratory and radiology facility, it could not electronically send orders to these facilities. The EHR generated paper communication documents for patients. Except for the return of results, all test results management steps available in the EHR had to be regularly monitored and performed by staff or physicians.
Interactions Between Performance of Test Results Management Steps and Quality
Office A and B can also serve as an example of how the quality of performance of test results management steps and their safety awareness and technology adoption interact. Although the tracking system at office A did have some functional and practical problems, its existence and the office’s commitment to its success demonstrated safety awareness. Even though they were committed to safety, without more advanced technology, their system remained cumbersome and open to frequent errors. In contrast, office B had an EHR, but the office did not use it or any other system to track their test orders. Office B’s staff and physicians expressed the belief that electronic-reported results do not get lost. They ignored, however, that several tasks still depend on individuals (eg, the patient, medical assistant, laboratory and radiology staff, etc) and that problems can occur. Even though they adopted an EHR, they did not possess a safety awareness that allowed them to look for and find weaknesses in their test tracking. Both these offices wanted to provide good results management, but they were limited from achieving the highest quality by the innate complexities of the results management steps.
DISCUSSION
Management of test results is a complex process that influences safety and quality in the primary care setting. Although problems and errors in the process have been previously described,3,5,9 this study is among those few to explore tests results management using multiple methods. At these 4 family medicine offices, we found that an office’s level of safety awareness and appropriate adoption of technology to be important factors in high-quality results management. Achieving such quality does not necessarily require high performance of every factor. Good quality can be achieved in an office without an EHR, but without some use of technology, many tasks require more staff, are more cumbersome, and have higher failure rates.3 Even so, an EHR alone never guarantees high quality.30
The complexity of results management makes clear that the steps, their tasks, and the staff involved are interconnected and that multiple components of the process must be evaluated to assess for quality.31 Despite a desire by all the offices to provide high-quality results management, none of them emerged as an exemplar office. Most of their standardized practices revolved around results management steps that dealt with communication with the testing facility (return of results to the office) and getting the results to the physician. Although these steps are necessary to assure that test results arrive at the office, the lack of standardization around patient notification and follow-up is disturbing. As these steps occur farther down the process (Figure 1 ▶), there will be less chance for staff and patients to mitigate or recover from errors32,33 before harm or adverse events occur.9
The work culture of an office is important, and in the hospital setting, the importance of assessing for safety has been well established.34 Recently, safety culture in ambulatory care has also been studied.27–29 Although our study assessed the performance of some common components of a safety culture as they relate to results management (leadership, teamwork, communication, and policy and procedures), we did not fully assess the offices’ safety culture. Our observations and interviews, however, allowed us to assess each office’s use and understanding of many safety culture components, what we called an awareness of safety. This awareness varied among the offices. Even though most of the offices performed most safety awareness factors fairly well (Table 3 ▶), office A showed strong leadership in quality control, whereas office D lacked many of the fundamentals of safety awareness, findings that were consistent with the data on the quality of results management. Further research comparing validated ambulatory safety culture surveys in family medicine offices with the actual performance of results management processes would be beneficial in finding additional ways to assess safety and quality.
Technology, specifically the incorporation of the EHR by primary care physicians, has been considered by some to be necessary to ensure high quality and safety in clinical care.30,35–39 Unfortunately, the ability to achieve such quality and safety with the EHR in real practice has fallen short of the promise.30,35,37–39 Our study confirms this finding. Although only office B had a fully implemented EHR, the office was not performing at the highest level of results management quality and safety, in part because the commercial EHR product was not designed primarily for these purposes.37,38 The office’s lack of strong safety awareness then led the staff to a sense of complacency. While enjoying the improved electronic return of results from the main laboratory and radiology facilities, they did not look for the other tasks and steps where errors and problems still occur.
All the offices used technology to some extent for test results management, but those without an EHR found that tracking, follow-up, and documentation required more staff time than often was available, findings consistent with other research focusing on the testing process in family medicine.3 These offices often found that their results management steps were not maintained at a high level when staff were pulled away for other purposes, that training was minimal, and that shortcuts were taken to save time. Dependence on staff for tracking and follow-up often meant dependence on human memory and double-checking,40 areas usually improved by technology.
There are limitations to this study. For financial and logistic reasons, we studied only 4 offices, and they were all in a single geographic region, limiting potential generalizability. Even though we purposefully chose offices with a range of characteristics, that they agreed to participate may indicate they were different from offices that chose not to participate. We collected data on processes, so we do not know whether the different processes we observed were correlated with different clinical outcomes. As an exploratory study, not all factors related to the quality of test results management may have been delineated, and we were unable to assess relationships between office characteristics, such as size and location and the quality of results management. Further research should help answer these important questions.
Family medicine offices today are asking for best practices and guidelines, not only for the best drug to prescribe, but also for the best systems or processes in which to practice.41–43 Unfortunately, these common, everyday processes, including test results management, are often lacking. As a result, even in this small sample of family medicine offices, we found great variation in how these tasks are performed, both by offices and by individuals within an office. We did find that 2 themes are important in assessing a high-quality results management process: an awareness of safety (exemplified by strong leadership, open communication, functional teams, and appropriate policies and procedures), and the adoption of appropriate technology. Although it is probably both impossible and unwise to assume there will ever be only 1 best way to manage test results in primary care, an increased awareness of quality factors and successful examples of results management13,26,44 would go a long way toward improving quality and safety in the office setting.
Conflicts of interest: none reported.
Funding support: This research is supported by a grant from the Agency for Healthcare Research and Quality: 1 K08 HS013914-01A2.
REFERENCES
- 1.Hickner JM, Fernald DH, Harris DM, Poon EG, Elder NC, Mold JW. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf. 2005;31(2):81–89. [DOI] [PubMed] [Google Scholar]
- 2.Boohaker EA, Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996;156(3):327–331. [PubMed] [Google Scholar]
- 3.Elder N, Graham D, Brandt E, et al. The testing process in family medicine: problems, solutions and barriers as seen by physicians and their staffs. A study of the American Academy of Family Physicians’ National Research Network. J Patient Saf. 2006;2(1):25–32. [Google Scholar]
- 4.Mold JW, Cacy DS, Dalbir DK. Management of laboratory test results in family practice. An OKPRN study. Oklahoma Physicians Resource/Research Network. J Fam Pract. 2000;49(8):709–715. [PubMed] [Google Scholar]
- 5.Poon EG, Gandhi TK, Sequist TD, Murff HJ, Karson AS, Bates DW. “I wish I had seen this test result earlier!”: Dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004;164(20):2223–2228. [DOI] [PubMed] [Google Scholar]
- 6.Dovey SM, Meyers DS, Phillips RL Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care. 2002;11(3):233–238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Fernald D, Pace W, Harris D, West D, Main D, Westfall J. Event reporting to a primary care patient safety reporting system: a report from the ASIPS Collaborative. Ann Fam Med. 2004;2(4): 327–332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Makeham MA, Dovey SM, County M, Kidd MR. An international taxonomy for errors in general practice: a pilot study. Med J Aust. 2002;177(2):68–72. [DOI] [PubMed] [Google Scholar]
- 9.Hickner J, Graham D, Elder N, et al. Testing process errors and their harms and consequences reported from family medicine practices: a study of the AAFP National Research Network. Qual Saf Health Care. 2008;17(3):194–200. [DOI] [PubMed] [Google Scholar]
- 10.Murff H, Bates D. Information Transfer in Making health Care Safer: A Critical analysis of Patient Safety Practices. University of California at San Franscisco–Stanford University Evidence Based Practice Center; 2001. AHRQ publication no. 01-E058.
- 11.Phillips RL Jr, Bartholomew LA, Dovey SM, Fryer GE Jr, Miyoshi TJ, Green LA. Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care. 2004;13(2):121–126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Matheny ME, Gandhi TK, Orav EJ, et al. Impact of an automated test results management system on patients’ satisfaction about test result communication. Arch Intern Med. 2007;167(20):2233–2239. [DOI] [PubMed] [Google Scholar]
- 13.The Physician Practice Patient Safety Assessment. Institute for Safe Medical practices; Health Research and Educational Trust; Medical Group Management Association; 2006. http://www.physiciansafe-tytool.org/. Accessed May 13, 2008.
- 14.Patient safety strategies: tracking of test results. CRICO/RMF. http://www.rmf.harvard.edu/patient-safety-strategies/office-practices/main/menu.aspx. Accessed May 13, 2008.
- 15.Hickner J. Reducing test management errors in primary care office practice. J Patient Safety. 2005;1(1):70–71. [Google Scholar]
- 16.West D, Westfall J, Araya-Guerra R, et al. Using Reported Primary Care Errors to Develop and Implement Patient Safety Interventions: A Report from the ASIPS Collaborative. Advances in Patient Safety: From Research to Implementation. Rockville, MD: Agency for Healthcare Research and Quality. Publication number 050021. [PubMed]
- 17.Murff HJ, Gandhi TK, Karson AK, et al. Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inform. 2003;71(2–3):137–149. [DOI] [PubMed] [Google Scholar]
- 18.Hanna D, Griswold P, Leape LL, Bates DW. Communicating critical test results: safe practice recommendations. Jt Comm J Qual Patient Saf. 2005;31(2):68–80. [DOI] [PubMed] [Google Scholar]
- 19.Steele AW, Eisert S, Witter J, et al. The effect of automated alerts on provider ordering behavior in an outpatient setting. PLoS Med. 2005;2(9):e255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Baldwin DM, Quintela J, Duclos C, Staton EW, Pace WD. Patient preferences for notification of normal laboratory test results: a report from the ASIPS Collaborative. BMC Fam Pract. 2005;6(1):11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Meza JP, Webster DS. Patient preferences for laboratory test results notification. Am J Manag Care. 2000;6(12):1297–1300. [PubMed] [Google Scholar]
- 22.Schofield MJ, Sanson-Fisher R, Halpin S, Redman S. Notification and follow-up of Pap test results: current practice and women’s preferences. Prev Med. 1994;23(3):276–283. [DOI] [PubMed] [Google Scholar]
- 23.Marcus AC, Kaplan CP, Crane LA, et al. Reducing loss-to-follow-up among women with abnormal Pap smears. Results from a randomized trial testing an intensive follow-up protocol and economic incentives. Med Care. 1998;36(3):397–410. [DOI] [PubMed] [Google Scholar]
- 24.Schiff GD, Kim S, Krosnjar N, et al. Missed hypothyroidism diagnosis uncovered by linking laboratory and pharmacy data. Arch Intern Med. 2005;165(5):574–577. [DOI] [PubMed] [Google Scholar]
- 25.Poon EG, Haas JS, Louise Puopolo A, et al. Communication factors in the follow-up of abnormal mammograms. J Gen Intern Med. 2004;19(4):316–323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform. 2003;36(1–2):80–91. [DOI] [PubMed] [Google Scholar]
- 27.Schutz AL, Counte MA, Meurer S. Development of a patient safety culture measurement tool for ambulatory health care settings: analysis of content validity. Health Care Manag Sci. 2007;10(2):139–149. [DOI] [PubMed] [Google Scholar]
- 28.Shostek K. Developing a culture of safety in ambulatory care settings. J Ambul Care Manage. 2007;30(2):105–113. [DOI] [PubMed] [Google Scholar]
- 29.Modak I, Sexton JB, Lux TR, Helmreich RL, Thomas E. Measuring safety culture in the ambulatory setting: The safety attitudes questionaire- ambulatory version. J Gen Intern Med. 2007;22(1):1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400–1405. [DOI] [PubMed] [Google Scholar]
- 31.Carayon P, Schoofs Hundt A, Karsh BT, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006; 15(Suppl 1):i50–i58. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Parnes B, Fernald D, Quintela J, et al. Stopping the error cascade: a report on ameliorators from the ASIPS collaborative. Qual Saf Health Care. 2007;16(1):12–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Woolf SH, Kuzel, A.J., Dovey, S.M., Philips, R.L. A String of mistakes: the importance of cascade analysis in describing, counting, and preventing medical errors. Ann Fam Med. 2004;2(4):317–326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Zohar D, Livne Y, Tenne-Gazit O, Admi H, Donchin Y. Healthcare climate: a framework for measuring and improving patient safety. Crit Care Med. 2007;35(5):1312–1317. [DOI] [PubMed] [Google Scholar]
- 35.Bates DW. Physicians and ambulatory electronic health records. U.S. Physicians are ready to make the transition to EHRs—which is clearly overdue, given the rest of the world’s experience. Health Aff (Millwood). 2005;24(5):1180–1189. [DOI] [PubMed] [Google Scholar]
- 36.Burstin HR. Achieving the potential of health information technology. J Gen Intern Med. 2008;23(4):502–504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Edsall RL, Adler KG. An EHR user-satisfaction survey: advice from 408 family physicians. Fam Pract Manag. 2005;12(9):29–35. [PubMed] [Google Scholar]
- 38.Miller RH, West C, Brown TM, Sim I, Ganchoff C. The value of electronic health records in solo or small group practices. Physicians’ EHR adoption is slowed by a reimbursement system that rewards the volume of services more than it does their quality. Health Aff (Millwood). 2005;24(5):1127–1137. [DOI] [PubMed] [Google Scholar]
- 39.Trachtenbarg D. EHRs fix everything—and nine other myths. Fam Pract Manag. 2007;14(3):26–30. [PubMed] [Google Scholar]
- 40.Elder N, McEwen T, Flach J, Gallimore J. Creating Safety in the Testing Process in Primary Care Offices. Advances in Patient Safety: New Directions and Alternative Approaches. Vol 2. Rockville, MD: Agency for Healthcare Research and Quality. AHRQ Publication No. 08-0034(2). http://www.ahrq.gov/qual/advances2.
- 41.Transfor MED. Empowering Medical Practice. 2007. A http://www.transformed.com/. Accessed Jul 13, 2007.
- 42.Mohr J, Batalden P, Barach P. Integrating patient safety into the clinical microsystem. Qual Saf Health Care. 2004;13(Suppl 2):ii34–ii38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Mohr JJ, Batalden PB. Improving safety on the front lines: the role of clinical microsystems. Qual Saf Health Care. 2002;11(1):45–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.2008 National Patient Safety Goals Ambulatory Care Program. The Joint Commission; 2007. http://www.jointcommission.org/Patient-Safety/NationalPatientSafetyGoals/08_amb_npsgs.htm. Accessed May 13, 2008.