Practice-audit-publish: A practice reflection

Abstract

Practice audits are useful opportunities to improve practice efficiency and effectiveness, reduce clinical errors, demonstrate quality care to stakeholders, promote high standards of practice, lower the risk of liability, and foster practice change. However, a benefit that is usually overlooked is the possibility of publication of the results of a practice audit. Publication (research) has a number of benefits for the clinician, including skill development as a scholar, communicator, professional, and collaborator. A practice audit is beneficial to an individual physician; furthermore, publication of the audit results could be beneficial for many others such as health care providers, patients, and other stakeholders in a health care system. The problem is that practice audits often begin without a clear plan. The important steps in planning and carrying out a practice audit can be captured by thinking about how a research publication evolves. Thus, a good researcher is a good practice auditor. This paper reviews the author’s experience and provides examples and directions of the process of practice-audit-publish.

Introduction

An often proposed tool of continual education is the practice audit (1, 2). The practice audit is, in effect, a performance review. It is an opportunity for the appraisal (by self or others) of patient care, with specific measures considered for improvement. Practice audits may improve practice efficiency and effectiveness, reduce clinical errors, demonstrate quality care to stakeholders, promote high standards of practice, lower the risk of liability, and foster practice change. However, a benefit that is usually overlooked is the possibility of publication of the results of a practice audit. Publication (research) has a number of benefits for the clinician, including skill development as a scholar, communicator, professional, and collaborator. A practice audit is beneficial to an individual physician; furthermore, publication of the audit results could be beneficial for many others such as health care providers, patients, and other stakeholders in a health care system. There is also the possibility of practice-audit-publish. Although there are extensive guidelines dealing with how to conduct a practice audit, they focus on practice improvement as the main goal; they also do not consider the audit as a means to become a researcher and to publish results (2). Yet, the important steps in planning and carrying out a practice audit can be captured by thinking about how a research publication evolves. Thus, a good researcher is a good practice auditor.

Although practice audits are not the “breakthrough” publications, they tell us about things that happen every day in our practices. Practice audits reflect the challenges of practice, and the results of a practice audit are often immediately relevant to practitioners in many centers who face the same challenges. To date, I have been conducting practice audits for several years with publications as one of the outcomes. Using the approach described below, I have published 30 original research papers (3–32). The subjects of these practice audits (and subsequent studies) are clearly the patients referred to my consultant practice. They have undergone a usual practice assessment, including history, physical examination, occasionally collection of health questionnaire data, investigations, and treatment with outcomes noted. The distinct feature of the practice-audit-publish program is the order of effort. Because the possibility of a publication is under consideration, the order of thinking changes. First, one asks a research question that a practice audit might be able to answer. Then, one checks if the data required to answer the question is already a part of usual practice or if an alteration of the usual practice is necessary (i.e., being more deliberate in collecting the same data in all patients). Data is then collected to answer a research question and a publication is possible. Hence, the practice audit begins with first considering how one would publish the results if they were of interest.

The process of practice-audit-publish

Step 1. Think about the paper you will write to publish your practice audit results

If one starts a practice audit without thinking like a researcher, chances are that the methodology will be haphazard, the question to be answered will be unclear, the data will be incomplete, and the results will be less useful; however, thinking about a practice audit as a research paper immediately brings to mind the many requirements of a research project including literature review with a clear research question and methodology (paying attention, particularly to how data is collected) as well as the compilation of the results in a manner such that the results are meaningful to oneself and to others. One of the first steps in any research project (this is how one must now think about practice audit) is to have a clearly stated objective. In research papers, this is often called the research question.

Example 1. The background of the issue in this published audit (24) was that in Canada, guidelines have advised that bone densitometry should be performed for all men aged ≥65 years. However, previous studies have indicated that it is not likely to be cost-effective to routinely screen men aged >65 years for osteoporosis. Cost-effectiveness depends on how prevalent osteoporosis is in men aged >65 years. If it is not at least 7%, screening in a primary care setting is not cost-effective (33). Thus, the research question became as follows: what is the prevalence of osteoporosis in an unselected primary care population of men aged ≥65 years? In this case, the author had been working as a consultant in a primary care clinic and, thus, was conducting an audit of the entire clinic by working with the primary care physicians to understand the prevalence of osteoporosis in the clinic’s population of men aged ≥65 years. More details of the literature review and background for this research can be found in the publication (24).

Example 2. The issue in this published practice audit dealt with the prevalence of primary hyperparathyroidism in fibromyalgia patients. A study had been published showing a very high prevalence of primary hyperparathyroidism in fibromyalgia patients referred to tertiary care (34). Primary hyperparathyroidism can be associated with chronic pain and may be an underdiagnosed condition that can be readily treated in fibromyalgia patients. As a practitioner, I thought that I should start screening all fibromyalgia patients for primary hyperthyroidism; however, was this the right thing to do? A practice audit would help answer this question. Thus, with the help of another rheumatology colleague, we undertook a study to assess whether primary hyperparathyroidism may play an etiological role in widespread pain in our fibromyalgia patients. Indeed, we wondered if this condition was common in widespread pain and in chronic pain patients. Or is the condition merely coincidental? Now, the research question was as follows: what is the prevalence of primary hyperparathyroidism in our population of referred fibromyalgia patients and non-specific chronic pain disorder patients?

Step 2. Write an introduction to the paper

In any research paper, a review of what is known on the topic and the importance of the topic are first expressed. The unanswered questions are also stated, and the motivation for the current study is described. The same should be done before the practice audit. A few examples of unanswered questions are as follows: why is this topic important to you? What do you already know about it? What do you still need to learn about it? Perhaps someone has already answered these questions.

Example 1. In the practice audit, to examine the prevalence of osteoporosis in men aged ≥65 years, a literature review was conducted for the problem that revealed that the prevalence of osteoporosis in this population in primary care is not known (24). I further learned that there was no specific evidence base for recommending osteoporosis screening in men in this age group, except for a 1998 publication that attempted an extrapolation of the prevalence [see (24) for more details].

Example 2. In the study of the prevalence of primary hyperparathyroidism in fibromyalgia subjects (27), the literature review revealed the prevalence of primary hyperparathyroidism in the general population, the risk factors of this condition, and what one would expect to be the prevalence in patients who do not have chronic pain [see (27) for details]. This review helped us to decide our methodology (see below).

Step 3. Write out the methodology you would use to answer this question as if you were writing the paper

A practice audit is more likely to be successful if you have a clear idea of exactly who you are going to study (the study patient), in which clinic will the study be conducted, what is the study duration, and which data (clinical and laboratory) will be collected. Mainly data is collected through chart data. It is helpful to know in advance as to what data will be in those charts and how it can be accessed. If a methodology is developed and when certain data is needed that is not in the charts, the practice audit will be difficult or impossible. It is better to be aware about this before starting the practice audit. The practice needs to be standardized for collecting information in the patients who are examined; then, the practice audit can be conducted at some point in the future.

To help focus the process, the following must be done exactly (in advance of your practice audit) as required in a research paper: describe your subject population and setting, the recruitment process, the criteria for inclusion or exclusion into the group that will ultimately be studied, the data that will be collected on each subject, and how the data will be analyzed, including a sample size calculation for knowing whether the obtained results differ from those expected; for example, the known proportion or mean of the result you are measuring. Again, if you do this first, you will be able to determine if your practice and charting contains the type of data you need and whether it is extractable. Alternatively, because you may be introducing something new into your practice, you have an opportunity to introduce a new intervention or concept into care and also do so in a manner that ensures that data relating to this new practice will be accessible. You essentially “set-up” your usual practice habits and charting for an audit in the future by knowing what type of data you will later need in the charts.

This is the most difficult part of the practice audit. Frequently, practice audits begin with opening charts and writing down data without knowing if you even have the data you need, if it is available in every patient’s chart, or how many charts you need to review. Considering every aspect of your methodology before you start, the practice audit is essential. If you cannot write down the procedure exactly, do not conduct the practice audit.

Example 1. For this study on osteoporosis (24), we had to first figure out how to find each man in the clinic’s registry aged 65–75 years (a specific study group). We had to then contact them to either undergo a bone density test or to find out whether they had already undergone the test. However, the following situations may arise. What if we could not find every patient? What if the patients attended the clinic 5 years previously and have not been seen since then? What if their current contact information was incorrect? Could we follow-up with them at a later date when they incidentally attend the clinic for some other reason and ask them about having a bone density? This was the most challenging part of the study. Other doctors, nurses, and myself in the clinic had to work to ensure that we recruited the population of men aged 65–75 years in the clinic and also that each one either had already undergone a bone density scan or would be undergoing it. That is the standard recommended by the Canadian guidelines. Thus, we performed the study as per the recommendation of the guidelines. We also had to describe our population. Are they similar to many other primary care populations of men aged 65–75 years? To understand this, we needed to know whether they smoked, which drugs were they taking, and whether they had diabetes, rheumatoid arthritis, or previous fractures. Fortunately, we learned that this data was being routinely collected as part of normal practice by the radiology clinics performing the bone density scan. They were not reporting it in their bone density results as usual practice, but I asked them to start doing so. Otherwise, the data would not be available and the practice audit may not have been performed. See reference (24) for more details.

Example 2. See paper (27). Here, we had to focus how many fibromyalgia patients we would check for primary hyperparathyroidism. Should we check 10, 100, or 1000? How long would it take us to perform this practice audit? This meant a formal sample size calculation. Often, the help of an experienced researcher is needed for this. In addition, we had to confirm that it was primary hyperparathyroidism, if at all it was detected. This meant checking renal function, serum vitamin D, and serum alkaline phosphatase because there are other causes of elevated parathyroid hormone. This in turn meant that our normal clinical practice had to be to screen all fibromyalgia and chronic pain patients for primary hyperparathyroidism, with all the proper laboratory indices. We had to do this in every patient, knowing that in the future, we would consider an audit. Again, thinking like researchers changed our practice, which then made the audit possible and the results publishable. Here is an example of from where thinking like a researcher needs to start (is the data going to be available and complete?).

Step 4. Get ethics approval

Obtain ethics approval where necessary. It is important that one should be able to describe who will access the chart data, how that data will be recorded, whether identifiable data (name, birthdate, phone numbers, health care numbers) will persist, or whether the data will be anonymized (i.e., each subject is given a number). Who will then have access to the data and where will it be stored? An ethics application is an opportunity to work out these details.

Example 1. For the osteoporosis study (24), as physicians, we decided to apply the recommended screening guidelines as this was considered as a good practice. We then had to inform the ethics board that we now wanted to check each bone density result and accompanying data by a chart review. The results would be entered into an Excel file in which each patient would ultimately receive a number rather than a name (anonymization). Hence, not even the researchers would know who the data belonged to and confidentiality can be maintained. I had to decide who would carry out the chart review and enter the data to ensure good ethical research. Who was going to keep that Excel file and who might have access to it are questions the ethics board typically raises.

Example 2. For the study involving primary hyperparathyroidism (27), we had the same issues above to explain to the ethics board. Here, we were conducting laboratory measures in every patient with chronic pain; the results of the measures would be entered in the chart. We then had to extrapolate those data along with age and diagnosis. Placing the data in an Excel file, we then had to anonymize (substitute a number for a name) each patient to ensure that even if someone else saw the data, they could not link it to a specific patient.

Step 5. Get help to collect data

A successful practice audit may mean that you need help from your reception and nursing staff as well as from colleagues in your practice to ensure that the data you need is present in every chart, for example, patients are not inadvertently excluded from the audit. If you are examining whether a new health questionnaire will be useful in your practice, how are you going to ensure that without introducing any bias, every patient you examine gets that questionnaire and can complete it? Who will explain the questionnaire to the patient and where will they complete it? How will the completed questionnaires be collected and collated? Who will check them for completeness?

Example 1. To study the prevalence of osteoporosis, men aged ≥65 years were asked to come to the clinic for a bone density scan requisition and were given an explanation for the same. When these men visited the clinic for other reasons, they were told about the need for a bone density scan. There was a concerted effort by the practitioners in the clinic to capture every man aged ≥65 years to ensure that he has either undergone a bone density scan or will be undergoing one.

Example 2. To study the prevalence of primary hyperparathyroidism in fibromyalgia patients, it was helpful to pool referred patients from more than one rheumatology practice to achieve sufficient sample size (by initially thinking like a researcher that led to the sample size calculation). Cooperation with a colleague made this possible.

Step 6. Analyze the data

You may need help to analyze the data you collect. One of the first steps in the analysis of data is to ensure that you have collected useful data. It is worthwhile to speak with an experienced researcher or a statistician before you start your practice audit. They can help you decide on your research question and also help by guiding how data can be reliably collected as well as how to organize it for analysis. Similar to writing a research paper, write the categories of data that you will collect. It is at this point that you may learn that some essential categories are needed but are not available in the chart data. There is little point in proceeding with the audit in this case. You may have to change your charting process for the future and revisit the practice audit.

You will need to develop a template into which you transcribe data from charts, Excel file, or similar tool where you can organize the entire sample population’s data that are ready for data analysis.

Example 1. To study the prevalence of osteoporosis, the items contained within the usual data collecting practice of the radiology centers that conducted bone density measurements were assessed. It was important to ensure that data was complete enough to characterize the population in terms of risk factors for osteoporosis and to allow others to generalize the results to their own clinical populations.

Example 2. To study the prevalence of primary hyperparathyroidism in fibromyalgia patients, it was necessary to ensure that the usual practice involved measuring all of the chemistries associated with the disturbances of parathyroid hormone levels. This list of laboratory investigations was developed prior to sampling consecutive patients so that the data would be available in every case upon chart audit.

Step 7. Get the credit

Practice audits and publications qualify for Continuing Professional Development credits in both Maninpro-M1 and M2 categories in a one-to-one credit for each hour you spend. It takes a lot of time to perform a practice audit and publish a manuscript.

Conclusion

Our practices are loaded with data. Patient’s charts are loaded with data. Every time a clinician introduces something new into his practice, there is an opportunity to measure an effect. If you think about a practice audit as an effort to publish the results of a data analysis, you can plan your practice audit by going through all the steps described and necessary for a research publication. Being able to put together the components of a research paper (an introduction and methodology with a clear research question) is a good test of how ready you are to perform a practice audit that will produce meaningful results.

By thinking about practice audits as research projects with the goal of publication, the clinician becomes a researcher, adopts the rigorous methodology of a researcher, and is likely to learn a great deal more and have a more successful audit.