Abstract
Background
Practice-based research networks (PBRNs) provide a venue to foster evidence-based care. We tested the hypothesis that a higher level of participation in a dental PBRN is associated with greater stated change toward evidence-based practice.
Methods
A total of 565 dental PBRN practitioner-investigators completed a baseline questionnaire entitled “Assessment of Caries Diagnosis and Treatment”; 405 of these also completed a follow-up questionnaire about treatment of caries and existing restorations. Certain questions (6 treatment scenarios) were repeated at follow-up a mean (S.D.) of 36.0 (3.8) months later. A total of 224 were “full participants” (enrolled in clinical studies and attended at least one network meeting); 181 were “partial participants” (did not meet “full” criteria).
Results
From 10% to 62% of practitioners were “surgically invasive” at baseline, depending on the clinical scenario. Stated treatment approach was significantly less-invasive at follow-up for 4 of 6 items. Change was greater among full participants and those with a more-invasive approach at baseline, with an overall pattern of movement away from the extremes.
Conclusions
These results are consistent with a preliminary conclusion that network participation fostered movement of scientific evidence into routine practice. PBRNs may foster movement of evidence into everyday practice as practitioners become engaged in the scientific process.
Keywords: Implementation research, practice-based research, dentistry, quality improvement, dental caries, dental restorations
INTRODUCTION
Many methods to incorporate current evidence into everyday clinical practice have been attempted. Methods that seem to be effective have been educational outreach done in the practice itself, reminder systems, financial incentives, and interactive meetings that encourage discussion and practice (1–7). Passive dissemination of materials, developing clinical guidelines, or simply attending didactic meetings generally have not been effective, and use of opinion leaders and audit and feedback approaches have had variable effectiveness (8,9). Practice-based research networks (PBRNs) have the potential to act as important means to move the latest scientific evidence into routine clinical practice. Demonstrating this potential would be important because it can take many years before evidence is incorporated into routine practice or become standard of care; consequently, patients may receive health care that is not consistent with current evidence, is not needed, or is even potentially harmful (10–12).
PBRNs have continued to grow in number because of the unique advantages they offer both to research and quality improvement, as well as because of their ability to bring practice-relevant topics onto the research agenda (13–16). One such PBRN is “The Dental Practice-Based Research Network (DPBRN)”. Many details about the network are publicly available at its web site (17), but briefly, it comprises practitioner-investigators and staff in outpatient dental practices from the United States and the Scandinavian countries of Denmark, Norway, and Sweden (18–21). With regard to the representativeness of its dentists, the network has a wide representation of practice types, treatment philosophies, and patient populations, including diversity with regard to the race, ethnicity, geography and rural/urban area of residence of both its practitioner-investigators and their patients. Analyses of these characteristics confirm that network dentists have much in common with dentists at large (22), while at the same time offering substantial diversity with regard to these characteristics (23). The network emphasizes doing “practical science” about, in, and for the benefit of “real world” clinical practice. This means that the practitioner-investigators themselves actively participate in developing ideas for studies as well as in designing, conducting, and communicating this research - all with the intent of having a direct, practical impact on everyday clinical practice. The network also holds annual regional meetings of its practitioner-investigators that provide opportunities to learn about and discuss the latest results from network studies and how they might impact practice. At these meetings, practitioner-investigators also design future studies, network with colleagues, and obtain continuing education credit. Additionally, these meetings encourage acceptance of responsibility for improving one’s own clinical practice, by understanding study results from one’s own practice and in the practices of fellow network practitioner-investigators.
The network has previously reported evidence that practitioner-investigators can serve as change agents (24). Those most “eligible” to act as change agents are those who have already incorporated recent scientific evidence into daily clinical practice, either because of their participation in the network or some other reason. When data from their own practice and from fellow practitioner-investigators are collected in a scientific manner as is done in network studies, practitioner-investigators have reported informally that they are more likely to use this information and are more likely to change practice if they are fully engaged in network activities (20,25). Instead of relying on informal, anecdotal reports, we sought to formally collect data on stated clinical practice (in this instance, clinical practice having to do with caries and restorative treatment), and its change over time. This allowed us to test the hypothesis that a higher level of participation in network activities is associated with greater stated change toward evidence-based practice -- the objective for this report.
METHODS
This study engaged network dentist practitioner-investigators on three key occasions: (1) an enrollment questionnaire; (2) a baseline questionnaire entitled “Assessment of Caries Diagnosis and Treatment”; and (3) a follow-up questionnaire entitled “Practice Impact Questionnaire”. This project was approved by the human participants institutional review boards (IRBs) at the University of Alabama at Birmingham and all of DPBRN’s regional IRBs. The informed consent of all human subjects who participated in this investigation was obtained after the nature of the procedures had been explained fully.
DPBRN Enrollment Questionnaire
As part of enrollment in the network, all practitioner-investigators complete a 101-item Enrollment Questionnaire about their practice characteristics and themselves. This questionnaire is publicly available (26), and the distribution of these characteristics for network dentists has been reported previously (22,23).
Baseline Questionnaire Entitled “Assessment of Caries Diagnosis and Treatment”
All network dentist practitioner-investigators who indicated on their Enrollment Questionnaire that they do at least some restorative dentistry (n = 998) were invited to participate in a baseline questionnaire entitled “Assessment of Caries Diagnosis and Treatment, of whom 565 did so. Methodologic particulars, such as sample selection, recruitment process, length of the field phase, data collection process, procedures used during a pilot study and questionnaire pre-testing, test/re-test reliability, and evaluation of non-response bias, have been reported previously (27–29). Among other items, this questionnaire included treatment scenarios in which practitioner-investigators stated how they would treat these hypothetical patients if they appeared in their practices. These scenarios are the subject of this paper. The full questionnaire is publicly available (26).
The first scenario (scenario 1 in Table 2) had a defective composite restoration with cementum-dentin margins and a description of a patient who had been a regular dental patient and who had existing dental restorations. A second scenario (scenario 2 in Table 2) had a defective composite restoration with enamel margins and a description of a patient at low caries risk. A third scenario (scenario 3 in Table 2) had a defective amalgam restoration and a description of the same patient at low caries risk. The nine treatment options that were provided in each scenario ranged from no treatment to replacement of the entire restoration. We categorized these nine treatment options into three categories: (1) no treatment or preventive treatment only; (2) polish, re-surface, or repair the restoration (as one provided choice); and (3) replace the entire restoration.
Table 2.
Distribution of responses at baseline to six treatment scenarios, among practitioner-investigators who participated for both the baseline and follow-up questionnaires (n=405)
| Do no treatment or preventive treatment only | Polish, re-surface, or repair | Replace the entire restoration | |
|---|---|---|---|
| Existing restoration scenarios | |||
| 1. Defective composite restoration with cementum-dentin margins | 12% (n=48) | 25% (n=101) | 62% (n=248) |
| 2. Defective composite restoration with enamel margins | 12% (n=49) | 54% (n=212) | 34% (n=134) |
| 3. Defective amalgam restoration | 50% (n=194) | 14% (n=56) | 36% (n=138) |
| Do no treatment or preventive treatment only | Do minimal intervention | Place a complete restoration | |
|---|---|---|---|
| Occlusal caries scenarios | |||
| 4. Occlusal caries scenario 1 | 36% (n=140) | 54% (n=209) | 10% (n=40) |
| 5. Occlusal caries scenario 2 | 9% (n=33) | 52% (n=194) | 40% (n=149) |
| Place a restoration at this lesion depth… | |||||
|---|---|---|---|---|---|
| Outer ½ of the enamel | Inner ½ of the enamel | Outer 1/3 of the dentin | Middle 1/3 of the dentin | Inner 1/3 of the dentin | |
| Proximal caries scenario | |||||
| 6. Proximal caries scenario | 2% (n=7) | 38% (n=145) | 53% (n=203) | 8% (n=29) | < 1% (n=1) |
Number of missing values: n = 8 for defective composite restoration with cementum-dentin margins; n = 10 for defective composite restoration with enamel margins; n = 17 for defective amalgam restoration; n = 16 for occlusal caries scenario 1; n = 29 for occlusal caries scenario 2; n = 20 for proximal caries scenario. Percentages in some rows do not add to 100% due to rounding.
The next scenarios involved two clinical images of increasingly severe occlusal caries (scenarios 4 and 5 in Table 2). Responses were again divided into three ordinal categories, by highest degree of clinical intervention recommended: (1) no treatment or preventive treatment only; (2) minimal intervention; and (3) complete restoration. The options for minimal intervention included: minimal drilling with sealant; minimal drilling with preventive resin restoration; air abrasion with a sealant; and air abrasion with preventive resin restoration. The options for complete restoration were amalgam restoration, composite restoration, and indirect restoration.
The next scenario (scenario 6 in Table 2) involved five radiographic images of a caries lesion of increasing depth located in the interproximal surface of a mandibular premolar (outer half of enamel; inner half of enamel; outer one-third of dentin; middle one-third of dentin; inner one-third of dentin). Practitioners were asked to indicate which image corresponded to the lesion depth at which they would recommend doing a permanent restoration rather than only preventive therapy.
Follow-up Questionnaire entitled “Practice Impact Questionnaire”
As part of an overall effort to determine the effect of participation in PBRN research, a follow-up questionnaire was also completed. This questionnaire repeated the same six scenarios, along with other items. The full questionnaire is publicly available (26).
To quantify test-retest reliability for the questionnaire at this time point, the questionnaire was completed twice by a total of 18 practitioners, who completed the second questionnaire a mean (S.D.) of 62.9 (30.1) days after the first questionnaire. For the scenarios used in this report, the mean value of the kappa statistic for the test-retest reliability was 0.87, with a range of 0.55–1.0.
Participation level
A key variable of interest was the level of participation in network activities. A “full participant” was defined as a practitioner-investigator who had attended a network orientation and training session or watched a video of it; had completed training in human participants research; had completed training and data collection for at least one study that involved recruitment of patients; and had attended at least one network meeting. A “partial participant” was a practitioner who did not meet all of the “full” criteria. A total of 405 practitioner-investigators participated for both the baseline and follow-up questionnaires, of whom 224 were full participants and 181 were partial participants.
Full participants attended a mean (S.D.) of 1.4 (1.1) annual meetings (range of 1–4 meetings), compared to 0.1 (0.4) annual meetings for partial participants (range of 0–2 meetings). Full participants participated in 2.8 (1.2) clinical studies (range of 1–5 studies), compared to none for partial participants. In addition to the current questionnaire study, full participants participated in 1.8 (0.4) questionnaire studies (range of 1–2 questionnaire studies), compared to 2.0 (0.1) for partial participants (range of 1–2).
Statistical Methods
All analyses were done using SAS (30). Statistical significance was assumed for a p-value less than 0.05. In addition to quantifying frequency distributions (Tables 1 and 2), baseline responses were compared with follow-up responses to the same scenario (Tables 3 and 4). The Bowker’s test of symmetry (31) was used to determine whether patterns of change between baseline and follow-up were statistically significant. This statistic does not depend on the proportion of respondents whose answers change. Instead, the statistic quantifies whether they are more likely to change in one direction than another. The statistic tests whether - among those whose answers changed - the direction of change was symmetrical. We also quantified concordance between baseline and follow-up responses to these scenarios using weighted kappa statistics; we observed kappa values of 0.38–0.47. Bowker’s tests of symmetry were statistically significant. Having determined that the pattern of change was statistically significant using Bowker’s tests, we next calculated a change score of “more-invasive than at baseline” (−1), “the same as that at baseline” (0), and “less-invasive than at baseline” (+1), and then tested for the statistical significance of the association between change score and the baseline level of invasiveness (Mantel Haenszel χ2 trend test).
Table 1.
Characteristics of practitioner-investigators who participated in both the baseline and follow-up questionnaires (n=405)
| Characteristic (n) | Percentage |
|---|---|
| DPBRN region | |
| Alabama/Mississippi (220) | 54% |
| Florida/Georgia (62) | 15% |
| Scandinavia (53) | 13% |
| Permanente Dental Associates (37) | 9% |
| Minnesota (25) | 6% |
| Other (8) | 2% |
| Missing (0) | |
| DPBRN participation level | |
| Full participant (224) | 55% |
| Partial participant (181) | 45% |
| Missing (0) | |
| General/specialist | |
| General dentist (382) | 95% |
| Pediatric dentist (19) | 5% |
| Prosthodontist (2) | 1% |
| Public health dentist (1) | <1% |
| Missing (1) | |
| Years since dental degree (at baseline) | |
| Less than 5 years (59) | 15% |
| 6–15 years (75) | 19% |
| 16–20 years (58) | 15% |
| 21–25 years (80) | 21% |
| More than 25 years (113) | 29% |
| Missing (20) | |
| Race of dentist | |
| White/Caucasian (373) | 92% |
| Other (32) | 8% |
| Missing (0) | |
| Gender of dentist | |
| Male (323) | 80% |
| Female (82) | 20% |
| Missing (0) | |
| Practice setting | |
| Urban (296) | 84% |
| Rural (55) | 16% |
| Missing (54) | |
Table 3.
Comparison of change between baseline and follow-up, by baseline level of invasiveness
| Baseline response to treatment scenario | Response at follow-up was … | ||
|---|---|---|---|
| … more-invasive than baseline | … the same as that at baseline | … less-invasive than baseline | |
| 1. Defective composite restoration with cementum-dentin marginsa | |||
| No treatment/preventive only (n=48) | 54% (n=26) | 46% (n=22) | np |
| Polish, re-surface, repair (n=101) | 29% (n=29) | 62% (n=63) | 9% (n=9) |
| Replace entire restoration (n=248) | np | 61% (n=151) | 39% (n=97) |
| 2. Defective composite restoration with enamel marginsa | |||
| No treatment/preventive only (n=47) | 68% (n=32) | 32% (n=15) | np |
| Polish, re-surface, repair (n=211) | 18% (n=38) | 69% (n=145) | 13% (n=28) |
| Replace entire restoration (n=130) | np | 55% (n=72) | 45% (n=58) |
| 3. Defective amalgam restorationa | |||
| No treatment/preventive only (n=191) | 24% (n=45) | 76% (n=146) | np |
| Polish, re-surface, repair (n=55) | 15% (n=8) | 35% (n=19) | 51% (n=28) |
| Replace entire restoration (n=136) | np | 54% (n=74) | 46% (n=62) |
| 4. Occlusal caries scenario 1b | |||
| No treatment/preventive only (n=135) | 19% (n=25) | 81% (n=110) | np |
| Minimal intervention (n=203) | 11% (n=22) | 38% (n=78) | 51% (n=103) |
| Place complete restoration (n=39) | np | 41% (n=16) | 59% (n=23) |
| 5. Occlusal caries scenario 2b | |||
| No treatment/preventive only (n=31) | 16% (n=5) | 84% (n=26) | np |
| Minimal intervention (n=187) | 22% (n=42) | 55% (n=102) | 23% (n=43) |
| Place complete restoration (n=144) | np | 51% (n=74) | 49% (n=70) |
| 6. Proximal caries scenarioc | |||
| Outer ½ of enamel (n=7) | np | 57% (n=4) | 43% (n=3) |
| Inner ½ of enamel (n=144) | 3% (n=5) | 67% (n=97) | 29% (n=42) |
| Outer 1/3 of dentin (n=202) | 17% (n=35) | 75% (n=152) | 7% (n=15) |
| Middle 1/3 of dentin (n=29) | 45% (n=13) | 55% (n=16) | 0% (n=0) |
| Inner 1/3 of dentin (n=1) | < 1% (n=1) | 0% (n=0) | np |
This table shows results for each of the six clinical scenarios separately. This is necessary because the percentages of change vary for each scenario. The left column shows what the practitioner’s response for that scenario was at baseline. The three right-most columns show what that practitioner’s response for that same scenario was at follow-up, describing whether or not the follow-up response was more-invasive, less-invasive, or the same.
Response categories were: (1) no treatment or preventive treatment only; (2) polish, re-surface, or repair the restoration; and (3) replace the entire restoration. A “less-invasive” response meant moving to a category 1 response after having provided at baseline a response of category 2 or 3; or to a category 2 response after having provided at baseline a response of category 3. A “more-invasive” response meant moving to a category 3 response after having provided at baseline a response of category 1 or 2; or to a category 2 response after having provided at baseline a response of category 1.
Response categories were: (1) no treatment or preventive treatment only; (2) minimal intervention; and (3) complete restoration. A “less-invasive” response meant moving to a category 1 response after having provided at baseline a response of category 2 or 3; or to a category 2 response after having provided at baseline a response of category 3. A “more-invasive” response meant moving to a category 3 response after having provided at baseline a response of category 1 or 2; or to a category 2 response after having provided at baseline a response of category 1.
Response categories were to intervene surgically (i.e., restore the tooth) at this caries lesion depth: (1) outer half of enamel; (2) inner half of enamel; (3) outer one-third of dentin; (4) middle one-third of dentin; (5) inner one-third of dentin. A “less-invasive” response meant moving to a category with a lower number after having provided at baseline a category with a higher number. A “more-invasive” response meant moving to a category with a higher number after having provided at baseline a category with a lower number.
np: not possible because of the scale’s floor or ceiling effects
The associations between baseline response and the change score (more-invasive at follow-up; same; less-invasive) are statistically significant at p<0.001 for all treatment scenarios, Mantel-Haenszel χ2 trend test. Percentages in some rows do not add to 100% due to rounding.
Table 4.
Comparison of change between baseline and follow-up, by baseline level of invasiveness, after stratifying by DPBRN participation level
| Full DPBRN participants | Partial DPBRN participants | |||||
|---|---|---|---|---|---|---|
| Baseline response to treatment scenario | Response at follow-up was … | Response at follow-up was … | ||||
| … more-invasive | … the same | … less-invasive | … more-invasive | … the same | … less-invasive | |
| 1. Defective composite restoration with cementum-dentin marginsa | ||||||
| No treatment/preventive only | 50% (n=18) | 50% (n=18) | np | 67% (n=8) | 33% (n=4) | np |
| Polish, re-surface, repair | 25% (n=17) | 66% (n=45) | 9% (n=6) | 36% (n=12) | 55% (n=18) | 9% (n=3) |
| Replace entire restoration | np | 47% (n=55) | 53% (n=61) | np | 73% (n=96) | 27% (n=36) |
| 2. Defective composite restoration with enamel marginsa | ||||||
| No treatment/preventive only | 59% (n=19) | 41% (n=13) | np | 87% (n=13) | 13% (n=2) | np |
| Polish, re-surface, repair | 15% (n=18) | 70% (n=87) | 15% (n=19) | 23% (n=20) | 67% (n=58) | 10% (n=9) |
| Replace entire restoration | np | 44% (n=26) | 56% (n=33) | np | 65% (n=46) | 35% (n=25) |
| 3. Defective amalgam restorationa | ||||||
| No treatment/preventive only | 16% (n=16) | 84% (n=87) | np | 33% (n=29) | 67% (n=59) | np |
| Polish, re-surface, repair | 10% (n=4) | 44% (n=18) | 46% (n=19) | 29% (n=4) | 7% (n=1) | 64% (n=9) |
| Replace entire restoration | np | 53% (n=36) | 47% (n=32) | np | 56% (n=38) | 44% (n=30) |
| 4. Occlusal caries scenario 1b | ||||||
| No treatment/preventive only | 18% (n=17) | 82% (n=75) | np | 19% (n=8) | 81% (n=35) | np |
| Minimal intervention | 8% (n=8) | 37% (n=39) | 56% (n=59) | 14% (n=14) | 40% (n=39) | 45% (n=44) |
| Place complete restoration | np | 13% (n=2) | 88% (n=14) | np | 61% (n=14) | 39% (n=9) |
| 5. Occlusal caries scenario 2b | ||||||
| No treatment/preventive only | 17% (n=4) | 83% (n=20) | np | 14% (n=1) | 86% (n=6) | np |
| Minimal intervention | 20% (n=22) | 56% (n=63) | 24% (n=27) | 27% (n=20) | 52% (n=39) | 21% (n=16) |
| Place complete restoration | np | 41% (n=27) | 59% (n=39) | np | 60% (n=47) | 40% (n=31) |
| 6. Proximal caries scenarioc | ||||||
| Outer ½ of enamel | np | 67% (n=2) | 33% (n=1) | np | 50% (n=2) | 50% (n=2) |
| Inner ½ of enamel | 6% (n=4) | 57% (n=37) | 37% (n=24) | 1% (n=1) | 76% (n=60) | 23% (n=18) |
| Outer 1/3 of dentin | 10% (n=12) | 80% (n=94) | 10% (n=12) | 27% (n=23) | 69% (n=58) | 4% (n=3) |
| Middle 1/3 of dentin | 38% (n=10) | 62% (n=16) | 0% (n=0) | 100% (n=3) | 0% (n=0) | 0% (n=0) |
| Inner 1/3 of dentin | 0% (n=0) | 0% (n=0) | np | 100% (n=1) | 0% (n=0) | np |
As in Table 3, This table shows results for each of the six clinical scenarios separately. This is necessary because the percentages of change vary for each scenario. The left column shows what the practitioner’s response for that scenario was at baseline. The three right-most columns show what that practitioner’s response for that same scenario was at follow-up, describing whether or not the follow-up response was more-invasive, less-invasive, or the same. This table shows results in Table 3, but after stratifying into two strata: full participants and partial participants. Percentages in some rows do not add to 100% due to rounding.
Response categories were: (1) no treatment or preventive treatment only; (2) polish, re-surface, or repair the restoration; and (3) replace the entire restoration. A “less-invasive” response meant moving to a category 1 response after having provided at baseline a response of category 2 or 3; or to a category 2 response after having provided at baseline a response of category 3. A “more-invasive” response meant moving to a category 3 response after having provided at baseline a response of category 1 or 2; or to a category 2 response after having provided at baseline a response of category 1.
Response categories were: (1) no treatment or preventive treatment only; (2) minimal intervention; and (3) complete restoration. A “less-invasive” response meant moving to a category 1 response after having provided at baseline a response of category 2 or 3; or to a category 2 response after having provided at baseline a response of category 3. A “more-invasive” response meant moving to a category 3 response after having provided at baseline a response of category 1 or 2; or to a category 2 response after having provided at baseline a response of category 1.
Response categories were to intervene surgically (i.e., restore the tooth) at this caries lesion depth: (1) outer half of enamel; (2) inner half of enamel; (3) outer one-third of dentin; (4) middle one-third of dentin; (5) inner one-third of dentin. A “less-invasive” response meant moving to a category with a lower number after having provided at baseline a category with a higher number. A “more-invasive” response meant moving to a category with a higher number after having provided at baseline a category with a lower number.
np: not possible because of the scale’s floor or ceiling effects
A multivariable regression of change score was done for each of the six scenarios; baseline response and participation level were the only covariates in each regression. The baseline response variable was statistically significant in each regression (p<0.001). Participation level was statistically significant in scenario 1 (p < 0.001), scenario 2 (p = 0.001), scenario 4 (p = 0.014), scenario 5 (p = 0.041), and scenario 6 (p=0.001). It was not significant for scenario 3 (p = 0.053).
To test for the significance of the association between participation level and change score, a multivariable regression of change score was done for each of the six scenarios; baseline response and participation level were the only covariates in each regression (GLM Procedure in SAS) (30).
RESULTS
Table 1 shows the characteristics of the 405 practitioner-investigators who participated for both the baseline and follow-up questionnaires. The mean (S.D.) number of months between baseline and follow-up was 36.0 (3.8), with a range of 19.8 – 43.0 months. The mean did not differ between the full and partial participants. The 405 participants for the follow-up questionnaire differed in some characteristics from the 160 baseline participants who did not: (1) 58% of Florida/Georgia region practitioners and 93% of Scandinavian region practitioners participated, compared to 72%–76% in the other network regions; (2) 93% of full participants responded to the follow-up questionnaire, compared to 58% of the partial participants. No differences were statistically significant with regard to years since dental school graduation, race of the dentist, gender of the dentist, or rural/urban practice location.
Distribution of baseline responses to the six treatment scenarios
Table 2 shows the baseline responses to each of the six treatment scenarios. For treatment scenarios 1–3, replacing the entire restoration was considered the “invasive” treatment. Doing no treatment, preventive treatment only, or only polishing, re-surfacing, or repairing the restoration, were not classified as invasive. From 34% to 62% of practitioners selected the invasive treatment option for these scenarios. For treatment scenarios 4 and 5, placing a complete restoration was considered the “invasive” treatment. From 10% to 40% of practitioners selected the invasive treatment option. For scenario 6, placing a restoration when the lesion was still in the enamel (either outer-half or inner-half) was considered the “invasive” treatment, which was selected by 40% of practitioners. Because there were substantive percentages of “invasive” responses for all six scenarios, this meant that there was substantial potential in this sample to effect change toward a non-invasive treatment approach.
Comparison of baseline responses to responses provided at follow-up (“change scores”)
Overall change was toward a less-invasive direction for all scenarios, but was statistically significant (p<0.05) only for scenarios 1, 3, 4, and 5 (results not shown; Bowker’s test of symmetry p<0.05).
After classifying change scores for all scenarios into three overall categories (less-invasive at follow-up; same as baseline; more-invasive at follow-up), it became evident that change score was strongly associated with the level of invasiveness stated at baseline (Mantel-Haenszel χ2 trend test p<0.05). This also meant that comparing change between full participants and partial participants needed to account for any differences between the two groups with regard to baseline levels of invasiveness.
Table 3 shows the change that occurred between baseline and follow-up, by baseline level of invasiveness. This table shows results for each of the six clinical scenarios separately. This is necessary because the percentages of change vary for each scenario. The left column shows what the practitioner’s response for that scenario was at baseline. The three right-most columns show what that practitioner’s response for that same scenario was at follow-up, describing whether or not the follow-up response was more-invasive, less-invasive, or the same. A salient percentage of practitioners who were in the “invasive” category at baseline had moved to a less-invasive category by follow-up. For scenarios 1–5, this ranged from 39% in scenario 1 to 59% in scenario 4. In scenario 6, 30% of practitioners in the two invasive categories combined (outer half of enamel or inner half of enamel) had moved to a less-invasive category at follow-up.
Table 3 also shows that significant percentages of persons in the least-invasive category (i.e., “no treatment/preventive treatment only” for scenarios 1–5, and “middle 1/3 of dentin” or “inner 1/3 of dentin” for scenario 6) had moved toward a more-invasive approach. There also was significant movement among persons who at baseline responded in the middle category (“polish, re-surface, repair” or “minimal intervention”).
Bivariate associations stratified by participation level and multivariable regressions of change score
Table 4 stratifies by participation level the analyses shown in Table 3. As in Table 3, this table shows results for each of the six clinical scenarios separately. The left column shows what the practitioner’s response for that scenario was at baseline. The three right-most columns show what that practitioner’s response for that same scenario was at follow-up, describing whether or not the follow-up response was more-invasive, less-invasive, or the same. To test for the significance of the association between participation level and change at follow-up, a multivariable regression of change score was done for each of the six scenarios; baseline response and participation level were the only covariates in each regression. The baseline response variable was a statistically significant predictor of change score in each regression. Participation level was also a statistically significant predictor of change score in all regressions except for scenario 3, in which it was 0.053. These results are not shown in tabular form here, but these are publicly available (26).
Other factors were also tested as covariates in multivariable regressions of change score for each of the 6 treatment scenarios, with baseline response and participation level accounted for. None of these covariates were statistically significant in these regressions: race of the practitioner, rural-urban location of the practice, DPBRN region, gender of practitioner, or the number of DPBRN surveys in which the practitioner participated.
DISCUSSION
The results in Table 2 make it clear that at baseline there were significant percentages of practitioner-investigators (from 10% to 62%, depending on the clinical scenario) who were more invasive than what the scientific evidence suggests that they should be. We have previously discussed the rationale for this conclusion and its implications for this sample (27–29). This meant that there was significant room for improvement (movement away from an invasive approach) during a three-year follow-up period as a result of DPBRN participation.
The results in Table 3 make it clear that there was a substantive amount of change during this three-year follow-up period. The Bowker’s tests of symmetry demonstrate that this overall change was indeed toward a less-invasive direction. However, it also became evident that baseline level of invasiveness was strongly associated with change score, and that there was also a substantive amount of movement toward more-invasive treatment among those who were non-invasive at baseline. Therefore, it would be best to describe the overall pattern as movement toward a less-invasive approach, but with substantial movement “toward the middle” (i.e, “polish, re-surface, repair” or “minimal intervention”) or away from the extremes. The pattern in Table 4 and the multivariable regression results suggest that full participation is indeed associated with likelihood of moving toward a less-invasive approach.
Because partial participants were less likely to have completed the follow-up questionnaire compared to full participants, it is possible that non-response bias influenced the results. Full participants may be more likely to adopt evidence-based recommendations. It is possible that PBRN participation motivates only a certain segment of dentists to adopt an evidence-based approach to care, or that full participants are more motivated to change independent of their participation, owing to a self-selected openness to incorporating scientific evidence into their practice routines. Other mechanisms (e.g., financial, contractual, or regulatory incentives) should also be investigated as means to facilitate adoption of evidence-based treatment, especially for other segments of the dentist population. Because only a minority of dentists participate in PBRNs, it is possible that the main contribution of PBRNs to the scientific advancement of the profession will ultimately lie in their encouragement of clinicians to include research and quality improvement as a fundamental component of their practices, having instilled a mindset that this is simply what dentists do as a profession to improve the care of their patients.
These findings are consistent with implementation methods used with other health care professionals. A change in knowledge about the latest evidence is necessary for change, but not sufficient (32). This is why passive dissemination of materials, developing clinical guidelines, or simply attending didactic meetings generally have not been effective (8). Interventions that target other barriers to change have a larger impact. These include interventions that take into account the context of the individual practice, such as that practice’s patients and the constraints and incentives due to its particular financial and health care system circumstance (5,33–37). This is why the PBRN research context might be especially useful – because practitioner-investigators are considering data from their own practices and are listening to views about these data from other practitioner-investigators who share many of the same constraints, incentives, and motivations. That is, participating in PBRN activities may create an openness to change and fellow practitioner-investigators can themselves act as change agents. The overall pattern of change in this sample suggests that practitioner-investigators are open to change, and that full participation in a PBRN is associated with positive change in the direction suggested by the latest scientific evidence, but that other factors may play important roles as well. To our knowledge, a movement “toward the middle” by some practitioner-investigators has not been described in the literature to date and this is the main contribution made by this report. This movement “toward the middle” also means less overall treatment variation or greater consistency in treatment, which is also positive, independent of an overall movement toward a less-invasive approach.
Movement “toward the middle” may occur because practitioners become uneasy with treatment approaches that they are using once they become aware that these approaches are not used by a large majority of their peers. The “movement toward the middle” phenomenon would comprise change due to real change and any change due to random measurement error. Because regression toward the mean is not based on cause and effect, but rather on random error in a natural distribution around a mean, having a high test-retest reliability argues against predominance of the change being due to regression toward the mean. Because of the items’ good to excellent test-retest reliability, it seems unlikely that much of this change could have been due to measurement error. “Floor” and “ceiling” effects are also possible, in that a practitioner cannot become more invasive than doing a replacement restoration, or less invasive than doing no treatment or preventive treatment only. Additionally, this report does rely on practitioners’ reports of how they would treat in a given clinical circumstance, instead of documenting whether this is what they actually are doing in a real clinical setting, so this limitation should be kept in mind.
The network provides to practitioners who participate in a particular study the key results from their own practice, as well as overall results from the entire network. This allows them to objectively compare their practices to other practices in their own region and to practices network-wide, and not rely on their own perceptions. Practitioners have reported that they find this information very enlightening and that this leads them to question why their practice differs from others, which then can lead to additional research questions and openness to change (20). Also, in our earlier work (24), we observed that participation in one particular type of PBRN activity (a highly-interactive meeting) was strongly associated with stated intention to change clinical practice. We judged that an appropriate next step would be to query all network practitioner-investigators, not just the ones who attended the highly-interactive meeting. The current report took this next step, and did indeed find that participation level was associated with change in practice. We used a full participation vs. partial participation dichotomy for the analysis in this report. However, for all participants we also knew how many network studies in which they participated overall (i.e., studies of any type, whether these studies were clinical studies that required data collection from patients or simply questionnaire studies done by the dentist), the number of questionnaire studies in which they participated, the number of clinical studies in which they participated, and the number of annual meetings of practitioner-investigators they attended. These can be used as additional measures of exposure to or participation in network activities. We used these measures in multivariable regressions of change scores, similar to what we did for the full/partial participation measure in Table 4 (results not shown). The number of annual meetings attended was significantly associated with stated change for 5 of the 6 clinical scenarios in Table 4 (not associated with scenario 4), We also hypothesized that the number of clinical studies participated in would be associated with change in stated practice, but that the number of questionnaire studies would not, because practitioners who only participated in questionnaire studies were not as engaged in network activities and/or may not have been as influenced by them. The multivariable regressions were consistent with this conclusion: the number of clinical studies participated in was significantly associated with stated change for 4 of the 6 clinical scenarios in Table 4 (not associated with scenarios 3 or 4), and the number of questionnaire studies participated in was not associated with stated change in any of the six scenarios.
Study designs exist on a continuum of inferential power. At the highest end of the spectrum lies the one with the greatest rigor, the well-done double-masked randomized clinical trial. The design of the current study is toward the other end of that spectrum. This study did increase scientific inferential power by moving from informal anecdotal reports (25) to collection of data from many practitioner-investigators, by collecting these data in a standardized manner, and by providing a before-and-after type of design. Although this design does provide limited inferential power and conclusions from this work therefore should be viewed as preliminary, we can say that if full engagement in network activities is effective in moving evidence into regular clinical practice, then the results we observed are consistent with that, although not conclusive of it. Consistent with an overall strategy of investigation that continues toward higher levels of scientific inferential power, an appropriate next step would be a randomized trial to test the effectiveness of different strategies to foster movement of the latest evidence into routine clinical practice.
Acknowledgments
This investigation was supported by NIH grants U01-DE-16746, U01-DE-16747, and U19-DE-22516. Opinions and assertions contained herein are those of the authors and are not to be construed as necessarily representing the views of the respective organizations or the National Institutes of Health.
References
- 1.O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis D, Haynes RB, Harvey E. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;(4):Art. No.: CD000409. doi: 10.1002/14651858.CD000409.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Forsetlund L, Bjørndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009 Apr 15;(2):CD003030. doi: 10.1002/14651858.CD003030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Chaix-Couturier C, Durand-Zaleksi I, Jolly D, Durieux P. Effects of financial incentives on medical practice: results from a systematic review of the literature and methodologic issues. Int J Qual Health Care. 2000;12(2):133–142. doi: 10.1093/intqhc/12.2.133. [DOI] [PubMed] [Google Scholar]
- 4.Clarkson JE, Turner S, Grimshaw JM, Ramsay CR, Johnston M, Scott A, Bonetti D, Tilley CJ, Maclennan G, Ibbetson R, MacPherson LMD, Pitts NB. Changing clinicians’ behavior: a randomized controlled trial of fees and education. J Dent Res. 2008;87(7):640–644. doi: 10.1177/154405910808700701. [DOI] [PubMed] [Google Scholar]
- 5.McGlone P, Watt R, Sheiham A. Evidence-based dentistry: an overview of the challenges in changing clinical practice. Brit Dent J. 2001;190(12):636–639. doi: 10.1038/sj.bdj.4801062. [DOI] [PubMed] [Google Scholar]
- 6.Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, Flottorp S, Robertson N. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Reviews. 2010 Mar 17;(3):CD005470. doi: 10.1002/14651858.CD005470.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Doumit G, Gattellari M, Grimshaw J, O’Brien MA. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007 Jan 24;(1):CD000125. doi: 10.1002/14651858.CD000125.pub3. [DOI] [PubMed] [Google Scholar]
- 8.Farmer AP, Légaré F, Turcot L, Grimshaw J, Harvey E, McGowan JL, Wolf F. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008;(3):Art. No.: CD004398. doi: 10.1002/14651858.CD004398.pub2. [DOI] [PubMed] [Google Scholar]
- 9.Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;(2):Art. No.: CD000259. doi: 10.1002/14651858.CD000259.pub2. [DOI] [PubMed] [Google Scholar]
- 10.Agency for Healthcare Research and Quality. U.S. Department of Health and Human Services. [Accessed August 14, 2012];National Healthcare Quality Report. 2010 AHRQ Publication No. 11-0004. Available at: “ www.ahrq.gov/qual/measurix.htm#quality”.
- 11.Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, McGlynn EA. Who is at greatest risk for receiving poor-quality health care? New Engl J Med. 2006;354(11):1147–1156. doi: 10.1056/NEJMsa044464. [DOI] [PubMed] [Google Scholar]
- 12.Schuster MA, McGlynn EA, Brook RH. How good is the quality of health care in the United States? Milbank Quart. 2005;83(4):843–895. doi: 10.1111/j.1468-0009.2005.00403.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Genel M, Dobs A. Translating clinical research into practice: practice-based research networks, a promising solution. J Invest Med. 2003;51(2):64–71. doi: 10.1136/jim-51-02-07. [DOI] [PubMed] [Google Scholar]
- 14.Mold JW, Peterson KA. Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005;3(supplement 1):S12–S20. doi: 10.1370/afm.303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lindbloom EJ, Ewigman BG, Hickner JM. Practice-based research networks: the laboratories of primary care research. Med Care. 2004;42(4 supplement):III45–III49. [PubMed] [Google Scholar]
- 16.Westfall JM, Fagnan LJ, Handley M, Salsberg J, McGinnis P, Zittleman LK, Macaulay AC. Practice-based research is community engagement. J Am Board Fam Med. 2009;22(4):423–7. doi: 10.3122/jabfm.2009.04.090105. [DOI] [PubMed] [Google Scholar]
- 17.The National Dental Practice-Based Research Network. [Accessed August 14, 2012]; Available at: “ http://nationaldentalpbrn.org”.
- 18.Gilbert GH, Williams OD, Rindal DB, Pihlstrom DJ, Benjamin PL, Wallace MA for The DPBRN Collaborative Group. The creation and development of The Dental Practice-Based Research Network. J Am Dent Assoc. 2008;139(1):74–81. doi: 10.14219/jada.archive.2008.0024. [DOI] [PubMed] [Google Scholar]
- 19.Gilbert GH, Qvist V, Moore SD, Rindal DB, Fellows JL, Gordan VV, Williams OD for The DPBRN Collaborative Group. Institutional Review Board and regulatory solutions in The Dental PBRN. J Public Health Dent. 2010;70(1):19–27. doi: 10.1111/j.1752-7325.2009.00139.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Gilbert GH, Richman JS, Gordan VV, Rindal DB, Fellows JL, Benjamin PL, Wallace-Dawson M, Williams OD for The DPBRN Collaborative Group. Lessons learned during the conduct of clinical studies in The Dental PBRN. J Dent Educ. 2011;75(4):453–465. [PMC free article] [PubMed] [Google Scholar]
- 21.Benjamin PL for the DPBRN Collaborative Group. Promoting evidence-based dentistry through The Dental Practice-Based Research Network. J Evidence-based Dent Pract. 2009;9(4):194–196. doi: 10.1016/j.jebdp.2009.06.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Makhija SK, Gilbert GH, Rindal DB, Benjamin PL, Richman JS, Pihlstrom DJ, Qvist V for The DPBRN Collaborative Group. Dentists in practice-based research networks have much in common with dentists at large: evidence from The Dental PBRN. Gen Dent. 2009;57(3):270–275. [PMC free article] [PubMed] [Google Scholar]
- 23.Makhija SK, Gilbert GH, Rindal DB, Benjamin PL, Richman JS, Pihlstrom DJ, Qvist V for the DPBRN Collaborative Group. Practices participating in a dental PBRN have substantial and advantageous diversity even though as a group they have much in common with dentists at large. BMC Oral Health. 2009;9(1):26–35. doi: 10.1186/1472-6831-9-26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Gilbert GH, Richman JS, Qvist V, Pihlstrom DJ, Foy PJ, Gordan VV for The DPBRN Collaborative Group. Change in stated clinical practice associated with participation in The Dental Practice-Based Research Network. Gen Dent. 2010;58(6):520–528. [PMC free article] [PubMed] [Google Scholar]
- 25.National Dental Practice-Based Research Network. [Accessed August 14, 2012];Testimonials. Available at “ http://nationaldentalpbrn.org/testimonials.php”.
- 26.National Dental Practice-Based Research Network. [Accessed August 14, 2012];Study results. Available at: “ http://nationaldentalpbrn.org/study-results.php”.
- 27.Gordan VV, Bader JD, Garvan CW, Richman JS, Qvist V, Fellows JL, Rindal DB, Gilbert GH for The DPBRN Collaborative Group. Restorative treatment thresholds for occlusal primary caries by dentists in The Dental PBRN. J Am Dent Assoc. 2010;141(2):171–184. doi: 10.14219/jada.archive.2010.0136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gordan VV, Garvan CW, Richman JS, Fellows JL, Rindal DB, Qvist V, Heft MW, Williams OD, Gilbert GH for The DPBRN Collaborative Group. How dentists diagnose and treat defective restorations: evidence from The Dental PBRN. Oper Dent. 2009;34(6):664–673. doi: 10.2341/08-131-C. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Gordan VV, Garvan CW, Heft MW, Fellows J, Qvist V, Rindal DB, Gilbert GH for The DPBRN Collaborative Group. Restorative treatment thresholds for interproximal primary caries based on radiographic images: findings from The Dental Practice-Based Research Network. Gen Dent. 2009;57(6):654–663. [PMC free article] [PubMed] [Google Scholar]
- 30.SAS Institute, Inc. SAS/STAT version 9.2. SAS Publishing; Cary, NC: 2009. [Accessed August 14, 2012]. Available at: “ www.sas.com/apps/pubscat/complete.jsp”. [Google Scholar]
- 31.Bishop YMM, Fienberg SE, Holland PW. Discrete Multivariate Analysis. Cambridge: MIT Press; 1975. p. 283. [Google Scholar]
- 32.Rohrbach LA, Grana R, Sussman S, Valente TW. Type II translation: transporting prevention interventions from research to real-world settings. Eval Health Prof. 2006;29(3):302–333. doi: 10.1177/0163278706290408. [DOI] [PubMed] [Google Scholar]
- 33.Crabtree BF, Miller WL, Tallia AF, Cohen DJ, DiCicco-Bloom B, McIlvain HE, Aita VA, Scott JG, Gregory PB, Stange KC, McDaniel RR., Jr Delivery of clinical preventive services in family medicine offices. Ann Fam Med. 2005;3(5):430–435. doi: 10.1370/afm.345. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Painter JE, Borba CPC, Hynes M, Mays D, Glanz K. The use of theory in health behavior research from 2000 to 2005: a systematic review. Ann Behav Med. 2008;35(3):358–362. doi: 10.1007/s12160-008-9042-y. [DOI] [PubMed] [Google Scholar]
- 35.Prochaska JO. Decision making in the transtheoretical model of behavior change. Med Decis Making. 2008;28(6):845–849. doi: 10.1177/0272989X08327068. [DOI] [PubMed] [Google Scholar]
- 36.Davies P, Walker AE, Grimshaw JM. A systematic review of the use of theory in the design of guideline dissemination and implementation strategies and interpretation of the results of rigorous evaluations. Impl Sci. 2010;9(5):14–19. doi: 10.1186/1748-5908-5-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Kahn K, Ryan G, Beckett M, Taylor S, Berrebi C, Cho M, Quiter E, Fremont A, Pincus H. Bridging the gap between basic science and clinical practice: a role for community clinicians. Impl Sci. 2011;6(1):34–44. doi: 10.1186/1748-5908-6-34. [DOI] [PMC free article] [PubMed] [Google Scholar]