Skip to main content
PMC home page
Bulletin of the Medical Library Association logoLink to Bulletin of the Medical Library Association
. 2001 Apr;89(2):194–203.

Changes in learning-resource use across physicians' learning episodes*

HB Slotnick 1,†, T Robert Harris 1, David R Antonenko 1
PMCID: PMC31727  PMID: 11337951

Abstract

Introduction: This study explores the numbers of learning resources physicians use at each stage in self-directed learning episodes addressing general problems.

Materials and Methods: A survey of a statewide random sample of doctors estimated the number of resources used at each stage in solving various general problems.

Results: The 50% response rate for faculty allowed generalization of findings to the population of these physicians; the rate for nonfaculty physicians was too low to allow generalization. Faculty findings showed (1) broader resource use in learning about diseases than diagnosis or therapeutics, (2) comparable numbers of resources used in deciding whether to take on the learning problem and learning the required skills and knowledge, (3) greater numbers of resources selected to evaluate the problem and to learn the required skills and knowledge than to gain experience with the newly learned skills and knowledge, and (4) support for assertions that doctors value learning resources that are accessible, applicable, familiar, and time effective.

Discussion: The findings were interpreted in light of theory describing physicians' self-directed learning episodes, and implications are presented for physicians-in-training, physicians, and medical librarians.

INTRODUCTION

Background

A rich literature exists describing the learning resources that doctors access (e.g., meta-analyses by Haug [1] and Verhoeven, Boerma, and Meyboom-de Jong [2]), and there is a growing literature on the stages practicing physicians work through in self-directed learning (e.g., see Fox and Bennett's summary [3] and the literature review in Slotnick's empirical study [4]). However, there is no mention of stages of learning in the first set of studies, while there is only one mention of numbers of learning resources used across stages in the second set. Thus, against a thin empirical background, the researchers sought an answer to the research question “how does learning-resource use change over the stages of physician learning?” Objectives for this study included exploring resource use (1) from one stage of physician learning to the next, (2) across different problems doctors encounter, and (3) for academic versus nonacademic physicians.

Physician learning

Physicians earn their living by solving problems that patients bring them, and they get better at this through both experience [5, 6] and the insights that come from reflecting on that experience. Indeed, we take learning to be precisely the process of deriving insights through reflection on experience, insights that are useful in solving problems in the future [7]. Though learning that occurs in the course of clinical practice has been described for curbside consultation [8, 9] and interactions with pharmaceutical representatives [10], we believe a more useful analysis for understanding how doctors learn arises from consideration of the stages of self-directed learning mentioned in the preceding paragraph. Such an analysis would also consider how learning varies with both the type of problem precipitating the learning episodes (i.e., specific problems, which typically concern individual patients and are resolved quickly, as opposed to general problems, which typically concern bodies of skills and knowledge and progress more slowly [11]), as well as the stage doctors are at in learning to solve the problem at hand [12–14]. Stages in individual learning episodes [15] are presented in Table 1.

Table 1 Stages in physicians' learning episodes

graphic file with name i0025-7338-089-02-0194-t01.jpg

Open in a new tab

The only literature bearing on the question of how many learning resources doctors use at each stage comes from Fox and colleagues' observation that the number decreases with advancing stages in self-directed learning [16]. This conclusion is limited, because the study considered a single specialty only, looked at adoption of innovation versus day-to-day learning, and came from a survey with a 32% response rate and no non-respondent study [17].

Physicians' use of learning resources

Physicians encounter the need for five kinds of information [18, 19], and they satisfy these needs by using particular learning resources in routine ways. They further value the learning resources according to criteria that are largely practical in nature.

Among the kinds of information, two are local varieties—logistic concerning issues such as how to order lab tests and epidemiologic concerning issues such as community prevalences for specific diseases. Two others bear on patients—patient information describing the patient specifically and patient-focused information such as what the doctor knows and can bring to bear to the patient at hand. Research on sources used to locate patient-focused information indicates that characteristics predictive of use by doctors include resource accessability [20], clinical applicability of the information provided [21], doctors' familiarity with the resource [22, 23], and the return of much useful information per unit time spent [24–26]. The attribute shared by these criteria is practicality, which we consider to be a defining characteristic of clinical medicine.

The last kind of information is general medical knowledge and life-long medical learning [27], and our reading of its description suggests it is used in addressing general problems. General medical knowledge and lifelong learning information commonly appear in the medical research literature (described as frequently unrelated to doctors' clinical practice questions [28, 29]) and in clinically useful materials such as review articles, consensus statements, practice guidelines, and textbooks. Criteria for evaluating clinically useful materials include the quality of the underlying science, the quality of the explanations of clinical principles and applications, and the fit of articles' recommendations to the clinicians' needs; the latter two criteria again reflect clinical medicine's practicality.

Discussing the kinds of information sought and the criteria for evaluating the resources providing it begs the question of which information sources physicians use, and so we now turn to that issue. The meta-analyses cited earlier summarize physicians' use of sources of skills and knowledge, one study dealing with doctors generally [30] and the other with family physicians specifically [31]. Both show that consultations and print materials are the primary information sources, though findings vary depending on how “print” is defined. This definition is important because textbooks are used differently than journals (probably reflecting the nature of the question precipitating the doctors' search for information [32]). Doctors use texts for basic topics that have not been visited in a while or when seeking a historical perspective and journal articles when more up-to-date information is needed [33].

Colleagues in the clinician-learner's specialty and other specialities are also sources of general medical knowledge and lifelong learning. These people help the clinician “evaluate and validate the medical developments” reported in research articles and reviews [34], a reality necessitated by the doctors' often having difficulty seeing how research findings apply to their clinical questions [35, 36]. Beyond helping apply research findings, colleagues also offer reassurance—an important outcome when doctors are trying things they have never done before [37, 38].

Continuing medical education (CME) is also a source of general medical information and skill. Doctors participate in CME for three reasons: to accrue CME credit, to learn solutions to specific and general problems, and to interact with other attendees [39]. At the same time, they decide to attend CME sessions based on five considerations: the availability of CME credit, the relevance of the topic to the doctor's existing needs, the doctors' scheduling conflicts, the speakers' reputations, and the costs of attendance [40].

The meta-analyses also offer observations on how doctors used learning resources. Concerning individual resources, physicians use available materials, so personal libraries are used more often than medical libraries [41]. Further, and as noted, consultants often link research findings to particular doctors' clinical needs [42]. Concerning resources more generally, Means (cited in Verhoven and colleagues [43]) reports that resource use is related to doctors' current stage in the information seeking process. Cost factors, such as time and energy, appear to be more important in selecting sources than quality issues, such as completeness and reliability of the information source [44]. Aspects of low costs, mentioned earlier as attributes of useful resources, include easy accessibility to the information (e.g., locating and searching the source), comprehensibility of the found information, and relevance of the information to the clinical issue at hand. Thus physicians are familiar with information sources they use regularly, and these are the sources that are easily accessible.

Other studies report that doctors use information sources in routine ways during learning that occurs in clinical practice (e.g., in curbside consultations [45, 46] or interactions with pharmaceutical representatives [47]), and they have procedures for valuing different information sources [48]. In reviewing reports of original research, for example, doctors entertain decreasing confidence in what they read moving from randomized controlled trials to prospective studies to retrospective studies to case studies [49]. These routine approaches can be related to physicians' responsibilities, because researchers and teachers make heavier use of learning resources than others [50, 51].

Finally, physicians decide whether and how to use learning resources based on prior experiences with those resources [52, 53]. This is important, because prior experiences lead to expectations that may foreclose their decisions to seek answers to patient care questions [54–56].

Research hypothesis

Doctors turn primarily to learning sources well known to them (journals in their offices and consultants they know) in solving specific problems, and they use those same resources as well as professional meetings in addressing general problems [57–59]. Given this background and Fox and colleagues' finding that the number of resources declined from stage to stage across learning episodes, we addressed the research question “How does learning resource use change over the stages of physician learning?” by testing the research hypothesis that “The number of resources declines from evaluating the problem to learning the skills and knowledge to gaining experience.”

We chose as our dependent variable the total number of resources used (rather than breaking the resources down into categories) for three reasons. First, it was not clear that the number of resources used did, in fact, change, and this issue needed to be addressed before categories of resources could be meaningfully examined. Second, it was not clear which categories of measures should be used because resources could be cataloged in a variety of ways (e.g., format, as in print, human, electronic; and frequency of use, as in commonly used, less commonly used, rarely used), and there was no evidence favoring one categorization scheme over the others. Third, we believed such an investigation was appropriate as a follow-up to studies such as this one, because we anticipated our findings might offer guidance in casting hypotheses for further investigations.

MATERIALS AND METHODS

Sampling

The population of 1,210 physicians practicing in North Dakota was stratified according to whether doctors held faculty appointments at the state's medical school. Seven hundred fifty-five held full-time and clinical appointees, and 455 doctors had no such appointments. Respondents were sampled randomly from within these strata [60].

Questionnaire development

We limited ourselves to general learning problems in the mailed survey, because specific problems resolved quickly and so were not well addressed by the kinds of data collection activities we used. We sent a questionnaire about solving general problems with a letter signed by the medical school dean and the chair of a committee studying the school's CME activities. Two weeks later, a second letter and questionnaire went to doctors who had not yet responded.

The questionnaire presented twelve scenarios representing four general problems at three learning stages (evaluating the problem, learning the skills and knowledge, and gaining experience). We did not consider the scanning stage, because it is completed quickly and on an affective basis [61], whereas the other stages are more deliberative, take longer to complete, and lend themselves better to the kind of data collection undertaken here. The problems concerned updates in the following areas:

  • Disease: The update covered a regularly encountered disease.

  • Diagnostic approach: The update covered an approach to diagnosing a commonly seen disease.

  • Therapeutic approach: The update covered a common therapeutic approach.

  • Therapeutic technique: The update covered a specific therapeutic technique.

Respondents were asked to identify learning resources (Table 2) Dr. A, a hypothetical physician in the respondent's specialty, might use to learn about each problem at each stage. This projective approach avoided respondents' indicating what they thought we wanted (because the decisions were Dr. A's and not theirs) and their being concerned about how we might view their choices (again, because the decisions were Dr. A's). When physicians were asked to indicate what a hypothetical doctor would do, they drew on their experiences and, we believed, they were less concerned about how researchers would view the hypothetical doctor. This point will be considered again under “Limitations of the Study.”

Table 2 Available learning resources

graphic file with name i0025-7338-089-02-0194-t02.jpg

Open in a new tab

Demographic questions collected data on respondents including undergraduate medical education, post-graduate training, board certification, years of experience, gender, marital status, and presence of children at home, which bears on attendance at away-from-home CME activities [62]. The draft instrument was prepared by the lead author of the research team and reviewed for clarity of meaning to physicians by the physician on the research team. Though the review specifically sought instances where wording was either unclear or ambiguous, no such instances were found. This fact, and the utter simplicity of the questionnaire (Appendix), militated against a full field test. We also invited participants to call us if there were problems or questions, so the telephone number of the senior author was provided. No such calls were received.

Data analysis

Demographics

Parameter estimates were to be computed within both faculty and nonfaculty strata and statewide using equations appropriate to stratified random sampling. We anticipated comparing faculty and nonfaculty demographics using t tests for continuous data, odds ratios, and differences between uncorrelated proportions as appropriate for dichotomous data.

Use of each learning resource

We expected to tabulate the proportion and standard error for each resource within each stratum and for the statewide population. The resources were then rank ordered and clusters of the most commonly selected resources identified through visual inspection of a scree plot [63].

Problem-by-stage scenarios

The dependent measure in each problem-by-stage scenario was the number of learning resources selected. We computed means and standard deviations for the dependent measures and created a frequency polygon displaying the mean number of resources selected for each problem-by-stage combination.

A fixed-effects, repeated-measures analysis of variance (ANOVA) was used to analyze the data. The statistical hypotheses to be tested concerned differences between faculty and nonfaculty physicians, differences among types of problems (a priori contrasts were disease with the combination of diagnosis and treatment problems, diagnosis with treatment, and treatment approach with treatment technique), stage of learning (a priori contrasts were based on the decline in the number of resources used moving from one stage to the next), and the various interactions among the main effects just described. We used Helmert contrasts to test these a priori comparisons [64, 65].

RESULTS

Response rate

The overall response rate was 50% for faculty and 27.7% for nonfaculty (Table 3). The nonfaculty response rate was too low to allow us to consider the sample representative of the stratum from which its members were drawn, and so no analyses for nonfaculty physicians were reported. Further, nine of seventy-eight faculty members did not provide complete data, so their cases were deleted for the ANOVA. They were, however, used in estimating the prevalence of use of each resource.

Table 3 Stratum sizes, numbers sampled, and numbers responding

graphic file with name i0025-7338-089-02-0194-t03.jpg

Open in a new tab

Demographic summary

Almost eight of every nine faculty physicians (π̂ = 0.8718, CI95 = [0.7965, 0.9479]) received their medical degree in the United States, and a comparable number were board certified (π̂ = 0.8846, CI95 = [0.7965, 0.9479]). The typical faculty member had just less than four-and-a-half years of post-medical degree training (μ̂ = 4.4324, CI95 = [4.0145, 4.8503]) and almost fifteen years' practice experience (μ̂ = 14.6670, CI95 = [12.4301, 16.9093]).

Physicians' selection of learning resources

Inspection of the scree plot identified three clusters capturing the most commonly selected learning resources (the estimated proportion of physicians selecting each of the remaining 33 learning resources is less than 40%). The most frequently identified cluster had one resource in it: same specialty physicians (π̂ = 0.7041, CI95 = [0.6012, 0.8070]). The second cluster contained two resources, both dealing with printed materials: journals the doctor subscribes to (π̂ = 0.5513, CI95 = [0.4392, 0.6634]) and review articles (π̂ = 0.5460, CI95 = [0.4338, 0.6582]). The last cluster also had two resources, and both concerned meetings: specialty society CME meetings (π̂ = 0.4896, CI95 = [0.3770, 0.6022]) and conversations with other doctors attending meetings (π̂ = 0.4544, CI95 = [0.3422, 0.5666]).

The mean number of resources selected by faculty members was 10.0350 (CI95 = [8.2143, 11.8557]). The distribution had a marked positive skew as reflected in the estimated population standard deviation of 8.0803.

Hypothesis testing

The analysis of variance (Tables 4 and 5) indicates a significant problem-by-stage interaction (Figure 1), and this necessitates comparison of problems at specific stages using complex Helmert contrasts as described in the methods section. Before examining these contrasts, however, it is helpful to visually assess the problem-by-stage interaction displayed in Figure 1. At stages 1 and 2 (evaluating the problem and learning the required skills and knowledge, respectively), the number of resources selected for disease is much greater than for the other problem types; while at stage 3 (gaining experience), the four problem types do not differ very much. In contrast, the curves for the three non-disease problem types seem roughly parallel, suggesting that these problem types may reasonably be combined when examining possible stage differences, and stages may be combined when testing for differences among these three problem types.

Table 4 Instructional resources selected for each problem-by-stage combination (standard deviations appear in parentheses)

graphic file with name i0025-7338-089-02-0194-t04.jpg

Open in a new tab

Figure 1.

Figure 1

Open in a new tab

Problem-by-stage means

Examination of the complex Helmert contrasts yield the following findings that are consistent with the visual inspection just described.‡

  1. The most likely “cause” of the interaction is the decline in number of resources used for disease at the first two learning stages versus the third stage, compared to the corresponding decline for all other problem types (F = 63.43, df = 1,68, P < 0.0001).

  2. The next most likely cause is the decline in number of resources used for therapeutic approach at the first two stages versus the third stage, compared to the corresponding decline for therapeutic technique (F = 11.13, df = 1,68, P < 0.0014).

All other contrasts were non-significant at α = 0.05.

DISCUSSION

Response rate

The 50% response rate for faculty meant that the sample of faculty participating in this study was likely representative of the population of faculty in North Dakota.

Demographics

We noted that faculty were most often board certified, and the numbers of years of post-graduate education were consistent with the fact that most faculty members were board certified.

Learning resources selected

The first two clusters of commonly used learning resources (same specialty physicians and both journals the doctor subscribes to and review articles) include the resources identified as most important in the meta-analyses performed by librarians [66, 67]. These resources are likely attractive to our physicians because, as noted earlier, they are accessible, are clinically relevant, are familiar to physicians, and offer a good information and skills return for the efforts invested in using them.

Though resources in the last cluster (specialty society CME activities, conversations with doctor at meetings) cannot be accessed without some kind of prior planning, we believe there are two reasons doctors nevertheless selected them frequently. First, the studied physicians come from a rural state, so they are interested in away-from-home CME activities, because fewer CME credit-bearing activities are available locally and because CME activities address doctors' problems [68]. Second, the survey has no questions about specific problems, problems where existing evidence suggests that CME activities are less likely to be used [69, 70].

Hypotheses bearing on numbers of resources accessed

The significant problem-by-stage interaction (Figure 1 and Table 4) and the post hoc analyses were the basis for five generalizations we cast summarizing the number of learning resources used as a function of stage. First, doctors indeed selected more learning resources for evaluating the problem and learning the needed skills and knowledge than for gaining experience regardless of problem type; and, second, doctors selected about the same number of learning resources while gaining experience (stage 3) regardless of problem type. An explanation for these findings came from the fact that while evaluating a problem required different information and different reflective activities than learning the skills and knowledge [71], both stages might well require the same learning resources and therefore the same number of resources. Simply stated, we suspected that doctors used the same resources, but they looked for different information and insights as a function of their learning episode stage. It was also possible that gaining experience (the last stage) required more experiential learning and so less use of learning resources than the other two stages.

The third generalization contrasted with the findings of the Canadian Radiologist Study, which indicated a drop in the number of learning resources moving from the first to the last learning stage [72]. While we documented the decline moving from the first and second stages to the last stage, we found no significant difference between stage 1 and stage 2 in mean number of resources used; indeed, the stage 2 mean was often larger than the stage 1 mean. Explanations would include design differences between the two studies, such as the Canadian Study's looking at innovation adoption in a single specialty nationally, while our study looked at day-to-day learning about four problem types in a statewide study.

The fourth generalization was that the more theoretical a problem type was (i.e., the less practical or clinical in nature), the more resources must be accessed to address it. Updating on disease, for example, required more learning resources than the other problem types for evaluating the problem and learning skills and knowledge (stages 1 and 2), because information on pathophysiology likely came from research reports while approaches to diagnosis and therapeutics were more likely available through consultants and journals doctors received. Further, there was no difference between the mean learning resources needed to update on diagnostic approach versus both therapeutic problem types because both were clinical issues and so likely required similar learning resources.

The fifth and final generalization was that fewer resources were needed when gaining experience was the primary learning activity. The difference between therapeutic approach and therapeutic technique, for example, depended on whether one was considering evaluating the problem (stage 1) and learning the skills and knowledge (stage 2). More specifically, while the mean number of learning resources for therapeutic approach and therapeutic technique were comparable at the evaluating the problem stage and gaining experience (stage 3), more resources were selected for approach than for technique at the learning the required skills and knowledge stage. Smaller numbers of resources were needed to learn skills and knowledge for techniques, because this learning was based more on experience than reading or talking with others.

Limitations of the study

The threat to the study's generalizability posed by the response rate speaks to the need for the study's replication, particularly in a variety of other settings. However, a more useful replication would be a mixed qualitative and quantitative effort looking at learning resources divided into categories, either on the basis of format or availability, with physicians interviewed about how they viewed and used each resource group. We expect such studies will explore learning resources as a function of problem type and stage. Such a replication would also look at actual learning activities rather than hypothetical ones, an approach that would reduce the bias due to both respondents' concerns for how they may be viewed and their forgetfulness. Care would need to be taken, of course, to ensure that such a study would not influence physicians' choices of learning resources.

Educational implications

Implications arising from this study for practicing clinicians derive from the fact that doctors are not reflective about how they learn [73]. Thus, it may be useful to identify learning problems physicians report (e.g., identifying time-efficient information sources) and to help them consider how they may use both learning resources they do not use (e.g., online journals§) and resources they do use but in different ways (e.g., ask consultants to help them identify CME sessions they may find useful) in developing more productive approaches to learning. Potential topics to consider include evaluating review articles, making the best use of consultation, and knowing when enough has been learned to stop seeking new skills and knowledge and begin resolving the problem at hand.

In contrast, activities for residents may include first having them solve a problem requiring learning resources and then reflect on what they have done from the perspective of selecting learning resources, integrating the information developed, and reflecting on what they have learned about the use of learning resources. Medical students can be helped in the same way by inviting them to reflect upon the resources they use and the results they observe. Such activities may begin with review articles and then work back to the original research on which the articles are based, so they can see the relationship of original research findings to what is prepared for clinical practice. Issues to be considered may include how clinical trials are “translated” into practice, the use of medical heuristics (i.e., general principles doctors use in making diagnoses and planning therapy [74]) and ways to work with and learn from consultants.

Implications for medical librarians

This study carries multiple implications for medical librarians and librarianship. First, knowing the stage physicians are at in given learning episodes carries suggestions for the numbers of resources that may be most useful to them. Second, knowing whether the doctors focus on a disease versus the disease's diagnosis or treatment implies an increase in resources needed in the former case and a decrease in the latter two. Third, knowledge of the fact that doctors are most likely to use information sources immediately available to them allows librarians to better understand the importance of recommending resources doctors need to have in their offices. Fourth, knowledge of learning episodes implies a better understanding of the clientele medical librarians serve and so an increased ability to serve them well.

CONCLUSIONS

We drew the following conclusions from the materials, methods, and data described here:

  • Physicians select more learning resources in evaluating a problem and learning the skills and knowledge than they do in gaining experience with the new skills and knowledge. This is likely due to the experiential nature of gaining experience versus the skill and knowledge required for evaluating a problem and learning the skills and knowledge needed.

  • Physicians select about the same number of learning resources for the evaluating problems and learning skills and knowledge stages. This is likely because doctors select the same learning sources despite the differences in what is needed at each stage. The reflection on what is learned is different as well.

  • General problems involving disease require more learning resources than do similar problems involving diagnosis and treatment, possibly because information about disease is developed at research centers, while information and skill diagnosis and treatment is likely to be more readily available locally.

  • Learning a therapeutic technique appears to require fewer learning resources than learning a therapeutic approach only at the stage where the doctor learned the required skills and knowledge.

  • Our findings appear consistent with the observations in the literature that physicians value resources that are accessible (i.e., readily locatable, scanable, and understandable), applicable, and are already well known to them and offer many useful skills and much knowledge for the invested resources.

Table 5 ANOVA summary table

graphic file with name i0025-7338-089-02-0194-t05.jpg

Open in a new tab

Acknowledgments

We thank Anne Taylor-Vaisey for her gracious help with the literature search, Barbara Knight and her colleagues for help in locating references, and Larry Gruppen for insightful comments on an early draft of this manuscript.

APPENDIX

The questionnaire used in this study

This appendix presents the instrument used to collect data bearing on the research question addressed in this paper.

The questionnaire used in the study was designed to elicit responses from physicians to hypothetical situations in which a colleague of theirs might require more skill and knowledge than they currently had. Respondents were asked to consider twelve situations and to select, for each of the situations, from the same list of thirty-nine resources.

The twelve questions were:

  1. It has been a while since Dr. A updated on a particular disease encountered regularly. Which sources would Dr. A use to decide whether an update is appropriate at this time?

  2. Dr. A has decided that updating on the disease is appropriate at this time. Which sources would you expect Dr. A to use in doing the update?

  3. Dr. A has completed the update on the disease and is interested in knowing about other clinicians' views of an experiences with the things learned during the update. Where do you expect Dr. A will turn to find out about others' views and experiences?

  4. It has been a while since Dr. A updated knowledge and skills bearing on a therapeutic approach. To which sources would Dr. A turn to figure out whether an update is appropriate at this time?

  5. Dr. A has decided that updating knowledge and skill on the therapeutic approach is appropriate at this time. Which sources would you expect Dr. A to use in doing the update?

  6. Dr. A has completed the update on the therapeutic approach and is interested in knowing about other clinicians' views of and experiences with what was learned during the update. Where do you expect Dr. A will turn to find out about others' views and experiences?

  7. Dr. A has heard about a diagnostic approach that may be of value in Dr. A's specialty. To which of the following resources would Dr. A turn to decide whether it is appropriate to learn about this approach at this time?

  8. Dr. A has decided that it is appropriate to learn about the new diagnostic approach. To which of the following resources would you expect Dr. A to turn in learning about the approach?

  9. Dr. A has finished learning about the new diagnostic approach and is interested in knowing about other clinicians' views of and experiences with the approach. Where do you expect Dr. A will turn to find out about others' views and experiences?

  10. Dr. A has heard about a new therapeutic technique that may be of value in Dr. A's specialty. To which of the following resources would Dr. A turn to figure out whether it is appropriate to learn about this technique at this time?

  11. Dr. A has decided that it is appropriate to learn about the new therapeutic technique at this time. To which of the following resources would you expect Dr. A to turn in learning about the technique?

  12. Dr. A has finished learning about the new therapeutic technique and is interested in knowing about other clinicians' views of and experiences with it. Where do you expect Dr. A will turn to find out about others' views and experiences?

The list of resources was:

Dr. A's personal materials:

  1. Files (e.g., articles, handouts)

  2. Personal experiences (e.g., patients seen)

Literature searches:

  1. Computer-based services

  2. Bibliographies in books and articles

  3. Librarian (e.g., in the medical library)

  4. Consultant-recommended reading

Human resources:

  1. Patients and their families

  2. Same-specialty physicians

  3. Consultants in other specialties

  4. Medical researchers, professors

  5. Pharmaceutical and medical device industry people

Journals:

  1. Journals the doctor subscribes to

  2. Journals available in the medical library

  3. Review articles

  4. Original research articles

  5. Case reports

  6. Abstracts and brief reports

  7. Editorials

Public domain information:

  1. Newspapers, magazines, etc.

  2. Web

Technological resources:

  1. MEDSTAR and IVN programs

  2. Audio-Digest and other audiotapes

  3. Commercially available videotapes

  4. Computer-based instructional activities including specialty society home pages

Texts:

  1. Texts in the doctor's specialty

  2. Texts in other specialties

  3. Standard references in the doctor's specialty

  4. Standard references in other specialties

  5. Research monographs

Meetings and courses:

  1. Local hospital CME activities

  2. University of North Dakota School of Medicine credit-bearing CME activities

  3. Specialty society CME activities

  4. Medical association CME activities

  5. Conversations with other doctors attending meetings

  6. Poster sessions at meetings

  7. Meetings run by pharmaceutical and device manufacturing companies

  8. Courses offered at regional centers of excellence

  9. On-the-job training supervised by colleagues

  10. Meeting proceedings

Footnotes

* This study was funded through an unrestricted educational grant from the American Medical Association Foundation, granted on behalf of the Audio-Digest Foundation.

‡ The complete set of Helmert contrasts are available from the authors; they are not reported here to reduce the length of the report.

§ The computer is not a time-efficient learning resource if a physician first has to learn how to use it to solve the problem he currently faces.

REFERENCES

  1. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  2. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  3. Fox RD, Bennett NL. Learning and change: implications for continuing medical education. Brit Med Journal. 1998 Feb 7; 316(7129):466–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  5. Elstein AS. Clinical reasoning in medicine. In: Higgs J, Jones MA, eds. Clinical reasoning in the health professions. Oxford, U.K.: Butterworth-Heinemann, 1995 49–59. [Google Scholar]
  6. Norman GR, Schmidt HG. The psychological basis of problem-based learning: a review of the evidence. Acad Med. 1992 Sep; 67(9):557–65. [DOI] [PubMed] [Google Scholar]
  7. Slotnick HB, Kristjanson AF, Raszkowski RR, Moravec R.. A note on mechanisms of action in physicians' learning. Professions Education Researcher Q. 1998;19(2):5–12. [Google Scholar]
  8. Golub RM. Curbside consultations and the viaduct effect. JAMA. 1998 Sep 9; 280(10):929–30. [DOI] [PubMed] [Google Scholar]
  9. Kuo D, Gifford DR, and Stein MD. Curbside consultation practices and attitudes among primary care physicians and medical subspecialists. JAMA. 1998 Sep 9; 280(10):905–9. [DOI] [PubMed] [Google Scholar]
  10. Slotnick HB, Kristjanson AF. How physicians “learn” from pharmaceutical representatives: an exploration. J Continuing Education in the Health Professions. 1999 Spring; 19(2):84–96. [Google Scholar]
  11. McClaran J, Snell L, and Franco E. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Continuing Education in the Health Professions. 1998 Spring; 18(2):107–18. [Google Scholar]
  12. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  13. Geertsma RH, Parker RC, and Whitbourne SK. How physicians view the process of change in their practice behavior. J Med Educ. 1982 Oct; 57(10 part 2):752–61. [DOI] [PubMed] [Google Scholar]
  14. Fox RD, Mazmanian PE, and Putnam RW. eds. . Changing and learning in the lives of physicians. New York, NY: Praeger Publishers. 1989 1–194. [Google Scholar]
  15. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  16. Fox RD, Rankin R, Costie KA, Parboosingh JT, and Smith E. Learning and the adoption of innovations among Canadian radiologists. J Continuing Education in the Health Professions. 1997 Summer; 17(3):173–86. [Google Scholar]
  17. Fox RD, Rankin R, Costie KA, Parboosingh JT, and Smith E. Learning and the adoption of innovations among Canadian radiologists. J Continuing Education in the Health Professions. 1997 Summer; 17(3):173–86. [Google Scholar]
  18. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  19. Gorman PN. Information needs of physicians. J Am Soc Inf Sci. 1995 Dec; 46(10):729–36. [Google Scholar]
  20. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  21. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  22. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  23. Gorman PN. Information needs of physicians. J Am Soc Inf Sci. 1995 Dec; 46(10):729–36. [Google Scholar]
  24. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  25. Connelly DP, Rich EC, Curley SP, and Kelly JT. Knowledge resource preferences of family physicians. J Fam Prac. 1990 Mar; 30(3):353–9. [PubMed] [Google Scholar]
  26. Curley SP, Connelly DP, and Rich EC. Physicians' use of medical knowledge resources: preliminary theoretical framework and findings. Med Decis Making. 1990 Oct–Dec; 10(4):231–41. [DOI] [PubMed] [Google Scholar]
  27. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  28. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  29. Slawson DC, Shaughnessy AF, and Bennett JH. Becoming a medical information master: feeling good about not knowing everything. J Fam Prac. 1994 May; 38(5):505–13. [PubMed] [Google Scholar]
  30. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  31. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  32. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  33. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  34. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  35. Thompson ML. Characteristics of information resources preferred by primary care physicians. Bull Med Libr Assoc. 1997 Apr; 85(2):187–92. [PMC free article] [PubMed] [Google Scholar]
  36. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  37. McClaran J, Snell L, and Franco E. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Continuing Education in the Health Professions. 1998 Spring; 18(2):107–18. [Google Scholar]
  38. Slotnick HB, Kristjanson AF, Raszkowski RR, and Moravec R. How doctors learn: mechanisms of action. Chicago, IL: American Educational Research Association, 1997 [Google Scholar]
  39. Slotnick HB, Raszkowski RR, and Lichtenauer D. Rethinking continuing medical education. J Continuing Education in the Health Professions. 1995 Fall; 15(1):8–22. [Google Scholar]
  40. Slotnick HB, Raszkowski RR, Jensen CE, and Wentz DK. Physicians' preferences in CME including insights into education versus promotion. J Continuing Education in the Health Professions. 1994 Summer; 14(3):173–86. [Google Scholar]
  41. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  42. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  43. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  44. Connelly DP, Rich EC, Curley SP, and Kelly JT. Knowledge resource preferences of family physicians. J Fam Prac. 1990 Mar; 30(3):353–9. [PubMed] [Google Scholar]
  45. Golub RM. Curbside consultations and the viaduct effect. JAMA. 1998 Sep 9; 280(10):929–30. [DOI] [PubMed] [Google Scholar]
  46. Kuo D, Gifford DR, and Stein MD. Curbside consultation practices and attitudes among primary care physicians and medical subspecialists. JAMA. 1998 Sep 9; 280(10):905–9. [DOI] [PubMed] [Google Scholar]
  47. Slotnick HB, Kristjanson AF. How physicians “learn” from pharmaceutical representatives: an exploration. J Continuing Education in the Health Professions. 1999 Spring; 19(2):84–96. [Google Scholar]
  48. Slotnick HB. How doctors know when to stop learning. Med Teacher. 2000 Mar; 22(2):189–96. [Google Scholar]
  49. Slotnick HB. How doctors learn: the anatomy and physiology of self-directed learning episodes. Ottawa, ON: Royal College of Physicians and Surgeons of Canada, 1998 1–118. [Google Scholar]
  50. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  51. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  52. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  53. McClaran J, Snell L, and Franco E. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Continuing Education in the Health Professions. 1998 Spring; 18(2):107–18. [Google Scholar]
  54. Covell DG, Uman GC, and Manning PR. Information needs in office practice: are they being met? Ann Intern Med. 1985 Oct; 103(4):596–99. [DOI] [PubMed] [Google Scholar]
  55. Gorman PN, Ash J, and Wykoff L. Can primary care physicians' questions be answered using the medical journal literature? Bull Med Libr Assoc. 1994 Apr; 82(2):140–6. [PMC free article] [PubMed] [Google Scholar]
  56. Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Making. 1995 Apr–Jun; 15(2):113–9. [DOI] [PubMed] [Google Scholar]
  57. Slotnick HB, Kristjanson AF, Raszkowski RR, Moravec R.. A note on mechanisms of action in physicians' learning. Professions Education Researcher Q. 1998;19(2):5–12. [Google Scholar]
  58. McClaran J, Snell L, and Franco E. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Continuing Education in the Health Professions. 1998 Spring; 18(2):107–18. [Google Scholar]
  59. Jennett P, Jones D, Mast T, Egan K, and Hotvedt M. Characteristics of self-directed learning methods. In: Davis DA, Fox RD, eds. The physician as learner: linking research to practice. Chicago, IL: American Medical Association. 1994 47–65. [Google Scholar]
  60. Schaeffer RL, Mendenhall W, and Ott L. Elementary survey sampling. 3d ed. Boston, MA: Duxbury Press, 1986 [Google Scholar]
  61. Slotnick HB. How doctors know when to stop learning. Med Teacher. 2000 Mar; 22(2):189–96. [Google Scholar]
  62. Slotnick HB, Raszkowski RR, Jensen CE, Christman TA.. Away-from-home CME: age and sex differences among physicians in the Dakotas. Teaching and Learning in Medicine. 1994;6(4):237–46. [Google Scholar]
  63. Johnson RA, Wichern DW. Applied multivariate statistical analysis. 4th ed. Upper Saddle River, NJ: Prentice Hall, 1998 [Google Scholar]
  64. Glass GV, Hopkins KD. Statistical methods in education and psychology, 2d ed. Englewood Cliffs, NJ: Prentice-Hall, 1984 [Google Scholar]
  65. Statistical Analysis System. SAS/STAT user's guide, version 6. 4th ed. Cary, NC: SAS Institute, 1990. 891–996. [Google Scholar]
  66. Haug JD. Physicians' preferences for information sources: a meta-analytic study. Bull Med Libr Assoc. 1997 Jul; 85(3):223–32. [PMC free article] [PubMed] [Google Scholar]
  67. Verhoeven AA, Boerma EJ, and Meyboom-de Jong B. Use of information sources by family physicians: a literature survey. Bull Med Libr Assoc. 1995 Jan; 83(1):85–90. [PMC free article] [PubMed] [Google Scholar]
  68. Slotnick HB, Raszkowski RR, and Lichtenauer D. Rethinking continuing medical education. J Continuing Education in the Health Professions. 1995 Fall; 15(1):8–22. [Google Scholar]
  69. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  70. McClaran J, Snell L, and Franco E. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Continuing Education in the Health Professions. 1998 Spring; 18(2):107–18. [Google Scholar]
  71. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  72. Fox RD, Bennett NL. Learning and change: implications for continuing medical education. Brit Med Journal. 1998 Feb 7; 316(7129):466–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Slotnick HB. How doctors learn: physicians' self-directed learning episodes. Acad Med. 1999 Oct; 74(10):1106–17. [DOI] [PubMed] [Google Scholar]
  74. McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996 Jan 1; 124(1 part 1):56–62. [DOI] [PubMed] [Google Scholar]

Articles from Bulletin of the Medical Library Association are provided here courtesy of Medical Library Association

RESOURCES