Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jul 13.
Published in final edited form as: AJOB Prim Res. 2011;2(1):42–46. doi: 10.1080/21507716.2011.564263

Content Analysis of Major Textbooks and Online Resources Used in Responsible Conduct of Research Instruction

Alexander A Kon 1, Debie A Schilling 2, Elizabeth Heitman 3, Nicholas H Steneck 4, James M DuBois 5
PMCID: PMC3135017  NIHMSID: NIHMS297882  PMID: 21766046

Abstract

Instruction in the responsible conduct of research (RCR) is required for all trainees funded by the National Institues of Health (NIH) or National Science Foundation (NSF). A recent Delphi study identified 53 key topics in 7 core areas that RCR education experts felt should be included in this instruction, which is required of many trainees in clinical and translational research. We performed a content analysis of major textbooks and online resources used in RCR instruction to determine the extent to which the 53 key topics identified in the Delphi study are covered by these resources. Textbooks and online resources used in RCR education at Clinical and Translational Science Award institutions were identified via survey. These resources were subjected to a content analysis. The 53 key topics identified in the Delphi study formed the basis of these analyses. We identified 10 textbooks and 1 online resource currently in use. Of the 53 key topics, only 4 were included in all 11 resources, and another 12 were included in 10. Twenty-three topics were covered in fewer than 65% of the resources, and two topics were absent from nearly all. Educators in clinical and translational research should be aware of key topics that are not covered in the RCR textbooks and online resources they may use and should consider augmenting discussion of such topics with other materials.

Keywords: biomedical research, professional education, research ethics, research misconduct, scientific misconduct, translational research

Since 1990, the National Institutes of Health (NIH) has required that all trainees supported by National Research Service Awards receive education and training in the responsible conduct of research (RCR) (U.S. Department of Health and Human Services 1989). This requirement has recently been updated and strengthened by the NIH (National Institutes of Health 2009). The 1990 requirement provided suggestions for areas that should be covered but left instructional content to institutions and RCR instructors. In 2000, the Public Health Service proposed nine core areas of instruction in RCR (PHS 2000): (1) data acquisition, management, sharing, and ownership; (2) mentor/trainee responsibilities; (3) publication practices and responsible authorship; (4) peer review; (5) collaborative science; (6) human subjects; (7) research involving animals; (8) research misconduct; and (9) conflict of interest and commitment. Although the policy detailing these specific core areas was suspended in 2001, it continues to have substantial influence, and many institutions continue to use these nine core areas as a guide in their own RCR programs (Heitman and Bulger 2005).

Recently, the NIH formally defined RCR as “the practice of scientific investigation with integrity. It involves the awareness and application of established professional norms and ethical principles in the performance of all activities related to scientific research” (National Institutes of Health 2009). This new notice includes the nine core areas already specified as essential and adds “safe laboratory practices” and “the scientist as a responsible member of society, contemporary ethical issues in biomedical research, and the environmental and societal impacts of scientific research” as additional recommended elements. The 2009 notice further specifies that Web-based learning may be used to augment instruction but by itself does not provide sufficient training (National Institutes of Health 2009).

In 2007, the first round of Clinical and Translational Science Awards (CTSAs) was funded by the National Center for Research Resources (NCRR 2006). This new funding mechanism replaced the former General Clinical Research Center (GCRC) funding, and rolled the training programs funded under the T32, K12, and K30 programs into the CTSAs. Placing these training programs within the CTSA framework necessitates the development of novel approaches to education in RCR specific to the clinical and translational research paradigm. Using Delphi survey methodology, DuBois and colleagues proposed topics that ought to be covered in seven of the core RCR areas when adapted to research training for clinical and translational science (DuBois and Dueker 2009). The survey identified 53 topics to be included in RCR instruction. Because institutions generally require training beyond basic RCR instruction for those individuals who perform research involving human subjects and those who perform research with animals, the survey did not address these core instructional areas. RCR instruction is generally considered merely introductory in these areas, and therefore such content overlaps significantly with other required instruction.

Previous work has also demonstrated that the content, format, and scope of RCR instruction vary widely (Heitman and Bulger 2005; Kalichman and Plemmons 2007; Steneck and Bulger 2007). Further, there is wide variability regarding whether textbooks are used in RCR courses and which textbooks are selected when instructors do incorporate such texts into their courses (DuBois et al. 2010). The Educational Materials Group of the NIH CTSA Consortium’s Clinical Research Ethics Key Function Committee therefore sought to determine the extent to which current textbooks and online resources address the 53 topic areas detailed in the DuBois et al. (2010) report in order to assist RCR educators in developing RCR courses tailored to the new clinical and translational research paradigm.

METHODS

A list of textbooks and online resources used in RCR instruction was developed by project members who have experience in developing RCR training programs (Heitman, Steneck, DuBois, Kon). The list was expanded to include all textbooks and online resources that are used in RCR education at CTSA institutions (DuBois et al. 2010). Resources produced by individual institutions that are not widely available were not included (e.g., handouts, Power-Point presentations, locally created online courses that are unavailable outside the host institution). Hard copies of all textbooks were obtained for analysis. Online resources were analysed via the Internet.

One member of the project team (Schilling), who has experience and training in content analysis, performed a content analysis of all textbooks and online resources based on the 53-topic list (DuBois and Dueker 2009). Each textbook and online resource was listed as covering a topic if that topic was presented to an extent that would provide a basic level of explanation for a novice reader. Merely mentioning a topic without presenting a definition and/or providing an explanation of its importance was not sufficient to be considered covering a topic for the sake of the analysis. The determination of whether a textbook’s description met this level of detail was determined subjectively by the project analyst based on discussions with other project members. There was no attempt to assess how well each topic was presented or the depth or acuracy of the information provided for each topic. Textbooks and online resources were scored on a “yes/no” grid for each topic.

RESULTS

Ten textbooks and one online resource were identified and included in the analysis. Table 1 presents these 11 resources and details the topics covered in each. Four of the 53 key topics were covered in all resources: significance of authorship; authorship assignment; significance of conflicts of interest; and types, definitions, and examples of conflicts of interest. Twelve other topics were covered by 91% (n = 10) of the resources: inappropriate authorship practices; dealing with controversies that arise in authorship; scientific responsibilities of authors; poor publication practices; significance of misconduct; factors that contribute to scientific misconduct; plagiarism; fabrication; falsification; regulations and policies addressing misconduct; institutional conflicts of interest; and managing conflicts of interest. Twenty-three of the 53 key topic areas were presented in fewer than 65% of the resources, including two topics (addressing compliance with ethical standards within articles; and studying taboo, controversial, or politically sensitive research topics) that were covered in only 18% (n = 2) and 27% (n = 3) of the resources, respectively.

Table 1.

Content analysis of RCR textbooks

Topic Book
n (%)
A B C D E F G H I J K
Core Area I. Data acquisition, management, sharing and ownership
 1. Ethical values behind the scientific standards for data acquisition, management, sharing, and ownership ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 2. Variations in lab practices—legitimate and illegitimate variations ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 3. Data acquisition issues ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 4. Data storage, protection, and archiving ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 5. Data sharing ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 6. Legal aspects of data ownership and rights ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 7. Data privacy ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 8. Scientific methodology issues, including research design, objectivity, and bias ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 9. Data reporting ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 10. Special issues related to scientific roles ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
Core area II. Mentor/trainee responsibilities
 11. Definitions and expectations of the mentor/trainee relationship ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 12. Power relationships and the potential problems they involve ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 13. Scientific responsibilities of the mentor ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 14. Nonscientific responsibilities of roles of the mentor ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 15. Responsibilities of trainees within the mentor–trainee relationship ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 16. How to get the most out of the mentor–trainee relationship ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 17. Addressing challenges and problems in the mentor–trainee relationship ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
Core area III. Publication practices and responsible authorship
 18. The significance of authorship ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 11 (100)
 19. Authorship assignment ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 11 (100)
 20. Inappropriate authorship practices ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 21. Dealing with controversies that arise in authorship ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 22. Scientific responsibilities of authors ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 23. Poor publication practices ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 24. Protecting privacy in publication ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 25. Addressing compliance with ethical standards within articles ✓ ✓ 2 (18)
 26. Responsible disclosure of scientific information within the popular press ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
Core area IV. Peer review
 27. The significance of peer review ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 28. Conflicts of interest and peer reviews ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 8 (73)
 29. Qualities of a good review/reviewer ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 30. Logistics of peer reviewing ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 31. Responding to reviewers ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 32. Reviewer roles in ensuring RCR ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 33. Editorial responsibilities ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
Core area V. Collaborative science
 34. The nature and advantages of successful collaborations ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 35. Types of collaboration ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 36. Working well with others ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 37. Dealing with challenges in collaborative relationships ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
 38. The role of institutions in collaborative science ✓ ✓ ✓ ✓ ✓ ✓ 6 (55)
Core area VIII. Research misconduct
 39. Significance of misconduct ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 40. Factors that contribute to scientific misconduct ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 41. Plagiarism ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 42. Falsification ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 43. Fabrication ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 44. Other serious deviations from scientific best practices ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 45. Regulations and policies addressing misconduct ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 46. Responding to observed misconduct ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
 47. Studying taboo, controversial, or politically sensitive research topics ✓ ✓ ✓ 3 (27)
Core area IX. Conflicts of interest and commitment
 48. The significance of conflicts of interest ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 11 (100)
 49. Types, definitions, and examples of conflicts of interest ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 11 (100)
 50. Conflicts of commitment ✓ ✓ ✓ ✓ ✓ ✓ ✓ 7 (64)
 51. Institutional conflicts of interest ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 52. Managing conflicts of interest ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 10 (91)
 53. Conflicts of interest law and policy ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 9 (81)
Open in a new tab

Note.

✓

= Topic presented. Books: A: Collaborative Institutional Training Initiative CITI (CITI 2009); B: ORI Introduction to the Responsible Conduct of Research, Revised (Steneck 2007); C: The Responsible Conduct of Research (Beach 1996); D: Making the Right Moves: A Practical Guide to Scientific Management for Postdocs and New Faculty (Howard Hughes Medical Institute and Burroughs Wellcome Fund 2006); E: The Ethical Dimensions of the Biological and Health Sciences (Bulger, Heitman, and Reiser 2002); F: Scientific Integrity: Text and Cases in Responsible Conduct of Research (Macrina 2005); G: On Being a Scientist: Responsible Conduct in Research (Committee on Science, Engineering, and Public Policy 1995); H: Responsible Conduct of Research (Shamoo and Resnik 2003); I: Ethical and Regulatory Aspects of Clinical Research: Readings and Commentary (Emanuel et al. 2003); J: Responsible Research: A Guide for Coordinators (Fedor, Cola, and Pierre 2006); K: Ethical Issues in Clinical Research: A Practical Guide (Lo 2009).

DISCUSSION

We found significant variability in topics covered by the 11 textbooks and online resources generally used in RCR education. Only 4 of the 53 key topics are covered in all of these resources, and merely 12 others are covered by over 90%. Fully 23 of the 53 key topic areas (43%) are presented in fewer than 65% of these resources, including two topics that were absent in nearly all.

Previous work demonstrates significant variability in topics covered in RCR education (Heitman and Bulger 2005; Kalichman and Plemmons 2007; Steneck and Bulger 2007). This project supports and adds to previous work by demonstrating that such variability is seen not only in individual courses but also in the textbooks and online resources used in many CTSA programs. These findings as a whole illuminate the lack of standardization in RCR instruction.

This work is limited by the scope of analysis. Merely presenting a given topic does not guarantee that such a topic is adequately addressed or that accurate information is presented within the discussion of that topic. The Delphi project, which developed the list of 53 key topics, did not include key topics in the core areas of human subjects research; research involving animals; the scientist as a responsible member of society; contemporary ethical issues in biomedical research; and the environmental and societal impacts of scientific research (for reasons presented earlier). As such, this project did not assess key topics within those core areas. Further, we did not analyze the readability or understandability of the text, which would also be of interest. Some of the texts were published prior to the identification of the core topics in RCR education, and all were published prior to the publication of the 53 key topic areas identified by the Delphi study. As such, it is perhaps not surprising that many of the key topic areas included in the content analyses were absent from many of the resources. Also of note, two texts published a later edition after completion of our analysis (Committee on Science, Engineering, and Public Policy 2009; Shamoo and Resnik 2009), and because the grant was limited to a single year, we were unable to extend the project and analyze these new editions. While these limitations are significant, educators may use the findings of this project to assist in course design and to help identify key topics that require specific attention either through classroom discussions or augmented readings.

This work in no way attempts to recommend any specific resource or to discourage the use of any specific resource. The decision of which resources to use, and how such texts and/or online content can be incorporated into any course, depends on many factors and cannot be based merely on a content analysis such as that presented in this paper.

The project team makes the following recommendations: (1) Instructors who use textbooks or online resources in RCR instruction should be cognizant of the topics that may be given at insufficient depth or are entirely lacking from the resource(s) they choose to use. (2) Instructors should consider many aspects in choosing an appropriate textbook and online resources (e.g., topics covered, readability, understandability, ability to facilitate the learner’s independent thought, inclusion of case-based learning examples, cost, availability). (3) Other resources (journal articles, book sections, online education tools, observation of mentors, etc.) should be considered to augment classroom work. (4) Merely relying on any single source or format (textbook, online course, single-instructor lectures, etc.) is likely insufficient to provide adequate breadth and depth in RCR instruction.

Acknowledgments

This project was conducted by members of the Educational Materials Group of the NIH CTSA Consortium’s Clinical Research Ethics Key Function Committee (CC-CRE-KFC). This project was funded primarily by an administrative supplement to the CC-CRE-KFC, grant 3UL1 RR024146-03S2 (administered by the University of California–Davis) with further support from grants UL1 RR024146, UL1 RR024992, 1UL1 RR024975, and 1UL1 RR024986-01. The views expressed are solely those of the authors and do not represent the official position of the United States government, NIH, ORI, DoD, or DoN.

Footnotes

Drs. Steneck and Heitman are each authors of textbooks included in this content analysis, which we believe represent conflicts of interest that should be disclosed to readers.

Contributor Information

Alexander A. Kon, University of California–Davis and Naval Medical Center San Diego

Debie A. Schilling, University of California–Davis

Elizabeth Heitman, Vanderbilt University Medical Center.

Nicholas H. Steneck, Michigan Institute for Clinical and Health Research

James M. DuBois, Washington University School of Medicine and Saint Louis University

References

  1. Beach D. The responsible conduct of research. New York: Wiley-VCH; 1996. [Google Scholar]
  2. Bulger RE, Heitman E, Reiser SJ. The ethical dimensions of the biological and health sciences. 2. Cambridge, UK: Cambridge University Press; 2002. [Google Scholar]
  3. CITI. The responsible conduct of research for biomedical investigators. 2009 Available at: https://www.citiprogram.org/default.asp?language=english.
  4. Committee on Science, Engineering, and Public Policy. On being a scientist: Responsible conduct in research. 2. Washington, DC: National Academy Press; 1995. [Google Scholar]
  5. Committee on Science, Engineering, and Public Policy. On being a scientist: Responsible conduct in research. 3. Washington, DC: National Academy Press; 2009. [Google Scholar]
  6. DuBois JM, Dueker JM. Teaching and assessing the responsible conduct of research: A Delphi consensus panel report. Journal of Research Administration. 2009;40(1):49–70. [PMC free article] [PubMed] [Google Scholar]
  7. DuBois JM, Schilling DA, Heitman E, Steneck NH, Kon AA. Instruction in the responsible conduct of research: An inventory of programs and materials within CTSAs. Clinical and Translational Science. 2010;3(3):109–111. doi: 10.1111/j.1752-8062.2010.00193.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Emanuel EJ, Crouch RA, Arras JD, Moreno JD, Grady C. Ethical and regulatory aspects of clinical research: Readings and commentary. Baltimore, MD: Johns Hopkins University Press; 2003. [Google Scholar]
  9. Fedor CA, Cola PA, Pierre C. Responsible research: A guide for coordinators. Chicago: Remedica; 2006. [Google Scholar]
  10. Heitman E, Bulger RE. Assessing the educational literature in the responsible conduct of research for core content. Accountability in Research. 2005;12(3):207–224. doi: 10.1080/08989620500217420. [DOI] [PubMed] [Google Scholar]
  11. Howard Hughes Medical Institute and Burroughs Wellcome Fund. Making the right moves: A practical guide to scientific management for postdocs and new faculty. 2. Research Triangle Park, NC: Howard Hughes Medical Institute and Burroughs Wellcome Fund; 2006. [Google Scholar]
  12. Kalichman MW, Plemmons DK. Reported goals for responsible conduct of research courses. Academic Medicine. 2007;82(9):846–852. doi: 10.1097/ACM.0b013e31812f78bf. [DOI] [PubMed] [Google Scholar]
  13. Lo B. Ethical issues in clinical research: A practical guide. Philadelphia, PA: Wolters Kluwer Health/Lippincott William & Wilkins; 2009. [Google Scholar]
  14. Macrina FL. Scientific integrity: Text and cases in responsible conduct of research. 3. Washington, DC: ASM Press; 2005. [Google Scholar]
  15. National Center for Research Resources. Clinical and Translational Science Award. 2006 Available at: http://www.ctsaweb.org.
  16. National Institutes of Health. Update on the requirement for instruction in the responsible conduct of research. 2009 Notice Number: NOT-OD-10-019. November 24. Available at: http://grants.nih.gov/grants/guide/notice-files/NOT-OD-10-019.html.
  17. PHS Policy on Instruction in the Responsible Conduct of Research (RCR)—Suspended. 2000 December 1; Available at: http://ori.hhs.gov/policies/RCR_Policy.shtml.
  18. Shamoo AE, Resnik DB. Responsible conduct of research. New York: Oxford University Press; 2003. [Google Scholar]
  19. Shamoo AE, Resnik DB. Responsible conduct of research. 2. New York: Oxford University Press; 2009. [Google Scholar]
  20. Steneck NH. Introduction to the responsible conduct of research, revised. Office of Research Integrity; 2007. Available at: http://ori.hhs.gov/documents/rcrintro.pdf. [Google Scholar]
  21. Steneck NH, Bulger RE. The history, purpose, and future of instruction in the responsible conduct of research. Academic Medicine. 2007;82(9):829–834. doi: 10.1097/ACM.0b013e31812f7d4d. [DOI] [PubMed] [Google Scholar]
  22. U.S. Department of Health and Human Services, National Institutes of Health and Alcohol, Drug Abuse, and Mental Health Administration. Requirements for programs on the responsible conduct of research in National Research Service Award institutional training programs. National Institutes of Health Guide for Grants and Contracts. 1989;18(45):1. [Google Scholar]

RESOURCES