Abstract
Background
Shared Decision Making (SDM) is increasingly advocated as a model for medical decision making. However, there is still low use of SDM in clinical practice. High impact factor journals might represent an efficient way for its dissemination. We aimed to identify and characterize publication trends of SDM in 15 high impact medical journals.
Methods
We selected the 15 general and internal medicine journals with the highest impact factor publishing original articles, letters and editorials. We retrieved publications from 1996 to 2011 through the full-text search function on each journal website and abstracted bibliometric data. We included publications of any type containing the phrase “shared decision making” or five other variants in their abstract or full text. These were referred to as SDM publications. A polynomial Poisson regression model with logarithmic link function was used to assess the evolution across the period of the number of SDM publications according to publication characteristics.
Results
We identified 1285 SDM publications out of 229,179 publications in 15 journals from 1996 to 2011. The absolute number of SDM publications by journal ranged from 2 to 273 over 16 years. SDM publications increased both in absolute and relative numbers per year, from 46 (0.32% relative to all publications from the 15 journals) in 1996 to 165 (1.17%) in 2011. This growth was exponential (P < 0.01). We found fewer research publications (465, 36.2% of all SDM publications) than non-research publications, which included non-systematic reviews, letters, and editorials. The increase of research publications across time was linear. Full-text search retrieved ten times more SDM publications than a similar PubMed search (1285 vs. 119 respectively).
Conclusion
This review in full-text showed that SDM publications increased exponentially in major medical journals from 1996 to 2011. This growth might reflect an increased dissemination of the SDM concept to the medical community.
Keywords: Shared decision making, Bibliometric analysis, Decision making, Full text search, Review, Information storage and retrieval, PubMed, Text mining
Background
Shared decision making (SDM) is increasingly advocated as a model of best practice for decision making in the medical encounter [1-3]. SDM has been defined as a process by which healthcare choices are made jointly by the physician and the patient [4]. Since the mid-1990s, an increasing number of publications on SDM have been published not only in the field of medicine, but also in sociology, psychology, economics, and ethics [5]. The Patient Protection and Affordable Care Act, signed into the law of the United States in March 2010, contains provisions aimed at encouraging the use of SDM, which represents an important entry point for SDM into public policy [6-8]. Despite these positive developments, the use of SDM remains low due to many barriers blocking its full implementation in clinical practice [9-11].
Research publications whose main topic is not necessarily SDM may not mention the phrase “shared decision making” in the title or the abstract, but only in the discussion. Editorials, reviews or debates, among other publication types, do not have abstracts but have a potential impact on the reader. Moreover, not every single phrase is indexed in PubMed. All these publications might then be missed out by a systematic search of titles, abstracts and keywords on PubMed [12,13]. In contrast, a search of full-text articles would enhance the publication retrieval [14].
Within the last years, SDM increasingly appears in editorials and articles’ discussions in high impact medical journals. However, no reliable data can support this assertion. Such journals play a major role in the dissemination of new medical evidence [15,16]. The frequency of SDM in top journals in general internal medicine may reflect its importance in the medical community. Therefore, we conducted a full-text search followed by bibliometric analysis to identify and characterize publication trends in SDM in 15 major general internal medicine journals.
Methods
We selected the 15 journals with the highest 5-year impact factor in 2010 based on the ISI Web of Knowledge Journal Citation Reports, [17,18] in the category “medicine, general and internal”, that already existed in 1996 and regularly published original articles, letters and editorials (Additional file 1: Table S1). We did not include journals publishing only reviews, like Cochrane Database of Systematic Reviews, in order to cover various types of publications, such as research publications and editorials, and to reduce heterogeneity between journals.
Search method and publication selection
We retrieved publications through the full-text search function on each journal website, usually located on the “advanced search” web page or on the publisher website if not available. We typed chosen keywords as a phrase in double quotes in the search box. We used this search method, defined as website full-text search, because it allows finding publications with a search term not necessarily in the title or abstract, but in the article sections in full text. This is not possible through a PubMed search as full texts are not available on the PubMed search engine. We included publications of any type, with the exception of cover pages, tables of content and indexes (of authors, keywords, etc.).
To identify publications relating to SDM, referred to as SDM publications, we first chose the phrases “shared decision making” and “informed decision making” because they are the most often encountered in medical literature [19]. To improve search sensitivity, we added variants with the word “medical” (“shared medical decision making”, “informed medical decision making”). We also studied combinations of those two phrases (“informed and shared decision making”, “informed shared decision making” [4]).
We included publications released between January 1996 and December 2011, because the concept of SDM began to appear significantly in medical literature since the mid-1990s [5,19]. Moreover, electronic publications only became widely available on journal websites around 1995–97 due to changes in the publishing framework that permitted the use of automated search engines [20].
Abstraction of bibliometric data
We abstracted the following bibliometric data from the full text of included SDM publications on a prespecified electronic form (EpiData Software, version 3.1, EpiData Association, Denmark): journal, publication year, publication type, publication topic, and exact phrase relating to SDM. Publications assessing decision aids were distinguished from those that did not. We categorized publications into 9 types adapted from the indexing methods of PubMed, Embase, and previous bibliometric studies [21-23]. We grouped publications into 9 clinical topics according to chapter titles of a textbook on SDM [24] and 4 non-clinical topics, derived from a study [21] and iterative processes. We considered topics as non-clinical when publications did not address any clinical specialty. We used the definition of the Cochrane Review on decision aids: “interventions designed to help people make specific and deliberative choices among options (including the status quo) by providing (at the minimum) information on the options and outcomes relevant to a person’s health status and implicit methods to clarify values” [25]. Two authors (X.B. and R.F.) carried out single data abstraction. To measure reliability of abstraction in our prespecified form, both authors independently abstracted a random selection of publications to calculate agreement rate and Cohen kappa [26].
Validation of website full-text search
This novel search method relied on journal/publisher websites whose search strategies were not explicit, in contrast to PubMed searches. To compare our method results with a validation dataset, we contacted the editorial board of each journal to request authorisation to obtain all published materials since 1996. After receiving authorisation from the journals (n = 6/15), we collected published materials and launched an automated search script (Python Software, version 2.6, Python Software Foundation, Wolfeboro Falls, NH, USA). We used the same six phrases relating to SDM for publication retrieval. This text retrieval method on a locally stored full-text corpus was defined as downloaded full-text search [27]. We assessed the numbers of retrieved publications and the reliability between the website and downloaded full-text searches.
To assess the performance of the website full-text search, we compared it with a PubMed search using a similar strategy, i.e. using the same six phrases relating to SDM in the 15 journals between 1996 and 2011. We typed the six phrases in double quotes in the search box, so that the PubMed search looked for the exact phrase in all fields, such as the titles and abstracts, but also indexed terms and other metadata recorded by PubMed database.
Statistical analyses
We first conducted simple descriptive statistics of publications in each journal. We compared the overall number of publications with the number of SDM publications from 1996 to 2011. We assessed descriptive statistics of bibliometric data of SDM publications. We used a polynomial Poisson regression model with logarithmic link function [28,29] to assess the evolution of the number of SDM publications, the dependent variable, according to different publication characteristics. These covariables were abstracted from the selected publications: publication type (dichotomized into research and non-research publications), topic (clinical and non-clinical), and variant of SDM phrase. A Poisson regression model with offset [30] was used to assess the evolution of the percentage of SDM publications with respect to the overall number of publications in a year. We used the R statistical computation and graphics package (version 2.12.1, GNU Project, University of Auckland, New Zealand) and considered P < 0.05 as significant.
Results
The results of the full-text search on the journal websites are shown in Figure 1 (adapted from PRISMA [31]). We retrieved 1331 publications out of a total of 229 179 released between 1996 and 2011 in the 15 journals with the highest 5-year impact factor in 2010. We screened the full texts and included 1285 publications in bibliometric analysis, referred to as SDM publications. The main reason for exclusion was that publications were not proper articles but previews of articles, tables of contents or indexes. Twenty other publications were excluded because they were not containing the SDM phrase or variant.
The number of SDM publications varied by journal from 2 in Annals of Medicine to 273 in the BMJ over 16 years (Table 1). The number of SDM publications relative to the total number of publications by journal ranged from 0.09% in The Lancet to 4.99% in Journal of General Internal Medicine. SDM publications increased both in absolute and relative numbers per year, from 46 (0.32%) in 1996 to 165 (1.17%) in 2011. In a Poisson model (see equations in figures footnotes), this increase in absolute numbers was significant (all coefficients P < 0.01) and exponential (Figure 2). In relative numbers, the growth was similar (Figure 3) and also significant (all coefficients P < 0.05, except for t2 coefficient P < 0.10).The number of SDM publications was much larger in 2011 than during previous years (Figure 2). Considering this value as a possible outlier, we performed a post-hoc sensitivity analysis without year 2011 and observed a similar pattern.
Table 1.
Journal | Total no publications | No SDM publications | % |
---|---|---|---|
British Medical Journal |
61 903 |
273 |
0.44 |
Journal of General Internal Medicine |
4267 |
213 |
4.99 |
Journal of the American Medical Association |
32 868 |
206 |
0.63 |
Annals of Internal Medicine |
11 950 |
117 |
0.98 |
Archives of Internal Medicine |
8388 |
105 |
1.25 |
American Journal of Preventive Medicine** |
3442 |
82 |
2.38 |
Canadian Medical Association Journal |
18 686 |
71 |
0.38 |
Journal of Pain and Symptom Management |
3814 |
48 |
1.26 |
The New England Journal of Medicine |
19 778 |
48 |
0.24 |
The Lancet |
45 325 |
41 |
0.09 |
Preventive Medicine |
3368 |
29 |
0.86 |
The American Journal of Medicine |
6780 |
28 |
0.41 |
Mayo Clinic Proceedings |
4810 |
19 |
0.40 |
Journal of Internal Medicine |
2544 |
3 |
0.12 |
Annals of Medicine |
1256 |
2 |
0.16 |
Overall (15 journals) | 229 179 | 1285 | 0.56 |
Abbreviations [for all tables]:SDM Shared Decision Making, SDM publications publications of any type containing in their full text the phrase “shared decision making” or five other variants.
*Journals ordered by absolute number of SDM publications.
**Publications not available for years 1996–1997 for technical reasons due to a change of publisher in 1998.
Of all 1285 SDM publications, there were fewer research publications (465, 36.2%) than non-research publications (Table 2). The dynamics of the growth was also significantly different: while the number of research publications increased almost linearly, from 9 in 1996 to 46 in 2011, the number of non-research publications showed an acceleration in the second part of the time period (Figure 4).
Table 2.
No | % | |
---|---|---|
Publication type* |
|
|
Research publications |
465 |
36.2 |
Observational study (case report/series, cohort study, case–control study, …) |
287 |
22.3 |
Interventional study (randomized controlled trial, before-after study, …) |
97 |
7.5 |
Systematic review with or without meta-analysis |
43 |
3.3 |
Guideline, consensus publication |
38 |
3.0 |
Non-research publications |
820 |
63.8 |
Non-systematic review (narrative, clinical case, debate, …) |
379 |
29.5 |
Letter, comment, book review |
250 |
19.5 |
Editorial |
145 |
11.3 |
Conference publication (abstract, paper, review) |
40 |
3.1 |
Other (animal study, biology study, government report, unclear study type, combination study design, …) |
6 |
0.5 |
Topic |
|
|
Clinical |
645 |
50.2 |
Gynaecology & Obstetrics (including prenatal testing and other foetal issues) |
118 |
9.2 |
End-of-life care & Cardiopulmonary Resuscitation |
114 |
8.9 |
Cardiovascular system (including diabetes, CV diseases prevention, CV risks management) |
101 |
7.9 |
Urology (including prostate issues) |
86 |
6.7 |
Gastroenterology (including colorectal cancer) |
54 |
4.2 |
Surgery, Orthopaedics & Rheumatology (including low back pain and osteoporosis) |
41 |
3.2 |
Psychiatry/Mental Health (including addiction medicine) |
28 |
2.2 |
Paediatrics & Genetics |
26 |
2.0 |
Other** |
77 |
6.0 |
Non-clinical |
640 |
49.8 |
Medical education† (including communication skills and medical management) |
203 |
15.8 |
Decision making (including shared decision making and ethics) |
199 |
15.5 |
Patient care management†† (including patient education) |
119 |
9.3 |
Public health‡ |
119 |
9.3 |
Publications assessing decision aids |
60 |
4.6 |
Overall | 1285 | 100.0 |
*Categories adapted from indexing methods of PubMed, Embase, and bibliometric studies [21-23].
**Other clinical topics: anaesthesiology, complementary and alternative medicines, emergency medicine, endocrinology, ENT, infectious diseases, intensive care, nephrology, neurology, oncology, ophthalmology, pathology, pharmacology, radiology, respiratory system.
†Education, training programs, and courses in various fields and disciplines in medicine.
††Generating, planning, organising, and administering medical and nursing care and services for patients.
‡Branch of medicine concerned with the prevention and control of disease and disability, and the promotion of physical and mental health of the population on the international, national, state, or municipal level, including health care delivery and health promotion, e.g. physical activity, smoking management.
SDM was approached through a range of clinical topics in the included publications; gynaecology/obstetrics (118 publications, 9.2%), end-of-life care (114, 8.9%), and cardiovascular system (101, 7.9%) were the most frequent clinical topics (Table 2). We found no statistically significant difference between SDM publications addressing a clinical topic and those addressing a non-clinical topic (P > 0.30 both at baseline and across the period). The progression pattern of both was similar to the general trend (data not shown).
We found 60 publications assessing decision aids over 16 years (4.6% relative to all SDM publications), most of them randomized controlled trials (Table 2). These publications showed the same growth pattern as that of the other publications (Figure 5).
Validation of website full-text search and data abstraction
The concordance rate was high (89.4%) between the downloaded and website full-text searches (Figure 1). Two publications were retrieved by the downloaded full-text search but not by the website full-text search. The reason was a defect in the automated Optical Character Recognition (OCR) of those publications. Both were research publications.
The website full-text search identified about ten times more publications than a systematic search through PubMed (1285 vs. 119). Out of 119 publications, 3 were missed out by the website full-text search; one because of unavailable data on the journal website for years 1996–97 and two were available on the journal website but the remote search engine failed to retrieve them for an unknown reason.
Two authors independently assessed a random selection of 173 publications out of the 1285 to measure the reliability of the dichotomized data of our prespecified form. The agreement rate and Cohen kappa between the two authors were as follows: for publication type, 95.4% and 0.89, respectively; for publication topic, 93.6% and 0.87, respectively; for publications assessing decision aids, 93.6% and 0.24, respectively.
Discussion
We found a significant and exponential increase in the number of SDM publications in the 15 highest impact factor journals in general internal medicine between 1996 and 2011. Nevertheless, the number of SDM publications across the journals varied widely. When focusing on research publications, the growth was not exponential but remained mostly linear since 1996. We observed few publications assessing decision aids, however their growth pattern was similar to the general trend.
Several studies have explored the frequency of concepts in medical literature [5,21] or focused on research productivity [18,32]. Through a PubMed search of the term SDM, a previous study showed a sharp growth of the number of publications in the overall literature from 1996 to 2003 [5]. Our results are concordant with these findings. We also obtained relative numbers to make sure that the increase was not only due to the growing number of overall publications in the studied journals.
More recently, a bibliometric analysis assessed the quality as well as the volume of research in primary care between countries [32]. Our review in full-text, however, identified not only research publications but other publication types as well, such as editorials or narrative reviews, which could also have an impact on the readers. Moreover, we showed that the exponential increase of SDM observed in the last 16 years was mainly due to these types of publications, while research publications followed a linear trend during the same period.
Another study measured the distribution of topics in two major medical journals by disease categories and domains over a calendar year [21]. They found that the topic prevention was underrepresented in the journals in comparison to the importance of this topic to patients and public health. In our study, we observed a large variation in the number of SDM publications across the 15 journals studied. In relative numbers, the Journal of General Internal Medicine released fifty times more publications than the last journal in the top 15. This variation may be partially explained by differences in editorial choices, which were not studied here. Nevertheless, the trends were analysed according to various components. Breaking down the increase in SDM publications into bibliometric data brought some keys to better understand it.
To our knowledge combining techniques of review through full-text search and bibliometric analysis has not been reported before. Full-text search techniques have been used in biomedicine, [12] especially genomics, [33] but not yet in clinical medicine. This new combined method offers both search precision and simplicity of use. The sensitivity of detecting phrases in full-text publications is much higher than when limited to a classical PubMed search [13,34]. We validated the method of full-text search on journal websites through a good concordance rate when comparing with the validation dataset. Moreover, this method did not present the difficulties that we met in collecting the validation dataset from some uneasy publishers, a known issue in the scientific field [35]. Validity of our abstraction form was also gained through high kappa values for the different categories of bibliometric data.
Limitations
Our study has some limitations. First, our study was limited to the 15 major medical journals chosen according to their impact factor. Despite the widespread use of the impact factor metric, [36] it has inherent limitations [37-39]. We considered an alternative journal selection process based on journal circulation as a reflection of readership [40]. Nevertheless, circulation counts are affected by the current increase in readers accessing journals online.
Second, categorising publication types from 15 different journals was limited by our a priori criteria as each journal has its own indexing method. Our category list was developed internally, inspired by PubMed and Embase classifications, yet we reached a high degree of agreement between both reviewers (agreement rate 95.4%, Cohen kappa 0.89). The discrepancy between agreement rate (93.6%) and Cohen kappa (0.24) on publications assessing decision aids could be explained by the imbalance of the 2×2 table and the low number of publications reporting on decision aid tools [41,42].
Third, we did not divide papers into those where the concept of SDM was the primary topic and those where the concept was briefly mentioned. Our aim was to perform a scoping review of the concept of SDM in the selected medical journals. Future, more detailed studies could aim at better understanding if the concept was used as the primary topic or if authors just mentioned it.
Fourth, two journals did not exist before 1996 and were therefore excluded from our analyses. These were two open access journals (PLOS Medicine and the Annals of Family Medicine). We cannot determine if the trends in SDM would have been different in open access journals. Future studies might aim at determining if the trends in SDM are different between open access and other journals.
Finally, if we showed an increase in the number of SDM publications, we did not assess to which extent the medical community reads them. Therefore, the impact of this increase remains unknown. A future study could alternatively focus on the number of citations per SDM publication compared to citations of other publications [43].
The growth in SDM publications found in our study nevertheless supports the call of experts in the field for the medical community to implement SDM in practice [2,3,44,45]. Some of these calls have been published in major medical journals. These journals are thought to have a large impact on a vast population of physicians [15,46]. More than three years after the enactment of the Patient Protection and Affordable Care Act, there are concerns that the SDM model has not been promoted enough [7,8]. Addressing SDM in major journals is therefore more important than ever, as it could be an efficient way to disseminate it among the medical community. Moreover, our study may capture the transition of interest and advocacy for SDM from experts to clinicians and policy makers [8,47].
SDM has been called to ensure evidence-based patient choice, especially for equipoise situation [48]. Even though some issues have been raised threatening the compatibility between EBM and SDM, such as practice or financial incentives to achieve quality standards, both approaches are now clearly justified to promote efficient care by integrating patient autonomy [49].
Conclusion
This study shows that SDM publications increased exponentially in major medical journals from 1996–2011. This growth might reflect an increased dissemination of the SDM concept to the medical community. We explored a novel methodology by combining review through full-text search with bibliometric analysis. The methodology permitted a thorough retrieval of SDM publications and a precise analysis of the dissemination of SDM in medical literature.
Competing interests
The authors declare that they have no competing interests.
Authors’ contributions
RA, THC, XB were involved in the design, implementation, and analysis of the study. FL and JC were involved in the design and analysis of the study. RF and XB abstracted data. IL did the statistical analysis. PI and JK were responsible for the downloaded full-text search. XB drafted the manuscript. All authors were involved in the interpretation of the findings. All authors critically read and approved the final manuscript.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Supplementary Material
Contributor Information
Xavier Blanc, Email: Xavier.Blanc2@chuv.ch.
Tinh-Hai Collet, Email: Tinh-Hai.Collet@chuv.ch.
Reto Auer, Email: Reto.Auer@hospvd.ch.
Roland Fischer, Email: Rolando.Pescatore@gmail.com.
Isabella Locatelli, Email: Isabella.Locatelli@chuv.ch.
Pablo Iriarte, Email: Pablo.Iriarte@chuv.ch.
Jan Krause, Email: pro@jankrause.net.
France Légaré, Email: France.Legare@mfa.ulaval.ca.
Jacques Cornuz, Email: Jacques.Cornuz@chuv.ch.
Acknowledgments
We gratefully acknowledge funding from the Department of Community Medicine and Public Health, Lausanne, Switzerland. Dr T-H Collet’s research is supported by grants from the Swiss National Science Foundation and the Swiss Foundation for Grants in Biology and Medicine (PBLAP3-145870, P3SMP3-155318). We thank Isabelle de Kaenel (Medicine Faculty Library, Lausanne, Switzerland) for assisting with the correspondence with the selected journals; Sylvie Payot (Department of Ambulatory Care and Community Medicine, University of Lausanne) for helping in the data management; the 6 journals which provided their published materials for us (American Journal of Preventive Medicine, the BMJ, Canadian Medical Association Journal, Journal of General Internal Medicine, Mayo Clinic Proceedings, Journal of Pain and Symptom Management).
References
- Charles C, Gafni A, Whelan T. Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango) Soc Sci Med. 1997;44(5):681–692. doi: 10.1016/s0277-9536(96)00221-3. [DOI] [PubMed] [Google Scholar]
- Salzburg Global S. Salzburg statement on shared decision making. BMJ. 2011;342:d1745. doi: 10.1136/bmj.d1745. [DOI] [PubMed] [Google Scholar]
- Stiggelbout AM, Van der Weijden T, De Wit MP, Frosch D, Legare F, Montori VM, Trevena L, Elwyn G. Shared decision making: really putting patients at the centre of healthcare. BMJ. 2012;344:e256. doi: 10.1136/bmj.e256. [DOI] [PubMed] [Google Scholar]
- Towle A, Godolphin W. Framework for teaching and learning informed shared decision making. BMJ. 1999;319(7212):766–771. doi: 10.1136/bmj.319.7212.766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Makoul G, Clayman ML. An integrative model of shared decision making in medical encounters. Patient Educ Couns. 2006;60(3):301–312. doi: 10.1016/j.pec.2005.06.010. [DOI] [PubMed] [Google Scholar]
- Taking shared decision making more seriously. Lancet. 2011;377(9768):784. doi: 10.1016/S0140-6736(11)60290-0. [DOI] [PubMed] [Google Scholar]
- Oshima Lee E, Emanuel EJ. Shared decision making to improve care and reduce costs. N Engl J Med. 2013;368(1):6–8. doi: 10.1056/NEJMp1209500. [DOI] [PubMed] [Google Scholar]
- Frosch DL, Moulton BW, Wexler RM, Holmes-Rovner M, Volk RJ, Levin CA. Shared decision making in the United States: policy and implementation activity on multiple fronts. Z Evid Fortbild Qual Gesundhwes. 2011;105(4):305–312. doi: 10.1016/j.zefq.2011.04.004. [DOI] [PubMed] [Google Scholar]
- Legare F, Ratte S, Gravel K, Graham ID. Barriers and facilitators to implementing shared decision-making in clinical practice: update of a systematic review of health professionals’ perceptions. Patient Educ Couns. 2008;73(3):526–535. doi: 10.1016/j.pec.2008.07.018. [DOI] [PubMed] [Google Scholar]
- Caldon LJ, Collins KA, Reed MW, Sivell S, Austoker J, Clements AM, Patnick J, Elwyn G, BresDex G. Clinicians’ concerns about decision support interventions for patients facing breast cancer surgery options: understanding the challenge of implementing shared decision-making. Health Expect. 2011;14(2):133–146. doi: 10.1111/j.1369-7625.2010.00633.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elwyn G, Legare F, van der Weijden T, Edwards A, May C. Arduous implementation: does the Normalisation Process Model explain why it’s so difficult to embed decision support technologies for patients in routine clinical practice. Implement Sci. 2008;3:57. doi: 10.1186/1748-5908-3-57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Divoli A, Wooldridge MA, Hearst MA. Full text and figure display improves bioscience literature search. PLoS One. 2010;5(4):e9619. doi: 10.1371/journal.pone.0009619. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blake C. Beyond genes, proteins, and abstracts: Identifying scientific claims from full-text biomedical articles. J Biomed Inform. 2010;43(2):173–189. doi: 10.1016/j.jbi.2009.11.001. [DOI] [PubMed] [Google Scholar]
- Kostoff RN. Expanded information retrieval using full-text searching. J Inf Sci. 2010;36(1):104–113. [Google Scholar]
- Callaham M, Wears RL, Weber E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002;287(21):2847–2850. doi: 10.1001/jama.287.21.2847. [DOI] [PubMed] [Google Scholar]
- Giguere A, Legare F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A, Makosso-Kallyth S, Wolf FM, Farmer AP, Gagnon MP. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;10 doi: 10.1002/14651858.CD004398.pub3. CD004398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- ISI Web of Knowledge Journal Citation Reports. [ http://admin-apps.webofknowledge.com/JCR/JCR]
- Rosmarakis ES, Vergidis PI, Soteriades ES, Paraschakis K, Papastamataki PA, Falagas ME. Estimates of global production in cardiovascular diseases research. Int J Cardiol. 2005;100(3):443–449. doi: 10.1016/j.ijcard.2004.11.005. [DOI] [PubMed] [Google Scholar]
- Moumjid N, Gafni A, Bremond A, Carrere MO. Shared decision making in the medical encounter: are we all talking about the same thing? Med Decis Making. 2007;27(5):539–546. doi: 10.1177/0272989X07306779. [DOI] [PubMed] [Google Scholar]
- Delamothe T. Is that it? How online articles have changed over the past five years. BMJ. 2002;325(7378):1475–1478. doi: 10.1136/bmj.325.7378.1475. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woolf SH, Johnson RE. A one-year audit of topics and domains in the Journal of the American Medical Association and the New England Journal of Medicine. Am J Prev Med. 2000;19(2):79–86. doi: 10.1016/s0749-3797(00)00180-x. [DOI] [PubMed] [Google Scholar]
- Almeida-Filho N, Kawachi I, Filho AP, Dachs JN. Research on health inequalities in Latin America and the Caribbean: bibliometric analysis (1971–2000) and descriptive content analysis (1971–1995) Am J Public Health. 2003;93(12):2037–2043. doi: 10.2105/ajph.93.12.2037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Falagas ME, Kouranos VD, Michalopoulos A, Rodopoulou SP, Batsiou MA, Karageorgopoulos DE. Comparison of the distribution of citations received by articles published in high, moderate, and low impact factor journals in clinical medicine. Intern Med J. 2010;40(8):587–591. doi: 10.1111/j.1445-5994.2010.02247.x. [DOI] [PubMed] [Google Scholar]
- Edwards A, Elwyn G. Shared-decision Making in Health Care. 2. Oxford, New York: Oxford University Press; 2009. [Google Scholar]
- Stacey D, Bennett CL, Barry MJ, Col NF, Eden KB, Holmes-Rovner M, Llewellyn-Thomas H, Lyddiatt A, Legare F, Thomson R. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2011;10 doi: 10.1002/14651858.CD001431.pub3. CD001431. [DOI] [PubMed] [Google Scholar]
- Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85(3):257–268. [PubMed] [Google Scholar]
- Kroeze JH, Matthee MC, Bothma TJD. Differentiating data- and text-mining terminology. Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT 2003) 2003. pp. 93–101.
- Loomis D, Richardson DB, Elliott L. Poisson regression analysis of ungrouped data. Occup Environ Med. 2005;62(5):325–329. doi: 10.1136/oem.2004.017459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parker RA. Analysis of surveillance data with Poisson regression: a case study. Stat Med. 1989;8(3):285–294. doi: 10.1002/sim.4780080309. discussion 331–282. [DOI] [PubMed] [Google Scholar]
- Gelman A, Hill J. Data analysis using regression and multilevel/hierarchical models. New York, NY, USA: Cambridge University Press; 2006. [Google Scholar]
- Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. doi: 10.1371/journal.pmed.1000097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glanville J, Kendrick T, McNally R, Campbell J, Hobbs FD. Research output on primary care in Australia, Canada, Germany, the Netherlands, the United Kingdom, and the United States: bibliometric analysis. BMJ. 2011;342:d1028. doi: 10.1136/bmj.d1028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin X, Li Z, Huang JX, Hu X. Promoting ranking diversity for genomics search with relevance-novelty combined model. BMC bioinformatics. 2011;12(Suppl 5):S8. doi: 10.1186/1471-2105-12-S5-S8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin J. Is searching full text more effective than searching abstracts? BMC bioinformatics. 2009;10:46. doi: 10.1186/1471-2105-10-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Van Noorden R. Trouble at the text mine. Nature. 2012;483(7388):134–135. doi: 10.1038/483134a. [DOI] [PubMed] [Google Scholar]
- Garfield E. Which medical journals have the greatest impact? Ann Intern Med. 1986;105(2):313–320. doi: 10.7326/0003-4819-105-2-313. [DOI] [PubMed] [Google Scholar]
- Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ. 1997;314(7079):498–502. doi: 10.1136/bmj.314.7079.497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith R. Commentary: the power of the unrelenting impact factor–is it a force for good or harm? Int J Epidemiol. 2006;35(5):1129–1130. doi: 10.1093/ije/dyl191. [DOI] [PubMed] [Google Scholar]
- Brown H. How impact factors changed medical publishing–and science. BMJ. 2007;334(7593):561–564. doi: 10.1136/bmj.39142.454086.AD. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jones TH, Hanney S, Buxton MJ. The journals of importance to UK clinicians: a questionnaire survey of surgeons. BMC Med Inform Decis Mak. 2006;6:24. doi: 10.1186/1472-6947-6-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feinstein AR, Cicchetti DV. High agreement but low kappa: I. the problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543–549. doi: 10.1016/0895-4356(90)90158-l. [DOI] [PubMed] [Google Scholar]
- Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990;43(6):551–558. doi: 10.1016/0895-4356(90)90159-m. [DOI] [PubMed] [Google Scholar]
- Lokker C, McKibbon KA, McKinlay RJ, Wilczynski NL, Haynes RB. Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study. BMJ. 2008;336(7645):655–657. doi: 10.1136/bmj.39482.526713.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elwyn G, Laitner S, Coulter A, Walker E, Watson P, Thomson R. Implementing shared decision making in the NHS. BMJ. 2010;341:c5146. doi: 10.1136/bmj.c5146. [DOI] [PubMed] [Google Scholar]
- Barry M, Levin C, MacCuaig M, Mulley A, Sepucha K. Shared decision making: vision to reality. Health Expect. 2011;14(Suppl 1):1–5. doi: 10.1111/j.1369-7625.2010.00641.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, Koepsell TD. Journal reading habits of internists. J Gen Intern Med. 2000;15(12):881–884. doi: 10.1046/j.1525-1497.2000.00202.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harter M, van der Weijden T, Elwyn G. Policy and practice developments in the implementation of shared decision making: an international perspective. Z Evid Fortbild Qual Gesundhwes. 2011;105(4):229–233. doi: 10.1016/j.zefq.2011.04.018. [DOI] [PubMed] [Google Scholar]
- Elwyn G, Edwards A, Kinnersley P, Grol R. Shared decision making and the concept of equipoise: the competences of involving patients in healthcare choices. Br J Gen Pract. 2000;50(460):892–899. [PMC free article] [PubMed] [Google Scholar]
- Barratt A. Evidence based medicine and shared decision making: the challenge of getting both evidence and preferences into health care. Patient Educ Couns. 2008;73(3):407–412. doi: 10.1016/j.pec.2008.07.054. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.