Development and Validation of Search Filters to Identify Articles on Family Medicine in Online Medical Databases

Abstract

Physicians and researchers in the field of family medicine often need to find relevant articles in online medical databases for a variety of reasons. Because a search filter may help improve the efficiency and quality of such searches, we aimed to develop and validate search filters to identify research studies of relevance to family medicine. Using a new and objective method for search filter development, we developed and validated 2 search filters for family medicine. The sensitive filter had a sensitivity of 96.8% and a specificity of 74.9%. The specific filter had a specificity of 97.4% and a sensitivity of 90.3%. Our new filters should aid literature searches in the family medicine field. The sensitive filter may help researchers conducting systematic reviews, whereas the specific filter may help family physicians find answers to clinical questions at the point of care when time is limited.

INTRODUCTION

Although many physicians use online medical databases to obtain biomedical information for clinical practice, the enormous volume and diversity of the available literature makes searching a challenging process. Lack of time and skills, as well as a clear preference for asking an expert colleague or consulting a print source, are considered barriers to the use of online databases.^1,2 A specific search filter might enhance the retrieval of relevant articles at the point of care by the physician. On the other hand, researchers completing a systematic review will need a sensitive search filter to avoid missing relevant articles. We conducted a study to develop and validate objective, sensitive, and specific search filters, applicable in frequently used databases, to identify studies that are conducted in or that apply or refer to family medicine and general practice settings.

METHODS

Definition of Family Medicine

A clear definition relevant to family medicine/general practice is needed to develop an efficient and objective search filter. The European branch of the World Organization of National Colleges, Academies and Academic Associations of General Practitioners/Family Physicians (WONCA Europe) provides a consensus statement in which they define the discipline of family medicine/general practice.³ On the basis of a short questionnaire that was sent to colleagues worldwide using the e-mail list of the Cochrane Primary Healthcare Field, we learned that many share this definition (Supplemental Appendix 1, available at http://www.annfammed.org/content/13/4/364/suppl/DC1). The respondents indicated, however, that 2 additional aspects should be taken into account. First, an inpatient hospital setting should explicitly be excluded. Second, one should be aware of the difference between primary care and family medicine/general practice. Respondents often regarded primary care as an umbrella term that includes family medicine/general practice, but could also include (among others) midwifery, psychology, and physiotherapy.

We considered any research article relevant to general practice if the article explicitly indicated the research was completed in a family medicine/general practice setting as defined by WONCA, excluded inpatient hospital care, and focused specifically on family medicine/general practice. Research articles that had family medicine/general practice as their research domain were also considered relevant (eg, research on the efficiency of general practices).

Creating and Validating Sensitive and Specific Filters

Two independent reviewers (D.H.J.P. and F.A.L.) classified 1,000 articles as being relevant or irrelevant to family medicine, creating a reference standard. From this reference standard, we derived a term identification set and a development set. Using specialized software, candidate filter terms and phrases (combination of words) were derived objectively from the term identification set based on frequency of occurrence. Sensitive and specific filters were created in the development set by combining these candidate terms and phrases to obtain optimal performance. The derived filters were then validated on the reference set and on 2 external validation sets (Supplemental Appendix, pages 4 and 5). Finally, we compared the performance of the filters with that of other family medicine/general practice filters.

RESULTS

We used the above methodology to construct strings of sensitive and specific filters and translated these filters for use in different databases (Table 1). In the validation process, the sensitive filter had an overall sensitivity of 96.8% (range = 95.4% to 100%), with an adequate overall specificity of 74.9% (range = 69.2% to 89.5%). For the specific filter, the overall specificity was 97.4% (range = 94.8% to 99.3%), with an adequate overall sensitivity of 90.3% (range = 83.9% to 96.0%). Both the sensitive and the specific filters performed better than other recently published filters on the same topic (Table 2).^4–7

Table 1.

Our Filters for Family Medicine/General Practice Translated for Different Databases

	Filter
Database	Sensitive Filter	Specific Filter
PubMed	(“family”[all fields] OR physician*[all fields] OR practice*[tw] OR “primary care”[all fields] OR “Primary Health Care”[mh] OR primary[tw] OR general pract*[tiab] OR gp[tiab] OR gps[tiab])	(“Primary Health Care”[mh] OR “primary care”[all fields] OR “Physicians, Family”[mh] OR general pract*[all fields] OR “family”[ad] OR family pract*[all fields] OR family physician*[tw])
Ovid (MEDLINE/ Embase)	(family.af. OR physician$.af. OR practice$.mp. OR primary care.af. OR exp Primary Health Care/ OR primary.mp. OR general pract$.af. OR gp.tw. OR gps.tw.)	(exp Primary Health Care/ OR primary care.af. OR exp Physicians, Family/ OR general pract$.af. OR family.in. OR family pract$.af. OR family physician$.mp.)
Embase.com	(family OR physician* OR practice*:de,it,lnk,ab,ti OR ‘primary care’ OR ‘Primary Health Care’/exp OR primary:de,it,lnk,ab,ti OR (general NEXT/1 pract*) OR gp:ab,ti OR gps:ab,ti)	(‘Primary Health Care’/exp OR ‘primary care’ OR (general NEXT/1 pract*) OR family:ad OR (family NEXT/1 pract*) OR (family NEXT/1 physician*):de,it,lnk,ab,ti)
Cochrane	(“family” OR physician* OR practice*:ti,ab,kw,pt OR “primary care” OR [mh “Primary Health Care”] OR “primary”:ti,ab,kw,pt OR general pract*:ab,ti OR “gp”:ab,ti OR “gps”:ab,ti)	([mh “Primary Health Care”] OR “primary care” OR [mh “Physicians, Family”] OR general pract* OR family pract* OR family physician*:ti,ab,kw,pt)

Notes: For easier copying and pasting of the search strings, the filters (along with field codes) are provided in the Supplemental Appendix as plain text.

Table 2.

Performance of Our Filters Compared With That of Other Published Filters for Family Medicine/ General Practice

	Our Filters		Other Published Filters
Standard Useda	Sensitive Filter	Specific Filter	Primary Health Care Search Filter4	Jelercic et al7	Glanville et al6	Gill et al5 High Sensitivity	Gill et al5 Balanced	Gill et al5 High Specificity
Review
Sensitivity, %	100.0	90.7	81.4	46.5	95.3	95.3	93.0	88.4
Specificity, %	69.2	97.9	99.0	94.4	77.0	61.0	99.1	99.5
Reference
Sensitivity, %	95.4	83.9	65.9	78.0	84.3	85.9	68.9	57.0
Specificity, %	69.5	94.8	96.3	89.4	89.4	47.6	96.7	98.6
Questionnaire
Sensitivity, %	97.4	96.0	81.4	87.4	97.0	96.2	92.2	78.0
Specificity, %	89.5	99.3	99.0	97.8	96.1	84.9	99.4	99.8
Overall
Sensitivity, %	96.8	90.3	80.8	81.9	92.3	91.9	83.7	70.9
Specificity, %	74.9	97.4	98.3	94.3	83.6	65.0	98.7	99.4

^aReview: created during the screening process of a family medicine–relevant systematic review. Reference: created by 2 independent reviewers (D.H.J.P. and F.A.L.). Questionnaire: created by sending an e-mail to the list of the Cochrane Primary Health Care Field.

DISCUSSION

We created and extensively validated 2 search filters for family medicine, both with good sensitivity and specificity.

Our specific filter was developed to help family physicians find answers to clinical questions at the point of care when time is limited. It provides the physician with references that are relevant, but with a small risk of missing articles. If an answer to the question is not found using the specific filter, the sensitive filter could be used next.

Our sensitive filter can also be used by researchers conducting a systematic review. It provides considerable efficiency. For example, we constructed a search string for a systematic review on atopic disorders in children through which 3,972 publications were found. If our sensitive filter had been applied, the number of relevant articles could have been limited to 1,478. In this example, no relevant articles were missed. Comparing our sensitive filter with the common practice of search strategies used when conducting, for example, Cochrane systematic reviews, all tested literature searches showed a lack of good sensitivity (see online Supplemental Appendix). Thus, it can be assumed that relevant references were missed in these reviews, which might have been found when applying our sensitive search filter.

Our filters do not use the Boolean operators AND or NOT, but combine single search terms and phrases in an OR relationship. In our methodology, however, phrases were already separately identified as combinations of words in an AND manner that could potentially discriminate between those that are relevant to family medicine/general practice and those that are not. For example “primary health care” was identified in this way. These 3 words are combined in an AND manner, but the quotes also demand they be in this specific order. Using an objective method, the developed filters did not always produce phrases that one would expect, such as “family physician.” Our objective method, however, suggested the single words “family” and “physician” to be more distinctive. Finally, we rejected using NOT because its use would carry a substantial risk of excluding relevant articles.

We conclude that the described objective method results in highly sensitive and specific filters for family medicine that should be helpful to researchers and clinicians using online medical databases to obtain biomedical information.