Abstract
BACKGROUND AND OBJECTIVE:
Implementing US Preventive Services Task Force and American Academy of Pediatrics preventive service guidelines within the short duration of a visit is difficult because identifying which of a large number of guidelines apply to a particular patient is impractical. Clinical decision support system integrated with electronic medical records offer a good strategy for implementing screening in waiting rooms. Our objective was to determine rates of positive risk screens during typical well-care visits among children and adolescents in a primary care setting.
METHODS:
Child Health Improvement through Computer Automation (CHICA) is a pediatric clinical decision support system developed by our research group. CHICA encodes clinical guidelines as medical logic modules to generate scanable paper forms: the patient screening form to collect structured data from patient families in the waiting room and the physician worksheet to provide physician assessments at each visit. By using visit as a unit of analysis from CHICA’s database, we have determined positive risk screen rates in our population.
RESULTS:
From a cohort of 16 963 patients, 408 601 questions were asked in 31 843 visits. Of the questions asked, 362 363 (89%) had a response. Of those, 39 176 (11%) identified positive risk screens in both the younger children and the adolescent age groups.
CONCLUSIONS:
By automating the process of screening and alerting the physician to those who screened positive, we have significantly decreased the burden of identifying relevant guidelines and screening of patient families in our clinics.
KEY WORDS: preventive services guidelines, CHICA, screening
What’s Known on This Subject:
Clinical decision support systems offer a way to help physicians use evidence-based guidelines for screening. Screening patients for common developmental, psychosocial, and behavioral issues informs the clinical decision-making process and may improve patient outcomes.
What This Study Adds:
The Child Health Improvement through Computer Automation system, a clinical decision support system and an electronic medical record, is able to effectively screen patient families in the waiting room by using a tailored questionnaire. The study reveals positive screening rates for identifiable risks in a very large representative urban population by using Child Health Improvement through Computer Automation’s questionnaire.
The body of guidelines for recommended preventive services from authoritative sources such as US Preventive Services Task Force and the American Academy of Pediatrics’ Bright Futures guidelines (www.brightfutures.org) is ever growing.1 Simultaneously, given the time constraints of an average office visit, providers find little to no time left to focus on preventive care after addressing an array of patient concerns, acute problems, and chronic conditions.2,3 This is especially true in pediatric primary care settings, where the number of recommended services for children far outnumber what can be done in a typical visit.4 This is perhaps why pediatricians provide fewer than 50% of recommended services.5 Recent adoption of electronic medical record (EMR) has improved the quality and delivery of some primary care services.6,7 However, some studies reveal that clinicians believe current EMRs cannot adequately capture the clinical decision-making process and care coordination required to deliver preventive services.8,9 This is mainly due to lack of processes that involve decision-making based on a patient’s risk profile.10–12
Health information technologies, such as computerized clinical decision support systems (CDSS), are designed to aid clinical decision-making by comparing characteristics of individual patients to a computerized knowledgebase of guidelines for the purpose of generating patient-specific assessments and recommendations for clinicians.13 CDSS have been shown to improve rates of preventive services in controlled trials and the overall process of care.14,15
Therefore, 1 potential strategy to deliver preventive services is to use CDSS to prioritize needed preventive services by integrating CDSS capabilities with the EMR.16 This strategy can become even more effective when the combined system fits within an existing clinical workflow allowing for targeted screening of patients. This combined system would allow for assessment of individual characteristics of patients along with their data from the EMR to prioritize the physician recommendations. We believe this strategy has the potential to focus on the most pertinent and important issues of each patient in a given visit. In the last 6 years, we have developed and deployed such a CDSS in the pediatric domain, the Child Health Improvement through Computer Automation (CHICA) system.
CHICA is both a CDSS and front end to our EMR, the Regenstrief Medical Record System.17 It has been used in 1 of the busiest primary care pediatric practices in our hospital system over the last 6 years and in 3 additional community health centers over the last 2 years. CHICA has been described in detail in a previous publication and is briefly described below.18 In previous randomized trials, CHICA was shown to improve rates of asthma case detection (2.8% absolute increase in ages 3–11 years),19 maternal depression screening (twofold increase in mothers of children ages 0–15 months; manuscript under review), detection of positive risk factors for tuberculosis (twofold increase in ages up to 11 years),20 and iron deficiency (twofold increase in ages up to 18 months).20 In another pilot trial of parental smoking cessation, CHICA produced a threefold increase in the parental quit rate among families in the intervention group.21 Thus, CHICA has proven to be an effective tool for targeted screening and surveillance in our initial 4 clinics.
In this study, we examined rates at which our patients and caregivers respond to CHICA’s screening questions and the rates at which the system finds positive screens.
Methods
The CHICA System
CHICA is a CDSS for pediatric health surveillance and disease management developed by our research group. CHICA integrates well into the high volume workflow of our pediatric practices by implementing age-appropriate screening of patients in the waiting room, and then combining this information with EMR data to generate patient-specific recommendations and reminders for the physician. CHICA uses a scanable paper-based interface we call adaptive turnaround documents (ATADs).22 To select which evidence-based assessments and provider recommendations are to be delivered, CHICA employs a global prioritization scheme based on the expected value (EV) of each question or prompt.23 EV is an attractive model for prioritizing these preventive services guidelines. Outcomes are valued by an objective function such as utility or cost, and alternatives that maximize EV are ranked with the highest priority.
The CHICA system consists of a knowledge base of guideline rules encoded as Arden Syntax medical logic modules19,24,25; a repository of patient data; a tailored document printing and scanning engine; and business rules to direct communication as well as the printing and scanning of patient-specific documents.19 When a patient checks into the clinic, the registration system sends a message to CHICA, which then queries the EMR and downloads clinical data for the patient. Upon receipt of these data, CHICA generates a highly tailored double-sided ATAD, the patient screening form (PSF) as shown in Fig 1.22 The top portion of the form is completed by nursing staff and contains a structured template for recording measurements and test results. The bottom portion, which is completed by the patient or caregiver in the waiting room, includes 20 yes/no health assessment questions (in English on 1 side and Spanish on the other) that CHICA determines to be most important for the particular patient.
FIGURE 1.
The CHICA PSF.
The questions that appear on the PSF are selected by an algorithm that first identifies all candidate questions based on the patient’s age. Each question has a priority score assigned by a panel of pediatricians based on their estimate of the EV of the question. Beginning with the highest priority question, the algorithm evaluates the underlying EMR data to determine whether each question is appropriate for the patient at hand. Once 20 questions have been selected, the algorithm stops, and the questions are printed. There are roughly 198 PSF questions, although this number changes as the system is developed. Depending on a particular child’s age and data in the EMR, the number of eligible questions may be only a few or over a hundred. This algorithm ensures that the questions with the highest EV will be asked but not that every question will be asked of every child. Higher priority questions will be asked more frequently.
The completed PSF is scanned into CHICA by the nursing staff before the physician encounter and is read by the computer by using Optical Character Recognition software (www.verity.com) with high mark sense22 and digit recognition accuracy.26 Once scanned into CHICA, data derived from responses to the PSF are analyzed along with previously existing medical record data by the CHICA library of Arden Syntax rules to generate the content for the second ATAD, the physician worksheet.19 The physician worksheet contains alerts and reminders to the physician regarding routine care, follow-up, or patient risk factors.
Study Design
Data for all patients who had a visit between June 2, 2009, and June 16, 2011, to 1 or more of the 4 clinics currently served by CHICA were extracted. These data were responses to PSF questionnaires, and we computed the response rate and rate of self-reported risk for each screening question as follows: First, we tallied the number of times each screening question was asked on a PSF during the study period and the number of times any responses were recorded by the system. Next, these data were used to calculate a response rate for each PSF question. From the response data, we calculated positive screen risk for each PSF question, ie, number of responses identifying positive risk (which can be a yes or no response depending upon the question) to calculate a positive screen rate measure for our population. We report a large but not complete set of screening questions that were analyzed. The questions or topics of questions reported here are selected by our domain experts (Drs Downs and Carroll). Where appropriate, some related questions were combined for calculating positive screen risks. We also report demographical (race, gender) as well as insurance characteristics of our population here. All methods for this study were approved by the Indiana University School of Medicine Institutional Review Board.
Results
Data analyzed for this study were collected from 16 963 patients who had at least 1 visit during the study period to 1 of the 4 pediatric outpatient clinics where CHICA is implemented. Table 1 gives details of the demographic characteristics of our study population.
TABLE 1.
Demographics and Visit Characteristics
| Characteristics | N (%) |
|---|---|
| Race/ethnicity (n = 16 963) | |
| African American | 8782 (52) |
| Latino | 5636 (33) |
| White | 1571 (9) |
| Other | 940 (6) |
| Unknown | 34 (<1) |
| Gender (n = 16 963) | |
| Girl | 8249 (49) |
| Insurance category (n = 16 963) | |
| Medicaid | 14 401 (85) |
| Self-pay | 1056 (6) |
| Commercial | 721 (4) |
| Other | 783 (<1) |
| Medicare | 2 (<1) |
| Age at visit (n = 31 843 visits) | |
| 0–30 d | 2880 (9) |
| 1–2 mo | 1224 (4) |
| 2–4 mo | 1484 (5) |
| 4–6 mo | 1421 (4) |
| 6 – 9 mo | 1885 (6) |
| 9–12 mo | 1482 (5) |
| 1–2 y | 4173 (13) |
| 2–4 y | 3991 (13) |
| 4–6 y | 3417 (11) |
| 6–8 y | 2581 (8) |
| 8–12 y | 5321 (17) |
| >12 y | 1984 (6) |
| Patients with this many visits | |
| 1 | 7011 (41) |
| 2 | 4294 (25) |
| 3 | 2365 (14) |
| 4 | 1389 (8) |
| 5 | 798 (5) |
| 6 | 505 (3) |
| >6 | 601 (3) |
For this study, we divided the questions on the PSF into the following topics: postnatal/early childhood/preteen, targeted screening, disease management, and adolescent screening questions. Collectively, 408 601 questions were asked for these topics of which 362 363 (89%) questions recorded a response in the system. Of these, 39 176 (11%) identified positive screens for actionable items to address during the visit. The details of these questions, their rates of response, and rate of positive screens are listed in Table 2. Among the highest rates of positive screens in the younger age group (0–12 years old) were the number not breast feeding (45.8%), the number not always wearing a helmet when riding a bike (39.5%), and lack of awareness of firearm safety (19.2%). Targeted screening revealed high rates of smokers in the home and positive screens for attention-deficit/hyperactivity disorder (ADHD) symptoms. In the ADHD module, a very new application, 70% of the children on ADHD medication reported side effects. Over 25% of children on asthma controller medications reported nonadherence. Among the older age group (13–21 years old), the most common positive screen is lack of birth control (56.6%) and condom use (28.3%) among sexually active adolescents (8.9%) followed by positive response to screening for depression (8.8%).
TABLE 2.
Patient Screening Identifiable Risks Based on Self Report
| No. | Screening Question (Age Group Asked) | Total Asked, N | Question Response Rate, N (%) | Positive Screens, N | % Positive (95% Confidence Interval ) (Of Those Answered) |
|---|---|---|---|---|---|
| Postnatal/early childhood/preteen | |||||
| 1 | Infant is not breastfed (0 d–6 mo) | 5987 | 5394 (90) | 2471 | 45.8 (44.5–47.1) |
| 2 | Do not always wear a helmet when riding bike or tricycle (2–12 y) | 17 599 | 15 599 (88.6) | 6173 | 39.5 (38.8–40.3) |
| 3 | Has ever used a walker (6–18 mo) | 6837 | 6136 (89.7) | 2168 | 35.3 (34.1–36.5) |
| 4 | Has ever slept on stomach (0–7 mo) | 8704 | 7800 (89.6) | 2266 | 29.0 (28.1–30.1) |
| 5 | Hot water heater not set to less than 120°F or 49°C (0–3 y) | 12 900 | 11 336 (87.8) | 2542 | 22.4 (21.7–23.2) |
| 6 | Car seat faces forward (0 d–2 y) | 8306 | 7429 (89.4) | 1287 | 17.3 (16.5–18.2) |
| 7 | Infant sleeps with bottle or is breastfed continuously throughout night (5–18 mo) | 469 | 438 (93.3) | 70 | 15.9 (12.5–19.4) |
| 8 | Smoke detectors batteries not tested monthly (0–21 y) | 14 337 | 12 600 (87.8) | 1598 | 12.6 (12.1–13.3) |
| 9 | Infant not always placed on back or side to sleep (0–7 mo) | 8462 | 7592 (89.7) | 479 | 6.3 (5.8–6.9) |
| 10 | Maternal depression screening: anhedonia, depressed mood, anxiety, self-blame, panicky (0 d–15 mo) | 23 296 | 21 080 (90.4) | 870 | 4.1 (3.85–4.39) |
| 11 | No working smoke detector in home or apartment (0–21 y) | 19 716 | 17 226 (87.3) | 582 | 3.3 (3.1–3.6) |
| 12 | Domestic violence concerns: partner kicked, hit, or slapped or parent does not feel safe at home (0 d–12 y) | 40 508 | 35 700 (88.1) | 542 | 1.5 (1.4–1.6) |
| 13 | Child abuse: partner or another adult threatened or hurt children (0 d–12 y) | 23 311 | 20 589 (88.3) | 128 | 0.6 (0.5–0.7) |
| Targeted screening | |||||
| 14 | Any smoker in child’s home (0 d–12 y) | 18 213 | 16 028 (88.0) | 3124 | 19.5 (18.9–20.1) |
| 15 | ADHD screening: makes careless mistakes, difficulty remaining seated, trouble paying attention (5–13 y) | 14 524 | 13 508 (93.0) | 2887 | 21.3 (20.7–22.1) |
| 16 | Firearms safety: has not talked about guns at school or on the streets (6–12 y) | 3852 | 3456 (89.7) | 665 | 19.2 (17.9–20.6) |
| 17 | Autism screening: have concerns about child's development (24–36 mo) | 475 | 453 (95.3) | 70 | 15.4 (12.1–18.8) |
| 18 | Development screening: 9 mo, 30 mo Ages and Stages Questionnaire (ASQ) Parental concerns about: arms and legs, hands and fingers to do things, talking or making speech sounds, learning, development, and behavior (2–66 mo) | 14 692 | 13 323 (90.6) | 1224 | 9.1 (8.7–9.7) |
| 19 | Developmental concerns: Not age appropriate based on Denver Development screening test – 90th percentile cutoffs (2–64 mo) | 38 603 | 34 420 (89.1) | 3073 | 8.9 (8.6–9.2) |
| 20 | Firearms safety: any firearms kept at home (1–12 y) | 12 721 | 11 257 (88.4) | 687 | 6.1 (5.7–6.5) |
| 21 | Lead screening: direct or indirect toxicity, pre 1950s home or childcare exposure, pre 1978 home with renovation (10–36 mo) | 1697 | 1508 (88.8) | 52 | 3.4 (2.5–4.4) |
| 22 | TB screening: direct or indirect - contact with TB and exposure to high risk country (11 mo–12 y) | 53 893 | 47 555 (88.2) | 956 | 2 (1.9–2.1) |
| Disease management | |||||
| 23 | ADHD: has side effects from ADHD medicine, including stomach ache, trouble sleeping, headache, or loss of appetite (5–13 y) | 23 | 20 (86.9) | 14 | 70 (49.9–90.1) |
| 24 | Asthma: does not always use a spacer when using inhaler (puffer) (3–12 y) | 1382 | 1287 (93.1) | 331 | 25.7 (23.3–28.1) |
| 25 | Asthma: does not use “controller” medication (eg, steroid inhaler or puffer, not Albuterol) every day (3–11 y) | 468 | 427 (91.2) | 92 | 21.5 (17.6–25.4) |
| 26 | Asthma: have asthma or wheezing, cough or breathing problems: (1) that won't go away or keep coming back, (2) a tight chest during the daytime 3 or more times a week, (3) a tight chest at night 3 or more nights a month or prevents usual activities in school or at home (3–11 y) | 15 328 | 13 791 (89.9) | 2843 | 20.6 (19.9–21.3) |
| Adolescent screening | |||||
| 27 | Not using birth control if had intercourse (12–21 y) | 232 | 212 (91.3) | 120 | 56.6 (49.9–63.2) |
| 28 | Do you and your partner always use condoms when you have sex (among sexually active) (12–21 y) | 239 | 219 (91.6) | 62 | 28.3 (22.3–34.3) |
| 29 | Ever had sexual intercourse (12–21 y) | 4766 | 4119 (86.4) | 370 | 8.9 (8.1–9.9) |
| 30 | Depression screening based on GAPS: (1) no fun during the past 2 weeks; (2) generally not happy with the way things are going these days; (3) during the past few weeks, have often felt sad or down or as though you have nothing to look forward to; (4) has seriously thought about killing, made a plan to kill, or actually tried to kill self (12–21 y) | 12 235 | 10 714 (87.5) | 945 | 8.8 (8.3–9.4) |
| 31 | Think might be pregnant (12–21 y) | 124 | 111 (89.5) | 6 | 5.4 (1.2–9.6) |
| 32 | Have been physically, sexually, or emotionally abused (12–21 y) | 3987 | 3399 (85.2) | 132 | 3.8 (3.2–4.5) |
| 33 | Used alcohol and then done any of the following: (1) driven a car, (2) gone swimming or boating, (3) gotten in a fight, (4) used tools or equipment, or (5) done something that later regretted (12–21 y) | 3464 | 2905 (83.8) | 92 | 3.1 (2.5–3.8) |
| 34 | Used marijuana, other drugs, or inhalants (12–21 y) | 3478 | 2917 (83.8) | 80 | 2.7 (2.1–3.3) |
| 35 | In the past year, have been in a car or other motor vehicle when the driver has been drinking alcohol or using other drugs (12–21 y) | 3468 | 2910 (83.9) | 66 | 2.2 (1.7–2.8) |
| 36 | Ever smoked cigarettes or use snuff or chewing tobacco (12–21 y) | 3374 | 2825 (83.7) | 53 | 1.8 (1.4–2.4) |
| 37 | Used nonprescription drugs to get to sleep, stay awake, calm down, or get high (12–21 y) | 1153 | 1060 (91.9) | 19 | 1.7 (1.0–2.6) |
| 38 | Have been criticized or gotten into trouble because of drinking (12–21 y) | 1164 | 1064 (91.4) | 10 | 0.9 (0.4–1.5) |
| 39 | In the past month, got drunk or very high on beer, wine, wine coolers, or other alcohol (12–21 y) | 3478 | 2917 (83.8) | 24 | 0.8 (0.5–1.6) |
| 40 | Used steroids (eg, roids or juice) without a doctor's permission (12–21 y) | 1136 | 1039 (91.4) | 3 | 0.2 (0–0.6) |
Discussion
By automating the process of screening and alerting the physician to those who screened positive with our decision support system, we have significantly decreased the burden of identifying relevant guidelines and screening of patient families in our clinics.
This study, like all research, has limitations. The most important limitation is that CHICA selects questions for the families to answer based on patient’s age and what is known about the patient at the time of the encounter. Therefore, every question is not asked of every patient, which could lead to some underestimates of prevalence in our population for some risks.
By contrast, because the system retains information from 1 visit to the next, CHICA may ask a question more than once over the course of several visits. For instance, if CHICA learns there is a smoker at home in 1 visit, in the next it may ask if that person has quit. Another example might be that CHICA learns that a patient has asthma in 1 visit, it may ask whether symptoms occur during the day or night or how often symptoms occur. Finally, CHICA will repeat a question after a sufficient period time (months or years) have passed. This may cause some risk rates to overestimate the population prevalence. Regardless, the rates of positive screening are informative as an indication of the rates at which pediatricians might find these problems if they had a process to ask the questions.
It should be noted that we have an urban patient population with high minority and Medicaid rates. Therefore, these risk rates may not apply to other populations. It could also be argued that adolescent patients may elect not to answer more sensitive questions honestly. In fact, our results reveal low rates of some risky behaviors in the adolescent group, lower than previously reported,27 for example, rates of illicit drug use or alcohol. The rates reported here are very low when compared with the national statistic from the most recent Youth Risk Behavior Surveillance survey (http://www.cdc.gov/HealthyYouth/yrbs) in 2009. On the other hand, reported rates of unprotected sex are close to national statistics, and identifying patients who think they might be pregnant (5.6%) is extremely valuable clinically. We have observed that when the PSF is handed to teenagers, they are often accompanied by adults. This undoubtedly limits a teenager’s willingness to disclose some behaviors. Interestingly, paper surveys typically achieve higher rates of disclosure of sensitive topics than face to face or telephone surveys.28 However, the logistics in our setting may not promote some disclosures.
Alternative approaches such as private kiosks with audio computer assisted structure interviews may be more promising in the adolescent population.29 However, in general, we found that all questions were answered ∼90% of the times they were asked. Moreover, we think that the rates at which risk factors are found (over 2 per patient per encounter) can help physicians focus their efforts. It is clear that the PSF provides an efficient way to capture risk data in the waiting room. It promises to increase both the sensitivity and the efficiency of routine screening for health risks in the primary care pediatric setting. Moreover, once the risk data are in the system, they can be tracked for future reminders and support can be generated for the physician at future visits.
This study provides useful information about how commonly certain risk factors are found in the clinical environment. Often, research about the prevalence of these risk factors is conducted outside of clinical care. These results represent the rates at which patients and families report risks when they are present in a clinical setting. A deeper understanding of the under-recognized risks can help formulate a rational strategy for approaching preventive care in pediatrics.
Conclusions
By automating the process of screening and alerting the physician to those who screened positive with our decision support system, we have significantly decreased the burden of identifying relevant guidelines and screening of patient families in our clinics. Moreover, we have been able to provide estimates as to the prevalence of various risk factors present in the pediatric primary care clinical setting that should be addressed in well-child visits.
Glossary
- ADHD
attention-deficit/hyperactivity disorder
- ATAD
adaptive turnaround document
- CDSS
clinical decision support system
- CHICA
Child Health Improvement through Computer Automation
- EMR
electronic medical record
- EV
expected value
- PSF
patient screening form
Footnotes
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was supported by the Robert Wood Johnson Foundation (grant 043628), Riley Memorial Association, Indiana University Health Partners (Clarian Health Values Fund, VFR – 123), Agency for Healthcare Research and Quality (R01HS018453, R01 HS017939), and the National Library of Medicine (5K22LM9160, R01 LM010031Z). Funded by the National Institutes of Health (NIH).
References
- 1.USPSTF Agency for Healthcare Research and Quality, 4th ed. United States Preventive Services Task Force. Guide to Clinical Preventive Services; 2008:2009–2010. Available at: www.ahrq.gov/clinic/uspstfix.htm. Accessed March 5, 2012 [Google Scholar]
- 2.Konrad TR, Link CL, Shackelton RJ, et al. It’s about time: physicians’ perceptions of time constraints in primary care medical practice in three national healthcare systems. Med Care. 2010;48(2):95–100 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kottke TE, Brekke ML, Solberg LI. Making “time” for preventive services. Mayo Clin Proc. 1993;68(8):785–791 [DOI] [PubMed] [Google Scholar]
- 4.Belamarich PF, Gandica R, Stein RE, Racine AD. Drowning in a sea of advice: pediatricians and American Academy of Pediatrics policy statements. Pediatrics. 2006;118(4). Available at: www.pediatrics.org/cgi/content/full/118/4/e964 [DOI] [PubMed] [Google Scholar]
- 5.Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357(15):1515–1523 [DOI] [PubMed] [Google Scholar]
- 6.Adams WG, Mann AM, Bauchner H. Use of an electronic medical record improves the quality of urban pediatric primary care. Pediatrics. 2003;111(3):626–632 [DOI] [PubMed] [Google Scholar]
- 7.Gioia PC. Quality improvement in pediatric well care with an electronic record. Proc AMIA Symp. 2001:209–213 [PMC free article] [PubMed] [Google Scholar]
- 8.O’Malley AS, Grossman JM, Cohen GR, Kemper NM, Pham HH. Are electronic medical records helpful for care coordination? Experiences of physician practices. J Gen Intern Med. 2010;25(3):177–185 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Damberg CL, Shortell SM, Raube K, et al. Relationship between quality improvement processes and clinical performance. Am J Manag Care. 2010;16(8):601–606 [PubMed] [Google Scholar]
- 10.Sharma A. Developmental examination: birth to 5 years. Arch Dis Child Educ Pract Ed. 2011;96(5):162–175 [DOI] [PubMed] [Google Scholar]
- 11.Li Y, Tian L, Wei LJ. Estimating subject-specific dependent competing risk profile with censored event time observations. Biometrics. 2011;67(2):427–435 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lowensteyn I, Joseph L, Levinton C, Abrahamowicz M, Steinert Y, Grover S. Can computerized risk profiles help patients improve their coronary risk? The results of the Coronary Health Assessment Study (CHAS). Prev Med. 1998;27(5 pt 1):730–737 [DOI] [PubMed] [Google Scholar]
- 13.Koppel R, Kreda DA. Healthcare IT usability and suitability for clinical needs: challenges of design, workflow, and contractual relations. Stud Health Technol Inform. 2010;157:7–14 [PubMed] [Google Scholar]
- 14.Lau F, Kuziemsky C, Price M, Gardner J. A review on systematic reviews of health information system studies. J Am Med Inform Assoc. 2010;17(6):637–645 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior? J Am Med Inform Assoc. 2009;16(4):531–538 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Biondich PG, Downs SM, Anand V, Carroll AE. Automating the recognition and prioritization of needed preventive services: early results from the CHICA system. AMIA Annu Symp Proc. 2005:51–55 [PMC free article] [PubMed] [Google Scholar]
- 17.McDonald CJ, Tierney WM, Overhage JM, Martin DK, Wilson GA. The Regenstrief Medical Record System: 20 years of experience in hospitals, clinics, and neighborhood health centers. MD Comput. 1992;9(4):206–217 [PubMed] [Google Scholar]
- 18.Anand V, Biondich P, Liu G, Rosenman M, SM D. Child health improvement through computer automation: the CHICA system. Stud Health Technol Inform. 107(Pt 1): 187–191 [PubMed] [Google Scholar]
- 19.Downs SM, Biondich PG, Anand V, Zore M, Carroll AE. Using Arden Syntax and adaptive turnaround documents to evaluate clinical guidelines. AMIA Annu Symp Proc. 2006:214–218 [PMC free article] [PubMed] [Google Scholar]
- 20.Carroll AE, Biondich PG, Anand V, et al. Targeted screening for pediatric conditions with the CHICA system. J Am Med Inform Assoc. 2011;18(4):485–490 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Downs SM, Zhu V, Anand V, Biondich PG, Carroll AE. The CHICA smoking cessation system. AMIA Annu Symp Proc. 2008:166–170 [PMC free article] [PubMed] [Google Scholar]
- 22.Biondich PG, Anand V, Downs SM, McDonald CJ. Using adaptive turnaround documents to electronically acquire structured data in clinical settings. AMIA Annu Symp Proc. 2003:86–90 [PMC free article] [PubMed] [Google Scholar]
- 23.Downs SM, Uner H. Expected value prioritization of prompts and reminders. Proc AMIA Symp. 2002:215–219 [PMC free article] [PubMed] [Google Scholar]
- 24.Jenders RA, Del Fiol G, Kawamoto K, Sailors RM. Standards in clinical decision support: activities in health level seven. AMIA Annu Symp Proc. 2008:1244–1245 [PubMed] [Google Scholar]
- 25.Jenders RA, Hripcsak G, Sideli RV, et al. Medical decision support: experience with implementing the Arden Syntax at the Columbia-Presbyterian Medical Center. Proc Annu Symp Comput Appl Med Care. 1995:169–173 [PMC free article] [PubMed] [Google Scholar]
- 26.Biondich PG, Overhage JM, Dexter PR, Downs SM, Lemmon L, McDonald CJ. A modern optical character recognition system in a real world clinical setting: some accuracy and feasibility observations. Proc AMIA Symp. 2002:56–60 [PMC free article] [PubMed] [Google Scholar]
- 27.Downs SM, Klein JD. Clinical preventive services efficacy and adolescents’ risky behaviors. Arch Pediatr Adolesc Med. 1995;149(4):374–379 [DOI] [PubMed] [Google Scholar]
- 28.Aquilino WS. Telephone versus face-to-face interviewing for household drug use surveys. Int J Addict. 1992;27(1):71–91 [DOI] [PubMed] [Google Scholar]
- 29.Paperny DM, Aono JY, Lehman RM, Hammar SL, Risser J. Computer-assisted detection and intervention in adolescent high-risk health behaviors. J Pediatr. 1990;116(3):456–462 [DOI] [PubMed] [Google Scholar]