Abstract
Objectives To determine the prevalence of hospital web sites, the types of information provided within these sites, and the relationship of information to institutional characteristics. Design Online search of hospital web sites over a 6-week period in late 1999. Web sites were abstracted for content. Bivariate comparisons were made of hospital profit status and ownership or operation by a multihospital network. Participants California acute care hospitals and their web sites. Main outcome measures Operation of web sites and web site content. Results Among 390 California hospitals, 242 (62%) had easily identifiable web sites, 59 (15%) had no web sites, and 89 (23%) had sites identified only after telephone follow-up. Hospitals without sites were more likely not-for-profit, small, rural, or unaffiliated. The presentation of information was inconsistent, although most (93%) provided basic contact information. Many hospitals provided health content information (70%) or mentioned health classes (65%), but few guaranteed the quality of this information. Patient care features (online health profiles, risk identification, e-mail) were infrequent (13%) and rudimentary. Product advertising was frequent (54%) but was often nonhealth-related and unobtrusive. Of the 36% of hospitals that reported information on quality, few of the designated measures were valid and reliable measures of quality. Overall, 21% of hospitals reported accreditation (Joint Commission on Accreditation of Healthcare Organizations) status, and for-profit hospital web sites were more likely to report this accreditation. Conclusion Consumers should be aware of current limitations in using information on hospital web sites. In the future, hospitals may better realize the potential of web sites for the delivery of health care information and patient care.
Individuals have embraced the use of the Internet for health care. Online health information is ubiquitous.1 More Americans use the Internet for health care information than for online shopping or financial advice.2 People can purchase pharmaceutical agents and health insurance through the Internet.3,4,5 Organ transplantation and egg harvesting have been sold online.6,7 Despite their central role in providing health care, the online activity of traditional “brick and mortar” institutions such as hospitals remains understudied.
Hospitals have incentives to “go on-line.” Information technology can improve care and promote consumer choice.8, 9 Web sites can facilitate services such as patient-clinician contact and appointment scheduling.10 Hospitals as providers of “well care” may use sites for patient education, especially preventive care and self-help. Nonprofit hospitals may promote services that support their institutional mission.11 Economies of scale may allow larger or network-affiliated hospitals to expend greater resources to create sophisticated web sites. Academic medical centers may combine “content” and advertising to promote cutting-edge technology.12,13
Consumers are perceived as seeking information regarding health care quality.14,15,16 Measures of quality are neither uniformly accepted nor universally available.17,18,19 Health care providers have been shown to provide incomplete or inaccurate quality information.20,21 Thus, the reporting of quality measures balances web sites between being information resources and vehicles for advertising.
We identified and abstracted information from web sites maintained by California acute care hospitals during 6 weeks in late 1999. We asked 3 salient questions regarding hospital web sites: What is the prevalence of web sites among hospitals? What is the content of these sites? Do institutional factors affect the presentation of information on web sites?
METHODS
Data collection
We identified and obtained information on all nonfederal general acute care hospitals in California licensed in 1999 from the annual report of the California Office of Statewide Health Planning and Development (OSHPD) (www.oshpd.cahwnet.gov). These data include hospital ownership, profit status, size, teaching status, and location.
Using the list of hospitals, we identified web sites by iterative searches, applying 3 Internet search engines that use different search strategies (Yahoo—index; Alta Vista—web crawler; and Netscape—index supplemented by external web crawler). Search terms included hospital and hospital owner names. Web-site text and graphics were downloaded for abstraction during 6 weeks in November and December 1999. A portion of sites maintained by 1 hospital chain could not be downloaded during the study period and were not abstracted. Hospitals without identified web sites were contacted by phone in January 2000, but web sites identified in this way were not abstracted for content (see figure and results).
Data abstraction
Content areas were identified for abstraction from each hospital web site and were grouped by contact information, online health care information, health care services, product links, measures of quality or accreditation, and web-site design. Online health care information included within-site online reference materials, links to external Internet web sites, and information for health classes. Information on common conditions treatable in hospitals was identified (heart disease, diabetes mellitus, and cancer).22 Because consumers consider health care services when selecting providers and because services are used in defining measures of quality of care, we abstracted information on health care services.23 We examined services for common conditions: cardiac care (bypass surgery, angiography, and other cardiac care), cancer care (cancer screening or treatment), obstetric care (birthing and neonatal care), and women's health (mammography, Papanicolaou smears, and osteoporosis care). We also abstracted direct advertising (product icons and direct links) and other marketing information (focused on “health benefits” for seniors or describing hospital-affiliated clinical trials). Similar approaches have been used for evaluating other health-related web sites.24
Content abstraction included hospital accreditation and quality measures, defined as self-identified awards or markers of excellence, or measures for quality improvement. Some measures could be verified. Every 3 years, OSHPD identifies hospitals as comparable, outperformers, and underperformers based on adjusted heart attack mortality rates.25 The Pacific Business Group on Health (PBGH) annually awards a Blue Ribbon Award to high-quality hospitals in California (www.healthscope.org). The weekly magazine US News and World Report (USNWR) produces an annual “Best Hospitals” issue that gives overall and specialty rankings of hospitals, based on a survey of medical providers.26,27 The Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) accredits hospitals every 3 years and makes results publicly available (www.jcaho.org). Among these, only OSHPD and USNWR are demonstrated valid measures of patient outcome, but only with regard to heart attack mortality.25,27
Two of us (D S Z and Y W L) independently abstracted information from hard copies of the web sites using a standardized abstraction form. Among disagreements, consensus was reached by reviewing inconsistencies on a case-by-case basis. Data reported reflect this consensus.
Analysis
Dependent variables were the presence of a hospital web site and abstracted content measures described earlier. Independent variables were hospital profit status, network affiliation, academic status, location, and size. Hospital networks (or chains) consisted of at least 2 hospitals owned or operated by a single organization.
We examined the characteristics of all hospitals that operated a web site and of those hospitals that operated a web site identified by phone follow-up. Univariate statistics were obtained for abstracted content. Reported OSHPD, JCAHO, USNWR, and PBGH measures were verified. Bivariate comparisons were made for abstracted content versus 2 hospital characteristics—profit status and hospital network ownership. Significance testing was by Pearson's χ2. We used multivariate logistic regressions with clustering corrections to account for hospital corporate affiliation to estimate the influence of profit status and network affiliation on content, after accounting for hospital size, location, and teaching status.28,29 Results did not differ significantly from bivariate comparisons and are not presented here. Analyses were performed using commercially available statistical software (Intercooled Stata 6.0; Stata Corporation, College Station, TX).
RESULTS
Characteristics of hospitals with web sites or web information
Our Internet search identified 242 of 401 hospitals as having web sites (see figure). Phone follow-up of the 158 hospitals without identified sites revealed 11 hospital closures and 89 additional web sites. Thus, 62% (242/390) were identified with web sites, 15% (59/390) had no web sites, and 23% (89/390) had sites not easily identifiable using search engines. Web sites identified by phone follow-up tended to be of smaller, independent, nonprofit, or district hospitals compared with those identified by Internet search.
Among the 242 Internet search-identified sites, 20 of 51 managed by 1 hospital network were inaccessible during the downloading period because of problems with that organization's network server (verified by their system administrator). Examination of these sites after the study period did not indicate substantive differences with the organization's other web sites. Thus, a total of 222 hospital web sites were abstracted and analyzed for content. Hospitals with web sites were most likely nonprofit, academic, associated with a hospital network, and located in an urban area (table 1).
Table 1.
Distribution of characteristics of 390 general acute care hospitals in California and their influence on whether a hospital operated a web site (December 1999)
| Hospital characteristics | Number of hospitals by category | Hospital has a web site, no. (%)* |
|---|---|---|
| Overall | 390 | 332 (85) |
| Size, no. of beds | ||
| <100 | 101 | 54 (53) |
| 100 to 199 | 121 | 95 (79) |
| 200 to 299 | 68 | 58 (86) |
| ≥300 | 100 | 98 (98) |
| Type of control | ||
| District | 50 | 31 (62) |
| Government | 22 | 15 (68) |
| For profit | 97 | 65 (67) |
| Nonprofit | 221 | 193 (87) |
| Hospital affiliation | ||
| Network | 221 | 212 (96) |
| Independent hospital | 169 | 93 (55) |
| Academic medical center | ||
| Major teaching hospital | 26 | 23 (88) |
| Not a major teaching hospital | 364 | 280 (77) |
| Location | ||
| Large urban area | 254 | 209 (82) |
| Outside of large urban area | 136 | 94 (69) |
Differences by category are all significant (P < 0.05).
Characteristics of hospital web sites
Many hospitals (57%) had web sites supported by hospital network web sites rather than their own stand-alone sites. Web sites were heterogeneous, with few fulfilling all content areas. Facility contact information (phone numbers and addresses) was consistently reported, but travel information (maps or directions) was less often given (table 2). Most hospitals provided either a referral phone number or online directory for affiliated physicians; 30% listed affiliated physician groups. Nearly half provided some insurance information, including contact numbers for eligible insurance plans, online lists of health plans, or online glossaries to explain health care plan options or terms. Opportunities for volunteering or charitable giving were also listed.
Table 2.
Content of hospital web sites, overall and by hospital profit and network status
| Hospital profit status, no. (%) | Hospital network status, no. (%) | ||||
|---|---|---|---|---|---|
| Web-site content | Overall, no. (%) | Nonprofit | For profit | Network* | Non network |
| Facility information | |||||
| History | 117 (53) | 108 (57) | 9 (28)† | 85 (53) | 32 (51) |
| Mission statement | 145 (65) | 128 (67) | 17 (53) | 99 (62) | 46 (73) |
| No. of beds | 111 (50) | 97 (51) | 14 (44) | 77 (48) | 34 (54) |
| Main phone no. | 207 (93) | 177 (93) | 30 (94) | 151 (95) | 56 (89) |
| Address | 205 (92) | 177 (93) | 28 (88) | 150 (94) | 55 (87) |
| Directions | 81 (36) | 73 (38) | 8 (25) | 60 (38) | 21 (33) |
| Map | 138 (62) | 121 (64) | 17 (53) | 105 (66) | 33 (52) |
| Referral services and assistance | |||||
| Physician referral no. | 124 (56) | 116 (61) | 8 (25)† | 99 (62) | 25 (40)† |
| Online physician directory | 107 (48) | 98 (52) | 9 (28)† | 90 (57) | 17 (27)† |
| Affiliated physician groups | 66 (30) | 61 (32) | 5 (16) | 48 (30) | 18 (29) |
| Financial information | 50 (23) | 48 (25) | 2 (6)‡ | 35 (22) | 15 (24) |
| Insurance information | 107 (48) | 100 (53) | 7 (22)† | 86 (54) | 21 (33)† |
| Contact phone no. | 50 (23) | 46 (24) | 4 (13) | 39 (25) | 11 (18) |
| List of plans | 47 (21) | 38 (20) | 9 (28) | 34 (21) | 13 (21) |
| Glossary of terms | 48 (22) | 49 (26) | 0 (0)† | 43 (27) | 5 (8)† |
| Hospital services | |||||
| Cardiac care | 88 (40) | 77 (40) | 11 (34) | 59 (37) | 29 (46) |
| Women's health | 86 (39) | 78 (41) | 8 (25) | 64 (40) | 22 (34) |
| Obstetric care | 105 (47) | 100 (53) | 5 (16)† | 73 (46) | 32 (51) |
| Cancer care | 87 (39) | 83 (44) | 4 (13)† | 67 (42) | 20 (31) |
| Patient information | |||||
| Overall patient-centered functions | 28 (13) | 27 (14) | 1 (3) | 28 (18) | 0 (0)† |
| Can make appointments | 24 (11) | 24 (13) | 0 (0)† | 24 (15) | 0 (0)† |
| Can e-mail | 26 (12) | 25 (13) | 1 (3) | 26 (16) | 0 (0)† |
| Online health profile | 11 (5) | 11 (6) | 0 (0) | 10 (6) | 1 (2) |
| Product information | |||||
| Product and designer links | 119 (54) | 86 (45) | 17 (53) | 77 (48) | 26 (41) |
| Product links | 96 (43) | 82 (43) | 14 (44) | 76 (48) | 20 (30)‡ |
| Web designer link | 19 (9) | 14 (7) | 5 (16) | 4 (3) | 15 (25)† |
| Exclusive senior “clubs” | 69 (31) | 65 (34) | 4 (13)‡ | 57 (36) | 11 (18)‡ |
| Clinical trials | 48 (22) | 49 (26) | 0 (0)† | 48 (30) | 0 (0)† |
| Accreditation, quality information, and profit status | |||||
| JCAHO | 47 (21) | 36 (19) | 11 (34)† | 31 (19) | 16 (25) |
| Other quality measures | 79 (36) | 72 (38) | 7 (22) | 68 (43) | 11 (17)† |
| Profit status reported | 171 (77) | 167 (88) | 4 (13)‡ | 128 (81) | 43 (68)‡ |
Network hospitals are portions of organizations owning at least 2 hospitals.
P < 0.01.
P < 0.05.
Information regarding particular health care services was limited (tables 2 and 3). Few sites involved patient-centered care features, and these were limited to e-mail or creating online health profiles for personal use (13%). A few reported the availability of cardiac services, women's health services, obstetric care, or cancer care. Although most hospitals (65%) promoted preventive or educational services to the community, few provided contact phone numbers or online course schedules. The most commonly reported courses were for maternity or parenting and cancer support.
Table 3.
Health information content of hospital web sites, overall and by hospital profit and network status
| Hospital profit status, no. (%) | Hospital network status, no. (%) | ||||
|---|---|---|---|---|---|
| Health information content | Overall, no., (%) | Nonprofit | For profit | Network* | Not network |
| Within site reference information | |||||
| All health references | 139 (63) | 122 (64) | 17 (53) | 110 (69) | 29 (46)† |
| Any heart preventive information | 102 (46) | 91 (48) | 11 (34) | 89 (56) | 12 (20)† |
| Healthy heart | 65 (29) | 63 (33) | 2 (6) † | 57 (36) | 7 (11) † |
| Cholesterol | 95 (43) | 84 (44) | 11 (34) | 84 (53) | 10 (16) † |
| Diabetes information | 93 (42) | 83 (44) | 10 (31) | 82 (52) | 10 (16) † |
| Cancer information | 80 (36) | 80 (42) | 0 (0) † | 73 (46) | 6 (10) † |
| Links to other web sites | |||||
| All health links | 107 (48) | 104 (55) | 3 (9) † | 83 (52) | 24 (38)† |
| Any heart disease site links | 29 (13) | 28 (15) | 1 (3) | 24 (15) | 5 (8) |
| Heart | 28 (13) | 27 (14) | 1 (3) | 24 (15) | 4 (6) |
| Cholesterol | 4 (2) | 4 (2) | 0 (0) | 3 (2) | 1 (2) |
| Diabetes site links | 18 (8) | 18 (9) | 0 (0) | 14 (9) | 4 (6) |
| Cancer site links | 43 (19) | 42 (22) | 1 (3) ‡ | 32 (20) | 11 (17) |
| Links to general health sites | 86 (39) | 85 (45) | 1 (3) † | 75 (47) | 11 (17)† |
| Links to other health topic sites | 90 (40) | 89 (47) | 1 (3) † | 77 (48) | 13 (21)† |
| Health classes schedule online | |||||
| Schedule information | |||||
| Any information | 143 (65) | 133 (70) | 11 (34)† | 107 (67) | 37 (59) |
| Phone no. | 75 (34) | 70 (37) | 5 (16)‡ | 52 (33) | 23 (36) |
| Schedule online | 54 (25) | 54 (28) | 1 (3) † | 39 (25) | 15 (25) |
| Class topics | |||||
| Healthy heart | 25 (11) | 21 (11) | 4 (13) | 18 (11) | 7 (11) |
| Tobacco cessation | 63 (28) | 61 (32) | 2 (6) † | 48 (30) | 14 (23) |
| Weight reduction | 49 (22) | 46 (24) | 3 (9) | 37 (23) | 12 (20) |
| Diabetes | 61 (27) | 56 (29) | 5 (16) | 40 (25) | 21 (33) |
| Women's health | 50 (23) | 48 (25) | 3 (9) ‡ | 38 (24) | 12 (20) |
| Maternity and prenatal care | 78 (35) | 72 (38) | 6 (19)‡ | 56 (35) | 22 (34) |
| Cancer support groups | 73 (33) | 69 (36) | 4 (13)‡ | 56 (35) | 17 (26) |
Network hospitals are portions of organizations owning at least 2 hospitals.
P < 0.01.
P < 0.05.
Preventive health care references were common: 70% of sites had either site-specific online health references or links to other health information resources (see table 3). Within hospital web sites, heart care information (either “healthy heart” or cholesterol control) was most common (46%), with diabetes care and cancer screening or treatment less common. Links to other health care web sites included general health information sites or search engines (39%), heart care (13%), diabetes (8%), cancer (19%), and other health conditions (40%).
Although hospitals often provided a mission statement or stated their profit status, few reported JCAHO accreditation or other quality measures. Only 9% of hospitals reported an OSHPD, USNWR, or PBGH quality measure. JCAHO accreditation was reported by 46 of 200 hospitals with neutral or positive scores but by 0 of 4 with negative scores. The OSHPD ranking was reported by 2 of 9 better-than-expected but 0 of 4 worse-than-expected hospitals. Two other hospitals incorrectly reported themselves as better than expected by citing unadjusted rather than adjusted mortality. No hospitals reported neutral OSHPD scores. A few hospitals reported their positive USNWR (4/11) or PBGH (1/5) rankings. USNWR and PBGH do not have negative rankings.
Product advertising was common (54%), but no type predominated—links to web site designers (9%), product links on commercial maps (Yahoo, Mapquest) inserted for directions (10%), and links to travel sites (6%). Hospitals did describe “exclusive” senior benefits clubs (31%) and ongoing clinical trials (22%).
Comparison of web-site characteristics by profit status and network affiliation
For-profit hospitals were less likely to provide web-site content than nonprofit hospital web sites. Nonprofit hospital sites were more likely to report referral information; provide health care information within their site or through web links; report more medical services in general, with obstetric and cancer care in particular; describe health education and prevention classes; and detail their charitable foundations.
The reporting of profit status, accreditation, and quality measures differed between nonprofit and for-profit hospital web sites. Although nonprofit hospitals were likely to report their profit status, for-profit hospitals infrequently provided such information. For-profit hospitals more likely reported JCAHO accreditation whereas non-profit hospitals more frequently reported either OSHPD or other measures. Only 1 for-profit hospital had a positive report from OSHPD, PBGH, or USNWR. Some for-profit hospitals reported selection to the annual HCIA/Mercer 100 Best-Run Hospitals List, a ranking emphasizing financial not clinical performance.
Network and non-network hospital web sites were generally similar. However, network hospital web sites presented more online referral information and health information by within-site reference materials and links to other sites. Some aspects of site sophistication—site search and consumer feedback—were more common among network hospital sites.
DISCUSSION
The Internet presents new opportunities for hospitals. Although most California hospitals had an online presence by early 2000, web sites were often difficult to locate, and site content was inconsistent and underdeveloped. Although many web sites had information for acute and preventive care, some were little more than “bookmarks” that provided minimal information.
Hospitals are marketing care to their potential patients as consumers, emphasizing well care and health information. They provide information on insurance and affiliated physicians and advertise special clubs and benefits to older people, a group of mostly insured, greater users of health care services. Finally, hospitals provide selected information regarding the quality of services at their institutions, an approach common to advertising. For-profit hospitals appear to report JCAHO accreditation because they lack more valid and reliable measures of quality.
Hospitals as health providers have a mission to give the best possible care. Yet, few appear to provide more sophisticated patient-centered integrative care options on their web sites, such as e-mail, scheduling, or health profiles. Lack of functionality is the result of technologic barriers, resource priorities, and privacy concerns.30 Web sites of hospitals with the greatest potential resources, network hospitals, tended to have more sophisticated content and functionality.
Hospitals should clearly present health information and uniformly report services and quality measures. Online health information is of varied quality.1 Efforts have been made to evaluate content and set standards for health care web sites, but given the fluidity of web sites, performing and updating evaluations is a sisyphean task.31 Few sites examined in our study mentioned any particular standards.
Limitations
Internet search engines did not identify web sites among some smaller independent hospitals. The Internet is a fluid medium, and observations made over a short period may not remain valid. California hospitals may not be generalizable.
Conclusions
Hospitals have a presence on the Internet. Yet, examination of their implementation demonstrates the dilemma that both hospitals and consumers face. Hospitals have the opportunity to make these sites true adjuncts to health care, 2-way streets for health information and care. At present, patients and possible consumers should temper their expectations regarding the usefulness of these sites for their health needs. Periodic reexamination of sites is indicated with the rapid changes on Internet web sites.
Summary points
Although most hospitals have web sites, these web sites are frequently difficult to locate
Information provided by web sites is highly variable and often unverifiable
Few web sites adhere to voluntary standards for certification of information presented
Most web sites do not include functions applicable to direct patient care
Hospitals associated with large hospital networks or non-profit status are more likely to provide detailed information and functionality within their web sites
Figure 1.
Identification of California general acute care hospital web sites
Figure 2.
Hospital web sites should uniformly report services and quality measures
Funding: This work was supported in part by a National Research Service Award
Competing interests: None declared
References
- 1.Berland GK, Elliott MN, Morales LS, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA 2001;285: 2612-2621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.California Health Line. Many web users look for medical advice online. California Healthcare Foundation; November 27, 2000.
- 3.Bloom BS, Iannacone RC. Internet availability of prescription pharmaceuticals to the public. Ann Intern Med 1999;131: 830-833. [DOI] [PubMed] [Google Scholar]
- 4.Armstrong K, Schwartz JS, Asch DA. Direct sale of sildenafil (Viagra) to consumers over the Internet. N Engl J Med 1999;341: 1389-1392. [DOI] [PubMed] [Google Scholar]
- 5.Weisbord SD, Soule JB, Kimmel PL. Poison on line: acute renal failure caused by oil of wormwood purchased through the Internet. N Engl J Med 1997;337: 825-827 [erratum published in N Engl J Med 1997;337:1483]. [DOI] [PubMed] [Google Scholar]
- 6.Harmon A. Illegal kidney auction pops up on eBay's site. New York Times September 3, 1999; pp A14(N), A13(L).
- 7.Goldberg C. Egg auction on Internet is drawing high scrutiny. New York Times October 28, 1999; pp A18(N), A26(L). [PubMed]
- 8.Tumlinson A, Bottigheimer H, Mahoney P, Stone EM, Hendricks A. Choosing a health plan: what information will consumers use? Health Aff (Millwood) 1997;16: 229-238. [DOI] [PubMed] [Google Scholar]
- 9.Hibbard JH, Slovic P, Jewett JJ. Informing consumer decisions in health care: implications from decision-making research. Milbank Q 1997;75: 395-414. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moyer CA, Stern DT, Katz SJ, Fendrick AM. “We got mail”: electronic communication between physicians and patients. Am J Manag Care 1999;5: 1513-1522. [PubMed] [Google Scholar]
- 11.Nudelman PM, Andrews LM. The “value added” of not-for-profit health plans. N Engl J Med 1996;334: 1057-1059. [DOI] [PubMed] [Google Scholar]
- 12.Iglehart J. Forum on the future of academic medicine: final session—implications of the information revolution for academic medicine. Acad Med 2000;75: 245-251. [DOI] [PubMed] [Google Scholar]
- 13.Tannery NH, Wessel CB. Academic medical center libraries on the Web. Bull Med Libr Assoc 1998;86: 541-544. [PMC free article] [PubMed] [Google Scholar]
- 14.Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA 1998;279: 1638-1642. [DOI] [PubMed] [Google Scholar]
- 15.Hibbard JH, Jewett JJ. What type of quality information do consumers want in a health care report card? Med Care Res Rev 1996;53: 28-47. [DOI] [PubMed] [Google Scholar]
- 16.Robinson S, Brodie M. Understanding the quality challenge for health consumers: the Kaiser/AHCPR Survey. Jt Comm J Qual Improv 1997;23: 239-244. [DOI] [PubMed] [Google Scholar]
- 17.McGlynn EA. Six challenges in measuring the quality of health care. Health Aff (Millwood) 1997;16: 7-21. [DOI] [PubMed] [Google Scholar]
- 18.Rainwater JA, Romano PS, Antonius DM. The California Hospital Outcomes Project: how useful is California's report card for quality improvement? Jt Comm J Qual Improv 1998;24: 31-39. [DOI] [PubMed] [Google Scholar]
- 19.Scanlon DP, Chernew M, Sheffler S, Fendrick AM. Health plan report cards: exploring differences in plan ratings. Jt Comm J Qual Improv 1998;24: 5-20. [DOI] [PubMed] [Google Scholar]
- 20.Wilkes MS, Bell RA, Kravitz RL. Direct-to-consumer prescription drug advertising: trends, impact, and implications. Health Aff (Millwood) 2000;19: 110-128. [DOI] [PubMed] [Google Scholar]
- 21.Himmelstein DU, Woolhandler S, Hellander I, Wolfe SM. Quality of care in investor-owned vs not-for-profit HMOs. JAMA 1999;282: 159-163. [DOI] [PubMed] [Google Scholar]
- 22.Health, United States, 1998 With Socioeconomic Status and Health Chartbook. Hyattsville, MD: National Center for Health Statistics; 1998.
- 23.Edgman-Levitan S, Cleary PD. What information do consumers want and need? Health Aff (Millwood) 1996;15: 42-56. [DOI] [PubMed] [Google Scholar]
- 24.Kim P, Eng T, Deering M, Maxfield A. Published criteria for evaluating health related web sites: review. BMJ (Clin Res Ed) 1999;318: 647-649. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Luft H, Romano P. Report on Heart Attack, 1991-1993. Vol 1. Sacramento California Office of Statewide Health Planning and Development; 1997.
- 26.Comarow A. How we selected the best hospitals. US News & World Report 2000; p 75.10977760
- 27.Chen J, Radford MJ, Wang Y, Marciniak TA, Krumholz HM. Do “America's Best Hospitals” perform better for acute myocardial infarction? N Engl J Med 1999;340: 286-292. [DOI] [PubMed] [Google Scholar]
- 28.Zeger S, Liang K. Longitudinal data analysis for discrete and continuous outcomes. Biometrics 1986;42: 121-130. [PubMed] [Google Scholar]
- 29.Liang K, Zeger S. Longitudinal data analysis using generalized linear models. Biometrika 1986;73: 13-22. [Google Scholar]
- 30.100 Top Hospitals: Cardiovascular Benchmarks for Success—2000. Available at www.100tophospitals.com/studies/cardio00/winners/ca.html. Accessed August 6, 2000.
- 31.Pennbridge J, Moya R, Rodrigues L. Questionnaire survey of California consumers' use and rating of sources of health care information including the Internet. West J Med 1999;171: 302-305. [PMC free article] [PubMed] [Google Scholar]
- 32.Delamothe T. Quality of websites: kitemarking the west wind. BMJ 2000;321: 843-844. [DOI] [PMC free article] [PubMed] [Google Scholar]