Evaluating Accountability in the Vaccines for Children Program: Protecting a Federal Investment

SYNOPSIS

The Vaccines for Children (VFC) program supplies health-care providers with federally purchased vaccines at no cost for administration to eligible children. Evaluation of vaccine accountability activities ensures appropriate and timely vaccinations are delivered. Program grantees in 50 states, Washington, five large U.S. metropolitan cities, and five U.S. territories and possessions completed a Web-based survey between December 2002 and January 2003 focused on current vaccine accountability operational systems.

Most grantees required providers to complete profiles describing the vaccination needs and demographics of their practices. More than half requested providers use benchmarking data, doses-administered reports, and/or claims or encounter data to determine their VFC program-eligible population size; however, >65% did not have written procedures for investigating and reconciling discrepancies between estimated vaccine needs and actual vaccine-use data. Most grantees had written standard policies requiring providers to report vaccine loss and wastage routinely and to explain why they occurred. Ninety percent of grantees did not have procedures to check providers for fraud and abuse sanctions, and 52% did not have written procedures to address complaints of vaccine fraud and abuse.

These results suggested specific areas in which the Centers for Disease Control and Prevention should work with grantees to improve vaccine accountability practices. As a result, enhancements to the VFC program are being implemented to address these areas and their impact evaluated for their effectiveness in ensuring the continued success of the VFC program in protecting the nation's most vulnerable children and adolescents.

The Vaccines for Children (VFC) program is a federal entitlement program created in 1994 to supply private and public health-care providers with federally purchased vaccines at no cost for administration to eligible children and adolescents in accordance with the Recommended Childhood and Adolescent Immunization Schedule.¹ Eligible children include those who receive Medicaid, lack health insurance, and have American Indian and/or Alaska Native ethnicities. In addition, children who have health insurance that does not cover vaccination are eligible for the VFC program if they are served through a federally qualified health-care center (FQHC) or rural health clinic (RHC). (An FQHC is certified through the Bureau of Primary Health Care of the Health Resources and Services Administration to provide health care to medically underserved populations. Such centers include community and migrant centers, special health facilities for the homeless and people with acquired immunodeficiency syndrome [AIDS], health centers within public housing, and Indian health centers.)

The VFC program is currently the largest child-targeted program administered by the Centers for Disease Control and Prevention (CDC), with nearly $1 billion expended during fiscal year 2004. More than 90% of VFC program funding is used to purchase vaccines. Remaining funds are used for operational activities (e.g., recruitment and enrollment of private providers, vaccine ordering and accountability efforts, vaccine distribution to public clinics and private vaccination providers, and program evaluation). Because of an increase in the number of eligible children as well as increases in the variety and costs of providing vaccines to them, efficient use of the VFC program funds is critical.

The VFC program helps to save Medicaid funds by purchasing vaccines using federal contracts, and, in addition, has accelerated introduction of new vaccines for children and adolescents. Several peer-reviewed studies have also documented that the VFC program has decreased provider referrals, relieved the burden on health department clinics, reintegrated immunization and primary care, and contributed to improving vaccination coverage rates in children.^2–8

State-based immunization programs and other immunization grantees (e.g., large urban areas and U.S. territories and possessions) implement their programs to ensure vaccine accountability. CDC provides tools and technical assistance to all grantees for the development of accountability activities to ensure that: (1) vaccines purchased with VFC program funds are administered only to VFC program-eligible children; (2) vaccine loss and wastage are minimized; and (3) fraud and abuse by immunization providers are mitigated or deterred. In 1997, CDC's National Immunization Program (NIP) enhanced accountability efforts by developing a provider site-visit protocol for grantees and VFC program staff and requiring states to submit annual management reports.

Evaluation of VFC program vaccine accountability activities is critical to ensuring that high-quality immunization services continue to be delivered through appropriate and timely vaccination of all vulnerable children. At the same time, accountability systems focused on vaccination provider practices need to be efficient and simple to implement, and strike a balance between the needs of providers and the VFC program to prevent development of disincentives for participation. Providers in private practices already often absorb a loss when administering vaccines to children enrolled in the Medicaid program because reimbursements for vaccine administration fall below actual expenses incurred for patient visits. If burdensome and expensive accountability systems, and excessive concerns about fraud and abuse associated with extensive auditing practices and unsubstantiated allegations become a part of the program, private providers may withdraw from the VFC program.

To identify ways to improve vaccine accountability, we undertook a study to: (1) determine the adequacy of grantees' current operational systems for vaccine accountability; (2) characterize the degrees of success among grantees at implementing vaccine accountability activities; and (3) enhance the available tools for improving vaccine accountability activities. Study findings will enable CDC to develop strategies and tools to improve accountability.

METHODS

The VFC Program Coordinator or a designated representative of 61 grantees (comprised of health departments in 50 states; Washington; the metropolitan cities of Philadelphia, New York, Chicago, Houston, and San Antonio; and the U.S. territories/possessions of Puerto Rico, Guam, American Samoa, the U.S. Virgin Islands, and the Northern Mariana Islands) was asked to complete a Web-based survey to evaluate their current operational systems for vaccine accountability. The survey was made available on the CDC website on December 9, 2002, and remained accessible for the grantees to complete until January 31, 2003.

The VFC program policies detailed in the 2002 Program Operations Guide were used to formulate questions included in the survey.⁹ The survey was pilot-tested with five grantees, and their suggestions were used to further refine it. The survey was designed to be administered and completed from a Web-page link. Logic checks programmed into the online survey ensured that a maximum number of grantee respondents provided complete responses.

RESULTS

All 61 grantees provided responses to the online survey, and only four required telephone, e-mail, and/or mail follow-up to clarify and/or provide missing responses; thus, there was a 100% response rate. The Table and the following sections summarize survey results based on accountability activities.

Table.

VFC program accountability activities among grantees,^a December 2002 (n=61)

^aGrantees include 50 states; Washington; five large metropolitan cities (Philadelphia, New York, Chicago, Houston, and San Antonio); and five U.S. territories/possessions (Puerto Rico, Guam, American Samoa, the U.S. Virgin Islands, and the Northern Mariana Islands).

^bOutlier practices are those of providers who report significant vaccine losses.

VFC = Vaccines for Children

Provider profile requirements and evaluations

In establishing a provider profile, the VFC program grantees ask vaccination providers to provide them with: (1) information relevant to their practice (e.g., administrative information such as tax and Medicaid identification numbers and enrollment information); (2) a description of the type of facility in which their practice is located (e.g., department of health, FQHC, RHC, hospital, and group or solo practice); (3) the number of children they serve by age; (4) the number of children served by VFC program eligibility status (i.e., Medicaid-eligible, no insurance, American Indian or Alaska Native, and insured but without coverage for vaccination); and (5) the method used to determine the size of their VFC program-eligible population (i.e., benchmarking, doses-administered reports, and registry data). (Benchmarking is a process whereby providers maintain logs in which they record all vaccines administered by type and number of doses, and by patient VFC program eligibility category over a predetermined period of time, e.g., one to three months. These data are used to establish projections of vaccine needs over a 12-month period and assess the appropriateness of vaccine orders placed.)

Of 61 grantees, 57 indicated they required profiles from either public or private providers enrolled in the VFC program; 51 required profiles from both. Of the grantees who required provider profiles, 52 indicated they required them at least annually, and 37 collected all five types of data previously described from providers. Only three of the 57 grantees requiring provider profiles indicated they had no procedures in place for providers to use when determining the size of their VFC program-eligible populations; 38 reported having such a policy that includes use of a combination of benchmarking data, doses-administered reports, claims or encounter data, and “other” methodologies. When asked how they used provider profile data, 49 of the 57 grantees indicated they entered the data into the vaccine management software (VACMAN), then compared it to past vaccine orders or other sources of data to validate the accuracy of providers' VFC program-eligible population estimates, ensure documentation, and estimate annual population.

Vaccine storage and handling management

When queried about the percentage of their project's vaccine reported as expired or wasted for the 2001 calendar year, 56 grantees reported a vaccine wastage level of 1% to 5%; only one project area reported a level of 11% to 15%. In addition, 59 grantees indicated they had a standard policy requiring the VFC program providers to report all incidents of vaccine loss and wastage. Among these grantees, 52 indicated having a written policy in place requiring providers to report incidents of vaccine loss and wastage routinely (e.g., quarterly, semiannually, or annually). In addition, 57 grantees indicated they required providers to explain why incidents of vaccine loss or wastage occurred and entered these reports into a database (e.g., VACMAN).

Forty-six of 59 grantees had established procedures for handling outlier practices (i.e., providers who reported significant vaccine loss, especially if they deviated from the practice patterns of providers in their peer group); however, 38 of 59 did not have procedures for identifying provider practices that had never reported or failed to report vaccine loss or wastage. When asked if data from vaccine loss and wastage reports were used to determine outlier practices, monitor loss/trends for their project, and monitor loss/wastage trends for VFC program-enrolled provider sites, 51 grantees indicated they used the data for at least two of these purposes.

Among the 61 VFC program grantees responding to the survey, 50 indicated they had procedures in place for handling providers who ordered vaccine in amounts greater than supported by the estimated population size included in their profile data. These grantees also checked provider vaccine inventories for unreported expired or wasted vaccine during VFC program site visits.

Vaccine reporting requirements

Of participating grantees, 54 reported returning all expired/wasted vaccine to manufacturers for excise tax credit at least some of the time; 14 of these grantees returned <500 doses and 30 returned >3,000 doses for the 2001 calendar year. Of grantees who returned vaccine, 46 submitted copies of vaccine credits for federally purchased vaccine to NIP at least some of the time.

CDC has no formal policy concerning the type of information grantees require of providers; however, the majority of grantees require both public and private providers to report information on doses administered, vaccine inventory, and expired/wasted vaccines. Thirty-five grantees reported having systems in place to determine how many doses of vaccine go unaccounted for; 22 of these 35 grantees make this determination at least once a year.

Fraud and abuse activities

When grantees were asked if they had procedures in place to check providers for past or current sanctions, 55 of 61 grantees indicated they had none. (A “sanction” is considered to have occurred when an individual or entity is suspended or excluded from participation or otherwise penalized under a state health-care program for reasons that bear on the individual's or entity's professional competence, professional performance, or financial integrity.) In addition, 32 grantees reported they did not have written procedures for addressing complaints of vaccine fraud and abuse. Of the remaining 29 grantees who indicated having written procedures, 23 indicated that their written procedures addressed the conduct of internal preliminary investigations, and reporting/referral to the state Medicaid program/state Medicaid Fraud-Control Unit, the state attorney general's office, and any other agencies responsible for investigating commercial insurance fraud.

When grantees were asked about issues related to fraud and abuse, 47 reported having at least one episode of suspicious vaccine fraud or abuse identified through their project's internal controls (e.g., data analysis and provider site visits) and/or reported by sources external to their immunization program. Since VFC program implementation, 21 of these grantees had investigated at least one episode of fraud that was referred to their Medicaid program or Medicaid Fraud-Control Unit, six had made referrals to their state agency responsible for investigating commercial insurance fraud, and five had at least one episode result in prosecution or felony conviction. Since VFC program implementation, 17 grantees had investigated at least one episode of abuse, which was subsequently referred to their Medicaid program, and 32 had at least one complaint result in disciplinary action taken by their immunization program.

Quality assurance checks

Of 61 grantees, 19 indicated they had a written protocol outlining the steps to be taken in the event that a provider vaccinated children not eligible for publicly purchased vaccine but did not maintain a privately purchased vaccine inventory. Of all 61 grantees, 58 reported using a methodology developed by their project or by CDC to ensure that VFC program-eligibility screening took place during VFC program site visits.

Project self-assessment

More than one-third (22) of grantees expressed being “somewhat confident” that their project areas had adequate systems in place for vaccine accountability (i.e., systems that would be considered acceptable by a state auditor).

CONCLUSIONS

Results from our survey revealed both strengths and areas in need of technical assistance to improve vaccine accountability activities.

Improvements in evaluating provider profiles, vaccine storage and handling, reporting requirements, and fraud and abuse

At the time this survey was fielded, CDC guidance suggested provider profiles be updated at least annually, that grantees have established policies in place to help providers determine the size of the VFC program-eligible population, and that they routinely compare individual provider profile estimates to other data sources to assess the validity of these estimates. However, data from our study indicated that less than two-thirds of participating grantees had established such a policy.

Eighty-eight percent of grantees reported routinely distributing a copy of their vaccine storage and handling management policy to providers, and 83% had established written vaccine loss and wastage reporting policies and procedures. Although CDC encourages grantees to report all instances of vaccine wastage and spoilage, many grantees do not, and thus do not have a clear understanding of how much of their vaccine supply is actually wasted or remains unaccounted for. This lack of accountability could be remedied by encouraging grantees to investigate providers whose vaccine ordering levels are unusually high, and requiring providers to report incidents of wastage routinely, provide explanations for why loss and wastage occurred, and enter this information into a database for tracking purposes. One strategy would involve ensuring that grantees have established policies for and collect data relevant to providers who report significant vaccine losses (i.e., outlier practices) or fail to report vaccine loss or wastage so that loss wastage trends can be monitored and interventions provided when needed.

Monitoring activities should also be established for handling outlier practices of providers who order vaccine in amounts greater than supported by the estimated population size in their profiles; and for checking provider inventories for unreported expired or wasted vaccine during provider site visits.

The survey showed most grantees do not check providers for past or current sanctions. Although CDC has no formal policy for the frequency with which sanction lists should be checked, program guidance suggests that grantees perform this activity at least every other year. In addition, grantees should establish policies and have processes in place to prevent and detect fraud and abuse, and work with their legal counsel and state and federal agencies to detect, investigate, and report allegations of fraud and abuse, and determine the best course of action for pursuing these cases.

To address fraud and abuse, grantees should also make efforts to educate their staff and investigative and enforcement agencies on the proper means of documenting allegations and encourage them to report fraud and abuse cases to their internal legal counsel and bodies, their state Medicaid program and attorney general's office, and any other agencies responsible for commercial insurance fraud.

VFC program changes and enhancements

The vaccine management and accountability needs of grantees have changed since the inception of the VFC program in 1994, and several program enhancements are currently underway to address this change in needs. First, CDC has implemented a number of changes in its guidance to states that directly address some of the findings from this survey. For example, CDC has provided guidance about the seven critical components to be included in a written fraud and abuse policy, and is requiring grantees to submit a copy of this policy to CDC no later than December 31, 2007. Components include identification of staff with lead responsibility for fraud and abuse issues, and incorporation of monitoring for fraud and abuse into the daily program operations. As these policies are submitted and reviewed by CDC, a model policy will be developed that includes best practices to provide technical assistance to grantees. In addition, CDC now requires grantees to report suspected cases of VFC program fraud and abuse referred to an external agency within the state to CDC within two days of the referral to allow for monitoring of allegations at both federal and state levels.

A second area of enhancement is a requirement for annual updating of provider profiles, rather than encouragement of grantees to do so. Assuring accurate information about the population of VFC program-eligible children served is critical for improving accountability and for fulfilling this important entitlement to vulnerable children.

Finally, the VFC program provider site visit questionnaire has been updated to query providers about notification of the immunization program in the event of expired or wasted vaccine. This update will assist grantees in gaining a better understanding of the provider practices in need of additional technical assistance about reporting and minimizing the amount of wasted and expired vaccine.

Another important initiative recently launched to improve vaccine management at federal, state, and local levels is the Vaccine Management Business Improvement Project (VMBIP). The goals of this federal/state collaboration are to: (1) simplify processes for ordering, distributing, and managing vaccines so that responses to public health crises such as disease outbreaks, vaccine shortages, and disruption of vaccine supply can take place more quickly and effectively; (2) implement a more efficient vaccine supply system that will direct vital public health resources away from vaccine distribution and toward public health activities that improve immunization coverage levels; and (3) significantly reduce the time between vaccine order and delivery by enabling direct delivery of vaccines to providers.¹⁰ It is anticipated that VMBIP will also address some of the issues highlighted in this study by requiring spending plans and systematically incorporating business rules into the vaccine order approval process.

Survey limitations

The vaccine accountability survey described here has several limitations. First, while the desired survey respondent was the VFC Program Coordinator, many grantees entrusted other types of personnel with completing the survey. These personnel may not have been the individuals most knowledgeable about their program activities. Second, grantees reported on their program practices relevant to the 2001 calendar year in late 2002 and early 2003; thus, the results reported here may not be reflective of current program activities.

Dissemination of study results

The original intent of this survey was to collect information that would identify best practices from selected grantees to be shared with others and identify grantees in particular need of technical assistance to improve their vaccine accountability activities. These objectives were achieved by disseminating the current survey results. Grantee-specific reports were generated and distributed to each respective grantee, as well as a brief report summarizing the data collected from all grantees, which CDC encouraged grantees to share with all staff members regardless of project site. To expedite dissemination of this information, the report was made available online for grantees to download and review for use in improving their program activities, sharing with enrolled providers, and recruiting additional providers into the program. This information has begun to be used by CDC staff to guide policy decisions, as described by the program enhancements currently being implemented. In addition, study findings highlight areas where additional evaluation and studies are needed to guide future activities.

The VFC program has aided the success of new vaccine introduction, helped reduce provider referrals from children's medical homes, and improved vaccination coverage rates. Nonetheless, the results of this survey revealed that important gaps in accountability exist, and identified where practices can be made more efficient. Of the 61 grantees that responded to the survey, only slightly more than one-third felt somewhat confident that their project had adequate vaccine accountability systems in place. Ongoing program enhancements, including strengthening grantee approaches for addressing fraud and abuse, updating provider profiles annually, focusing on vaccine wastage during provider site visits, and automating the vaccine ordering and approval process as part of VMBIP, will address many of these gaps. Continued efforts to monitor the impact of these enhancements and to address remaining gaps are necessary to ensure that the VFC program is able to meet its goal of serving vulnerable children to the fullest extent possible.