Table 2.

Summary of Independent Meta-Evaluation of the Swift Worksite Assessment and Translation (SWAT) Project^a

Category	Criteria
Key questions	What are the strengths and weaknesses of the SWAT process, including each of its components in terms of producing its intended results? How can SWAT be improved? How effective is SWAT for producing its intended results? To what degree does the SWAT framework enable an evaluator to produce an evaluation that satisfies accepted program evaluation standards?
Key activities	Before any site visits are conducted, convene an independent expert panel to duplicate the SWAT rating process to assess its efficacy. Review SWAT documents, including protocols and rating forms. Provide "just-in-time" feedback to CDC's SWAT methods development team, especially with regard to the program evaluation standards of the Joint Committee on Standards for Educational Evaluation. "Observe" weekly project telephone calls and attend 4 of the initial 9 SWAT site visits to observe implementation of SWAT protocols. Conduct telephone interviews with SWAT site visitors, project team members, and staff from the sites visited. After the initial 9 site visits, the expert panel stages a duplicate review of the SWAT documents (1-page program summary and full site-visit reports) to assess which steps of the SWAT process are essential in order to judge promising practices.
Key findings	SWAT is an effective method for rapid evaluation of worksite health promotion programs. Sites reported that SWAT was well-organized but took longer than expected. Site staff reported that the SWAT reports, phone calls with the SWAT project team, and dissemination activities were accurate, useful, and of high quality. The primary benefit that accrued to the sites was that SWAT forced sites to be introspective about their programs. Site personnel reported that the nature of the questions and the site-visit protocols required them to think about both their program and their evaluation practices. For some sites, this took the form of an "internal audit," which might not have occurred without SWAT.
Strengths of SWAT process	Used a systematic approach based on recognized evaluation principles and practices. Adhered to program evaluation standards. Had adequate and accurate data, especially considering short amount of time for collection. Included worksite key staff as stakeholders in reviewing and interpreting SWAT site-visit reports and conclusions. Had a service orientation, both to serve public health goals and to provide feedback to worksites on ways to enhance or improve their programs and program evaluation.
Recommendations to strengthen SWAT process	CDC staff should personally communicate with worksite staff to ensure that the purpose of the project and the site visits are fully understood. Reduce bias by interviewing both participants and nonparticipants in worksite health promotion as key informants in the SWAT protocols. Streamline interview protocols to eliminate redundancy in questions. Shorten the time from initial contact with a site to a site visit and reporting.

^a Conducted by Western Michigan University Evaluation Center, Kalamazoo, Michigan (15).