Table 2.

Comparison of Potential Analysis Methods

	Analysis Method	Description	PROs	CONs
Approach #1	Analyze and report each project separately	Each PNRP project would be separately analyzed and reported as an independent study. There would be no attempt to combine data from different projects.	Accounts for differing study designs	No summary measure of effect Leaves synthesis of findings to readers Summary of overall program may be biased by projects with favorable results.
Approach #2	Individual-level analysis ignoring any intraclass correlation	Combine data from all the projects, analyze using standard methods, ignoring any intraclass correlation All individuals receive equal weight in the analysis. Primary analysis compares mean difference in time to diagnostic resolution for navigated versus control subjects.	All data are utilized – no subjects excluded	Pooling data ignores each project’s internal validity Ignores “group” in group-randomized designs and non-randomized group designs Projects with large sample sizes could overwhelm results
Approach #3	Pool data from projects having similar designs	Data from two projects that used group-randomized designs would be combined and analyzed. Two projects that randomized at individual level would similarly be pooled and analyzed.	Interpretation more straightforward Restricts analysis to projects with true experimental designs Increases generalizability of findings	Only four of the nine projects contribute to the analysis Analyses yield two separate results
Approach #4	Prospective meta analysis	Data from each project analyzed separately to estimate effect size. Effect sizes would be combined using meta-analytic techniques to obtain overall measure of program effect.	All data are available, not just that in published literature Measures of effect for each program are computed Allows comparisons across projects (by design, by type of navigation) Avoids publication bias	Meta-analysis in advance of the publication of project results risks inconsistencies with the reports by the projects
Approach #5	Simulated Group Randomized Design	For projects that did not randomize by group, pairs of matched, and balanced groups will be created from the individual data simulating a group randomized design.	Utilizes all data from nine projects Insures balance between intervention and control arms Allows for control of confounding at level of ‘group’ across projects Efficient use of available data	Untested and novel approach. Less transparent approach Does not reflect original study designs in individual projects. Projects with more groups have greater influence on analysis Small number of observations per group may require combining groups