TABLE 2.

Key Findings and Updates from Internal Verification Process

Technique	Implementation	Findings and Changes
Structured“walk-throughs”	Went through main input and output tabs in the model spreadsheet with each face validity expert	Findings, such as wrong market share percentages in the input tab 3 of the model, were reported in the face validity section
Unit tests	Looked through each tab, checked equations, compared data with input data, and searched for errors	Found and corrected mistakes in input data where values were out of order, missing, or misclassified Found and corrected errors in equations that did not account for OHI versus non-OHI after recommendations from the face validity process Changes to improve formatting and readability
Extreme condition tests: sensitivity and subgroup analyses	Examined results for each sensitivity and subgroup analysis. Some of these sensitivity and subgroup analyses had very little utilization Created additional scenarios combining sensitivity analyses, subgroup analyses, and artificial data. Compared these results to what would be expected	Found odd trends over time in budget and budget impact. Traced these back to errors in equations; fixed these errors Found DIV/0 errors in the output tab. Traced these back to the lack of utilization of a drug class for some subset models. Fixed
Traces	Traced back key budget and budget impact results on output tabs When errors were found, we traced errors back to the original input data	See above
Replication tests	Checked that running all macros (e.g., reset buttons, sensitivity analysis buttons, cost driver buttons) would lead to the same results	No errors found
Double programming	Reprogrammed sections of tabs and checked results with original tab Reprogrammed calculation of the baseline year’s net budget in SAS and found errors in data input values on copay tabs	Corrected data input values on copay tabs

DIV/0 = a division by 0 error that occurs in Microsoft Excel; OHI = other health insurance.