Table: 1.

Criteria to be included in the new cross-institutional scale (ranked by > 70% of participants as "very important" or "important.	% participants that ranked criterion as "very important" or "important"	Type of Assessment for Quantification of the metric
1. Maintains visibility (of tool) while removing bone	100%	Contextual/Procedural
2. Selects appropriate burr type and size	98.4%	Contextual
3. Antrum entered	98.4%	% of structure (volume violated)
4. No violation of facial nerve sheath	98.3%	Proximity - % of volume structure violated
5. Sigmoid sinus is not entered	96.7%	% of volume of structure violated
6. Identifies tympanic segment of the facial nerve	96.6%	% correlated with expert removal Procedural
7. Does not drill on ossicle	91.8%	Proximity - % of volume of structure violated
8. Firm, low, good hand position and grip on drill	91.8%	Procedural
9. Does not use excessive drill force near critical structures	91.8%	Procedural Proximity
10. Identifies the chorda tympani or stump	91.7%	% correlated with expert removal Procedural
11. Drills in best direction (clear understanding of cutting edge)	90.2%	Procedural/Contextual
12. Canal wall up (EAC)	89.8%	% correlated with expert removal
13. Identifies the facial nerve at the cochlearform process	88.1%	% correlated with expert removal
14. Appropriate depth of cavity (Cortex)	88%	% correlated with expert removal
15. Drills with broad strokes	86.9%	Procedural/Contextual Technique
16. No holes in EAC	86.4%	% of volume of structure violated
17. Complete saucerization (Cortex)	83.3%	% correlated with expert removal
18. Posterior canal wall thinned	81.4%	% of volume of structure violated
19. Facial recess completely exposed (overlying bone sufficiently thinned so nerve can be seen, located and safely avoided)	80.4%	% correlated with expert removal
20. Identifies the facial nerve at the external genu	79.3%	% correlated with expert removal Procedural
21. Low frequency of drill "jumps" (jumps defined as drilling further than 1 cm from previous spot)	77%	Procedural (localization of tool)
22. No holes in tegmen	73.8%	% of volume of structure violated
23. Use of diamond burr within 2 mm of facial nerve	73.3%	Proximity/Localization Procedural
24. No cells remain on sinodural angle	72.5%	% correlated with expert removal
25. Sinodural angle sharply defined	71.2%	% correlated with expert removal

Categories of Metrics:

Contextual = Contingent on Sequence (Steps)

Technique = Speed, force, angle, etc.

Localization = Position of tool

Proximity = Distance of tool to structure (s)

Violation = % of structure (volume) removed

Identified = % correlated with expert removal