Table 2.
Summary characteristics of the 13 techniques for assessing quality decision making listed in descending order of total QDMPs evaluated by the technique.
| No. | Refernces | Title | Decision area | Study subject | Subject type | Method | QDMPs evaluated | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | |||||||
| 1 | Matheson and Matheson, 1998 | Organizational IQ test | Corporate | Industry (including pharm.) | Org. | 45-item questionnaire assessing nine principles for strategic decision making in an organization (N = 100s) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| 2 | Donelan et al., 2016 | Quality of Decision Making Orientation Scheme (QoDoS) instrument | Medicines R&D/Reg. Review | Regulatory agency + pharm. industry | Org. + Ind. | Questionnaire with 47 items assessing organizational decision making culture and approach, as well as individual competence and style (N = 76) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| 3 | Mindtools, 2013 | “Good Are Your Decision-Making Skills?” Questionnaire | General | ND | Ind. | Questionnaire, “How Good Are Your Decision-Making Skills?” containing 18 items (N = ND) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |
| 4 | Garbuio et al., 2015 | Survey on strategic decision making | Corporate | Industry (including pharm.) | Org. | Survey (28 items) assessing relationship between robustness of analysis, dialogue, and decision-making effectiveness (N = 634) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||
| 5 | Open University, 2013 | Decision making Questionnaire | General | ND | Org. + Ind. | Questionnaire containing 12 items in three areas: decision-making process, psychological perspective and the role of social influences (N = ND) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | |||
| 6 | Fischer et al., 2011 | A structured tool to analyze coverage decision making | HTA | HTA agencies | Med. | Ten indicators for a structured empirical comparison of coverage decisions with corresponding ordinal rankings (N = 6) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||||
| 7 | Wood, 2012 | Study exploring individual differences in decision-making styles as predictors of good decision making | General | University students | Ind. | Three part study: 1. General Decision-Making Style measure (25 items) 2. The BFI personality test (50 items) 3. Peer ratings of decision-making quality (26 items). (N = 315) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ||||
| 8 | Blenko et al., 2010 | Decision and organizational scorecard | General | ND | Org. | Two web-based questionnaires, with 4 and 10 items respectively assessing decision effectiveness and organizational drivers (N = 1,065) | ✓ | ✓ | ✓ | |||||||
| 9 | Cowlrick et al., 2011 | Questionnaire for assessing perception of risk through phases of medicine R&D | Medicines R&D | Pharm. industry | Ind. | Questionnaire with five sets of judgment statements to assess case studies for four medicines (N = 52) | ✓ | ✓ | ✓ | |||||||
| 10 | McIntyre et al., 2012 | Questionnaire for assessing how US FDA Advisory Committee Members prepare and what influences them | Regulatory Advisory Committee | Regulatory agency (US FDA) | Ind. | 26-item questionnaire assessing US FDA committees' preparatory practices, influencers and preferences (N = 101) | ✓ | ✓ | ✓ | |||||||
| 11 | Marangi et al., 2014 | Survey of the Italian Medicines Agency (AIFA) 2013 | Regulatory Advisory Committee | Regulatory agency (AIFA) | Ind. | Questionnaire, “Survey AIFA 2013” consisting of 17 questions, 4 regarding participant information and 13 assessing influences on AIFA committees (N = 72) | ✓ | ✓ | ✓ | |||||||
| 12 | Beyer et al., 2015 | A field study using the Domain Specific Risk Taking (DOSPERT) scale and the Big Five Jackson Inventory (BFI) scale | Reg. review | Regulatory agency (EU) | Ind. | Three part questionnaire: 1. Demographic data and DOSPERT scale; 2. Medicine case Study; 3. The BFI personality test consisting of 44 items to assess risk perceptions of assessors (N = 75) | ✓ | ✓ | ||||||||
| 13 | Salek et al., 2012 | Scorecards to assess the quality of a regulatory submission and its review | Reg. submission and review | Reg. agencies + pharm. industry | Med. | Two scorecards containing 50 items grouped into seven domains: application format, content of the dossier, labeling, scientific advice, conduct of the review, communication, and overall assessment (N = 4) | ✓ | |||||||||
US FDA, Food and Drug Administration; HTA, health technology assessment; ind., individual; med., medicine; N, sample subject size used in testing; ND, not defined; org., organization; pharm., pharmaceutical; QDMP, quality decision-making practice; reg., regulatory; R&D, research and development; ✓, QDMP evaluated by the technique; the QDMPs are: 1, Have a systematic, structured approach to aid decision making (consistent, predictable and timely); 2, Assign clear roles and responsibilities (decision makers, advisors, contributors); 3, Assign values and relative importance to decision criteria; 4, Evaluate both internal and external influences/biases; 5, Examine alternative solutions; 6, Consider uncertainty; 7, Re-evaluate as new information becomes available; 8, Perform impact analysis of decision; 9, Ensure transparency and provide a record trail; 10, Effectively communicate the basis of the decision.