Table 1.
Frequency of identified issues, grouped by platform and cost
| Frequency by platform (%) | Frequency by cost (%) | |||||||
|---|---|---|---|---|---|---|---|---|
| Comment | All apps (n = 46) | Android (n = 21) | iOS (n = 25) | P value* | Free (n = 21) | Paid (n = 25) | P value* | |
| Input issues | 44 (96%) | 21 (100%) | 23 (92%) | 0.493 | 20 (95%) | 24 (96%) | 1.000 | |
| Numeric validation lacking | Inputs unconstrained by physiologically or logically plausible limits, for example, negative values accepted. | 42 (91%) | 21 (100%) | 21 (84%) | 20 (95%) | 22 (88%) | ||
| No validation of any input | For example, textual values allowed in all numeric fields. | 24 (52%) | 14 (67%) | 10 (40%) | 12 (57%) | 12 (48%) | ||
| Some validation | Inconsistency reflecting use of validation for only some inputs. | 18 (39%) | 7 (33%) | 11 (44%) | 8 (38%) | 10 (40%) | ||
| Calculation despite missing inputs | For example, correction bolus calculated even though insulin to blood glucose ratio not provided. | 27 (59%) | 14 (67%) | 13 (52%) | 10 (48%) | 17 (68%) | ||
| Ambiguous terminology | Use of ambiguous labeling or descriptions creates risk of misinterpretation. | 22 (48%) | 8 (38%) | 14 (56%) | 8 (38%) | 14 (56%) | ||
| Data entry issues | User data cannot be entered faithfully or are not stored correctly for calculation. | 6 (13%) | 3 (14%) | 3 (12%) | 4 (19%) | 2 (8%) | ||
| Precision issues | For example, a data field for measurements in mmol/L that accepts only whole integers. | 4 (9%) | 1 (4%) | 3 (14%) | 4 (19%) | 0 (0%) | ||
| Data not stored correctly | Data change unexpectedly, for example, transposed into a different data field. | 2 (4%) | 2 (8%) | 0 (0%) | 0 (0%) | 2 (8%) | ||
| Output issues | 31 (67%) | 17 (81%) | 14 (56%) | 0.115 | 14 (67%) | 17 (68%) | 1.000 | |
| Clinical model violation | Calculation does not conform to principles of underlying clinical conceptual model, for example, by assuming an omitted blood glucose implies a measured value of zero. | 22 (48%) | 13 (62%) | 9 (36%) | 10 (48%) | 12 (48%) | ||
| Formula inconsistency † | Calculation does not proceed in accordance with the stated formula. | 3 (14%) | 1 (11%) | 2 (17%) | 0 (0%) | 3 (23%) | ||
| Input–output mismatch | Calculator output is not consistently synchronized with changing user inputs. | 17 (37%) | 9 (43%) | 8 (32%) | 8 (38%) | 9 (36%) | ||
| Automatic calculator | A calculator which normally refreshes output automatically in response to changing inputs does not under certain circumstances. | 3 (7%) | 2 (10%) | 1 (4%) | 0 (0%) | 3 (12%) | ||
| Manual calculator | App does not respond reliably to manual commands to recalculate dose, for example, tapping a “Calculate” button. | 14 (30%) | 7 (33%) | 7 (28%) | 8 (38%) | 6 (24%) | ||
| Other software errors | Software unexpectedly crashes or becomes unresponsive during normal use. | 11 (24%) | 8 (38%) | 3 (12%) | - | 4 (19%) | 7 (28%) | - |
*Two-tailed P value for the comparisons between Android and iOS and Free and Paid apps respectively, calculated using Fisher’s exact test.
† The denominator used for calculating percentages for this issue was the number of apps for which a formula was identified (n = 21, Android 9, iOS 12, Free 8, Paid 13).
Input issues affect the entry of data and increase the risk that calculation will proceed without correct parameters. Output issues affect interpretation of the calculated value. They create a risk of incorrect dose recommendation even when all input parameters have been supplied correctly. While input issues may be amenable to corrective action taken by a user, for example, by double-checking data entry, output issues may only be effectively addressed through changes to the software.