Table 3.
Proposed core effectiveness metrics for electronic consultation programs, using the Quadruple Aim framework
| Arm of the Quadruple Aim | Measure | Definition | Rationale |
|---|---|---|---|
| Financial | E-consult management | Percentage of e-consults that are not scheduled for a face-to-face visit in the ensuring 12 months/Total number of e-consults per year | Calculate the number of avoidable face-to-face visits |
| Financial | Out of network specialty care requests | Number of out-of-network specialty care requests/Total number of specialty care requests | Examine changes in out-of-network specialty care visits |
| Population Health | Time to third next available new in-person appointment for e-consult specialties | Third next available new patient appointment if patient calls to make appointment | Direct measure of impact on specialty care access |
| Population Health | Demographics of patients who received an e-consult compared to in-person specialty care | Insurance status of patients who received at least one e-consult/Insurance status of all patients who received specialty expertise (e-consult + in-person specialty visits) | Program reach |
| Impact on equity | |||
| Population Health | PCP capacity | Percentage of PCPs who self-report educational value of the e-consult program on a survey | Effectiveness of e-consult |
| Population Health | Number of specialties offering e-consult and what they are | Raw number of specialties offering e-consult | Measure of adoption |
| Population Health | Unclosed loop by PCP | Number of specialist responses that are not read by PCP per year/Total number of specialist responses via e-consult per year | Patient safety; unanticipated impact |
| Population Health | Unclosed loop by Specialist | Number of e-consults that did not receive a specialist response per year/total number of e-consults per year | Patient safety; unanticipated impact |
| Population Health | Average time to e-consult response | Average lapsed number of days between time e-consult was generated and time specialist responded | Access to specialty care |
| Care team experience | PCP satisfaction/dissatisfaction | Percentage of PCPs who report satisfaction with the program on a survey | Program sustainability |
| Care team experience | Specialist satisfaction/dissatisfaction | Percentage of specialists who report satisfaction with the program | Program sustainability |
| Care team experience | Medical Assistant/Nurse/Referral Coordinator satisfaction/dissatisfaction | Percentage of non-MD team primary care team members who report satisfaction with the program | Program sustainability |
| Patient experience | Satisfaction with access to specialty care in general | Percentage of patients who report satisfaction with access to specialty care pre- and post- implementation. | Program sustainability |
| Patient experience | Concerns about limitations in care | N/A | |
| Patient experience | Patient acceptability of having an e-consult | N/A | |
| Patient experience | Travel/time saved by patients for avoided clinic visits | Number of hours that patients must forgo for each in-person visit | Business case for managed care plans |