Abstract
Objective
This study estimates the costs and potential savings associated with all eConsult cases completed between 1 April 2014 and 31 March 2015.
Design
Costing evaluation from the societal perspective estimating the costs and potential savings associated with all eConsults completed during the study period.
Setting
Champlain health region in Eastern Ontario, Canada.
Population
Primary care providers and specialists registered to use the eConsult service.
Main outcome measures
Costs included (1) delivery costs; (2) specialist remuneration; (3) costs associated with traditional (face-to-face) referrals initiated as a result of eConsult. Potential savings included (1) costs of traditional referrals avoided; (2) indirect patient savings through avoided travel and lost wages/productivity. Net potential societal cost savings were estimated by subtracting total costs from total potential savings.
Results
A total of 3487 eConsults were completed during the study period. In 40% of eConsults, a face-to-face specialist visit was originally contemplated but avoided as result of eConsult. In 3% of eConsults, a face-to-face specialist visit was not originally contemplated but was prompted as a result of the eConsult. From the societal perspective, total costs were estimated at $207 787 and total potential savings were $246 516. eConsult led to a net societal saving of $38 729 or $11 per eConsult.
Conclusions
Our findings demonstrate potential cost savings from the societal perspective, as patients avoided the travel costs and lost wages/productivity associated with face-to-face specialist visits. Greater savings are expected once we account for other costs such as avoided tests and visits and potential improved health outcomes associated with shorter wait times. Our findings are valuable for healthcare delivery decision-makers as they seek solutions to improve care in a patient-centred and efficient manner.
Keywords: eConsult, referral, access to care, wait times, cost
Strengths and limitations of this study.
The eConsult service has been operational for 5 years and has the largest menu of specialty services available worldwide, allowing for a rich and diverse data set.
Our use of administrative records allowed for accurate calculation of delivery costs.
Our estimate of net potential societal savings is conservative, as it does not include all factors that may result in cost savings (eg, decreases in number of laboratory tests, parking/caregiver costs for patients).
The study population is primarily urban and cannot be generalised to more rural regions.
Introduction
Poor access to specialist care can be detrimental to patients' health. Excessive wait times and delays in care can cause anxiety in patients and their families.1 They can also reduce patients' ability to carry out day-to-day activities and lead to deterioration in patients' overall health as important diagnoses are delayed.2 3 In addition, many tests or services may be duplicated needlessly, causing frustration for patients and providers alike.2
Electronic consultation (eConsult) systems are an innovative approach to potentially improve wait times for specialist care by facilitating secure online communication between primary care providers (PCPs) and specialists.4–7 Such services have been implemented in health regions all over the world, including the USA,8 9 the UK,6 10 Ireland,11 the Netherlands12 and Spain.13 Studies of these services have demonstrated that eConsult is an acceptable form of care, increases access to specialists, reduces the need for face-to-face specialist visits and improves communication between providers.14–18
Given the focus on the ever-increasing costs of delivering health services, there is a need to understand the cost implications of new models of care delivery. There is however a lack of literature assessing the overall costs associated with eConsult.19 20 Much of the research published in the area of telemedicine has been focused on real-time telemedicine and not on asynchronous services such as eConsult, and the resulting cost implications have been worrisome despite the promising benefits. Few telemedicine services have been found to generate cost savings compared with traditional referrals, with the majority of studies finding that greater costs were incurred due largely to the substantial technological investments and infrastructure required to support these systems.21 Among asynchronous systems, the few studies that have been published reported mixed findings.22 Depending on the perspective taken and the cost elements included, eConsult systems have been reported to incur either higher or lower costs when compared with traditional referrals.23–26 Furthermore, the vast majority of these economic evaluations focused solely on single specialty eConsult systems, most often dermatology. As such, the economic impact and potential net societal savings of multispecialty eConsult services remain uncertain.
We previously reported on an economic evaluation of the Champlain BASE eConsult service from the perspective of the payer over the first 3 years of the service being implemented in our health region. While we found that overall the system did not generate cost savings, the average cost per eConsult decreased each year as the number of eConsults conducted increased.27 We estimated that the service had the potential to be cost saving if the annual number of eConsults exceeded 7818.27 Further, we hypothesised that eConsult would generate societal cost savings if the payer and patient perspectives were taken into account.
This paper builds on our prior work by examining if, and to what extent, the Champlain BASE eConsult service can generate potential cost savings from the societal perspective. There is greater emphasis being placed on health system improvements that are patient centred and lead to an improved experience of care. The use of a broader economic lens (societal vs payer only) is a first step in incorporating this approach into the economic analysis of eConsult. The results will be relevant for policymakers and decision-makers interested in the overall economic impact of multispecialty eConsult systems with a patient-centred perspective.
Methods
Study design
This is a costing evaluation from the societal perspective of all eConsults submitted between 1 April 2014 and 31 March 2015. This 1-year time frame represented the fourth year of operation of the service which had reached a stable point from a usage, payment, service delivery and impact perspective.
The eConsult service
Our service was initially launched in 2009–2010 with a small proof of concept,16 which subsequently led to a larger pilot which has now grown into a full service in our health region funded through the Ontario Ministry of Health. Plans are now underway to offer eConsult in other regions of our province and across Canada. The full details of the Champlain BASE eConsult service proof of concept and pilot studies have been published previously.15 16 28
To use the service for a patient, the PCP logs onto a secure web-based platform, completes a four-field electronic form detailing the patient history and their clinical question and any images and attachments as needed, and submits to a specialty service. The eConsult is then assigned to a specialist based on their availability or by rotation. When responding to the eConsult, the specialist has the ability to ask for additional information or clarification, can provide recommendations or suggest a face-to-face consultation. Specialists are asked to reply within a week and are remunerated quarterly at a rate of $200 per hour prorated to their self-reported time spent responding to eConsults.
The Canadian healthcare system
In the Canadian universal healthcare system, access to a specialist physician for a face-to-face visit requires a referral from a PCP or another physician. Several models of specialist remuneration exist across the country; however, in most cases, specialists are remunerated on a fee-for-service basis, whereby they receive a standard fee for each service provided. Physicians may also be compensated through various Alternative Funding Plans, which use more stable payments that typically include a combination of a salary, incentive/bonus payments, population or capitation funding, and additional fee-for-service payments.29 Healthcare spending in Canada is in the top quartile among Organization for Economic Cooperation and Development (OECD) countries, estimated to be $6105 per capita in 2015 (10.9% of gross domestic product).30 Canada ranks last in terms of access to specialist service among commonwealth countries, with 7 out of 10 PCPs reporting difficulty with accessing specialist care.31
Setting
The Champlain health region is one of 14 regional health districts located in the province of Ontario, Canada. The area includes Ottawa (Canada's capital city) and its surrounding communities.32 The region has a population of ∼1.2 million people, half of whom live in Ottawa. The majority of specialist physicians practise in Ottawa, requiring many individuals who live in outlying communities to travel to attend specialist visits. Median wait times in Ontario to see a specialist range from 39 to 76 days for medical specialties and 33 to 66 days for surgical specialties.33
All PCPs (family physicians and nurse practitioners) in the Champlain region are eligible to use the eConsult service for free.
Data collection
The data used in this study were extracted from usage data continuously collected by the eConsult service. Specifically, information on the number of eConsults directed to the different specialty groups, specialists' self-reported completion time for each eConsult and the postal codes of the PCPs' practice location was captured from administrative records. In addition, information about the ‘outcome’ of the eConsult (whether a face-to-face referral was either avoided or not originally contemplated but recommended by the specialist) was collected through a mandatory brief closeout survey that PCPs complete at the conclusion of each eConsult (figure 1).
We choose to focus on a 1-year period encompassing the most recent year of data (April 2014—March 2015). This was done to reflect the costs of a ‘stable service’, as during this period the service experienced minimal growth in terms of providers added to the service and the variety of specialty groups available. Only eConsults submitted from PCPs in the Champlain health region were included (93.1% of total number of eConsult submitted). Specifically, we excluded the eConsults associated with a pilot project in Nunavut (Canada's far north) and also from providers outside our region, as the distance travelled and travel costs for patients in those regions are greater. This was done to also support the assumption that all face-to-face specialist visits would take place in Ottawa.
Cost analysis
The total societal cost of eConsult included estimated direct (ie, costs to the payer) and indirect costs (ie, costs to the patient). Direct costs consisted of three elements: (1) delivery costs, (2) consultation-specific costs and (3) added referral costs. Delivery costs were obtained from administrative records and represent the costs required to support and operate a fully functional eConsult service. Delivery costs include administrative costs, support costs and costs associated with registration and orientation of new users during the study period. Administrative costs consist of time spent on data extraction and usage monitoring, preparation and dissemination of communication materials, and specialist updates. Support costs consist of time spent following up with PCPs/specialists on outstanding cases and processing user support requests. User registration costs consist of staff time to train each new user, which typically takes 45 min to complete. All costs were calculated based on a human resource remuneration rate of $46 per hour. Consultation-specific costs include staff time to assign each new eConsult to the appropriate specialist and the costs required to remunerate specialists for answering eConsults. Added referral costs were calculated based on those eConsults where the PCP was not contemplating a referral, but as a result of eConsult, one was initiated (option 4 in question 2 of figure 1). This cost was estimated by taking the cost of a traditional referral and multiplying it by how many referrals the system generated. To estimate what the cost of a traditional referral would have been, we used the general listing consultation fees for each specialty group set by the Ontario Ministry of Health and Long-Term Care.34 All eConsult costs were estimated in 2015 Canadian dollars and costs were not discounted due to the short time horizon. We did not include start-up costs of the eConsult service, which are reported elsewhere,27 as they are a one-time cost in the first year of the service based on an existing regional platform that was already deployed for a number of other applications. Costs to the patient associated with travel time and lost wages/productivity are also included as described below.
The total potential savings due to eConsult were calculated based on direct (ie, savings for the payer) and indirect savings (ie, savings for the patient). Potential direct savings consisted of all referrals that were avoided due to eConsult (option 1 in question 2 of figure 1). For each referral that was avoided, we multiplied the number of referrals to each specialty group by the cost of a traditional referral. Potential indirect savings were calculated only for those referrals that were avoided for patients under the age of 65 and consisted of (1) travel costs and (2) costs associated with lost wages/productivity. Travel costs were estimated first by calculating the hypothetical driving distance between the patient location and Ottawa using Google Maps. We used the PCPs' practices' postal code as a proxy for the patient's location since patient address information is not collected through the system. We summed the total round-trip distance travelled and estimated the cost of gas as $0.54 per km in 2015 dollars.35 To estimate the potential savings in terms of lost wages/productivity, we assumed that patients who were living in rural areas would need 1 day off work and those living in urban areas would need half a day off work. Rurality was defined based on the Ontario Medical Association's Rurality Index of Ontario (RIO).36 The RIO score was used to categorise practices as either urban or rural. However, not all patients who attend specialist appointments would be required to take time off work; we assumed based on a patient waiting room survey that only 44% of urban and rural patients would have had to miss work to attend their appointment.37 Furthermore, we calculated wage/productivity losses only for patients under 65 at the time the eConsult was submitted. To be conservative, our calculations assumed no patient savings for those over 65. This assumption clearly suggests that no one in that age group is working and thus none are missing work to attend medical appointments. Ignoring patient savings for this group necessarily underestimates the potential cost savings associated with eConsult. According to Statistics Canada, 78% of Canadians aged 15–65 and 13.4% of Canadians older than 65 participate in the labour force.38 39 The methodology above made use of an estimate of the proportion of patients who need to miss work to attend their appointments. If we assume that the same proportion of patients needed to miss work in both groups, we would then add the productivity losses for 8% of patients 65 and older. To estimate the impact of including patient savings for those 65 and older on our results, we calculated the cost savings using the same methodology as above, but including the travel for patients 65 and over and the productivity loss for 8% of patients 65 and over. Median income in the province of Ontario was used to estimate lost wages/productivity.40
Similarly, indirect costs incurred by added referrals were estimated through assessing travel costs and lost wages/productivity. Although this was done for consistency, these costs do not reflect wasteful expenditure. Rather, through the eConsult process, a referral was initiated where one was never contemplated by the PCP. These eConsults constitute the avoidance of a medical referral delay, which, had it gone unnoticed, may have resulted in even greater costs to the patient and/or the healthcare system in terms of poorer outcomes or increased healthcare usage. We calculated the impact of excluding these added referral costs on the total potential cost savings of the eConsult service.
The overall net potential societal cost savings were estimated by subtracting the total costs (direct and indirect costs) from the total savings (direct and indirect savings). All costs are reported in Canadian dollars and were adjusted to 2015 dollars using the Consumer Price Index.41
In our main analysis, we assumed that all recommended referrals would be attended. Less than full compliance would affect our estimated societal savings by overestimating the potential cost savings associated with avoided referrals and the additional costs associated with added referrals (recall that costs in both cases include the referrals themselves and the indirect patient savings/costs associated with saved/lost wages and productivity). As we do not collect follow-up data on patients after the eConsult was completed and thus have no reliable information on actual compliance rates, we instead calculated the compliance rate below which eConsult would not generate any cost savings, that is, the compliance rate at which the costs and savings are equivalent. That is:
The findings of this calculation are reported separately in the Results section.
Results
A total of 3487 eConsults were completed between 1 April 2014 and 31 March 2015 by specialists from 44 different specialty groups. In 39.9% (n=1393) of eConsults, a referral was originally contemplated by the PCP but, as a result of the eConsult, a face-to-face referral was no longer required. In 109 (3.1%) eConsults, a face-to-face referral was not originally contemplated but as a result of eConsult, one was initiated. A total of 377 (10.8% of total) eConsults originated from PCPs practicing in rural areas located at an average distance of 128.0 km (SD=42.2) from Ottawa. The median patient age was 46.9 years (IQR 28.1–62.5).
Costs due to eConsult
The total direct and indirect costs of eConsult were $202 735 and $5052, respectively; the overall societal cost was $207 787. Seventy-eight per cent of the costs to the payer were attributed to specialty remuneration fees while the remaining costs were related to operational costs associated with running the service (figure 2). Two hundred and thirty-six new users were registered during the year, amounting to $8142 in human resource time. Additionally, support and administrative costs amounted to $15 078 and $6615, respectively. The cost of the 109 referrals initiated as a result of eConsult amounted to $13 873.
Savings due to eConsult
The total direct and indirect cost savings of eConsult were $177 909 and $68 607, respectively. Direct cost savings resulted from 1393 specialist referrals that were no longer needed as a result of eConsult. Indirect savings were only calculated for those who were under the age of 65. This amounted to 1105 eConsults, which resulted in patients avoiding an estimated 61 877 km of travel and saving $33 787. Had these referrals not been avoided, we estimated that ∼47 patients from rural areas would have missed 1 day of work and 934 would have missed half a day, leading to a savings of $34 820.
The total societal savings, less the total societal costs, resulted in an overall net savings of $38 729. This amounts to a saving of $11 per eConsult (figure 2). We found that at a compliance rate lower than 83%, the societal costs are greater than the societal savings.
Exclusion of added referral costs
Excluding added referral costs decreases the total costs to the payer by $13 873 and to the patient by $5052. The resulting net cost savings is $57 654, or $17 per eConsult. This increases the overall societal cost savings per eConsult by 55%.
Inclusion of patients aged 65 and older
Including patient savings for patients aged 65 and over increases the cost savings by $9700–48 430.
Discussion
Healthcare costs are increasing and there is a need to innovate and implement new models of care delivery that not only improve care but are also cost effective.42 Our findings demonstrate cost savings for eConsult from the societal perspective attributable to patient avoided costs, as patients whose PCPs had originally considered a referral but ultimately chose not to refer them avoided the travel costs and lost wages/productivity associated with face-to-face specialist visits. Our results are conservative, in that we only attributed lost wages and avoided travel in those under 65 years, and we also expect greater savings will be incurred once costs associated with system usage metrics are accounted for, such as a potential reduction in imaging, laboratory testing, healthcare visits, and potential further deterioration of patient condition while awaiting face-to-face visits. Furthermore, health systems who currently subsidise patient travel for specialist care43 44 could see substantial cost savings through the implementation of eConsult. For example, the Government of the Northwest Territories spent $30.5 million dollars in 2013–2014 related to medical travel. A reduction of 40% of these transfers would create a cost saving of $12 million dollars in 1 year alone.45 In such cases, avoiding even one visit by accessing a specialist through an eConsult service has the potential to generate substantial overall system cost savings as well as saving the patient from burdensome travel that may, at times, be for one brief face-to-face specialist visit yet take longer than 1 day to attend. Furthermore, the trip will generate substantial out-of-pocket expenses for the patient and their family, not to mention the additional stress and anxiety associated with travelling.
While few other economic evaluations of eConsult have been published in the literature,19 the studies we identified support our findings of eConsult's potential cost effectiveness from a societal perspective. A study of an eConsult service in the US Veteran Affairs (VA) setting found that, within a randomised controlled trial, the costs for teledermatology were comparable to traditional referrals from the payer perspective and lower from a societal perspective.26 Likewise, an economic evaluation of a store-and-forward teledermatology service in Spain found the service to be cost saving from a societal perspective. The study found a cost ratio of 1.6 between teledermatology and conventional care, with electronic consultations costing €79.78 per patient and conventional consultations €129.37 per patient.46
This study has several strengths. We used our fourth year of data from the service at which point the system was stable in terms of avoidance of referral rates, system costs and remuneration costs. As well, the service was being used by over 50% of the PCPs in the region who had access to the largest selection of specialty services in an eConsult system worldwide. By using administrative records, we were able to calculate our delivery costs accurately and only included the human resource time that was needed to fully support the eConsult service. By estimating the costs to the patient, we have provided a more complete overview of the potential cost savings of the eConsult service. Additionally, our estimates on the overall net societal savings are conservative in that we did not include all factors that may generate cost savings. We would have demonstrated a greater cost saving if some additional, harder to quantify elements had been accounted for, such as decreases in the number of laboratory tests/repeated primary care visits, and lowered risk of further deterioration of patient condition while waiting to see the specialist. Furthermore, our calculations assume no productivity losses associated with patients 65 and over.
Our study has limitations. The population distribution of our health region is concentrated primarily in urban areas32 with outlying communities up to 2 hours away by vehicle. Many other regions in Canada face limited access to care and require much longer commutes to access such services. Therefore, our results may not be generalisable to those regions. We made simplifying assumptions in order to estimate some of the included costs, such as using the PCP postal code as a proxy for the patient address. Patients do not necessarily live within a short distance of their PCP, making the PCP address only a rough approximation of distance in some cases. Notably, since rural patients are more likely required to travel to visit urban doctors, we likely underestimated travel and lost wages/productivity costs. Additionally, we did not account for other costs potentially borne by patients when attending their specialist appointments including parking, meal cost, caregiver costs, hotel, nor were we able to include costs related to laboratory tests or medication, which likely further underestimates the potential cost savings to patients. The eConsult service operates on a multipurpose regional platform, thus there is no direct costs associated with operation or maintenance of the shared infrastructure that is directly attributable to the service itself.
Our findings represent an important contribution to the emerging literature evaluating the economic benefits and potential cost savings associated with eConsult. As healthcare delivery decision-makers seek solutions to improve care delivery in an efficient manner, an understanding of the cost implications and the overall potential savings is important. This is especially true for innovations incorporating novel technology, as they often require a substantial initial investment. Our approach quantifies costs from a broad prospective and includes the patient experience. We have also adopted a conservative stance when calculating potential savings. For instance, we included cases where a referral was not originally considered but ultimately requested as added costs. However, it is worth considering what outcomes this cohort would have experienced had they not received a referral at this early stage. It is likely they may have incurred greater costs to the patient and/or the healthcare system in terms of poorer outcomes or increased healthcare usage such as an emergency department visit.47
Future research should examine and monetise other ways eConsult could lead to cost savings including the avoidance of unnecessary medical tests and procedures and the overall costs of a delayed medical referral.
Conclusion
This study found that the multispecialty Champlain BASE eConsult service generated cost savings from the societal perspective; total estimated societal costs were estimated at $207 787, while estimated savings were $246 516, leading to a net societal savings of $38 729 ($11 per eConsult) over a 1-year period. Our findings suggest that eConsult services have the potential to lower the costs for the healthcare system and make care more affordable for patients by reducing indirect costs of care. Patient-centred health services should consider incorporating eConsult services into their practices in order to reduce the economic burden of care.
Acknowledgments
The authors would like to thank all the primary care providers and specialists who participated in the service. They also wish to thank Winchester District Memorial Hospital who hosts the servers, The Ottawa Hospital, and the Champlain Local Health Integration Network.
Footnotes
Contributors: CL conceived of, helped collect the data, participated in the analysis, synthesis, drafting of the paper and final approval of submitted manuscript. PD contributed to the data analysis, drafting of the manuscript and approved the final copy of the submitted manuscript. CDA contributed to the data analysis plan, helped revise the manuscript and approved the final manuscript. FM contributed to the data analysis, drafting of the manuscript and approved the final copy of the submitted manuscript. AA provided input related to operational details of the service, participated in the data analysis, critically revised the manuscript and approved the final copy of the manuscript. EK contributed to the establishment of the service, study design, data analysis, review of the manuscript and final approval of the manuscript.
Funding: Funding for this study was provided by the Ontario Ministry of Health and Long-Term Care's Health Service Research Fund. The authors affirm their independence from these funders. The views expressed are those of the author and do not necessarily reflect those of the funder. The funders played no part in the study design, collection, analysis or interpretation of the data, in the writing of the report or in the decision to submit the article for publication. All authors had full access to all the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Ethics approval: This study was approved by the Ottawa Health Science Network Research Ethics Board (Protocol # 2009848-01H).
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: No additional data are available.
Transparency declaration: The lead author, CL, affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained. All authors had full access to the data and can take responsibility for the integrity of the data and the accuracy of the analysis.
References
- 1.Barua B, Esmail N. Waiting your turn: wait times for health care in Canada. Fraser Institute, 2013. https://www.fraserinstitute.org/sites/default/files/waiting-your-turn-2013.pdf [Google Scholar]
- 2.Day B. The consequences of waiting. In: Globerman S, ed. Reducing wait times for health care—What Canada Can Learn From Theory and International Experience. Fraser Institute, 2013. https://www.fraserinstitute.org/sites/default/files/reducing-wait-times-for-health-care.pdf [Google Scholar]
- 3.Barua B, Esmail N, Jackson T. The effect of wait times on mortality in Canada. Fraser Institute, 2014. https://www.fraserinstitute.org/sites/default/files/effect-of-wait-times-on-mortality-in-canada.pdf [Google Scholar]
- 4.North F, Uthke LD, Tulledge-Scheitel SM. Integration of e-consultations into the outpatient care process at a tertiary medical centre. J Telemed Telecare 2014;20:221–9. 10.1177/1357633X14533886 [DOI] [PubMed] [Google Scholar]
- 5.Palen TE, Price D, Shetterly S et al. Comparing virtual consults to traditional consults using an electronic health record: an observational case-control study. BMC Med Inform Decis Mak 2012;12:65 10.1186/1472-6947-12-65 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Stoves J, Connolly J, Cheung CK et al. Electronic consultation as an alternative to hospital referral for patients with chronic kidney disease: a novel application for networked electronic health records to improve the accessibility and efficiency of healthcare. Qual Saf Health Care 2010;19:e54 10.1136/qshc.2009.038984 [DOI] [PubMed] [Google Scholar]
- 7.Kim-Hwang JE, Chen AH, Bell DS et al. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med 2010;25:1123–8. 10.1007/s11606-010-1402-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hsueh MT, Eastman K, McFarland LV et al. Teledermatology patient satisfaction in the Pacific Northwest. Telemed J E Health 2012;18:377–81. 10.1089/tmj.2011.0181 [DOI] [PubMed] [Google Scholar]
- 9.McFarland LV, Raugi GJ, Reiber GE. Primary care provider and imaging technician satisfaction with a teledermatology project in rural Veterans Health Administration clinics. Telemed J E Health 2013;19:815–25. 10.1089/tmj.2012.0327 [DOI] [PubMed] [Google Scholar]
- 10.Bowns IR, Collins K, Walters SJ et al. Telemedicine in dermatology: a randomised controlled trial. Health Technol Assess 2006;10:1–39. 10.3310/hta10430 [DOI] [PubMed] [Google Scholar]
- 11.Williams L, O'Riordan S, McGuigan C et al. A web-based electronic neurology referral system: a solution for an overburdened healthcare system? Ir Med J 2012;105:301–3. [PubMed] [Google Scholar]
- 12.Scherpbier-de Haan ND, van Gelder VA, Van WC et al. Initial implementation of a web-based consultation process for patients with chronic kidney disease. Ann Fam Med 2013;11:151–6. 10.1370/afm.1494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moreno-Ramirez D, Ferrandiz L, Nieto-Garcia A et al. Store-and-forward teledermatology in skin cancer triage: experience and evaluation of 2009 teleconsultations. Arch Dermatol 2007;143:479–84. 10.1001/archderm.143.4.479 [DOI] [PubMed] [Google Scholar]
- 14.Straus SG, Chen AH, Yee H et al. Implementation of an electronic referral system for outpatient specialty care. AMIA Annu Symp Proc 2011;2011:1337–46. [PMC free article] [PubMed] [Google Scholar]
- 15.Keely E, Liddy C, Afkham A. Utilization, benefits, and impact of an e-consultation service across diverse specialties and primary care providers. Telemed J E Health 2013;19:733–8. 10.1089/tmj.2013.0007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Liddy C, Rowan MS, Afkham A et al. Building access to specialist care through e-consultation. Open Med 2013;7:e1–8. [PMC free article] [PubMed] [Google Scholar]
- 17.Liddy C, Afkham A, Drosinis P et al. Impact of and satisfaction with a new eConsult service: a mixed methods study of primary care providers. J Am Board Fam Med 2015;28:394–403. 10.3122/jabfm.2015.03.140255 [DOI] [PubMed] [Google Scholar]
- 18.Eminovic N, de Keizer NF, Wyatt JC et al. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial. Arch Dermatol 2009;145:558–64. 10.1001/archdermatol.2009.44 [DOI] [PubMed] [Google Scholar]
- 19.Vimalananda VG, Gupte G, Seraj SM et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare 2015;21:323–30. 10.1177/1357633X15582108 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Liddy C, Drosinis P, Keely E. Electronic consultation systems: worldwide prevalence and their impact on patient care—a systematic review. Fam Pract 2016. [DOI] [PubMed] [Google Scholar]
- 21.Wade VA, Karnon J, Elshaug AG et al. A systematic review of economic analyses of telehealth services using real time video communication. BMC Health Serv Res 2014;10:233 10.1186/1472-6963-10-233 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Deshpande A, Khoja S, Lorca J et al. Asynchronous telehealth: a scoping review of analytic studies. Open Med 2009;3:e69–91. [PMC free article] [PubMed] [Google Scholar]
- 23.Eminovic N, Dijkgraaf MG, Berghout RM et al. A cost minimisation analysis in teledermatology: model-based approach. BMC Health Serv Res 2010;10:251 10.1186/1472-6963-10-251 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Whited JD, Datta S, Hall RP et al. An economic analysis of a store and forward teledermatology consult system. Telemed J E Health 2003;9:351–60. 10.1089/153056203772744671 [DOI] [PubMed] [Google Scholar]
- 25.Pak HS, Datta SK, Triplett CA et al. Cost minimization analysis of a store-and-forward teledermatology consult system. Telemed J E Health 2009;15:160–5. 10.1089/tmj.2008.0083 [DOI] [PubMed] [Google Scholar]
- 26.Datta SK, Warshaw EM, Edison KE et al. Cost and utility analysis of a store-and-forward teledermatology referral system: a randomized clinical trial. JAMA Dermatol 2015;151:1323–9. 10.1001/jamadermatol.2015.2362 [DOI] [PubMed] [Google Scholar]
- 27.Liddy C, Deri Armstrong C, Drosinis P et al. What are the costs of improving access to specialists through eConsultation? The Champlain BASE experience. Stud Health Technol Inform 2015;209:67–74. [PubMed] [Google Scholar]
- 28.Liddy C, Maranger J, Afkham A et al. Ten steps to establishing an e-consultation service to improve access to specialist care. Telemed J E Health 2013;19:982–90. 10.1089/tmj.2013.0056 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.HealthForceOntario. Family Practice Models 2015. http://www.healthforceontario.ca/en/Home/Physicians/Training_%7C_Practising_Outside_Ontario/Physician_Roles/Family_Practice_Models (accessed 1 Dec 2015).
- 30.Canadian Institute for Health Information. National Health Expenditure Trends, 1975 to 2015, 2015. https://www.cihi.ca/sites/default/files/document/nhex_trends_narrative_report_2015_en.pdf (accessed 1 Dec 2015).
- 31.Canadian Institute for Health Information. How Canada Compares: Results From The Commonwealth Fund 2014 International Health Policy Survey of Older Adults. https://www.cihi.ca/sites/default/files/document/commonwealth_fund_2015_pdf_en.pdf (accessed 16 May 2016).
- 32.Champlain LHIN. Population Characteristics for Champlain Health Link Areas. http://www.champlainlhin.on.ca/~/media/sites/champlain/Accountability/Integration/HL%20Docs/201410ChHLAsPopCharEN.pdf?la=en (accessed 16 May 2016).
- 33.Jaakkimainen L, Glazier R, Barnsley J et al. Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Fam Pract 2014;15:16 10.1186/1471-2296-15-16 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Schedule of Benefits for Physician Services under the Health Insurance Act. Ministry of Health and Long-Term Care, 2014. http://www.health.gov.on.ca/english/providers/program/ohip/sob/physserv/a_consul.pdf (accessed 1 Dec 2015).
- 35.Automobile allowance rates. Canada Revenue Agency, 2014. http://www.cra-arc.gc.ca/tx/bsnss/tpcs/pyrll/bnfts/tmbl/llwnc/rts-eng.html (accessed 1 Dec 2015).
- 36.Kralj B. Measuring rurality for purposes of health care planning: an empirical measure for Ontario. Ontario Medical Association, 2005. [Google Scholar]
- 37.Keely E, Traczyk L, Liddy C. Patient perspectives on wait times and the referral-consultation process while attending a tertiary diabetes and endocrinology centre: is eConsultation an acceptable option? Can J Diabetes 2015;39:325–9. 10.1016/j.jcjd.2014.12.010 [DOI] [PubMed] [Google Scholar]
- 38.Statistics Canada. Labour force survey estimates (LFS), by sex and age group, seasonally adjusted and unadjusted, annual (persons unless otherwise noted). Statistics Canada, 2016. http://www5.statcan.gc.ca/cansim/a26?id=2820087 (accessed 16 May 2016). [Google Scholar]
- 39.Statistics Canada. Labour force survey estimates (LFS), by sex and detailed age group. Statistics Canada, 2016. http://www5.statcan.gc.ca/cansim/a26?lang=eng&id=2820002 (accessed 16 May 2016). [Google Scholar]
- 40.Individuals by total income level, by province and territory (Ontario). Statistics Canada, 2014. http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/famil105g-eng.htm (accessed 21 Aug 2015). [Google Scholar]
- 41.Statistics Canada. Consumer Price Index, by province (monthly). Statistics Canada, 2016. http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/cpis01g-eng.htm (accessed 12 Apr 2016). [Google Scholar]
- 42.Berwick D, Nolan T, Whittington J. The triple aim: care, health and cost. Health Aff (Millwood) 2008;27:759–69. 10.1377/hlthaff.27.3.759 [DOI] [PubMed] [Google Scholar]
- 43.Ambulance Services. Department of Health and Community Services 2015. http://www.health.gov.nl.ca/health/findhealthservices/ambulance_services.html#air (accessed 12 Nov 2015).
- 44.Medical Travel. Northwest Territories Health and Social Services, 2015. http://www.hss.gov.nt.ca/health/nwt-health-care-plan/medical-travel (accessed 12 Nov 2015).
- 45.Measuring Success and Focusing on Results—NWT Health and Social Services System 2013/2014 Annual Report. Northwest Territories Health and Social Services, 2015. http://www.hss.gov.nt.ca/sites/default/files/nwt_health_and_social_services_2013–2014annual_report.pdf [Google Scholar]
- 46.Moreno-Ramirez D, Ferrandiz L, Ruiz-de-Casas A et al. Economic evaluation of a store-and-forward teledermatology system for skin cancer patients. J Telemed Telecare 2009;15:40–5. 10.1258/jtt.2008.080901 [DOI] [PubMed] [Google Scholar]
- 47.Olayiwola JN, Anderson D, Jepeal N et al. Electronic consultations to improve the primary care-specialty care interface for cardiology in the medically underserved: a cluster-randomized controlled trial. Ann Fam Med 2016;14:133–40. 10.1370/afm.1869 [DOI] [PMC free article] [PubMed] [Google Scholar]