Abstract
Objective: Asynchronous telepsychiatry (ATP) is an integrative model of behavioral health service delivery that is applicable in a variety of settings and populations, particularly consultation in primary care. This article outlines the development of a training model for ATP clinician skills.
Methods: Clinical and procedural training for ATP clinicians (n = 5) was provided by master's-level, clinical mental health providers developed by three experienced telepsychiatrists (P.Y. D.H., and J.S) and supervised by a tele-psychiatrist (PY, GX, DL) through seminar, case supervision, and case discussions. A training manual and one-on-one sessions were employed for initial training. Unstructured expert discussion and feedback sessions were conducted in the training phase of the study in year 1 and annually thereafter over the remaining 4 years of the study. The notes gathered during those sessions were synthesized into themes to gain a summary of the study telepsychiatrist training recommendations for ATP interviewers.
Results: Expert feedback and discussion revealed three overarching themes of recommended skill sets for ATP interviewers: (1) comprehensive skills in brief psychiatric interviewing, (2) adequate knowledge base of behavioral health conditions and therapeutic techniques, and (3) clinical documentation, integrated care/consultation practices, and e-competency skill sets. The model of training and skill requirements from expert feedback sessions included these three skill sets. Technology training recommendations were also identified and included: (1) awareness of privacy/confidentiality for electronic data gathering, storage, management, and sharing; (2) technology troubleshooting; and (3) video filming/retrieval.
Conclusions: We describe and provide a suggested training model for the use of ATP integrated behavioral health. The training needs for ATP clinicians were assessed on a limited convenience sample of experts and clinicians, and more rigorous studies of training for ATP and other technology-focused, behavioral health services are needed. Clinical Trials number: NCT03538860.
Keywords: asynchronous, primary care, education, telepsychiatry, training
Introduction
Asynchronous telepsychiatry (ATP) is a psychiatric consultation that is conducted asynchronously or not in real time. The ATP process is embedded within a primary care or outpatient setting without requiring an onsite psychiatrist. Behavioral health care can be maintained in the primary care setting, and the need for an in-person or live telemedicine appointment with an onsite or offsite psychiatrist for medication management can be largely eliminated for many patients.
ATP uses a behavioral health clinician who meets with the patient in the primary care clinic. The clinician, who is trained to do a semistructured ATP interview, receives the referral information from the primary care provider (PCP). They meet with the patient and discusses the patients history, concerns, and other pertinent information in a video-recorded session that is then sent electronically, through a secure cloud-based system, to a consulting psychiatrist. The psychiatrist then reviews the video and any other patient data available and sends a detailed consultation note back to the PCP.1,2 The two providers can also consult through electronic medical record messaging or phone1,2 (Fig. 1).
The idea behind the ATP care model is twofold: (1) to provide efficient and high-quality, distance psychiatric consultation at any time and (2) better integrate mental health care efficiently with the use of technology. A 2019 systematic review of ATP found that asynchronous services can be feasibly integrated into clinical care settings, improve access to care, provide equivalent satisfaction ratings as face-to-face care, and may reduce the cost of services.3 Many asynchronous options are now being added to in-person and telepsychiatric care conducted through live video (e.g., asynchronous video, internet-based care, e-mail/telephone, and mobile apps) often in collaborative, stepped, and integrated care models.4 ATP is currently the best studied, most reliable, and valid asynchronous option to diagnose and help PCPs initiate treatment and can be utilized across languages.5
Models of integrated care have been proposed to address behavioral health care needs for primary care populations,6,7 those with substance misuse or severe mental illness, and those with comorbid chronic conditions.7,8 Many health care systems are working to increase clinical operating efficiencies by integrating care and providing care at multiple points of service.9 The ATP model is consistent with established integrated care models (Table 1) and overcomes organizational and administrative barriers that can impact nonvirtual, integrated care models.10,11 ATP applied broadly can make the delivery of integrated care more accessible and less costly than other modalities,12 particularly for patients with comorbid chronic diseases and mental illness.13,14 With the increasing use of asynchronous telemedicine and other technology-based care, clinicians need a versatile skill set and training model to address these new and changing models of care.
Table 1.
COMPONENTS OF INTEGRATED CARE13,20 | DESCRIPTION OF THE ASYNCHRONOUS TELEPSYCHIATRY MODEL1,16 |
---|---|
1. Meet patients where they seek health care | Psychiatric services are offered in a flexible setting, which can be completed in the patient's home primary care clinic, behavioral health center, or in the home, according to the patient's preference. |
2. Identify those at risk and offer preventative or early intervention services for emerging, mild, or moderately severe problems | PCPs identify at-risk patients and refer them for an ATP consultation. The PCP and consulting psychiatrists develop a coordinated care plan, including the level of care needed for the patient (e.g., assess if they need to be followed up and how frequently; referrals for therapy; long-term medication management; and patient education). Those at risk are identified and a stepped care plan is developed to address the patient's care relative to need. |
3. For those patients who have problems that are too severe or complex to manage effectively, link or coordinate care with specialists | More severe cases are identified and are referred to be managed directly in the behavioral health setting (either inpatient or outpatient) and mild to moderate cases are managed in the primary care setting with a graduated level of consultation from the psychiatrist relative to patient and PCP need. |
ATP, asynchronous telepsychiatry; PCP, primary care provider.
Development of Clinical Training Requirements for ATP
Two primary approaches to training have been developed over the course of several studies on ATP.2,15,16 These models initially included utilizing a PCP conducting the interview directly with a structured clinical instrument,17 such as the Mini International Neuropsychiatric Interview (MINI)18 or Montreal Cognitive Assessment,19 and later included the use of a behavioral health clinician integrated within the primary care environment and trained to conduct psychiatric interviews (Table 2). The behavioral health clinician interviewer model can be implemented with or without adjunct therapy by the behavioral health clinician.
Table 2.
LEVELS OF ATP INTERVIEWING | INTERVIEWER | INTERVIEW FORMAT | LEVEL OF CARE | DESCRIPTION OF DATA GENERATED |
---|---|---|---|---|
PCP with MINI | PCP/RN | Structured with instrument (MINI)21 | Collaborative, consultation liaison model, with limited option for therapy. | The PCP is trained on technology procedures and conducts the ATP interview. The PCP provides a video of the structured assessment and any consultation notes for the psychiatrist's review19 |
Behavioral health clinician (current study) | MFT, LCSW, NP, and RN | Semistructured/comprehensive history and assessment | Integrated, collaborative care model with option for integrated therapy in tandem with psychiatric consultation | A trained mental health clinician is trained to conduct the ATP interview in a broader population (i.e., primary care outpatient and specialty clinics) and provides targeted progress notes based on the interview and in collaboration with the PCP for the psychiatrist to review along with the video1 |
LCSW, licensed clinical social worker; MA, medical assistant; MFT, marriage and family therapist; MINI, Mini International Neuropsychiatric Interview; NP, nurse practitioner; PCP, primary care provider; RN, registered nurse; SCID, Structured Clinical Interview for DSM Disorders.
Evaluation of the efficacy of the ATP training model to ensure clinician skill and knowledge is important. This article will describe the ATP training approach and training model, with a focus on the skills required for the ATP clinician interviewer. The training model and skill sets were defined in the initial start-up phase of a clinical trial to evaluate the clinical efficacy of ATP compared with synchronous telepsychiatry (NCT02084979). This was the first clinical outcome-based study of ATP evaluating the clinical efficacy of this model of care in the primary care setting, as approved by the University of California Davis Institutional Review Board. An ATP training model utilizing behavioral health providers as ATP interviewers was developed and a training framework was defined in this study through expert feedback.
Methodology
ATP Interviewer Sample
Clinical and procedural training for ATP clinicians (a small convenience sample of five clinicians) was provided by an experienced, master's-level, clinical mental health provider and was supervised by an experienced psychiatrist on the study (P.Y, D.L, G.X) through a seminar, case supervision, and case discussions. The ATP clinicians employed on the trial were mental health clinicians who were trained at a graduate degree level equivalent to mental health, counseling, social work, or related field.
ATP Interview Procedure
The interview procedure has been previously described.2 For the purposes of this study, the interviewer and patient would meet in a room in the primary care clinic and conduct a semistructured interview that was recorded using a webcam and laptop. Recordings were captured through a web-based recording platform and stored securely in cloud-based storage. Local backup recordings were also captured. The interviewers sent the recording and a clinical note to the psychiatrist for review. The interviewer's note included sections providing details about the following information: chief complaint, history of present illness, psychiatric and medical review of symptoms, past medical history (weight, height, nicotine use, exercise, head injuries, and chronic/significant medical issues), past psychiatric history (substance abuse, family psychiatric and medical history, current relationships and children, key developmental history, hobbies, and social support and trauma/abuse history), current medication, medication history, and finally, the overall impression.
ATP Interviewer Training Procedure
The ATP interview is a semistructured interview that we developed in our pilot research.2 We have refined our ATP interview with input from ATP clinicians and psychiatrists over time. The behavioral health provider training model was developed over several years and refined through an expert feedback process on this clinical trial. A training manual and one-on-one sessions were employed for initial training, and supervision was provided to the ATP clinicians by supervising telepsychiatrists.
Before starting the trial, each ATP clinician received ∼10 h of didactic training on interviewing, which included shadowing a trained ATP clinician. Additionally, each ATP clinician was supervised by the treating telepsychiatrists who could ask for additional patient information to be collected at the next patient visit as well as give direct feedback to the ATP clinician about their interviewing style, process, and information collected.
The time invested in training was ∼75 h in total across five interviewers, with each ATP interviewer receiving an average of 15 h (two full training days) of didactic, shadowing, and supervised feedback before commencing ATP interviewing. The skills of the ATP interviewers were iteratively assessed through supervision by the study telepsychiatrists.
Expert Discussion and Feedback
Expert discussion and feedback sessions were conducted at three time points early in the study, and annually thereafter three 1-h information sessions were conducted over the course of the study with the treating telepsychiatrists and expert telepsychiatrists (n = 6) providing supervision on the study to gain expert feedback on the interviewing methods and compare cases. Notes were taken during the expert discussion and feedback sessions that were summarized and synthesized into themes by two researchers.
Results
Expert Feedback and Discussion Results
We found that three educational themes were recommended for ATP interviewers: (1) comprehensive skills in brief psychiatric interviewing; (2) adequate general knowledge base in behavioral health conditions and therapeutic techniques; and (3) clinical documentation, integrated care/consultation practices, and e-competency skill sets.
The framework for training and skill recommendations resulting from the expert feedback sessions included these themes (Table 3) as well as three technology training recommendations: (1) awareness of privacy/confidentiality for electronic data gathering, storage, management, and sharing; (2) technology troubleshooting; and (3) video filming/retrieval (Table 4).
Table 3.
SKILL | REQUIREMENT | SKILL AND COMPETENCY EVALUATION PROCEDURE |
---|---|---|
Theme 1: Comprehensive Skills in Brief Psychiatric Interviewing | ||
1a. Brief semistructured interview and history taking | Training on brief methods for identifying the symptomology of common disorders such as SIG-E-CAPS and DIGFAST8 and the MMSE23 or MoCA22 | Training on brief semistructured interviewing with feedback from supervising psychiatrists |
AND/OR | ||
1b. Structured and/or semistructured mental health interviewing skills | Structured interviewing training module (e.g., trained on the SCID24 or MINI21). | Training on reliability on standardized measures |
AND | ||
1c. Asynchronous interviewing | Trained to competency in asynchronous interviews | Interviewers initially shadow and observe the ATP interview process by a trained interviewer. Once prepared to administer the ATP interview, interviewers are shadowed by a trained ATP interviewer or supervised by an attending psychiatrist until competency is determined. |
Theme 2: Adequate General Knowledge Base in Behavioral Health Conditions and Therapeutic Techniques | ||
2a. Clinical knowledge | Knowledge of common psychological and psychiatric disorders and training in basic therapeutic communication techniques such as active listening, paraphrasing, reflection, use of silence, and asking for clarification. | Assessed by a supervising clinician during initial training and through a practicum review of the captured video |
2b. Clinical training | Adequate level of clinical training with a demonstrated understanding of common psychiatric disorders, in addition to understanding how to conduct a targeted clinical intake interview and record the patient's history, and basic therapeutic techniques. | As evidenced by academic training such as specific training in clinical mental health conditions, master's-level clinician in psychology, LCSW, psychiatric nurse, or adequate training to support the needs of the patient population. |
Theme 3: Clinical Documentation, Integrative Care/Consultation Practices, and e-Competency Skill Sets | ||
3a. Clinical documentation | Training on review of video and note taking to provide an adequate representation of the patient's history that is documented for the psychiatrist to review. | Note taking assessed by the supervising clinician with captured video |
3b. Consultation and integrated care practice skill sets | Training in integrated care and consultation care practices to facilitate adequate communication between the psychiatrist, the PCP, and the interviewer | Assessed by the supervising clinician during initial training and through a practicum review |
3c. e-Competency | e-Competency skill sets required to securely record and store video and other relevant clinical information/communication (skill sets outlined in Table 4) | e-Competency skills assessed by the supervising clinician with captured video |
DIGFAST, an acronym for bipolar disorder screening; MINI, Mini International Neuropsychiatric Interview; MMSE, Mini-Mental State Examination; SIG-E-CAPS, an acronym for depression screening: sleep disorder, interest deficit, guilt, energy, concentration, and appetite, psychomotor, and suicidal ideation.
Table 4.
SKILL | REQUIREMENT | SKILL EVALUATION PROCEDURE |
---|---|---|
1. Technology use for recording, storage, and retrieval | ATP interviews should have training and knowledge on how to set up, execute, and troubleshoot the recording program used. If using a web-based program to record and store, it is best to use a secure VPN network if possible. | One-on-one training with an experienced ATP clinician and technology support as needed |
2. Privacy and confidentiality (e.g., HIPAA25 compliance) | All clinical professionals should have training on HIPPA privacy and security, which covers what patient information should be protected under state and federal privacy laws as well as how to maintain the privacy and security of PHI. | Expanded CITI26 and compliance training |
3. Technology troubleshooting | It is also advised that clinicians have an alternate option to record interviews if there is no secure connection or if wireless internet is not available. When using this method of recording and storing, videos should be kept in a password-protected locked file or flash drive until it is properly transferred to a secure location. | One-on-one training with an experienced ATP clinician and technology support as needed |
CITI, Collaborative Institutional Training Initiative; EHR, electronic health record; HIPAA, Health Insurance Portability and Accountability Act; PHI, protected health information.
Theme 1: Comprehensive Skills in Brief Psychiatric Interviewing
The most important training that is needed for the ATP clinician is how to conduct a modified psychiatric interview. The purpose of the asynchronous interview is to help gather relevant information for the psychiatrist to review. The clinician is not responsible for diagnosis or treatment, but during the interview, the ATP clinician must have skills to gather adequate information on the chief symptom/complaint, history of the presenting illness, family history, medical history, social history, and substance use. Importantly, the clinician should know how to conduct a review of psychiatric symptoms for disorders such as depression (SIG-E-CAPS mnemonic),1 mania (DIGFAST mnemonic),21 anxiety, and psychosis. It should be noted that ATP has been used primarily for common low-risk disorders; when presented with a patient who is actively psychotic, the protocol implemented was to ensure the safety of the patient and the clinician and refer the patient directly to the supervising psychiatrist for in-person or synchronous video care. Finally, the ATP clinician should know how to properly document the ATP interview with the information mentioned above. In cases where there is cognitive impairment such as dementia or early onset of Alzheimer's disease, the ATP clinician should know how to properly conduct a mental state examination.19,22
Theme 2: Adequate General Knowledge Base in Behavioral Health Conditions and Therapeutic Techniques
An ATP clinician should have formal education or training in behavioral health. This includes, but is not limited to, marriage and family therapists, licensed clinical social workers, licensed professional counselors, and mental health registered nurses who have additional academic training in therapeutic techniques, diagnostics, and psychopharmacology. ATP clinicians should have knowledge of common psychological and psychiatric disorders and training in basic therapeutic communication techniques such as active listening, paraphrasing, reflection, use of silence, validation, and asking for clarification. In addition, they need skills and knowledge to help identify common psychological and psychiatric disorders (e.g., depression, bipolar disorder, anxiety disorders, and substance use) as well as common symptoms, behaviors, appearances, and comorbidity. Clinicians should have basic knowledge of the names and types of medications that are used to treat mental health disorders. Finally, for patients experiencing crisis, ATP clinicians should be properly trained in de-escalation techniques, crisis intervention, and suicide assessment to ensure the safety of the patient and the clinician.
Theme 3: Clinical Documentation, Integrated Care/Consultation Practices, and e-Competency Skill Sets
ATP clinicians must have training on clinical documentation and comprehensive note taking to adequately review the interview video and provide an adequate representation of the patient's presenting problem and history for the psychiatrist to review. The ATP interviewer must also have skill sets in integrated care and consultation care practices to facilitate adequate communication between the psychiatrist, the PCP, and the interviewer. In subsequent visits, the ATP clinician may need to gather additional information at the request of the consulting psychiatrist and may need to liaise between the providers and the patient. The clinician must also understand the context of care, including primary care workflow and consultation approach. e-Competency skills are required for the ATP clinician to adequately capture, retrieve, and forward patients' electronic information for consultation and care management. The e-competency skill sets identified as part of the training regimen for this clinical trial include (1) technology use for recording, storage, and retrieval; (2) privacy and confidentiality (e.g., Health Insurance Portability and Accountability Act [HIPAA] compliance); and (3) technology troubleshooting (Table 4).
Technology Training Recommendations for ATP Interviewers
Technology training is necessary for individuals conducting ATP consults (the clinical ATP interviewer and the psychiatrist conducting the consultations). The goal is to ensure e-literacy2,23,24 competencies for clinicians to orient them specifically to asynchronous technology. The Guidelines for Videoconferencing from the American Telemedicine Association provide training resources for telepsychiatrists, and the ATP technology workflow and requirements have been outlined in previous studies.17 The e-competency skill sets identified during this study are outlined in Table 4.
Discussion
This study establishes a model of training for ATP interviewing skills as achieved through expert feedback. ATP is an increasingly important care option, in that it is transferable, flexible, and adaptable. This study has evaluated and defined an ATP training model for PCP integrated behavioral health clinicians. The asynchronous approach could be utilized to improve care from a variety of consultants in the primary care workflow. Interview skills could be developed and adapted to encompass data relevant to multiple specialties, and by using the strategy with a secure shareable video, multidisciplinary specialty consultations could be achieved through one concise and consolidated asynchronous dataset. Thus, the ATP model could create a more efficient and less costly model of care across the spectrum of patient care services.
There are limitations to this article, including that the comprehensive method of obtaining expert consensus by proposing the model to clinical, telehealth, and educational leaders beyond the experts engaged in the study was not conducted as part of the study. This study examined training methods for behavioral health clinicians only. It is feasible that other health professionals who have training in behavioral health such as nurse practitioners or registered nurses could be given additional ATP training to assist with the interview process and this should be further evaluated in future studies. The training approach in this study was designed as part of a clinical trial protocol and the necessary training dose (e.g., time required for training) was not assessed. Furthermore, this study did not obtain an objective rating of expert feedback, but used a convenience sample of experts participating in the study and included a very small sample of ATP clinicians and experts.
Further evaluation is needed to determine the training dose requirement before adaptation of this training approach for staff in the context of larger ATP implementation, and generalizability of this model should be established. We describe what we learned from training five mixed discipline interviewers in the ATP approach and provide a summary of our recommendations for technical and integrated care/consultation practices appropriate to the context of care. The skill set identified by the study experts for an ATP interviewer is a broad skill set that could create positions that can be filled by a range of clinicians in many areas of integrated health. This has the potential to open up new career paths in integrated health.
Acknowledgments
The authors are thankful for the support of CommuniCare Health Care Centers and UC Davis School of Medicine and posthumously acknowledge the contributions of Alberto Odor, MD.
Disclosure Statement
No competing financial interests exist.
Funding Information
This project was funded by the Agency for Health Care Research and Quality, Grant# 1R01HS021477-01A1.
References
- 1. Blenkiron P. A mnemonic for depression. BMJ 2006;332:551. [Google Scholar]
- 2. Yellowlees P, Shore JH. Telepsychiatry and health technologies: A guide for mental health professionals. Washington, DC: American Psychiatric Publishing, 2018. [Google Scholar]
- 3. O'Keefe M, White K, Jennings JAC. Asynchronous telepsychiatry: A systematic review. J Telemed Telecare 2019. [Epub ahead of print]; DOI: 10.1177/1357633X19867189 [DOI] [PubMed] [Google Scholar]
- 4. Hilty DM, Uno J, Chan S, Torous J, Boland RJ. Role of technology in faculty development in psychiatry. Psychiatr Clin North Am 2019;42:493–512. [DOI] [PubMed] [Google Scholar]
- 5. Yellowlees PM, Odor A, Parish MB. Cross-lingual asynchronous telepsychiatry: Disruptive innovation? Psychiatr Serv 2012;63:945. [DOI] [PubMed] [Google Scholar]
- 6. Balasubramanian BA, Cohen DJ, Jetelina KK, et al. . Outcomes of integrated behavioral health with primary care. J Am Board Fam Med 2017;30:130–139. [DOI] [PubMed] [Google Scholar]
- 7. Richardson LP, McCarty CA, Radovic A, Suleiman AB. Research in the integration of behavioral health for adolescents and young adults in primary care settings: A systematic review. J Adolesc Health 2017;60:261–269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Flanagan S, Damery S, Combes G. The effectiveness of integrated care interventions in improving patient quality of life (QoL) for patients with chronic conditions. An overview of the systematic review evidence. Health Qual Life Outcomes 2017;15:188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Davis KE, Brigell E, Christiansen K, et al. . Integrated primary and mental health care services: An evolving partnership model. Psychiatr Rehabil J 2011;34:317. [DOI] [PubMed] [Google Scholar]
- 10. Butler TN, Yellowlees P. Cost analysis of store-and-forward telepsychiatry as a consultation model for primary care. Telemed J E Health 2012;18:74–77. [DOI] [PubMed] [Google Scholar]
- 11. Zalpuri I, Liu H, Stubbe D, Wrzosek M, Sadhu J, Hilty D. A competency-based framework for social media for trainees, faculty and others. Acad Psychiatry 2018;42:808–817. [DOI] [PubMed] [Google Scholar]
- 12. Archer J, Bower P, Gilbody S, et al. Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev 2012:CD006525. [DOI] [PubMed] [Google Scholar]
- 13. Coventry PA, Hudson JL, Kontopantelis E, et al. . Characteristics of effective collaborative care for treatment of depression: A systematic review and meta-regression of 74 randomised controlled trials. PLoS One 2014;9:e108114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Panagioti M, Bower P, Kontopantelis E, et al. . Association between chronic physical conditions and the effectiveness of collaborative care for depression: An individual participant data meta-analysis. JAMA Psychiatry 2016;73:978–989. [DOI] [PubMed] [Google Scholar]
- 15. Xiong GL, Iosif A-M, Godwin HT, et al. . A pilot randomized trial of asynchronous and synchronous telepsychiatry in skilled nursing facilities. J Am Med Dir Assoc 2018;19:461–462. [DOI] [PubMed] [Google Scholar]
- 16. Yellowlees P, Burke Parish M, González Á, et al. Asynchronous telepsychiatry: A component of stepped integrated care. Telemed J E Health 2018;24:375–378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Yellowlees P, Odor A, Burke Parish M, Iosif AM, Haught K, Hilty D. A feasibility study of the use of asynchronous telepsychiatry for psychiatric consultations. Psychiatr Serv 2010;61:838–840. [DOI] [PubMed] [Google Scholar]
- 18. Sheehan DV, Lecrubier Y, Sheehan KH, et al. . The Mini-International Neuropsychiatric Interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998;59 Suppl 20:22–33. [PubMed] [Google Scholar]
- 19. Nasreddine ZS, Phillips NA, Bedirian V, et al. . The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–699. [DOI] [PubMed] [Google Scholar]
- 20. McGovern M, Dent K, Kessler RA. Unified model of behavioral health integration in primary care. Acad Psychiatry 2018;42:265–268. [DOI] [PubMed] [Google Scholar]
- 21. Caplan JP, Stern TA. Mnemonics in a mnutshell: 32 aids to psychiatric diagnosis. Curr Psychiatry 2008;7:27–33. [Google Scholar]
- 22. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12:189–198. [DOI] [PubMed] [Google Scholar]
- 23. Torous J, Roberts LW. The ethical use of mobile health technology in clinical psychiatry. J Nerv Ment Dis 2017;205:4–8. [DOI] [PubMed] [Google Scholar]
- 24. Hilty D, Yellowlees PM, Parrish MB, Chan S. Telepsychiatry: Effective, evidence-based, and at a tipping point in health care delivery? Psychiatr Clin North Am 2015;38:559–592. [DOI] [PubMed] [Google Scholar]
- 25. Health Insurance Portability and Accountability Act. Pub. L. No. 104–191, § 264, 110 Stat.19361.
- 26. Braunschweiger P, Hansen K. Collaborative institutional training initiative (CITI). J Clin Res Best Pract 2010;6:1–6. [Google Scholar]