Table 2.
Item content, content domains, and consensus ratings of the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ)
| Original Item | Content domain | Research team rating | Final item, if modified from original | Delphi rating |
|---|---|---|---|---|
| 1. During clinical encounters with patients, I see patients for 30 minutes or less. |
Practice and Session Management |
Essential |
N/A |
83% Essential |
| 2. I manage patients reporting mild and moderate symptoms and refer those with more severe symptoms to other behavioral health services. |
Practice and Session Management |
Essential |
2. I manage patients reporting mild and moderate symptoms in primary care, and I refer those with more severe symptoms to specialty mental health services when possible. |
88% Essential |
| 3. During clinical encounters with a patient, I discuss barriers to implementing a plan or adhering to treatment recommendations. |
Practice and Session Management |
Essential |
3. During patient appointments, I discuss barriers to implementing a plan or adhering to treatment recommendations. |
80% Essential |
| 4. I collaborate with primary care team or PACT staff to provide group medical visits to patients |
Consultation, Collaboration, and interprofessional communication |
Compatible |
4. I collaborate with primary care team staff to provide group medical visits (or shared medical appointments) to patients. |
80% Compatible |
| 5. I accept referrals for patients with traditional mental health problems (i.e. depression, anxiety, PTSD, etc.). |
Referral management and care continuity |
Essential |
5. I accept referrals for patients with common mental health problems (i.e. depression, anxiety, etc.). |
80% Essential |
| 6. During clinical encounters with a patient, I implement behavioral and/or cognitive interventions. |
Clinical scope and Interventions |
Essential |
N/A |
86% Essential |
| 7. In introducing my role in the clinic to patients, I explain that I want to get an idea of what is and what is not working for the patient and then together develop a plan to help them manage their concerns. |
Practice and Session Management |
Essential |
7. In introducing my role in the clinic to patients, I explain that I want to get an idea of what is and what is not working for the patient and then together develop a plan to help them manage their concerns. |
80% Essential |
| 8. During clinical encounters with patients, I triage patients to determine if they can be treated in primary care or should be referred to a specialty mental health or a community agency. |
Practice and Session Management |
Essential |
N/A |
86% Essential |
| 9. I accept referrals for patients in need of behavioral health interventions for chronic pain. |
Referral Management and Care Continuity |
Essential |
9. I accept referrals for patients who might benefit from brief, targeted behavioral health interventions for chronic pain. |
80% Essential |
| 10. I accept referrals for patients in need of behavioral health interventions for adjustment to illness (i.e., diabetes, heart disease, spinal cord injury, TBI, etc.). |
Referral Management and Care Continuity |
Essential |
10. I accept referrals for patients who might benefit from brief, targeted behavioral health interventions for adjustment to illness (i.e., diabetes, heart disease, spinal cord injury, TBI, etc.). |
96% Essential |
| 11. My progress notes include focused recommendations for the PCP |
Practice and Session Management |
Essential |
11. My progress notes in the shared medical record include focused recommendations for the Primary Care Provider and/or primary care team. |
88% Essential |
| 12. I huddle with the primary care team or PACT staff to provide both a behavioral health perspective and behavioral data. |
Consultation, Collaboration, and interprofessional communication |
Essential |
12. I meet briefly with primary care staff as a team to provide both a behavioral health perspective and behavioral data. |
88% Essential |
| 13. My progress notes include focused recommendations for the patient. |
Practice and Session Management |
Essential |
N/A |
86% Essential |
| 14. During clinical encounters with patients, I provide educational handouts to the majority of patients. |
Clinical Scope and Interventions |
Essential |
14. During patient appointments, I provide educational handouts when appropriate. |
72% Compatible |
| 15. I routinely consult with primary care team or PACT staff other than the PCP (i.e., pharmacist, dietician) about behavioral aspects of medical conditions (i.e., medications that cause nightmares.) |
Consultation, Collaboration, and interprofessional communication |
Essential |
15. I consult with various members of the primary care team (i.e., pharmacist, dietician) in addition to the Primary Care Provider about behavioral aspects of medical conditions (i.e., medications that cause nightmares.) |
80% Essential |
| 16. At follow-up encounters with patients, I inquire about progress on goals or action plans set at the previous appointment. |
Practice and Session Management |
Essential |
N/A |
90% Essential |
| 17. During clinical encounters with patients, I routinely complete standardized measures for an initial screening (e.g., PHQ-9, PCL, or brief cognitive screening). |
Practice and Session Management |
Essential |
17. I administer one or more brief validated measures (e.g., Patient Health Questionnaire-9, or PHQ-9) for an initial screening of symptoms of interest, or I review these findings if measures were administered by other primary care staff. |
84% Essential |
| 18. I document a full-length treatment plan with multi-axial diagnosis after the initial encounter. |
Practice and Session Management |
Prohibited |
18. It takes 30 minutes or more for me to complete all documentation following the initial appointment. |
80% Prohibited |
| 19. During a clinical encounter with a patient, I use reflection of affect and silence to promote emotional exploration. |
Clinical Scope and Interventions |
Prohibited |
19. During patient appointments, I promote emotional exploration. |
88% Compatible |
| 20. Following clinical encounters with patients, I provide feedback to the PCP within 1 business day of initial patient contact. |
Consultation, Collaboration, and interprofessional communication |
Essential |
20. Following patient appointments, I provide feedback to Primary Care Providers (based on their preferred method of communication) within 1 business day of an initial appointment. |
96% Essential |
| 21. During clinical encounters with patients, I clarify, confirm, and discuss the patient’s concerns. |
Practice and Session Management |
Essential |
N/A |
86% Essential |
| 22. My progress notes include a brief clinical conceptualization or impressions. |
Practice and Session Management |
Essential |
22. My progress notes include brief clinical impressions of the patient’s presenting problem(s). |
80% Essential |
| 23. During a clinical encounter with a patient, I provide full neuropsychological, cognitive, or personality assessments. |
Clinical Scope and Interventions |
Prohibited |
N/A |
96% Prohibited |
| 24. I see patients for weekly, open-ended therapy. |
Practice and Session Management |
Prohibited |
N/A |
86% Prohibited |
| 25. In introducing my role in the clinic to patients, I explain that our sessions will be less than 30 minutes. |
Practice and Session Management |
Essential |
25. In introducing my role in the clinic to patients, I explain that our appointments typically will be 30 minutes or less. |
84% Essential |
| 26. I provide behavioral health crisis or emergency intervention (i.e. suicide intervention) as the CCC BHP. |
Clinical Scope and Interventions |
Compatible |
26. I provide suicide risk assessment for primary care patients in crisis and refer to a higher level of care as indicated. |
84% Essential |
| 27. During clinical encounters with patients, I see patients for 50-minute appointment. |
Practice and Session Management |
Prohibited |
27. I typically see patients for 50-minute appointments. |
80% Prohibited |
| 28. During clinical encounters with patients, I use local community resources to assist me in meeting the behavioral health needs of patients. |
Practice and Session Management |
Essential |
28 During patient appointments, I use local community resources to assist me in meeting the behavioral health needs of patients. |
84% Essential |
| 29. I provide education to the primary care team or PACT staff on behavioral health issues (e.g., presentations and handouts). |
Consultation, Collaboration, and interprofessional communication |
Essential |
N/A |
86% Essential |
| 30. I provide advice to primary care team or PACT staff about appropriate referrals to specialty behavioral health services. |
Referral Management and Care Continuity |
Essential |
N/A |
86% Essential |
| 31. I conduct follow-up sessions via telephone. |
Practice and Session Management |
Compatible |
31. I conduct follow-up appointments via telephone when appropriate. |
88% Compatible |
| 32. During a clinical encounter with a patient, I provide traditional family therapy to patients and couples. |
Clinical Scope and Interventions |
Prohibited |
32. I provide family or couples therapy for 10 or more appointments per episode of care. |
84% Prohibited |
| 33. During a clinical encounter with a patient, I primarily use open-ended questions. |
Clinical Scope and Interventions |
Prohibited |
33. During patient appointments, I use open-ended questions. |
72% Compatible |
| 34. On average, I see patients for only 2–3 consultations. |
Practice and Session Management |
Essential |
34. I typically see patients for 6 or less appointments per episode of care. |
79% Essential |
| 35. My progress notes include findings from functional assessments and brief screening instruments. |
Practice and Session Management |
Essential |
N/A |
96% Essential |
| 36. During clinical encounters with patients, I complete standardized measures for assessing change at follow up (e.g., PHQ-9, PCL, or brief cognitive screening). |
Practice and Session Management |
Essential |
36. I administer one or more brief validated measures (e.g., Patient Health Questionnaire-9, or PHQ-9) for follow up screening of symptoms of interest, or I review these findings if measures were administered by other primary care staff. |
84% Essential |
| 37. I routinely consult with PCPs to increase my knowledge about behavioral aspects of medical conditions, such as the role of anxiety in cardiac distress. |
Consultation, Collaboration, and interprofessional communication |
Essential |
N/A |
83% Essential |
| 38. During a clinical encounter with a patient, I provide supportive interventions without addressing cognitive or behavioral change. |
Clinical Scope and Interventions |
Compatible |
38. During a patient appointment, I provide supportive interventions without addressing cognitive or behavioral change. |
84% Compatible |
| 39. During a clinical encounter with a patient, I provide full-length empirically supported treatments (ESTs), such as Prolonged Exposure or Cognitive Processing Therapy. |
Clinical Scope and Interventions |
Prohibited |
39. During a patient appointment, I provide full-length empirically supported treatments, such as Prolonged Exposure or Dialectical Behavior Therapy. |
88% Prohibited |
| 40. Following clinical encounters with patients, I continue to provide feedback to the PCP about follow-up appointments when needed. |
Consultation, Collaboration, and interprofessional communication |
Essential |
N/A |
90% Essential |
| 41. During clinical encounters with patients, I work with the patient to develop a specific plan to address their presenting problem and document this plan. |
Practice and Session Management |
Essential |
N/A |
83% Essential |
| 42. I accept referrals for patients who need lifestyle interventions (e.g., tobacco cessation, weight control, stress management). |
Referral Management and Care Continuity |
Essential |
N/A |
97% Essential |
| 43. I accept referrals for patients in need of behavioral health interventions for medication issues (i.e., adherence). |
Referral Management and Care Continuity |
Essential |
N/A |
86% Essential |
| 44. During a clinical encounter with a patient, I provide traditional psychotherapy and specialized behavioral interventions. |
Clinical Scope and Interventions |
Prohibited |
44. I typically see patients for 10 or more appointments per episode of care. |
88% Prohibited |
| 45. Following clinical encounters with patients, I schedule follow-ups at least two weeks apart. |
Practice and session management |
Essential |
45. Following patient appointments, I typically schedule follow-ups at least two weeks apart. |
84% Compatible |
| 46. I accept referrals for patients in need of behavioral health interventions for adjustment to aging and issues specific to older patients. |
Referral Management and Care Continuity |
Essential |
46. I accept referrals for patients in need of behavioral health interventions for adjustment to aging and issues specific to older patients. |
80% Essential |
| 47. During a clinical encounter with a patient, I provide a highly structured encounter to address functional assessment, focused intervention, and disposition. |
Practice and Session Management |
Essential |
47. During a patient appointment, I provide functional assessment, focused intervention, and address disposition. |
92% Essential |
| 48. I provide brief psycho-education and symptom management groups as part of my role as CCC BHP. |
Clinical Scope and Interventions |
Compatible |
48. I provide brief psycho-educational groups or classes on specific topics (such as mood management, stress reduction, etc.). |
80% Compatible |
| 49. During a clinical encounter with a patient, I provide brief consultation to couples or families. |
Clinical Scope and Interventions |
Compatible |
49. I have appointments with couples and families as appropriate. |
68% Compatible |
| 50. I accept referrals for patients from PCPs as a warm hand off (i.e., the PCP introduces me to the Veteran). |
Referral management and Care continuity |
Essential |
N/A |
100% Essential |
| 51. In introducing my role in the clinic to patients, I explain that I work with the PCPs in situations where good healthcare involves paying attention to physical health, habits, behaviors, emotional health and how those things interact. |
Practice and Session Management |
Essential |
N/A |
83% Essential |
| 52. I provide long-term (i.e., greater than 6 sessions) group psychotherapy, such as DBT, as part of my role as CCC BHP. |
Clinical Scope and Interventions |
Prohibited |
52. I provide long-term (i.e., greater than 8 sessions) group psychotherapy. |
80% Prohibited |
| 53. During a clinical encounter with a patient, I obtain a full psycho-social history. |
Clinical scope and Interventions |
Prohibited |
53. I meet with a patient for greater than 50 minutes to gather a full psycho-social history and comprehensive psychiatric interview. |
88% Prohibited |
| 54. During a clinical encounter with a patient, I provide medical social work services. |
Clinical Scope and Interventions |
Prohibited |
54. During a patient appointment, I typically provide medical social work services, including, but not limited to, assistance with disability claims, obtaining health insurance, and/or assisting with housing. |
88% Prohibited |
| 55. During clinical encounters with patients, I address the PCPs reason for referral. |
Practice and Session Management |
Essential |
N/A |
93% Essential |
| 56. I employ strategies to identify and prevent exacerbation of at-risk, sub-syndromal behaviors and symptoms. |
Clinical Scope and Interventions |
Essential |
N/A |
83% Essential |
| 57. Not included in Round 1 Survey |
Consultation, Collaboration, and interprofessional communication |
|
57. I provide information regarding a patient’s symptoms and functioning to assist Primary Care Providers (and/or clinical pharmacists, primary care psychiatrists, psychiatric nurse practitioners) in initiating or modifying common psychotropic medications, such as antidepressants. |
84% Essential |
| 58. Not included in Round 1 Survey | Clinical Scope and Interventions | 58. I participate in primary care based clinical pathways for common health conditions, such as chronic pain or comorbid depression and cardiovascular disease. A clinical pathway is an approach to managing patients with common conditions by utilizing empirically supported interventions in a pre-defined sequence among a multidisciplinary group of providers. | 80% Essential |
Notes: Items 57 and 58 were created during the Delphi process based on qualitative feedback suggesting new items were necessary. Content domain labels for these items were subsequently applied by the research team.