Validating administrative records in post‐traumatic stress disorder

Abstract

There is insufficient data on the accuracy of administrative coding data (ACD) for post‐traumatic stress disorder (PTSD). Medical records were reviewed for (1) a diagnosis of PTSD; (2) treatment for PTSD. The records were compared against the Veterans Health Administration (VHA) data in order to determine the positive predictive value (PPV) and negative predictive value (NPV) of three commonly used approaches. The PPV and NPV varied according to the ACD approach. Relative to a medical records review, the ACD approach of one or two PTSD coded outpatient encounters had a PPV of 78% and an NPV of 91%; whereas the PPV was 97% and the NPV was 98% for three or more PTSD codes. For pharmacotherapy, the ACD approach with one or two codes for PTSD had a PPV of 33% (NPV = 93%), whereas three or more PTSD coded encounters improved the PPV to 85% (NPV = 100%). When using VHA data, we recommend tailoring the identification strategy according to the research aims. An ACD approach identifying one or more PTSD outpatient encounters should be considered sufficient for a diagnosis of PTSD. Assessments for PTSD associated pharmacotherapy require using an ACD approach that identifies veterans with the presence ≥ 3 outpatient PTSD encounters. Published 2015. This article is a U.S. Government work and is in the public domain in the USA.

Background

Post‐traumatic stress disorder (PTSD) affects hundreds of thousands of service members following military deployment and is one of the signature wounds of today's Veteran (Bernardy et al., 2012). Veterans Health Administration (VHA) administrative data has been used to inform PTSD research and policy. In general terms, VHA administrative data contain a broad number of clinical and demographic variables which are routinely uploaded into a central data base reflecting each and every documented patient–health care system interaction. The data can then be assembled and analyzed providing investigators and policy‐makers with insights into particular comorbid patterns of illnesses, disparities in health care outcomes between sample populations, or variations in treatment delivery across sites of care. Obviously, the quality of the data and the ability of the administrative coding data (ACD) methods to accurately identify the condition(s) of interest directly impact the strength of these insights.

Specific illnesses or processes of care can be identified from the VHA administrative data using a variety of approaches. For example, clinical contacts can occur either over the telephone, in inpatient, or outpatient settings; with every encounter generating a clinical record with a diagnosis code accounting for the services being delivered. These codes can be further associated with the type of encounter (e.g. provider, laboratory, or nurse‐based), dates of services, and with other types of care (e.g. prescriptions, procedures, or diagnostic testing). Each of these data points can then be manipulated to generate varying ACD algorithms in order to define the presence/absence of a variable of interest; an effective ACD algorithm will identify the variable of interest with a high degree of accuracy lending confidence to the results. Unfortunately, relative to other medical conditions (e.g. diabetes and heart disease) where multiple studies have well validated ACD methods (Iezzoni et al., 1992; Newton et al., 1999; Longenecker et al., 2000), valid methods in the field of PTSD are lacking (Gravely et al., 2011; Yeager et al., 2007; Magruder et al., 2005; Frayne et al., 2010).

The existing studies on PTSD examining the accuracy of ACD methods using VHA data have several key weaknesses. First, there are wide discrepancies in the positive predictive values (PPV) and negative predictive values (NPV). For example, Magruder et al. (2005) noted that only 46.5% of primary care clinicians coded a diagnosis of PTSD (e.g. an ACD method) in the medical charts of veterans determined to have PTSD from a clinical assessment delivered by telephone. In other work by Gravely et al. (2011) and Frayne et al. (2010) self‐reported scales of the PTSD checklist (PCL) and a life‐time self‐report of a diagnosis of PTSD were compared to an ACD method. The authors report discrepant PPV depending on the location of visit, ranging from 69% to 88% for a mental health location and 80% to 85% for a primary care location. Also, each of these examples were originally designed with different study aims in mind, thus introducing sampling biases and limiting generalizability.

Because of the lack of data and inconsistent reports, we conducted this study and ereport on an electronic medical records (EMR) abstraction compared to varying PTSD ACD algorithms. The research was driven by two objectives: (1) to define the PPV and NPV of three commonly used ACD definitions of PTSD compared to an EMR abstraction; (2) to explore differences in the demographic and clinical characteristics of veterans with PTSD as identified by each of these three ACD approaches. The EMR abstraction was selected as the standard to determine PTSD because it allowed us to sample the entire VHA population of veterans and is considered an acceptable approach to identify the existence of medical conditions (Quan et al., 2009). A secondary outcome aimed to evaluate the accuracy of the ACD algorithms to determine the presence of a medication for a PTSD diagnosis. This is an important consideration because several papers have been written on patterns of pharmacotherapy for PTSD in the VHA with little validation of the ACD methods (Harpaz‐Rotem et al., 2008; Lund et al., 2012; Mohamed and Rosenheck, 2008; Rosenheck and Fontana, 2007). The aim of this research is to inform researchers and policy‐makers relying on VHA administrative data to conduct PTSD analyses by providing insights on the performance characteristics of differing ACD algorithms.

Methods

Study design

VHA administrative and EMR data recorded during the fiscal years (FY) 2006–2008 were used to examine the NPV and PPV of three ACD algorithms against EMR evidence of: (1) diagnosis of PTSD; (2) medications for PTSD. We also explored associated characteristics of each sample according to each algorithm. Sensitivity analyses examined the number of encounter driven PTSD codes (e.g. ≥ 1, ≥ 2, ≥ 3, … etc.) to generate a receiver operator characteristic (ROC) curve.

Data sources and elements

Inpatient and outpatient VHA administrative data were used for the sampling and initial PTSD identification (Bernardy et al., 2012). A diagnosis of PTSD was initially identified by the International Classification of Medical Diseases, ninth revision (ICD‐9) diagnostic code of 309.81 placed in either the primary or secondary position for any inpatient or outpatient clinical encounter. Geographic region assignment was determined using the VHA enrollment files (The Dartmouth Atlas of Health Care, 2012).

Sample selection process

Initial eligibility criterion required at least one VHA encounter in FY 2006 identifying n = 4,780,188 veterans which were categorized into five geographical regions (west, midwest, north, south, and other) based on the location of VHA facility. Veterans with missing facility information were excluded, leaving n = 4,276,044 eligible patients. Patients were then divided into smaller sampling pools according to one of three administrative coding algorithms as defined by (1) one or two outpatient ICD‐9 codes for PTSD (n = 103,854), (2) three or more outpatient codes for PTSD (n = 301,617), and (3) one or more inpatient codes for PTSD (n = 43,977).

The final study cohort was created by randomly selecting equal portions of patients based on each of the algorithm definitions. In every geographical area and according to the specific algorithm, a randomly selected patient with PTSD was matched with a randomly selected control patient without PTSD. Every time a patient was sampled from each initial pool those patients were excluded from being selected over again since the ACD algorithms were not mutually exclusive. This process was then repeated for each of the algorithms resulting in a final sample of n = 573 unique individuals representing equal geographic distribution and equal numbers with and without PTSD according to one of the three ACD algorithms. To accommodate for insufficient records (e.g. non‐veterans, deaths during the study window) an additional n = 150 were identified to serve as alternates of which 9.1% contributed to the final sample (n = 52/573).

Administrative data algorithms and chart abstraction process

Three ACD algorithms were identified from existing literature (Frayne et al., 2010; Gravely et al., 2011; Harpaz‐Rotem et al., 2008; Lund et al., 2012; Magruder et al., 2005; Mohamed and Rosenheck, 2008; Rosenheck and Fontana, 2007; Yeager et al., 2007). Algorithm one identified veterans with PTSD with one or two ICD‐9 codes 309.81 in any outpatient encounters; coding algorithm two assigned PTSD to patients with three or more PTSD codes on outpatient encounters; coding algorithm three assigned PTSD to any patient with at least one code from any inpatient encounter.

The EMR of each patient was reviewed using a very detailed structured protocol. For the first 400 charts, two trained psychology graduate students performed a blinded, independent review of each medical record using a double entry mirror technique in Microsoft Access Database®. To improve efficiency, an experienced VHA nurse abstractor conducted the final 173 charts reviews. For all abstractors, training was conducted by the primary author (TEA) and involved education on the structure of the medical record with examples on where to find important clinical information, education on pharmacological therapies used in PTSD, education on how screening instruments are implemented in clinical notes, and education on how to recognize language that represents templates in notes.

Patients were considered to have a possible past diagnosis of PTSD if provider notes indicated a diagnosis in the past medical history section, whereas a current diagnosis was determined by the presence of a PTSD diagnosis listed in the assessment and plan section. Linking pharmacotherapy to a diagnosis of PTSD required an explicit statement in the assessment and plan connecting the medication to the diagnosis. In situations where medications were written for other mental health conditions and PTSD, the use was attributed to both PTSD and the other mental health condition. All medications included in the VHA PTSD clinical practice guidelines were examined.

The raters reviewed all clinical notes from FY 2006 to 2008 and abstracted clinical data according to a structured abstraction tool (see Appendix 1). The final assessment recorded each reviewer's judgment according to a five‐point Likert scale ranging from “highly likely” to “highly unlikely” for evidence of a PTSD diagnosis, receipt of PTSD treatment, or delivery of different forms of behavioral therapy. In instances where the evidence was equivocal the reviewers recorded “impossible to determine”. For the 400 abstracted charts performed by the graduate students any disagreements in the final assessment rating were adjudicated in an independent review of the record by an experienced, board certified VHA psychiatrist (TEA). For the charts (n = 173) abstracted by the research nurse, a 20% random sample was independently reviewed for agreement. Inter‐rater agreements were not performed, however in general, there was excellent agreement regarding the presence of a diagnosis of PTSD.

Analyses

Initial comparative descriptive analyses were conducted using chi square test statistics in order to identify differences in demographics and clinical characteristics between each of the sampled ACD algorithms (one through three) with PTSD relative to those without PTSD by any method. Next, PPV and NPV were generated to assess the match between PTSD algorithms assigned using each of the ACD coding algorithms versus the EMR review. The analyses were designed to evaluate the ability of each of the three administrative data coding algorithms to fulfill our primary aim of the validity of a PTSD diagnosis and the secondary aim of determining the performance characteristics an ACD algorithm to identify medication treatment for PTSD. To facilitate the analyses, the Likert responses were collapsed into a binomial outcome with “highly likely” and “likely” representing evidence of PTSD and “impossible to determine”, “unlikely”, and “highly unlikely” representing no evidence of PTSD.

For the current study, PPV and NPV were calculated through comparison of chart against each algorithm (one through three) where EMR review was the gold standard and the coding identification algorithms represent the test result. Statistical Analysis Software (SAS®) was used to generate ROC curves for medical records evidence of PTSD diagnosis and the receipt of PTSD medication for when administrative PTSD codes were greater or equal to one (e.g. ≥ 1, ≥ 2, ≥ 3, … etc.).

Results

Overall from the 573 charts abstracted, the study sample consisted of veterans with a mean age of 55.1 (standard deviation [SD] ± 15.3) and were mostly male (92.1%) and White (63.1%). Table 1 demonstrates a comparison of the demographical and clinical characteristics across each of the three cohorts as defined by each of the three ICD‐9 coding algorithms against patients with no PTSD diagnoses. Veterans with PTSD, as identified by ACD algorithm one, were younger, more likely to be White, and were identified with fewer comorbid medical and psychiatric conditions compared to the cohort of veterans with PTSD by ACD algorithm three. Somewhat in contrast were veterans identified by ACD algorithm three, who tended to be older, more racially diverse, and with greater levels of medical and psychiatric comorbidity compared to veterans identified by ACD algorithms one or two.

Table 1.

Clinical and demographical characteristics stratified by the three administrative coding data (ACD) algorithms

Characteristic	Algorithm 1 N = 95	Algorithm 2 N = 98	Algorithm 3 N = 97	No PTSD N = 283	p Value*
Demographic
Age categorical (%)					<0.001
<35	13.7	3.1	8.3	4.6
35–44	10.5	8.2	13.4	7.1
45–54	11.6	8.2	12.4	14.5
55–64	44.2	48.0	50.5	19.8
65–74	10.5	18.4	10.3	23.0
75–84	7.4	9.2	4.1	23.0
>84	2.1	5.1	1.0	8.1
Gender (%)					0.205
Male	87.1	92.7	91.7	95.0
Race					0.009
White	67.4	63.3	63.9	57.2
Black	12.6	13.3	17.5	13.8
Hispanic	5.3	11.2	6.2	4.6
Other	5.3	8.2	8.3	7.1
Unknown	9.5	4.1	4.1	17.3
Marital Status (%)					0.094
Single	16.8	11.2	14.4	9.9
Married	39.0	50.0	45.4	58.7
Separated	34.7	29.6	35.1	22.6
Widowed	6.3	6.1	4.1	7.1
Unknown	3.2	3.1	1.0	1.8
Rurality (%)					0.83
Urban	62.1	63.3	61.9	63.3
Rural	32.6	29.6	32.0	33.6
Highly rural	4.2	5.1	6.2	2.8
Unknown	1.1	2.0	0.0	0.4
Medical comorbidities (%)
Alcohol (%)	24.2	26.5	60.8	7.4	<0.001
Liver disease	17.9	10.2	15.5	3.9	<0.001
Hypertension	62.1	67.4	56.7	63.6	0.473
Diabetes	26.3	35.7	27.8	23.3	0.122
COPD	30.5	20.4	24.7	20.5	0.204
CVD	2.1	6.1	9.3	9.5	0.101
Depression	49.5	60.2	66.0	16.6	<0.001
Anxiety	14.7	16.3	19.6	7.8	0.007
Bipolar	5.3	5.1	25.8	0.7	<0.001
Psychosis	4.2	5.1	15.5	2.5	<0.001
Substance	43.2	40.8	62.9	13.1	<0.001
Other	16.8	25.5	21.7	9.9	<0.001

^* p‐Value generated using the chi‐square test statistic refers to any two comparisons.

Note: COPD, chronic obstructive pulmonary disease; CVD, cardiovascular disease.

The test characteristics for each of the three ACD algorithms varied according to the specified condition (e.g. diagnosis of PTSD, medications for PTSD). For algorithm one (1–2 outpatient ICD‐9 PTSD codes) the sensitivity and specificity varied somewhat for a diagnosis or medication treatment (Table 2). For example, we found a PPV of 77.9% and a NPV of 90.5% for evidence of a diagnosis of PTSD in the medical chart. In comparison when evaluating for evidence of medications for PTSD, for algorithm one, we found a PPV of 33.7% and an NPV of 92.6%.

Table 2.

Test characteristics for each coding algorithm for evidence of a PTSD diagnosis and evidence of pharmacotherapy for PTSD

	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
PTSD Diagnosis
ACD Algorithm 1 (n = 190)	89.2	80.4	77.9	90.5
ACD Algorithm 2 (n = 191)	97.9	96.8	96.9	97.9
ACD Algorithm 3 (n = 192)	88.0	90.2	90.7	87.4
PTSD Pharmacotherapy
ACD Algorithm 1 (n = 190)	82.1	58.3	33.7	92.6
ACD Algorithm 2 (n = 191)	100.0	86.1	84.7	100.0
ACD Algorithm 3 (n = 192)	89.9	77.0	73.2	91.6

Sensitivity analyses

We also examined the ROC for each of the diagnostic algorithms for the chart review agreement with a PTSD diagnosis and medication treatment for PTSD continuously varying the number of outpatient diagnostic ICD‐9 codes (e.g. ≥ 1, ≥ 2, ≥ 3, … etc.). Figures 1 and 2 demonstrate the ROC for each of the specified conditions. Increasing the number of codes required to make a diagnosis in general improves PPV but at the cost of more false negatives (decreased sensitivity). For example, requiring two or more outpatient codes to consider PTSD a diagnosis in a study would improve the likelihood the medical record has evidence of PTSD being diagnosed, however in our study this would result in the assignment of patients with only one PTSD code as being negative for PTSD. Examining the data, requiring two or more outpatient codes for evidence of a diagnosis of PTSD in our study would have resulted in coding 16.5% of outpatient samples as false negatives (Appendix 2, Table A1).

Figure 1.

Receiver operator characteristic (ROC) curves for continuous measure of the number of ICD‐9 codes agreement when evaluating evidence for PTSD diagnosis.

Figure 2.

Receiver operator characteristic (ROC) curves for continuous measure of the number of ICD‐9 codes agreement when evaluating evidence of medications linked to PTSD.

Discussion

VHA administrative data dominate the literature as the main source for exploring novel associations between PTSD and health related outcomes. Gaining insight into the inherent biases introduced by ACD coding strategies is a vital area of study. This report highlights important clinical differences between the different coding algorithms and illustrates the test properties for three commonly used PTSD coding algorithms compared to a structured EMR review.

Three main findings are emphasized. First, each of the algorithms appeared to identify populations of veterans with somewhat differing demographic and clinical characteristics. Veterans coded with PTSD from ACD algorithm one were, on average, younger and somewhat more likely to be female. Similarly, the medical and psychiatric comorbidity rates varied somewhat by PTSD coding algorithm (Table 1). Second, PPV varied according to the ACD when assessing for the presence of a diagnosis of PTSD. For example, algorithm one had a PPV of 77.9% followed by a PPV of 97% for algorithm two, and 90.7% for algorithm three. The differences in PPV according to each ACD algorithm were substantially greater when examining for PTSD medication treatment. For example, algorithm one had a PPV of only 33.7%, whereas algorithm two resulted in a PPV of 84.7%. Third, similar differences between the ACD algorithms were evident when examining NPV for both a PTSD diagnosis and for PTSD medication treatment. Regarding a PTSD diagnosis, algorithm one revealed n = 9 false negatives (NPV = 90.5%), algorithm two had n = 2 false negatives (98%), and algorithm three had n = 12 false negatives (87.4%). For PTSD medication treatment, algorithm two had zero false negatives (NPV = 100%) compared to n = 7 (NPV = 92.6%) and n = 8 (NPV = 91.6%) for algorithms one and three, respectively.

This study provides deeper insight into the importance of understanding the operational definition of PTSD applied using administrative data sources to conduct PTSD research. While it is clear that administrative data can readily provide answers to important policy questions, the administrative coding definition used will ultimately impact the relationship between health‐related outcomes observed. In general our findings of a PPV of 77.9% for algorithm one and 96.9% for algorithm two, for PTSD diagnosis, are similar to those reported by prior authors (Frayne et al., 2010; Gravely et al., 2011; Magruder et al., 2005; Yeager et al., 2007). For example, Gravely et al. (2011) reported a PPV of 82% when ≥ 2 PTSD (309.81) codes were used as the administrative coding algorithm. In this study the gold standard for a diagnosis of PTSD was defined as a score of ≥ 50 on the PCL. The authors went on to recommend researchers use at least two PTSD codes when searching administrative data for cases of PTSD. However, upon closer examination of the study, the PCL scores of patients with only one PTSD code had substantial PTSD symptoms; patients ultimately considered negative for PTSD by this coding algorithm (e.g. false negatives). Yeager et al. (2007) suggests that a PCL cutoff of ≥ 50 may be too high for a Veteran primary care population; thus as a substantial number of the veterans analyzed in the Gravely et al. (2011) study had PCL scores less than 50 (e.g. 30–49) and would have been defined as false negatives (e.g. only one PTSD coded encounter).

Given the PTSD symptom burden of these patients, one could argue that clinicians were correctly identifying (and coding) PTSD in the administrative record but such cases were of moderate severity or potentially well treated thus resulting in a “less than threshold” level of PTSD symptoms needed to code a diagnosis of PTSD. Comparing our results to the Gravely et al. (2011) study we argue that this could be the case. For example, using the ACD algorithm one we found a PPV of 77.9%; suggesting that patients with only one or two codes are indeed believed to have PTSD by providers generating the medical records.

To more directly compare our study to Gravely et al. (2011) we performed a series of sensitivity analyses in order to determine the sample size changes with increasing the stringency of the number of PTSD codes. Going from at least one PTSD code to at least two PTSD codes, n = 63 (16.5%) veterans with medical chart evidence for PTSD would be categorized as not having PTSD (false negatives). This was a lower proportion to the 40% rate as reported by Gravely et al. (2011). The ROC displayed in Figure 1 graphically displays the tradeoff between patients identified with versus without PTSD by assigning different cutoff values for the number of PTSD codes (e.g. one or more, two or more, three or more PTSD codes, etc.). Moving from the right upper corner of the ROC to the left, the first data point represents one or more PTSD codes and the second data point represents two or more PTSD codes, thus the sensitivity drops off substantially without a substantial decrease in the number of false positive cases.

In general, we concur that having two or more outpatient PTSD diagnosis codes as a criteria will improve the PPV, however we do not find adequate evidence that enough patients with only one PTSD diagnosis code lack substantial evidence of PTSD by medical records review. Assigning a requirement of two or more PTSD diagnosis codes will effectively assign a substantial portion of patients with PTSD symptoms and medical chart evidence into the negative PTSD category which, depending on the research question, could move the study results towards the null.

This study also moves the knowledge base forward in an area that is currently lacking decisive data. We demonstrate the need to calibrate the administrative data algorithm to the research question being asked (Table 2). For example, if a researcher is interested in identifying rates of medications used in (for) PTSD, assigning one or two diagnosis codes (algorithm one) for the identification of PTSD and then examining for medications used in this population may result in a high false positive rate, thus potentially falsely attributing a medication for a diagnosis of PTSD. As our study demonstrated that the evidence of medications for PTSD revealed a PPV of only 33.7% (false positives > true positives) suggesting that this approach may be inadequate for research questions pertaining to PTSD medication treatment rates or outcomes. However, raising the threshold to three or more PTSD diagnosis codes improved the PPV to 84.7% while maintaining zero false negatives (NPV = 100%).

This is an important finding considering the number of recent studies that have been focused on examining the delivery of evidenced‐based medication therapies in PTSD. Of the published studies, nearly all have used less stringent criteria in identifying PTSD (e.g. using one more codes) (Bernardy et al., 2012; Harpaz‐Rotem et al., 2008; Mohamed and Rosenheck, 2008; Seal et al., 2010; Spoont et al., 2010). Our findings highlight an important question that has been raised in prior publications, “how can we be sure that the use of these medications is for PTSD and not for a competing comorbidity?” Given that nearly 84% of veterans with PTSD have additional psychiatric comorbidity, this is a crucial limitation of the extant literature (Kessler et al., 1995). These findings emphasize the need to use more stringent criteria (e.g. requiring three or more PTSD codes) when assessing meaningful associations between PTSD and medication use.

There are a few limitations of this study that require disclosure. Our sample was initially selected based on the administrative record identifying PTSD thereby one of the three ACD algorithms. In general, when reporting on sensitivity and specificity a random selection of patients in the population representing the true prevalence of the disease under examination should be validated against a gold standard. With an estimated PTSD prevalence rate of 15% over 1000 charts would have been needed to estimate a sensitivity and specificity with a 95% confidence interval; thus in order to reduce the number of charts reviewed, we created a sample with the prevalence of 50% based on whether or not each patient met one of the three defined ACD algorithms for PTSD. An alternative approach could have used a sample population enriched with PTSD (e.g. mental health clinics), but the findings would not have been broadly applicable to VHA outpatient population. Another limitation reflects the choice to employ medical records as the gold standard as documentation quality may vary from chart to chart. Finally, we did not report the level of agreement between the independent abstractors or between the nurse abstractor and the 20% random assessments by the primary adjudicator (TEA). As a result of these notable limitations we focus the results of this paper on the PPV and NPV test results which remain valid and less sensitive to disease prevalence in the context of this study.

In summary, performance characteristics for identifying a diagnosis of PTSD and medication therapy for PTSD were different depending on the coding algorithm employed. We recommend, for research questions pertaining to the presence of a diagnosis of PTSD, one or two codes of PTSD in an administrative record may suffice. In contrast, we recommend for research questions pertaining to PTSD medication therapy, three or more codes be considered.

Declaration of interest statement

The authors report no conflicts of interest.

Appendix 1. Medical records chart abstraction form

Section 1. Demographics

1. Study ID Number	│ │ │ │ │ │
2. Social Security Number	│ │ │ │ │ │ │ │ │ │
a. Age	│ │ │
b. Gender	1 = Male	2 = Female 99 = Unknown/not doc
c. Race	1 = White	3 = Asian 99 = Unknown/not doc
	2 = Black	4 = Other 99 = Unknown/not doc
a. Ethnicity	1 = Hispanic	2 = Not Hispanic 99 = Unknown/not doc
d. Marital status (most recent documentation)	1 = Single	3 = Divorced/Separated
	2 = Married	4 = Widowed 99 = Unknown/not doc
e. Service connection	1 = 0% 1 = 1‐20% 2 = 21‐50% 3 = 51‐70% 4 = >70% 99 = Unknown/not doc

Section 2. Is there a Mental Health Consult Note? 0 = No 1 = Yes 99 = Unknown/not doc

If YES continue, If NO proceed to Section 3
If both inpatient and outpatient mental health consults present: Select the consult that occurs first.
If YES continue, If NO proceed to Section 3
Location consult:	0 = Inpatient	1 = Outpatient	99 = Unk/not doc
HISTORY AND MENTAL STATUS EXAM:
Depression Assessment (PHQ‐9/Diagnostic symptoms):
a. Low mood (dysphoria)/feeling down/depressed	0 = No	1 = Yes	99 = Unknown/not doc
b. Low interest (anhedonia)/little pleasure	0 = No	1 = Yes	99 = Unknown/not doc
c. Change in sleep (insomnia)/trouble falling asleep or staying asleep	0 = No	1 = Yes	99 = Unknown/not doc
d. Change in energy/ feeling tired/little energy	0 = No	1 = Yes	99 = Unknown/not doc
e. Change in guilt/feeling bad about yourself	0 = No	1 = Yes	99 = Unknown/not doc
f. Change in concentration/trouble concentrating on television or newspaper	0 = No	1 = Yes	99 = Unknown/not doc
g. Change in appetite/poor appetite/overeating	0 = No	1 = Yes	99 = Unknown/not doc
h. Change in psychomotor activity/moving slowly/speaking slowly or very restless/can't sit still.	0 = No	1 = Yes	99 = Unknown/not doc
i. Suicidal ideation	0 = No	1 = Yes	99 = Unknown/not doc
PTSD ASSESSMENT:C2, C4, C5, C6, C7, D1, and D5
Criterion A: Was a trauma history elicited?	0 = No	1 = Yes	99 = Unknown/not doc
Criterion C: Avoidance and numbing.
a. Avoidance of thoughts, feelings, and conversations	0 = No	1 = Yes	99 = Unknown/not doc
b. Greatly reduced interest in related activities	0 = No	1 = Yes	99 = Unknown/not doc
c. Detachment or estrangement feelings	0 = No	1 = Yes	99 = Unknown/not doc
d. Restricted range of affect	0 = No	1 = Yes	99 = Unknown/not doc
e. Sense of a foreshortened future	0 = No	1 = Yes	99 = Unknown/not doc
Criterion D: Arousal
f. Difficulty sleeping	0 = No	1 = Yes	99 = Unknown/not doc
g. Exaggerated startled response	0 = No	1 = Yes	99 = Unknown/not doc

Section 3. Psychiatric History Documented psychiatric history from available encounter notes.

(Abstracted from clinic encounter note, inpatient, d/c summary, or problem list)

9. a.) Documented history of previous mental illness?0 = No 1 = Yes 99 = Unk/not doc
(IF YES, CHOOSE ALL THAT APPLY)
	1 = Depression	2 = Traumatic Brain Injury
3 = Anxiety Disorder	4 = PTSD	5 = Bipolar Disorder
6 = Psychotic Disorder	7 = Substance Abuse Disorder	8 = Personality Disorder
9 = Other _____________
b.) Previous hospitalization for mental health condition? 0 = No 1 = Yes 99 = Unk/not doc
(IF YES, CHOOSE ALL THAT APPLY)
	1 = Depression	2 = Traumatic Brain Injury
3 = Anxiety Disorder	4 = PTSD	5 = Bipolar Disorder
6 = Psychotic Disorder	7 = Substance Abuse Disorder	8 = Personality Disorder
9 = Other _____________
c.) Past suicidal ideation or attempt present? 0 = No 1 = Yes 99 = Unk/not doc
b) If yes, indicate all classes of medications taken.
…First examine Medications Tab.
….Next in each clinic note examine for active medications.
….Next go to Reports, clinical reports, pharmacy, all outpatient medications.
(CHOOSE ALL THAT APPLY)
1 = SSRI	2 = SNRI	3 = Mixed Class	4 = Antipsychotics
5= Mood stabilizers	6=Benzodiazepines	7 = other psychotropic (list ____________)	99 = Unk/not doc
(list specific drugs associated with PTSD Rx)______________________________________________________________
c) Is there evidence for past psychotherapy?	0 = No	1 = Yes	99 = Unk/not doc
(If yes list type of therapy provided or planned (e.g. PE, CBT, CPT, Supportive)____________________________)
10. Psychiatric history
1. First look on cover sheet for diagnoses – ensure that date of diagnosis falls within specified time frame if no MI
2. Look in primary care notes under Past Medical History section if no MI
3. Examine D/C summary section and look for admissions to psychiatry service
Documented history of previous mental illness?0 = No 1 = Yes 99 = Unk/not doc
(IF YES, CHOOSE ALL THAT APPLY)
	1 = Depression	2 = Traumatic Brain Injury
3 = Anxiety Disorder	4 = PTSD	5 = Bipolar Disorder
6 = Psychotic Disorder	7 = Substance Abuse Disorder	8 = Personality Disorder
9 = Other _____________
b.) Previous hospitalization for mental health condition? 0 = No 1 = Yes 99 = Unk/not doc
(IF YES, CHOOSE ALL THAT APPLY)
	1 = Depression	2 = Traumatic Brain Injury
3 = Anxiety Disorder	4 = PTSD	5 = Bipolar Disorder
6 = Psychotic Disorder	7 = Substance Abuse Disorder	8 = Personality Disorder
9 = Other _____________
c.) Past suicidal ideation or attempt present? 0 = No 1 = Yes 99 = Unk/not doc
	FINAL ASSESSMENT
1. Has a past or current diagnosis of PTSD		_____________
2. Has a past or current diagnosis of Depression		_____________
Score 1 – 5
3. Is currently being managed for PTSD and being monitored for symptoms of PTSD			_____________
4. Is currently being managed for PTSD and being monitored for symptoms of Depression			_____________
Score 1 – 5
5. Is currently being treated for PTSD with medications		_____________
6. Is currently being treated for Depression with medications		_____________
Score 1 – 5
7. Is currently being treated for PTSD with behavioral therapies		_____________
8. Is currently being treated for Depression with behavioral therapies		_____________
Score 1 – 5

Each of the four parameters will be assessed using a 5‐point Likert scale with the following categories: 1=definitely; 2=highly likely; 3=possibly; 4=highly unlikely; and 5=definitely not.

Appendix 2.

Table A1. Additional table demonstrating the numbers of veterans with a specific number of PTSD coded encounters. These veterans would represent the potential icrease in false negatives as the number of PTSD coded encounters increases

Number of PTSD coded encounters	N	Percent
1	63	16.5
2	32	8.4
3	10	2.6
4	11	2.9
5	7	1.8
≥6	70	18.4

Examining only cohorts one and two.

Abrams, T. E. , Vaughan‐Sarrazin, M. , Keane, T. M. , and Richardson, K. (2016) Validating administrative records in post‐traumatic stress disorder. Int. J. Methods Psychiatr. Res., 25: 22–32. doi: 10.1002/mpr.1470.