Optimizing Symptom Surveys with Machine Learning to Predict PTSD One Year Post-trauma

Abstract

Post-traumatic stress disorder (PTSD), a lasting mental health disorder, can be a burden to monitor for patients and clinicians. This work introduces AI-driven methods to understand the predictive value of behavioral symptom surveys for PTSD diagnosis one year post-trauma. We find that surveys issued at regular time intervals can predict PTSD diagnosis with moderate accuracy of around 75% (AUC ~ 0.83). Then, using machine learning algorithms for feature selection, we develop a survey minimization method in which a patient is only required to answer up to seven questions at each time point. Finally, we introduce a tree-based algorithm for individualized, adaptive surveys that approximate PTSD checklist (PCL-5) scores with high precision, allowing for prediction of diagnoses with comparable accuracy to the full survey set. These findings offer resource-efficient AI interventions for PTSD monitoring that are readily deployable in a clinical setting.

I. Introduction

Post-traumatic stress disorder (PTSD) is a trauma-induced mental health disorder that varies in severity over time [1]. Diagnosis of PTSD requires patients to undergo structured diagnostic interviews with clinicians, such as the clinician-administered PTSD scale (CAPS) [2]. As an alternative, survey-based psychological assessments are easier to administer and do not require a clinician; the 20-item PCL-5 is a common diagnostic survey [3]. PTSD has comorbidities with other mental disorders, such as depression and general anxiety. Moreover, some pre-existing factors, like childhood traumatic experiences, have been thought to be risk factors for PTSD in adulthood. Therefore, PTSD self-report assessments are usually accompanied by other mental health survey assessments for clinical and differential diagnoses. Regularly spaced clinical surveys are an important tool to understand changes in PTSD symptoms, but these surveys are often relatively long, placing a burden on patients.

Some of the questions in survey assessments may be redundant or uninformative, so we propose AI-driven feature selection algorithms that provide an upper bound on the number of questions asked at each time point. Furthermore, different questions may be relevant to different patients, a problem often addressed by computerized adaptive testing/diagnosis (CAT/CAD) [4], [5]. We propose a tree-based CAT algorithm to predict PCL-5 scores, which in turn can be used to predict a clinical CAPS diagnosis. With these methods, we show that combining AI-driven learning with robust clinical evidence can support personalized, efficient mental health monitoring.

II. Related Work

There is a strong body of work that applies machine learning to the diagnosis of PTSD and other mental health disorders [6]. This work includes survey minimization algorithms that aim to reduce the length of questionnaires, such as the PCL-5 [7], [8]. These approaches apply a variety of methods, including random forest, stepwise regression, and exploratory factor analysis. Results show that the PCL-5 may be shortened by as much as 80% with a minor loss in utility, supporting the development of survey reduction approaches. Another study used feature selection to minimize the number of supplementary survey questions given to a patient and achieved a high reduction in question count without compromising CAPS diagnosis at three months post-trauma [9]. Other approaches include natural language analysis of data from clinical interviews [10].

Researchers have also applied machine learning to understand how PTSD diagnoses change over time. Relevant approaches include using classification algorithms to predict diagnosis immediately after trauma and six months after trauma [11], as well as the deployment of clustering algorithms to identify prototype disease progression trajectories over two years [12]. However, there has not yet been a significant effort to reduce the length of surveys in the context of longitudinal PTSD trajectory assessment.

Prior work has also developed CAD/CAT algorithms for PTSD diagnoses. To develop an adaptive PTSD survey for Iraq war veterans, one CAT algorithm used item response theory, a mathematical framework linking observed question responses to latent traits. [13]. Approaches beyond item response theory often rely on tree-based algorithms, which have a significant history in the field of machine learning. Decision trees have proved useful for a host of mental health disorders, including depressive disorder, anxiety, and schizophrenia, and they may be deployed either as individual trees or as ensemble models [14]–[16]. A recent approach used the M5Prime algorithm, an extension of foundational decision tree algorithms [17] that incorporates a linear model at each leaf node to predict a continuous diagnosis measure. This approach uses smoothing and pruning procedures to stabilize predictions, while providing individualized surveys through distinct paths from the root of the tree to the leaf node. In this sense, tree-based algorithms provide interpretable, adaptive surveys with minimal computational requirements.

The aforementioned approaches have yet to establish how minimized or adaptive surveys can ease longitudinal monitoring of PTSD, a process that historically involves repeated administration of full surveys.

III. Methodology

A. Data Description

Developing survey minimization and adaptive testing algorithms requires appropriate clinical data. We make use of a longitudinal PTSD development dataset collected at the University of Toledo, where participants were monitored for a year after trauma-inducing events such as motor vehicle accidents and interpersonal assaults [18], [19]. The study was approved by the University of Toledo Institutional Review Board (protocol #201575). Participants were recruited from trauma survivors admitted to the hospital within 48 hours of the traumatic event. They completed the PCL-5 at enrollment as an initial evaluation (IS), and follow-up surveys were administered at two weeks (2W), three months (3M), six months (6M), nine months (9M), and twelve months (12M) post-trauma, respectively. They also completed the ASDS [20], QIDS [21], PASS [22], DERS [23], CTQ [24], and PSQI-A [25] questionnaires. Given that PTSD has many comorbidities, such as anxiety and depression, as well as additional risk factors, these surveys are valuable supplements to the PCL-5 [26]. See Table I for summary statistics and a full list of the questionnaires administered at each time point.

TABLE I.

Demographic and Clinical Characteristics

Variable	Value
Demographics & Trauma History
Age (Mean ± SD)	34.0 ± 10.5
Gender (% Female)	70.5%
Trauma Type (%)
Motor Vehicle Crash	50.9%
Physical Assault	40.2%
Sexual Assault	6.3%
Other	2.7%
Race (%)
Black	51.8%
White	40.2%
Other	8.0%
First CAPS Diagnosis (% PTSD)	45.5%
Second CAPS Diagnosis (% PTSD)	41.1%
Clinical Assessments	Included Time Points
PCL-5	IS, 2W, 3M, 6M, 9M, 12M
ASDS	2W
QIDS	2W, 3M, 6M, 9M, 12M
PASS	2W, 3M, 6M, 9M, 12M
DERS	2W
CTQ	2W
PSQI-A	2W, 3M, 6M, 9M, 12M

Note. CAPS = Clinician-Administered PTSD Scale; PCL-5 = PTSD Checklist; ASDS = Acute Stress Disorder Scale; QIDS = Quick Inventory of Depressive Symptomatology; PASS = Panic and Agoraphobia Scale; DERS = Difficulties in Emotion Regulation Scale; CTQ = Childhood Trauma Questionnaire; PSQI-A = Pittsburgh Sleep Quality Index Addendum for PTSD. Timepoints: IS = Initial survey administered days post-trauma; 2W = 2 weeks post-trauma; 3M, 6M, 9M, 12M = 3, 6, 9, 12 months post-trauma

Participants also received a CAPS diagnosis at three and twelve months post-trauma. With a total of 112 participants, 50 received a negative diagnosis at both time points, 35 received a positive diagnosis at both points, 11 received a negative diagnosis at three months and a positive diagnosis at twelve months, and 16 received a positive diagnosis at three months and a negative diagnosis at twelve months. Recent work with the two-week survey data demonstrates that survey and demographic data are sufficient to predict the CAPS diagnosis at three months [9], [27]. The present study aims to extend this work by predicting the CAPS diagnosis at twelve months using the longitudinal survey data.

Data Preprocessing:

To handle missing data, we use mean imputation. However, there are some time points with extensive missing data due to low response rate, particularly at six months and nine months. Therefore, we remove features where at least 90% of values are missing. All features are standardized based on the distribution of the appropriate training set. Study participants reported one of four types of trauma (i.e., motor vehicle crash, physical assault, sexual assault, other), which were mapped to integer values between 0 and 3.

B. Full Survey Analysis

First, to provide an explanatory baseline, we train models to predict the twelve-month CAPS diagnosis using all the demographic data, past diagnoses, and survey responses up to a given time point. For example, for the three-month classification, we use all the survey data from days after the initial trauma, two weeks post-trauma, and three months post-trauma to predict the CAPS diagnosis twelve months post-trauma. We merge the initial days and two-week features due to the temporal proximity, meaning we obtain results from five time points: initial survey/two weeks, three months, six months, nine months, and twelve months.

We consider four machine learning algorithms for PTSD prediction: logistic regression with elastic net penalty [28], random forest [29], gradient boosting [30], and support vector machine with varying kernel types [31]. We use a 5 × 5 nested cross-validation design, where the inner folds are used to tune basic hyperparameters for the models and the outer folds are used for model assessment (Table II). To account for class imbalance in the diagnoses, we apply equal class weights for all algorithms. This design makes use of the scikit-learn library in Python [32] for classification models and cross-validation functionality. In theory, this approach should provide an upper bound on the predictive accuracy of a model at a given time point, since all of the survey data is being used. In practice, this may not be the case, e.g., due to noise in the feature set, as found by [9] in their analysis of the two-week survey data.

TABLE II.

Hyperparameter Tuning Grid

Hyperparameter	Values
Logistic Regression
Inverse Regularization Strength C	10−4, 10−3, …, 102
Penalty Type	Elastic Net
L1 Ratio	0.75, 0.85, 0.95
Random Forest
# Trees	5, 15, …, 95
Split Criterion	Gini Impurity
Maximum Depth	1, 2, … ,9
Maximum Features (n) per Split	None, $\sqrt{n}$, 0.5n, 0.7n
Gradient Boosting
Maximum Depth	None, 3, 5, 10
Learning Rate	0.01, 0.1, 0.2
L2 Regularization	0, 1, 10
SVM
Kernel Type	Linear, RBF, Polynomial, Sigmoid
Inverse Regularization Strength C	0.1, 1, 10, 100
Gamma (if applicable)	Scale, Auto
Degree (if applicable)	2, 3
Regression Tree
Minimum Leaf Size	3, 5, 7, 10
Ridge Alpha	0.1, 1, 10
Maximum Depth	3, 4, 5, 6, 7

C. Top 7 Analysis

In our second experiment, we use AI-driven feature selection to minimize the length of the survey at each time point. To do this, we apply a temporal extension of recursive feature elimination [33]. Recursive feature elimination is a greedy algorithm that iteratively trains a model on a set of features and removes the feature with the lowest weight. This process continues until the top k features are selected.

For our temporal extension, we select the top k features at the first time point. Then, we train an algorithm to classify the twelve-month CAPS diagnosis with these k features. The k features propagate to the next time point, forming a propagation set. Therefore, at the second time point, there is a new feature set containing (1) k features from the propagation set and (2) all the features at the second time point. From this feature set, a new set of k features is selected, and any of the k features obtained from the second time point are added to the propagation set. Therefore, at any time t, our feature set consists of at most k(t − 1) propagated features, in addition to all the features at t. We apply recursive feature elimination again at t to select a new set of k features, which are used to predict twelve-month CAPS diagnosis, and any of the k features obtained from time t are added to the propagation set for t + 1.

Using this approach, a patient has to answer at most k additional questions at each time point. If the questions at a given time point have high utility, they will replace the propagated questions in the top k, and the patient will answer a significant body of questions. However, if the questions have low utility compared to the propagated questions, they will not be added to the top k, and the patient may not be asked any questions.

For our analysis, we select k = 7 based on previous work with this dataset [9]. We make use of the same cross-validation design as the previous experiment, maintaining a different set of k features for each outer fold of the cross-validation in order to prevent test set leakage. Recursive feature elimination is implemented using the same model type as the classification algorithm. For example, a logistic regression can select the standardized features with the highest magnitude model coefficients at each recursive step. Then, another logistic regression classifies using the top k features, with the inner cross-validation now used for hyperparameter tuning.

D. Tree-based Adaptive Survey

To move from survey minimization to adaptive surveys, we first implement a regression tree algorithm to predict PCL-5 scores at a target time. We do not drop any features for this experiment, as we are using a restricted feature set consisting of PCL-5 questions asked at the target time. Our decision tree implementation functions largely as a standard regression tree, making splits at each node based on the maximum child-weighted variance reduction. We control the complexity of the tree by setting thresholds for the maximum depth and the minimum number of samples at a leaf node. However, differing from standard implementations, our tree includes a ridge regression model [34] to predict the PCL-5 score at each leaf node. This model is trained on the features that appear on the path from the root of the tree to the leaf node. Given that there may only be a small number of samples at each leaf node, we utilize ridge regression in an effort to prevent overfitting. The α parameter for the ridge regression, defined in scikit-learn to scale the strength of regularization, is tuned during the cross-validation.

To train this algorithm, we again use a nested cross-validation design. In the hyperparameter grid, the highest maximum depth parameter is set to seven, meaning the patient must answer at most seven questions. To augment the size of the training dataset, we use PCL-5 responses from up to six time points for training set patients. Then, we can make more robust PCL-5 outcome predictions at each of the time points, even if there is significant missing data at any given point. The PCL-5 is offered at both the initial survey and two-week phases, whereas the remaining surveys are only offered at two weeks, so we do not combine the initial survey and two-week time points for this analysis.

Then, using demographic information, the three-month diagnosis, and the predicted PCL-5 scores at the initial survey, two weeks, three months, and twelve months as features, we can predict CAPS diagnosis at twelve months. Since we do not have available data to predict PCL-5 results for many participants at six months and nine months, these predictions are excluded from the feature set. We use the same cross-validation and classification algorithms as previous analyses. The PCL-5 prediction tree is embedded in the cross-validation to prevent test set leakage, meaning the classification models are trained on prediction tree training outputs and tested using prediction tree testing outputs. This mechanism serves to verify that the prediction tree is not overfitting, while simultaneously providing a minimal feature set to the classification algorithm.

IV. Results and Discussion

A. Full Survey Analysis

As seen in Table III, the primary evaluation metrics are accuracy, which can be somewhat misleading given the class imbalance, and AUC, the area under the receiver operating characteristic (ROC) curve, which better reflects how well an algorithm is able to separate the two classes. There is a substantial improvement in twelve-month prediction performance at three months, as the three-month CAPS diagnosis becomes a feature at this time, providing an important source of prior information. The next substantial increase occurs at twelve months, at which point we observe the survey responses closest in time to the twelve-month CAPS diagnosis we are trying to predict. Overall, logistic regression performs best (twelve-month mean AUC = 0.83), although AUC is relatively consistent across algorithms. Performance is also similar across cross-validation folds.

TABLE III.

Classification Accuracy: Full Survey Analysis

Algorithm	Metric	IS/2W	3M	6M	9M	12M
Logistic Regression	Accuracy	0.590 ± 0.033	0.733 ± 0.034	0.716 ± 0.038	0.725 ± 0.037	0.750 ± 0.027
	AUC	0.648 ± 0.017	0.797 ± 0.032	0.801 ± 0.030	0.782 ± 0.026	0.832 ± 0.031
Random Forest	Accuracy	0.651 ± 0.051	0.715 ± 0.040	0.698 ± 0.042	0.688 ± 0.024	0.704 ± 0.070
	AUC	0.659 ± 0.040	0.759 ± 0.031	0.774 ± 0.033	0.749 ± 0.017	0.780 ± 0.046
Gradient Boosting	Accuracy	0.564 ± 0.039	0.716 ± 0.039	0.725 ± 0.035	0.698 ± 0.040	0.713 ± 0.067
	AUC	0.598 ± 0.032	0.799 ± 0.028	0.820 ± 0.032	0.805 ± 0.035	0.767 ± 0.063
SVM	Accuracy	0.697 ± 0.021	0.661 ± 0.045	0.599 ± 0.025	0.680 ± 0.057	0.723 ± 0.048
	AUC	0.514 ± 0.101	0.707 ± 0.037	0.699 ± 0.043	0.621 ± 0.102	0.809 ± 0.028

Note. AUC = Area Under the ROC Curve. Data are presented as Mean ± SE. The best AUC value at each time point is bolded.

B. Top 7 Analysis

Table IV demonstrates that the top 7 algorithm suffers from a drop in AUC as compared to the full feature set. This trend is most evident at the twelve-month diagnosis point, where AUC is lower across the leading algorithms (random forest twelve-month AUC = 0.80). Given the number of cumulative features available at twelve months, it may be that the feature set needs to be expanded at later time points to achieve comparable AUC. Despite the decrease in AUC, this finding demonstrates that survey minimization is a valid approach at twelve months, just as it is at three months [9].

TABLE IV.

Classification Accuracy: Top 7 Analysis

Algorithm	Metric	IS/2W	3M	6M	9M	12M
Logistic Regression	Accuracy	0.552 ± 0.036	0.741 ± 0.016	0.698 ± 0.056	0.661 ± 0.061	0.721 ± 0.059
	AUC	0.616 ± 0.024	0.787 ± 0.021	0.722 ± 0.080	0.702 ± 0.048	0.763 ± 0.072
Random Forest	Accuracy	0.608 ± 0.033	0.661 ± 0.026	0.644 ± 0.025	0.653 ± 0.032	0.706 ± 0.015
	AUC	0.573 ± 0.037	0.709 ± 0.051	0.716 ± 0.039	0.705 ± 0.015	0.795 ± 0.025
Gradient Boosting	Accuracy	0.502 ± 0.045	0.687 ± 0.025	0.643 ± 0.041	0.634 ± 0.040	0.677 ± 0.045
	AUC	0.533 ± 0.037	0.695 ± 0.029	0.709 ± 0.027	0.702 ± 0.028	0.775 ± 0.044
SVM	Accuracy	0.579 ± 0.038	0.679 ± 0.043	0.660 ± 0.029	0.633 ± 0.054	0.731 ± 0.054
	AUC	0.578 ± 0.036	0.738 ± 0.043	0.703 ± 0.022	0.709 ± 0.068	0.783 ± 0.044

Note. AUC = Area Under the ROC Curve. Data are presented as Mean ± SE. The best AUC value at each time point is bolded.

The mean predicted PTSD probability over time using logistic regression for the top 7 analysis is shown in Fig. 1. We observe some interesting trends by separating the data into subgroups based on the three and twelve-month CAPS diagnoses. First, notice that the predictions at the initial time point are very uncertain, as the mean predicted probability for each subgroup is very close to 0.5. This trend is expected, as it is difficult to say whether a patient who has just undergone trauma will be diagnosed with PTSD in twelve months. At three months, the four subgroups separate into two groups, with subjects having positive three-month diagnoses separated from those with negative three-month diagnoses. This pattern persists until the twelve-month survey data are available, at which point we see a separation among patients diagnosed with PTSD at three months. These results suggest that our logistic regression models are also able to pick up changes in PTSD diagnoses over time, separating subjects who recover from PTSD (positive at three months, negative at twelve months) from those who do not. However, we do not see this strong trend when comparing late onset PTSD patients (negative at three months, positive at twelve months) to patients who are never diagnosed with PTSD.

Fig. 1.

Mean predicted PTSD probability over time for the Top 7 Analysis, using logistic regression. Dashed vertical lines denote CAPS diagnosis times. Subgroups are denoted by their CAPS diagnoses at three and twelve months, e.g., P-N for positive at three months and negative at twelve months. Error bars denote standard error of the mean.

C. Tree-based Adaptive Survey

As demonstrated by the strong root-mean squared error and R² scores across time points, the regression tree successfully predicts PCL-5 scores (Table V). This result is intuitive, given that PCL-5 scores are a sum of the component questions. However, we are able to make predictions using at most seven personalized questions, and the model does not significantly overfit at the leaf nodes. The R² varies across the models due to changes in the variance of PCL-5 scores, even as the root-mean squared error is consistent. A comparison between the actual and predicted PCL-5 scores for each subject is shown in Fig. 2.

TABLE V.

Regression Accuracy: Tree-based Adaptive Survey

PCL Score Prediction
Time Point	R2	RMSE
IS	0.612 ± 0.063	7.593 ± 0.730
2W	0.828 ± 0.021	6.104 ± 0.387
3M	0.833 ± 0.019	7.546 ± 0.299
6M	0.788 ± 0.063	7.351 ± 0.906
9M	0.725 ± 0.031	8.899 ± 1.020
12M	0.881 ± 0.022	6.653 ± 0.485
CAPS Diagnosis with Predicted PCL Scores
Algorithm	Accuracy	AUC
Logistic Regression	0.796 ± 0.028	0.891 ± 0.010
Random Forest	0.777 ± 0.041	0.832 ± 0.029
Gradient Boosting	0.785 ± 0.055	0.839 ± 0.030
SVM	0.760 ± 0.031	0.856 ± 0.033

Note. RMSE = Root Mean Squared Error; AUC = Area Under the ROC Curve. Data are presented as Mean ± SE. The best AUC is bolded.

Fig. 2.

Actual vs. predicted PCL-5 scores at each time for the Adaptive Survey Analysis.

Interestingly, the models are able to at least match models using the full feature set at twelve months (logistic regression mean AUC = 0.89). One feasible explanation is that the adaptive PCL-5 surveys serve as a form of feature extraction, engineering the most relevant information from PCL-5 scores and removing noise from individual question responses. Future work should explore whether the same results would extend to a larger, more representative sample.

D. Limitations

Despite these findings, our sample size is limited by the difficulty of longitudinal PTSD monitoring. Our results may fluctuate based on the partitioning of the dataset during cross-validation, meaning that further research should pursue expanded data collection and explore sensitivity analyses related to data partitioning. Furthermore, we choose to model the trauma type as an arbitrary integer encoding, as introducing more features through one-hot encoding would have complicated the interpretation of the survey minimization results. Our encoding mechanism may cause unintended ordering effects. Finally, we cannot account for real-world treatment conditions in our analysis. Participants who are diagnosed with PTSD three months post-trauma may receive additional clinical treatment, impacting a later diagnosis. In clinical settings, participants may respond differently to a question in a full survey and a question in a shortened and/or adaptive survey, necessitating further controlled experiments with optimized surveys in patient populations.

V. Conclusion

In this paper, we determine that survey responses are a valuable tool to predict PTSD diagnosis twelve months post-trauma (best mean AUC = 0.83), especially when surveys are administered close to the diagnosis date. Then, we show that feature selection algorithms for survey minimization can approach this performance (best mean AUC = 0.80). By simplifying the feature space, adaptive surveys can match full survey performance (best mean AUC = 0.89). While past diagnoses and survey responses obtained at twelve months contribute substantially to these results, we demonstrate that AI interventions driven by machine learning can ease the burden of clinical monitoring and diagnosis of PTSD.

Although we developed an adaptive survey algorithm, future work may expand this algorithm to an adaptive survey across time points. Prior work reveals the benefits of temporal ordering in decision trees [35], which may allow for early diagnosis in some participants [36]. Such work would provide another adaptive tool for clinical professionals and their subjects.