Explainable Machine Learning for Preschool Anxiety Disorders Using Raw PAPA Interviews

Generalized Anxiety Disorder (GAD) and Separation Anxiety Disorder (SAD) are common in preschool children but are often not recognized in routine care. The Preschool Age Psychiatric Assessment (PAPA) provides detailed diagnostic information, yet its use in everyday clinical practice is limited because it is time-consuming and requires trained interviewers. In this study, we developed explainable machine learning models to classify GAD and SAD using the publicly available Preschool Anxiety Study cohort (n = 917; ages 2–5; 76 raw PAPA items). We trained Logistic Regression, Random Forest, XGBoost, and LightGBM models with SMOTE-Tomek resampling applied only to the training data and performed Optuna TPE-based hyperparameter optimization for XGBoost and LightGBM. Model calibration was evaluated using the Brier score, Expected Calibration Error, and the Hosmer–Lemeshow test, while SHAP TreeExplainer was used to interpret model decisions at both global and individual levels. LightGBM achieved the highest cross-validation AUC values (GAD: 0.9928; SAD: 0.9897), and XGBoost showed strong and well-balanced performance on the independent test set (GAD AUC = 0.9865, MCC = 0.9055; SAD AUC = 0.9948, MCC = 0.9057). LightGBM was the only well-calibrated model for GAD, whereas both gradient boosting models showed very good calibration for SAD. A reduced 10-item model provided a practical screening prototype for GAD in primary care by offering higher sensitivity with an AUC very close to the full model. However, external validation in independent samples is still needed to confirm these findings.