Metacognitive training and AI-based Socratic scaffolding in pre-service teachers’ reflective practice: a dual-study experimental design

This study examined whether metacognitive strategy training strengthens the contribution of an AI-based Socratic reflection system to pre-service teachers’ reflective practice. Study 1 used a 2 (metacognitive training: present vs. absent) × 2 (tool condition: AI-based Socratic scaffolding vs. traditional digital tools) randomized between-subjects design ( N = 60; n = 15 per cell). Written reflection quality at post-test was analyzed by ANCOVA with pre-test performance as the covariate. The metacognitive-training main effect was significant, F (1, 55) = 33.89, p < 0.001, η 2 p = 0.381, as was the tool-condition main effect, F (1, 55) = 9.63, p = 0.003, η 2 p = 0.149. More importantly, the metacognition × tool interaction was significant, F (1, 55) = 10.91, p = 0.002, η 2 p = 0.165. Under metacognitive training, AI-based Socratic scaffolding produced a significant model-estimated advantage over traditional digital tools (Δ M = 0.56, p < 0.001), whereas the tool difference was not significant without metacognitive training (Δ M = −0.03, p = 0.821). Study 2 examined dialogue logs from the two AI conditions. Participants who received metacognitive training generated a substantially larger proportion of higher-order prompts than participants who received placebo training (44.1% vs. 12.2%), t(28) = 7.14, p < 0.001, d = 2.61. These process data are consistent with the proposed account that metacognitive preparation changes how learners engage with AI, although they do not constitute a formal mediation test. The findings suggest that, within this configured Socratic AI environment, metacognitive preparation is an important condition for realizing higher-quality reflective practice.