Beyond Content Safety: A Scenario-Based Dual-Layer Risk Evaluation Framework for In-Vehicle Large Language Model Voice Assistant Responses

In-vehicle voice assistants are shifting from fixed-intent command systems to conversational agents powered by large language models for open-ended spoken interaction. This shift changes safety evaluation requirements: a response may be unsupported by facts, or it may be factually correct yet still meet the study’s response-form risk criteria when it is too long, information-dense, or choice-heavy for the current driving state. This study proposes a scenario-based dual-layer risk evaluation framework that separates semantic content risks from Proxy-Based Cognitive-Overload Risk (R1), an operational response-form proxy rather than a direct measure of driver cognitive workload. We evaluated 900 responses from GPT-4o, Qwen-Plus, and Spark 4.0 across 300 scenarios spanning four driving states and five task domains. Factual/Contextual Error (R3) and R1 had inclusive presence rates of 41.2% and 39.6%, respectively. State effects and task-by-ordered-state interaction were analyzed; the interaction was not significant $(p=0.179)$ . Sensitivity analyses tested alternative thresholds and trigger rules; the main results and state patterns were largely stable. We also conducted a held-out reproducibility study using 60 new scenarios, 180 new responses, and two new annotators independent of guideline development. Pre-adjudication agreement was 85.6% for six-category primary-risk labels ( $\kappa =0.792; 95 \%$ confidence interval, 0.705–0.871), with quadratic-weighted $\kappa =0.847$ for severity and 81.7% exact agreement for multilabel classification. Overall, the proposed framework offers a repeatable way to assess both content risks and driving-state-related response-form risks for in-vehicle voice assistants powered by large language models.