Latent World Models for Autonomous Driving: A Taxonomy and Survey

Autonomous driving is evolving from fragmented modular pipelines toward more integrated system-level architectures for perception–prediction–planning–control coordination. Latent world models (LWMs) provide a compact systems-oriented modeling paradigm for connecting perception, prediction, and decision-making by compressing high-dimensional sensory inputs into latent states that support downstream prediction and planning. This survey reviews LWMs in autonomous driving and establishes a taxonomy based on dominant alignment objectives: feature alignment, value alignment, and behavior alignment. We discuss their applications in latent dynamics, anticipatory driving, predictive control, and safety verification, with particular attention to their integration with model predictive control (MPC) for closed-loop autonomous-driving systems. Furthermore, we summarize benchmark datasets and multidimensional evaluation metrics required by latent architectures, covering latent quality, open-loop accuracy, closed-loop robustness, and reasoning-oriented assessment. Finally, we identify key challenges, including latent physical reliability, latent drift and misalignment, and compositional generalization, and outline future directions such as physics-constrained dynamics, drift-aware representations, and compositional closed-loop training.