SE2Image: Generating Images from Classical Chinese Poetry by Combining Scene and Emotion Descriptions

Classical Chinese poetry is a valuable cultural heritage of humanity, but its comprehension is often challenging due to the need for specialized knowledge. To better disseminate and promote classical Chinese poetry, this paper proposes a framework (SE2Image) that integrates scene and emotion information to visually present the meaning of poems. We utilize large language models to extract scene description and emotion description from poetry translation and poetry appreciation, and these descriptions are then used as prompts for a diffusion model to generate images. To support the implementation of this framework, we have constructed a dataset enriched with knowledge of Chinese poetry. Qualitative and quantitative analyses show that SE2Image effectively generates images that capture the essence of classical Chinese poetry, offering a new avenue for its modernization and dissemination.