voc2vec: A Foundation Model for Non-Verbal Vocalization
Alkis Koudounas, Moreno La Quatra, Sabato Marco Siniscalchi, Elena Baralis
Politecnico di Torino Università degli Studi di Enna Kore University of Palermo
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
Speech foundation models have demonstrated exceptional capabilities in speech-related tasks. Nevertheless, these models often struggle with non-verbal audio data, such as vocalizations, baby crying, etc., which are critical for various real-world applications. Audio foundation models well handle non-speech data but also fail to capture the nuanced features of non-verbal human sounds. In this work, we aim to overcome the above shortcoming and propose a novel foundation model, termed voc2vec, specifically designed for non-verbal human data leveraging exclusively open-soruce non-verbal audio datasets. We employ a collection of 10 datasets covering around 125 hours of non-verbal audio. Experimental results prove that voc2vec is effective in non-verbal vocalization classification, and it outperforms conventional speech and audio foundation models. Moreover, voc2vec consistently outperforms strong baselines, namely OpenSmile and emotion2vec, on six different benchmark datasets. To the best of the authors’ knowledge, voc2vec is the first universal representation model for vocalization tasks.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AIMusic and Audio Processing
Speech Recognition and Synthesis · Emotion and Mood Recognition