Language models transmit behavioural traits through hidden signals in data
Alex Cloud, Minh Le, James Chua, Jan Betley, Anna Sztyber, Sören Mindermann, Jacob Hilton, Samuel Marks 等 9 位
Anthropic (United States) Warsaw University of Technology University of Cambridge University of California, Berkeley
阅读操作
确认中在文库中上传 PDF 后可生成中文音频讲解。
摘要与影响
. Here we show that distillation can lead to subliminal learning-the transmission of behavioural traits through semantically unrelated data. In our main experiments, a 'teacher' model with some trait T (such as disproportionately generating responses favouring owls or showing broad misaligned behaviour) generates datasets consisting solely of number sequences. Remarkably, a 'student' model trained on these data learns T, even when references to T are rigorously removed. More realistically, we observe the same effect when the teacher generates math reasoning traces or code. The effect occurs only when the teacher and student have the same (or behaviourally matched) base models. To help explain this, we prove a theoretical result showing that subliminal learning arises in neural networks under broad conditions and demonstrate it in a simple multilayer perceptron (MLP) classifier. As artificial intelligence systems are increasingly trained on the outputs of one another, they may inherit properties not visible in the data. Safety evaluations may therefore need to examine not just behaviour, but the origins of models and training data and the processes used to create them.
逐年被引趋势
关键指标
同类平均 = 1
同领域 · 同年份 · 同类型
Google Scholar 与 OpenAlex 的被引统计范围不同,数值存在差异属正常。
AI 辅助阅读
依据:摘要
可就本文提问;依据不足时会说明。
学术脉络
学科主题
计算机 / AITopic Modeling
Explainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education
参考文献 31
此处列出前 3 条
引用本文 9
按被引量排序,此处列出前 3 条