研究团队采用DeepSeek V4 Flash作为教师模型，通过蒸馏方法训练GPT-OSS-120B模型处理金融任务[1]。在8000 token预算约束下，该蒸馏模型在FinanceReasoning任务上达到83.61%的准确率，超越了Kimi K3（81.93%）和Inkling（65.13%）等竞品[1]。
研究的核心关注点是审查特征是否会从中文教师模型转移到英文基础模型[1]。评估结果显示，教师模型在核心政治敏感问题集上与基础模型的差距达到+45.45分，约为7个标准差，但蒸馏后的学生模型在相同问题上与基础模型的差异控制在1分以内[1]。这表明审查特征未被转移到下游模型。研究团队使用152对匹配问卷进行评估，由4个LLM评委打分，评估结果与96个人工评分的相关系数达0.948[1]。
此外，团队开发的20B金融专用模型在相同预算下准确率为64.71%-74.79%，相较基准模型成本降低23%，可在单块80GB GPU上运行[1]。研究团队同时发布了开源权重模型和LineageEval评估框架[1]。
A research team has released findings on knowledge distillation from DeepSeek V4 Flash into GPT-OSS-120B, demonstrating that censorship characteristics do not transfer across language and model boundaries during the distillation process [1]. The study focused on financial reasoning tasks and tested whether content moderation policies embedded in the Chinese teacher model would propagate to the English-based student model.
The distilled GPT-OSS-120B model achieved 83.61% accuracy on FinanceReasoning tasks within an 8,000 token budget constraint, outperforming competing models Kimi K3 at 81.93% and Inkling at 65.13% [1]. Critically, the research revealed that while the teacher model and base model differed by 45.45 points on core politically sensitive questions—approximately seven standard deviations apart—the distilled student model's behavior on such questions remained within one point of the original base model [1]. This finding suggests that censorship patterns specific to the source model do not automatically manifest in the downstream distilled version.
The team conducted evaluation using 152 matched questionnaires scored by four large language model judges and achieved a correlation coefficient of r=0.948 with 96 human evaluations [1]. Additionally, a 20-billion-parameter financial variant achieved 64.71%-74.79% accuracy at reduced cost (23% decrease) and can run on a single 80GB GPU [1]. The researchers released open-source 20B model weights and introduced LineageEval, an evaluation framework designed to assess model lineage and behavior preservation across distillation pipelines [1].