开源工具Soup发布v0.72.4版本，通过层流式处理技术使用户能在仅有4GB显存的笔记本GPU上微调8B参数的大语言模型[1]。新版本新增对DPO、ORPO、SimPO和KTO等多种偏好学习算法的支持[1]。
该项目采用了创新的内存管理方案来降低显存占用[1]。根据在RTX 3050 4GB显卡上的测试，流式DPO的峰值显存为同等条件下监督微调的0.914倍[1]。通过使用相同的流式基座并禁用适配器作为参考模型，对比模型所需的额外显存仅为730MB[1]。该工具还采用4bit NF4量化设置（约4倍压缩）来适配4GB显卡的限制[1]。Soup采用Apache-2.0开源许可证[1]。
Soup, an open-source tool, enables users to fine-tune 8-billion-parameter language models on laptops equipped with just 4 GB of GPU memory through layer streaming technology [1]. The project's latest release, version 0.72.4, introduces support for preference learning algorithms including DPO, ORPO, SimPO, and KTO [1].
The tool achieves this capability through innovative memory management techniques. When tested on an RTX 3050 4 GB graphics card using streaming DPO, peak GPU memory consumption reached 0.914 times that of supervised fine-tuning [1]. For preference learning scenarios, the reference model requires only an additional 730 MB of GPU memory when using the same streaming architecture with adapters disabled [1]. The quantization approach employs 4-bit NF4 compression, reducing the 8B model to fit within 4 GB constraints [1]. Soup is distributed under the Apache 2.0 open-source license [1].