PROMETHEVS引擎近日发布v0.2版本,宣称在配备Ryzen 9 3950X CPU、64GB DDR4内存及RTX 3060显卡(12GB显存)的普通PC上成功运行完整的DeepSeek-V4.1-Flash-1017大模型 1。该突破得益于由Ryan Flavell发明的“Compress-then-Correct”(先压缩后校正)技术,该方法通过2-bit副本进行初步计算,并在模型置信度较低时利用存储的校正平面恢复原始权重以执行精确计算,从而在保持接近全模型输出质量的同时显著降低了硬件门槛 1。测试数据显示,该技术使代码提示词下的生成速度最高达到7.4 tokens/s(预热缓存状态下),相比同一设备上直接运行全模型的2.14 tokens/s有显著提升 1。
在准确度方面,针对960个审计单词的测试表明,经过校正后的输出与完整模型选择相同单词的概率为98.3%,而纯2-bit副本的匹配率为97.4% 1。尽管性能表现优异,但该方案对存储资源要求较高,模型原始权重约510 GB,构建后的存储库需占用约1,009 GiB磁盘空间 1。Promethevs提供了兼容OpenAI API的本地端点(http://127.0.0.1:8100/v1),支持流式传输和工具调用功能 1。引擎本身采用Apache-2.0许可完全免费开源,报告和数据使用CC BY 4.0许可,而被运行的DeepSeek模型则遵循MIT许可 1。目前该引擎存在一定局限性,仅支持Windows系统和NVIDIA RTX 30/40系列显卡,且为单请求处理模式,长提示词的首字延迟较高,例如处理1,163 token的提示词需耗时约126秒 1。
PROMETHEVS, an engine associated with MAGNVS Ltd and utilizing technology invented by Ryan Flavell, has released version 0.2 to run the complete DeepSeek-V4.1-Flash-1017 model locally on hardware equipped with only 12GB of video RAM 1. The breakthrough relies on a "Compress-then-Correct" technique that allows the model to operate on a standard PC configuration featuring an AMD Ryzen 9 3950X CPU, 64 GB of DDR4 RAM, an NVIDIA RTX 3060 GPU, and dual NVMe drives 1. This approach significantly lowers the barrier to entry for running large language models locally while maintaining output quality close to that of the full model 1.
The core mechanism involves initial computations performed using a 2-bit compressed copy of the model weights 1. The original model weights occupy approximately 510 GB, while the constructed repository requires about 1,009 GiB of disk space 1. When the model's confidence in its predictions is low, the system utilizes stored correction planes to restore original weight values for precise calculation 1. Performance metrics indicate that this method achieves word selection accuracy matching the full model in 98.3% of cases across a 960-word audit sample, compared to 97.4% accuracy when relying solely on the pure 2-bit representation 1. In terms of speed, the engine generates up to 7.4 tokens per second on code prompts with warmed caches, a significant improvement over the 2.14 tokens per second observed when attempting to run the full model directly on the same hardware 1.
PROMETHEVS provides an OpenAI-compatible local endpoint at http://127.0.0.1:8100/v1, supporting streaming responses and tool calls 1. The software is distributed under the Apache-2.0 license, while accompanying reports and data are licensed under CC BY 4.0; the underlying DeepSeek model itself remains available under the MIT license 1. Despite its efficiency gains, the current release has notable limitations, including exclusive support for Windows operating systems and NVIDIA RTX 30/40 series graphics cards 1. Additionally, the engine processes requests one at a time, resulting in high latency for the first token when handling long prompts, with a 1,163-token input requiring approximately 126 seconds before generation begins 1.
评论
还没有评论,欢迎留下第一条。