Janas-LLM是一款开源工具，可在普通计算机上运行超过内存容量的大型混合专家语言模型1。该工具通过从NVMe固态硬盘流式传输模型权重的方式实现这一功能1。在搭载32GB内存、配备Intel Core Ultra 9 185H处理器和1TB NVMe固态硬盘的笔记本电脑上，Janas-LLM成功运行了80亿参数的Qwen3-Next模型1。
该模型的运行速度达到提示读取时37 tok/s（每秒令牌数）和无草稿生成时24 tok/s，明显优于llama.cpp的2.8 tok/s1。在所有对比模型上，Janas-LLM均快于llama.cpp1。工具支持Linux x86-64（Ubuntu 22.04、Debian 12及以上）和Windows 10/11系统1，支持code completion、MCP客户端集成和HTTP API等功能1。
该项目采用GNU GPL v3或更高版本许可证，由Maurizio "camauri" Cammalleri设计编写1。支持的模型来自Qwen、Google等开源项目，包括Qwen3、Qwen3.5、Qwen3.6、Qwen3-Next和Gemma 4等1。性能测试于2026年9月26日至10月12日期间进行1。
Janas-LLM, an open-source tool, enables users to run large mixture-of-experts language models exceeding their computer's RAM capacity by streaming model weights from NVMe SSDs 1. The software successfully demonstrates running Qwen3-Next with 8 billion parameters on a 32GB laptop, achieving performance that surpasses llama.cpp 1.
On a notebook equipped with 32GB DDR5 RAM, Intel Core Ultra 9 185H processor, and 1TB NVMe SSD running Debian 13, Janas-LLM delivers 37 tokens per second during prompt ingestion and 24 tokens per second for draft generation, compared to llama.cpp's 2.8 tokens per second 1. Performance benchmarks across all tested models show Janas-LLM consistently outperforming llama.cpp under identical hardware conditions, with testing conducted between September 26 and October 12, 2026 1.
The tool supports models from open-source projects including Qwen3, Qwen3.5, Qwen3.6, Qwen3-Next, and Gemma 4 1. Janas-LLM is distributed under the GNU GPL v3 or later license and was designed and developed by Maurizio "camauri" Cammalleri 1. The software runs on Linux x86-64 systems such as Ubuntu 22.04 and Debian 12 or above, as well as Windows 10 and 11, and includes features for code completion, MCP client integration, and HTTP API access 1.
评论
还没有评论，欢迎留下第一条。