淘宝旗下MNN团队于11月6日发布并开源了面向手机应用的轻量级大语言模型MNN-Pocket-0.3B 1。该模型参数量约为3亿,专为有限算力、内存和能耗设计,并提供多个版本以支持从预训练到端侧部署的全流程开发 1。在性能表现上,MNN-Pocket-0.3B在苹果iPhone 18 Pro上实现了平均约350 token/s的离线解码速度;在iPad Pro M5(Metal后端)下,Decode吞吐达466 token/s 1。
通过结构优化与量化技术,该模型在资源占用与能效方面展现出显著优势 1。架构上采用28层结构,结合分组查询注意力(GQA),词表精简至64K(缩减约57%)1。其INT4 MNN部署文件体积为173MB,峰值内存增量控制在300MB以下(iPad Pro测试中增加228MB)1。在一加13手机上,执行100次同等负载任务时,累计增量能耗约占电池总电量的0.3% 1。相较于Qwen3-0.6B,MNN-Pocket-0.3B部署体积减少约49%,峰值内存增量减少约47%,同条件下Decode吞吐为其1.63倍 1。
该模型在中文指令遵循、知识推理等评测中表现优异 1。具体指标显示,其在英文IFEval得分73.20,中文IFEval-zh得分74.59;在中文数学MGSM-zh得分51.20,代码生成HumanEval得分53.66 1。开发者可通过GitHub地址 https://github.com/alibaba/mnn 获取该模型及相关资源 1。
The MNN team under Taobao released and open-sourced MNN-Pocket-0.3B on November 6, a lightweight large language model designed specifically for mobile devices with constraints on computing power, memory, and energy consumption 1. The model features approximately 300 million parameters and utilizes a 28-layer architecture incorporating Grouped Query Attention (GQA), while reducing the vocabulary size to 64K, which is about a 57% reduction compared to standard models 1. The project is publicly available at https://github.com/alibaba/mnn 1.
In terms of performance, MNN-Pocket-0.3B achieves an average decoding speed of roughly 350 tokens per second in offline mode on the Apple iPhone 18 Pro 1. On the iPad Pro equipped with the M5 chip using the Metal backend, the decode throughput reaches 466 tokens per second 1. Resource efficiency is also notable: the INT4 deployment file size is only 173MB, and the peak memory increment is kept below 300MB, with tests showing a 228MB increase on the iPad Pro 1. Energy consumption remains minimal; on the OnePlus 13 smartphone, completing 100 tasks of equivalent load resulted in an incremental energy usage of approximately 0.3% of the total battery capacity 1.
Compared to Qwen3-0.6B, the new model reduces deployment volume by about 49% and cuts peak memory increment by roughly 47%, while delivering 1.63 times the decode throughput under similar conditions 1. Evaluation metrics show strong capabilities across various tasks, including scores of 73.20 on English IFEval, 74.59 on Chinese IFEval-zh, 51.20 on Chinese math task MGSM-zh, and 53.66 on code generation HumanEval 1. To support developers throughout the pipeline from pre-training to edge deployment, the team provides multiple versions of the model 1.
评论
还没有评论,欢迎留下第一条。