Homebench是一款开源命令行工具，旨在对本地部署的大语言模型进行全面基准测试[1]。该工具通过测量tokens生成速度、首个token延迟、内存占用和质量评分等指标，帮助用户在本地硬件上找出性能最优的模型[1]。
Homebench采用零配置设计，用户可通过pip install homebench快速安装，无需配置、API密钥或云端依赖[1]。该工具支持Ollama、LM Studio、llama.cpp、vLLM及OpenAI兼容服务等多个本地运行器[1]。用户可根据需要选择默认快速套件（包含8个任务）或完整套件（包含31个任务）进行评测，所有测试结果自动保存至~/.homebench/runs目录，支持历史对比分析[1]。此外，该工具还支持JSON或YAML格式的自定义评估任务包，内置模型目录涵盖约50个热门开源模型，参数规模从135M至141B不等[1]。
Homebench采用MIT开源许可证发布，需要Python 3.9或更高版本运行[1]。项目源代码托管在GitHub上（https://github.com/david-g-3654/homebench）[1]。
Homebench is an open-source command-line tool designed to benchmark large language models running locally on users' hardware [1]. The tool measures three critical performance dimensions: processing speed, memory consumption, and output quality [1].
Installation is straightforward via pip with no configuration, API keys, or cloud dependencies required [1]. Homebench supports multiple local LLM runners, including Ollama, LM Studio, llama.cpp, vLLM, and OpenAI-compatible services [1]. The benchmarking process tracks metrics such as tokens per second, time-to-first-token latency, memory usage, and quality scores [1]. Users can choose between a quick suite containing 8 tasks or a comprehensive suite with 31 tasks, and all results are automatically saved to ~/.homebench/runs for historical comparison [1].
The tool comes with an embedded model directory covering approximately 50 popular models ranging from 135 million to 141 billion parameters [1]. Advanced users can create custom evaluation task packages in JSON or YAML format to tailor benchmarks to their specific needs [1]. Homebench is released under the MIT open-source license and requires Python 3.9 or higher [1]. The project is available on GitHub at https://github.com/david-g-3654/homebench [1].