Momentic(YC W24)发布了一款基于自主 AI 代理的 QA 工具,旨在通过自动规划和探索来发现 Web、iOS 和 Android 应用中的缺陷 1。该产品由曾就职于 Robinhood 和 Retool 等公司的 Wei-Wei 与 Jeff 创立,其核心功能是模拟用户操作以捕捉真实 Bug 1。除了上述平台,该工具还支持即将推出的桌面端应用测试 1。
此次发布重点展示了名为“Mo”的新代理架构,该架构构建自组织有向无环图(DAG),支持并行执行及专用子代理协同工作 1。在与 Codex、Claude Code 和 Devin 等编码代理进行的基准测试中,Mo 的表现尤为突出 1。数据显示,在 60 分钟的运行时间内,Mo 共发现了 269 个 Bug,召回率为 19.3%,精度为 84%,单个 Bug 的发现成本为 2.67 美元 1。相比之下,Codex 在同一时间段内仅发现 86 个 Bug,召回率为 6.2%,精度为 86%,单个 Bug 成本为 0.80 美元;而 Claude Code 和 Devin 分别发现了 59 个和 27 个 Bug 1。尽管 Mo 的单次发现成本高于 Codex,但其在真实 Bug 的绝对数量上具有明显优势 1。此外,测试表明随着时间推移,Mo 的 Bug 发现能力显著增强,从 30 分钟到 60 分钟期间,发现的 Bug 数量增长了 52% 1。相关数据还引用显示,Sonnet 5.5 模型在禁止编写测试时的通过率为 66.2%,允许编写测试时则为 65.3% 1。目前,新用户注册可获得 250 美元的使用额度 1。
Momentic, a Y Combinator Winter 2024 startup, has introduced an autonomous quality assurance tool powered by AI agents designed to automatically plan, explore, and identify bugs in web, iOS, and Android applications 1. The product is built around a new agent architecture named "Mo," which utilizes a self-organizing directed acyclic graph (DAG) to enable parallel execution and specialized sub-agents for efficient testing workflows 1. Founded by Wei-Wei and Jeff, who previously worked at companies including Robinhood and Retool, the team aims to provide desktop support in addition to mobile and web platforms in the near future 1.
Benchmark Performance Against Coding Agents
In a comparative benchmark test conducted over a 60-minute period, Mo identified 269 bugs with a recall rate of 19.3% and a precision of 84%, resulting in a cost of $2.67 per bug discovered 1. This performance surpassed other prominent coding agents such as Codex, which found 86 bugs with a 6.2% recall rate and 86% precision at a lower cost of $0.80 per bug, as well as Claude Code (59 bugs) and Devin (27 bugs) 1. Although Mo's unit price is higher than that of Codex, it demonstrated superior capability in discovering actual bugs, with its detection count increasing by 52% when the runtime extended from 30 to 60 minutes 1.
To illustrate the limitations of standard models without structured agent frameworks, the company cited data showing that Sonnet 5.5 achieved a pass rate of 66.2% when prohibited from writing tests, compared to 65.3% when allowed to do so 1. To encourage adoption, Momentic is offering new users a $250 credit upon registration 1.
评论
还没有评论,欢迎留下第一条。