来自Letta和UC Berkeley的研究人员提出了一种名为“睡眠计算”(Dreaming)的机制,旨在利用AI智能体的空闲时间对记忆进行重组与修正 1。这项研究深入探讨了为Claude等模型设计记忆系统的核心组件,揭示了智能体从被动记录信息向主动判断显著性、构建结构化Wiki以辅助检索的进化过程 1。研究发现,允许模型自主决定何时记录比强制规则更有效 1。
在不同基准测试中,采用先进记忆策略的模型展现出显著的效率提升与性能差异。在Continual Learning Bench (CL-Bench) 1.0中,Claude Opus 5是首个闭合学习循环的模型,能生成经过验证的规则并标记陷阱 1。而在Pokémon Red测试中,Claude Sonnet 3.7写了413个记忆文件仅获1枚徽章,而Claude Fable 5仅写9个文件加1个全局文件获4枚徽章 1。值得注意的是,Claude Opus 5.5和Sonnet 5.5使用记忆后得分与Opus 5相当,但无记忆基线性能略高 1。
针对大规模文档处理任务,研究人员以Stainless公司的约35万份文件(约4.2亿token)为例,通过128次“做梦”会话构建Wiki,评分标准达成率从13%提升至48% 1。结合搜索工具与Wiki,Claude的聚合分数从单独使用搜索工具的53%升至61%,每题节省约20,000 tokens 1。在法律领域Harvey LAB基准测试中,对于需要跨文档信息的合并案问题,查阅Wiki得分为87%,仅用搜索为73% 1。此外,为了控制注入上下文的成本,Claude Code自动加载MEMORY.md的前200行或25KB作为索引 1。在合同审查任务中,通过迭代指令让每个合同拥有独立页面并逐字引用条款,使简报标准满足率从72%提升至100% 1。
Researchers from Letta and UC Berkeley have introduced a conceptual framework for AI agent memory systems, proposing a "sleep computation" mechanism that allows models to reorganize and correct their memories during idle periods. This approach shifts the paradigm from passive recording to active judgment of information significance, aiming to optimize retrieval through structured knowledge bases. 1
Experimental results across various benchmarks highlight the evolution of memory strategies in different model versions. In tests on Pokémon Red, Claude Sonnet 3.7 generated 413 memory files yet secured only one badge, whereas Claude Fable 5 achieved four badges using just nine specific files plus one global file. 1 On the Continual Learning Bench (CL-Bench) 1.0, Claude Opus 5 became the first model to close the learning loop by generating verified rules and identifying pitfalls, while subsequent iterations like Claude Opus 5.5 and Sonnet 5.5 maintained comparable scores with memory but showed slightly higher baseline performance without it. 1
The study demonstrates significant efficiency gains when combining search tools with Wiki-style memory structures. For Stainless’s corpus of approximately 350,000 documents (roughly 420 million tokens), conducting 128 "dreaming" sessions to build a Wiki increased scoring criterion satisfaction rates from 13% to 48%. 1 Integrating these Wikis with search capabilities raised Claude's aggregate score from 54% to 61%, surpassing the 53% achieved by search alone, while saving about 20,000 tokens per question. 1 Similarly, in the Harvey LAB benchmark for merger cases requiring cross-document information, Wiki consultation yielded an 87% score compared to 73% for search-only methods. 1
Practical implementation details reveal how memory injection is managed to control context costs. Claude Code automatically loads the first 200 lines or 25KB of MEMORY.md as an index. 1 Furthermore, iterative instructions allowing each contract to have its own page with verbatim clause citations improved brief standard compliance from 72% to 100% in contract review tasks. 1 The research suggests that letting models autonomously decide when to record information is more effective than enforcing rigid rules. 1
评论
还没有评论,欢迎留下第一条。