Pi编码助手通过Compaction（压缩）机制来解决LLM对话中的上下文窗口限制问题[1]。当对话历史超过模型的上下文容量时，该机制将旧消息总结为压缩表示，保留最近的消息和新用户输入，使对话能够持续进行[1]。
Pi的默认token预算为20000个tokens，通常对应5到20个对话轮次[1]。在实现Compaction时，Pi使用独立的LLM请求生成结构化摘要，将系统提示改为"你是一个上下文总结助手"，摘要包含目标、进展和关键决策等信息，并将其存储为纯文本[1]。
这一压缩方案需要处理与提示词缓存的兼容性问题——Compaction会破坏提示词缓存的前缀匹配，需要重新计算缓存[1]。
Pi, an AI coding assistant, implements a compaction mechanism to manage conversation history when it exceeds the model's context window constraints.[1] Large language models face inherent limitations in the amount of input they can process, and Pi's default token budget of 20,000 tokens typically accommodates only five to twenty conversation turns.[1] To address this constraint while maintaining conversation continuity, Pi summarizes older messages into a compressed representation while preserving recent exchanges and new user input.[1]
The compaction process operates through an independent language model request that generates structured summaries containing objectives, progress made, and key decisions.[1] The system prompt for this summarization task is modified to "you are a context summarization assistant."[1] These compressed summaries are stored as plain text within the conversation history.[1] However, this approach introduces a compatibility challenge: compaction disrupts prefix matching used by prompt caching, necessitating recalculation of the cache when compression occurs.[1]