一位技术开发者在Hacker News上发布了将35kb大规模预提示词从OpenAI/Anthropic等前沿LLM提供商迁移到自托管Ollama系统的技术笔记。1在这一迁移过程中，他遇到了多项性能瓶颈。该35kb预提示词在128GB AMD Ryzen系统上立即消耗了总上下文窗口的14%。1由于本地模型的上下文窗口限制为65k tokens，预提示词加上会话历史会迅速超出容量。1
在测试中，Ollama在3分钟内开始出现资源耗尽，代理会重复调用工具、重复读取文件。1针对这些问题，开发者提出了一系列优化策略，包括将预提示词分解为单目标单元、显式配置上下文长度，以及减少单步工具调用的频率。1此外，开发者指出前沿提供商在用户不知情的情况下使用会话数据进行训练，这促使许多用户选择迁移至自托管方案以保护数据隐私。1
A developer has documented significant hurdles encountered when moving 35 kilobytes of preprompt material from commercial large language model providers to a self-hosted Ollama installation.1 The preprompt, when deployed on a 128GB AMD Ryzen system, immediately consumed 14 percent of the total context window available.1
The core issue stems from context window limitations inherent to locally-hosted models, which operate with a ceiling of 65,000 tokens.1 When combined with preprompts and ongoing session history, this constraint is exceeded rapidly, degrading performance.1 In testing, the Ollama instance began experiencing resource exhaustion within three minutes, manifesting as repetitive tool invocations and redundant file-reading operations by the agent.1
To address these constraints, the author proposes several optimization strategies: decomposing preprompts into single-objective units, explicitly configuring context length parameters, and reducing the number of tool calls executed in each step.1 The author also expresses concerns about data usage practices at leading LLM providers, suggesting they may train on session data without explicit user awareness.1
评论
还没有评论，欢迎留下第一条。