亚马逊工程师 Carr 和 Mehul 将内部工单系统的 AI 分类模型从 Claude Sonnet 4.6 替换为自研的决策模型 Strands Decider。这一变更在保持同等性能的前提下,使单次请求成本降低约 98%(约为原来的 1/44),预计每年可节省数百万美元的 Token 费用 1。新采用的 Strands Decider 属于决策模型而非生成式大语言模型,其核心机制是返回概率和置信度而不生成文本,具体实现基于 E4B Gemma 4 构建版本,并在 ml.g5.xlarge 实例上运行,单次请求耗时约为 640 毫秒 1。
为确保迁移过程的安全与可控,团队通过 AWS AppConfig 实现了包含 BEDROCK(仅使用 Sonnet)、DUAL(双跑对比)和 DECIDER(仅使用 Strands)的三向开关切换机制 1。在技术优化细节方面,测试发现 JSON 键的顺序会影响输出概率,差异幅度可达 0.19,因此系统强制对请求中的键进行排序以消除不确定性 1。此外,此次模型替换还带来了显著的性能提升,Lambda 函数的平均执行时长减少了 74% 1。
Amazon engineers Carr and Mehul have replaced the Claude Sonnet 4.6 model in their internal ticketing system’s classification workflow with a proprietary decision model called Strands Decider 1. This change reduced per-request costs by approximately 98%, equivalent to about 44 times less expense, while maintaining equal performance 1. The company expects to save several million dollars annually in token fees through this optimization 1.
Unlike generative large language models, Strands Decider is a non-generative classifier that returns probabilities and confidence scores instead of generating text 1. The model utilizes a customized version of E4B Gemma 4 and runs on ml.g5.xlarge instances 1. Individual requests take approximately 640 milliseconds to process, contributing to an average reduction of 74% in Lambda execution duration 1.
The transition was managed through rigorous A/B testing using AWS AppConfig, which enabled three operational modes: running only Sonnet via Bedrock, executing both models simultaneously for comparison (DUAL), or utilizing only the Strands Decider 1. During development, engineers discovered that the order of JSON keys within the request payload could influence output probabilities by up to 0.19 1. To ensure consistent results, they enforced strict key sorting in the requests 1.
评论
还没有评论,欢迎留下第一条。