Manifest公司日前宣布停用其LLM路由器，这项曾被视为核心网关功能的系统已于9月1日完全关闭1。该路由器于今年3月推出，在6月被正式弃用，服务期间覆盖7000名云用户、运行时长达4个月1。
经过实际应用，Manifest团队发现LLM路由的成本效益未达预期1。该路由器原设计将用户请求分为四个复杂度等级——simple、standard、complex和reasoning——但公司发现复杂度无法仅从提示词内容推断1。更关键的是，缓存机制的效果超越了路由策略：缓存读取的成本比未缓存输入便宜75%-90%1。此外，路由功能还破坏行为一致性并增加不可预测性1。
Manifest has discontinued its LLM router, a feature it launched as a critical component of its LLM gateway in March 2024. 1 The company deprecated the tool in June and shut it down completely on September 1st, following four months of usage across approximately 7,000 cloud users. 1
The router had categorized requests into four complexity levels—simple, standard, complex, and reasoning—to optimize routing decisions. 1 However, Manifest determined that the cost-benefit proposition did not justify the feature's continued operation. 1 The company identified several fundamental limitations: complexity could not be reliably inferred from prompts alone, caching proved more cost-effective than routing at reducing expenses (with cached reads costing 75-90% less than uncached input), routing undermined behavioral consistency, and the approach introduced unpredictable outcomes. 1
评论
还没有评论，欢迎留下第一条。