TIME杂志网站根据用户身份采用差异化策略对待网络请求[1]。人类用户访问该网站时获得完整的HTML页面，而AI爬虫（包括ClaudeBot、PerplexityBot和OAI-SearchBot）则被提供经过大幅精简的markdown版本[1]。这些针对AI模型的markdown页面文件大小仅为对应HTML版本的二十三分之一——markdown版本约13,409字节，而HTML版本则为303,235字节[1]。
TIME已与广告技术公司Mobian建立合作关系，在提供给AI爬虫的markdown页面中嵌入广告商赞助内容[1]。这些广告包括Ally Bank常见问题和项目管理学会的相关内容[1]。Mobian采用技术手段确保每次爬虫请求都被计为一次独立的广告展示：系统为每个请求生成唯一的UUID标识符，配合"cache-control: no-store"缓存控制策略，这些广告展示不会被人类用户看到[1]。markdown页面的请求头还包含token计数信息，以模型输入的token数而非页面浏览次数作为计数单位[1]。
TIME杂志表示，来自爬虫的流量已在大多数日子超过人类用户流量[1]。此外，OpenAI的GPTBot和ChatGPT-User遭到网站返回的406错误阻止，但OpenAI旗下的OAI-SearchBot被允许访问markdown版本[1]。
TIME magazine is delivering two distinct versions of its website based on user type, according to reports of the publication's traffic management practices [1]. Human visitors receive the full HTML page, while AI crawlers including ClaudeBot, PerplexityBot, and OAI-SearchBot are served a simplified markdown version that is roughly one-twenty-third the size of the standard page [1]. The markdown variant weighs approximately 13,409 bytes compared to the 303,235-byte HTML edition [1].
The markdown pages served to AI bots contain embedded sponsored content from advertisers such as Ally Bank and the Project Management Institute, marked as "Sponsored content" [1]. TIME has partnered with ad-tech company Mobian to facilitate this arrangement, which allows sponsored material to reach AI systems in contexts where human readers would not encounter it [1]. Each crawler request generates a unique identifier that counts as a separate ad impression, with the system tracking token counts rather than traditional page views, according to response headers indicating token volumes for input to language models [1]. The cache-control protocol is configured to prevent storage, ensuring each request registers as a distinct advertising exposure [1].
The traffic dynamics have shifted significantly, with TIME noting that bot traffic has exceeded human traffic on most days [1]. Notably, while OpenAI's GPTBot and ChatGPT-User receive 406 blocking errors, the company's OAI-SearchBot is permitted to access the markdown version [1].