大语言模型水印机制被发现影响AI代理行为

研究发现，Anthropic在Claude模型中部署的基于Google DeepMind SynthID-Text的隐形文本水印会改变大语言模型的token生成过程，导致AI Agent的工具调用和拒绝行为发生偏移1。该水印机制采用Tournament sampling方法改变token选择过程，在不修改模型权重和提示词的情况下产生不同的生成结果1，旨在符合EU AI Act第50(2)条关于合成文本标记的要求1。

在BFCL v4工具调用基准测试中，水印导致6个模型的准确性均出现下降，其中4个模型下降幅度显著，平均采样漂移率达到6.5%1。在200个HarmBench有害行为和100个JailbreakBench良性对照测试上，水印下的拒绝行为出现改变，特别是在提示词注入攻击下表现更为明显1。具体而言，Llama-3.1-8B模型在低温度条件（T=0.001）下，面对裸露有害请求的采样漂移为7.5%，但在提示词注入攻击下增加至11.0%；Gemma-3-27b的漂移则从6.0%增加到23.5%1。

水印效果因模型和密钥而异，相同的Llama-3.1-8B模型在不同密钥下的攻击成功率变化范围为−4.5到+14.5个百分点1。值得注意的是，水印引起的拒绝采样漂移在多个模型上高于温度变化引起的漂移，以Granite-3.2-8B为例，水印采样漂移达21.5%，而温度诱导漂移为15.5%1。这些发现表明，文本水印机制可能对AI安全防护能力构成潜在威胁。

Anthropic's deployment of invisible text watermarks in Claude models has unintended consequences for artificial intelligence agent performance, according to research findings. The watermarks, based on Google DeepMind's SynthID-Text technology and designed to comply with EU AI Act requirements for synthetic text labeling, modify the token generation process in ways that shift how models make tool calls and refuse harmful requests.1

The watermarking mechanism uses Tournament sampling to alter token selection without changing model weights or prompts, resulting in different generation patterns across the same model.1 In benchmarking tests using the BFCL v4 tool-calling standard, watermarking reduced accuracy across all six tested models, with four experiencing significant declines and an average sampling drift rate of 6.5 percent.1 Beyond tool performance, the watermarks also change refusal behavior on harmful and benign requests, with particularly pronounced effects under prompt injection attacks.1

The impact varies substantially depending on the specific model and watermarking key used. For the Llama-3.1-8B model at temperature 0.001, the sampling drift for raw harmful requests reached 7.5 percent, increasing to 11.0 percent under prompt injection attacks, while for Gemma-3-27b the drift expanded from 6.0 percent to 23.5 percent.1 In some cases, the same model showed attack success rate variations between negative 4.5 and positive 14.5 percentage points across different keys.1 Notably, watermark-induced refusal sampling drift exceeded the drift caused by temperature variation on multiple models, with Granite-3.2-8B exhibiting a watermark-induced sampling drift of 21.5 percent compared to 15.5 percent from temperature changes.1