研究者探索了从概念词元在雅可比空间（J space）中反演通用激活转向向量的方法1。J space被定义为激活层L到非嵌入矩阵之间的线性映射1。在Qwen3-1.7B模型上的实验中，研究人员采用了一套系统化的算法流程：收集20至30个相关词元、随机抽取其中5个词元用于计算激活向量、重复此过程后对结果取平均，最终得到转向向量1。
实验结果显示，该方法对相对简单的行为表现出有效性1。在全大写输出任务中，生成的转向向量相比使用恒等操作（logit lens）表现出明显更高的余弦相似度1。然而，该方法在处理复杂行为时遭遇显著局限1。以拒绝回应为例，转向向量的准确率仅为五分之一，且在生成回复时出现严重幻觉现象1。研究者指出，复杂行为难以仅通过词元级别的J space信息充分表示1。
Researchers have developed a method to extract universal activation steering vectors from concept tokens using Jacobian space (J space), demonstrating both promising results and significant limitations.1 Experiments conducted on the Qwen3-1.7B model successfully applied this approach to simple behaviors such as producing all-uppercase output, where steering vectors derived from the Jacobian space showed substantially higher cosine similarity compared to traditional logit lens methods using identity operations.1
The methodology involves collecting 20-30 related tokens, randomly sampling C=5 tokens to compute activation vectors, repeating this process K times, and averaging the results to generate steering vectors.1 However, the approach reveals considerable constraints when applied to complex behaviors.1 Steering vectors for refusal behavior achieved only a 1-in-5 accuracy rate and produced severe hallucination phenomena.1 These limitations stem from the fact that complex behaviors cannot be adequately represented solely through token-level information in J space, which functions as a linear mapping between the activation layer L and non-embedding matrices.1
评论
还没有评论，欢迎留下第一条。