一位研究员通过创建包含50,000个完全虚假统计数据的网站StateGlobe，揭示了大语言模型在信息核查和可信度评估方面的严重缺陷。1这个成本仅约50美元的实验网站在7个月内被ChatGPT检索超过10,000次。1虽然Google在两次搜索算法更新后迅速将该网站从搜索结果中删除，但ChatGPT却持续从该网站抓取数据。1
这些虚假数据最终通过多个渠道广泛传播，被330多个引用域链接，其中包括Shopify、Bluehost、DHL等知名品牌。1更令人担忧的是，这些来自虚假网站的统计数据被引用在学术论文和卡塔尔最大金融集团的投资研究报告中。1研究发现，LLM在检索信息时并不进行事实核查，而是采用相关性重排器来评估内容价值，页面中包含的年份信息则为模型提供了新鲜度信号。1该研究员随后用真实数据创建的统计页面同样获得了AI引用和链接，进一步表明LLM缺乏对信息真实性的根本性验证机制。1
A researcher from Peec AI created StateGlobe, a website containing 50,000 entirely fabricated statistics, to investigate how large language models retrieve and cite information 1. The experiment revealed a critical vulnerability in AI systems' ability to verify facts, as the false data was subsequently cited by academic papers, investment reports from major financial institutions, and references from well-known brands 1.
Over a seven-month period, ChatGPT accessed the website more than 10,000 times across AI sessions, with an additional 72,000 requests from the ChatGPT-User crawler 1. Despite Google removing the site from search results after two algorithm updates, ChatGPT continued retrieving data from it 1. The fabricated statistics accumulated over 330 citing domain links, including references from companies such as Shopify, Bluehost, and DHL, and were incorporated into academic papers and research reports produced by Qatar's largest financial groups 1.
The research underscores fundamental gaps in how language models process information. Rather than performing fact-checking, LLMs rely on relevance ranking mechanisms instead of authenticity verification 1. The presence of year information within the statistical pages served as a freshness signal to the models, making older content appear current 1. Notably, when the researcher subsequently created a similar page containing verified statistics, it also accumulated AI citations and incoming links through the same mechanisms 1. The entire experiment cost approximately fifty dollars to establish 1.
评论
还没有评论，欢迎留下第一条。