Research in progress

Knowledge in the age of AI

Which sources do AI systems surface, how do they attribute knowledge, and what do they overlook?

A review of six published studies on how AI assistants cite their sources, followed by a method for a study Project 3rd Rock has not yet run. Each figure links to the study it came from.

Summary

What the evidence says so far

Across the studies reviewed, AI assistants get their sources wrong often enough that a citation cannot be taken on trust, and few readers check. The studies cover news and one restoration domain. None asked climate or sustainability questions across languages, which is the gap the proposed method is written for.

• Asked to identify the article behind a news excerpt, eight AI search tools answered more than 60 percent of 1,600 requests incorrectly. Perplexity was wrong 37 percent of the time and Grok 3 94 percent.

  Columbia Journalism Review, Tow Center, March 2025

• In an audit of four generative search engines, about half of generated sentences were fully supported by their citations, and about three in four citations supported the sentence they sat beside.

  Liu, Zhang and Liang, April 2023

• When Google showed an AI summary, about 1 percent of visits included a click on a link inside it. Clicks on ordinary results fell from 15 percent of visits to 8 percent.

  Pew Research Center, July 2025

• Asked about ecological restoration, ChatGPT drew on male academics at US universities in over two-thirds of 30,000 answers. Indigenous and community experience appeared in 2 percent.

  Urzedo and colleagues, 2024

• Across six chatbots, Hindi news questions scored lowest, and models cited English Wikipedia more often than any Hindi-language outlet.

  Suzgun and colleagues, preprint, May 2026

Why it matters

Answers now arrive before sources

More people meet knowledge as a composed answer than as a list of pages. What the answer cites, and what it leaves out, shapes which institutions, places and languages are visible. For climate and sustainability work, where practical knowledge is often local, multilingual and held outside the largest publishers, the question is not only whether an answer is correct. It is whose knowledge was available to be found.

What existing evidence says

Six studies, read side by side

Three of the studies produced figures that can be drawn. They differ in task, date, systems and language, so the numbers do not sit on one scale. Each chart names its source and what it cannot show.

What is still unknown

Open questions

• No source reviewed here measures which sources AI systems cite when asked about climate and sustainability practice across languages. The closest evidence concerns news, one restoration domain, and systems that have since changed.
• It is not established how much source concentration comes from model training, from the retrieval layer, or from publisher access rules such as crawler blocking.
• Citation accuracy figures come from different tasks (identifying an article, supporting a sentence, answering a news question). They are not directly comparable and are not shown as one ranking.
• A grounded climate assistant built on IPCC reports has been published (ChatClimate, Communications Earth and Environment, 2023). Its findings were not verified during this review and are not summarised here.
• Whether people check citations when the answer looks plausible is only partly known. The Pew data show low click-through to summary links on Google, not why.

Proposed method

How an independent study would work

This is a plan, not a result. No data have been collected under it.

1. 01

   Fix a question set

   Draft a small, public set of climate and sustainability questions, written with practitioners, in several languages and covering both widely reported and locally documented topics. Publish the questions before any results.

2. 02

   Fix the systems and the settings

   Record each assistant, version, interface, region, language setting and date of query. Repeat each question more than once, since outputs vary between runs.

3. 03

   Capture everything

   Save the full answer, every cited link, and any sources shown but not cited. Note links that fail to resolve.

4. 04

   Code the sources

   Classify each cited domain by type (government, multilateral, academic, news, advocacy, encyclopaedia, commercial), country and language. Two coders, with disagreements recorded.

5. 05

   Check attribution

   For a sample of sentences, test whether the cited page supports the claim, using the recall and precision definitions from Liu and colleagues so results can be compared with earlier work.

6. 06

   Compare against known knowledge

   Where an authoritative local or regional source exists for a question, record whether it was surfaced, and what was surfaced instead.

7. 07

   Publish with limits

   Release data, prompts and coding rules with a plain account of what the sample cannot show. Invite correction before conclusions are drawn.