What the GEO research found, and what it did not test

The GEO paper is the study most generative-engine optimisation advice traces back to. Aggarwal et al. had a language model rewrite web pages nine different ways and measured how much of an AI-generated answer each rewritten page earned. Adding quotations and statistics earned the most, with source attributions and plain fluency edits close behind; keyword stuffing earned less than doing nothing.

The paper is GEO: Generative Engine Optimization by Pranjal Aggarwal, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan and Ameet Deshpande, from Princeton University, IIT Delhi and independent researchers, published at KDD 2024. It is careful work, and it is usually quoted more confidently than it was written. This guide reads it closely: what the experiment did, what the numbers mean, where the benchmark stops, and which findings hold up well enough to act on. The figures below are from the final version on arXiv (v3, June 2024).

What the experiment actually did

The setup is worth understanding before any result, because every number depends on it.

• The queries. GEO-bench holds 10,000 queries drawn from nine sources: real search queries (MS MARCO, ORCAS-I, Natural Questions), essay questions from All Souls College, Oxford, debate prompts, LIMA's open-ended tasks, Perplexity's Discover feed, the ELI5 subreddit, and queries generated by GPT-4. It is split 8,000/1,000/1,000 for training, validation and test, and is 80% informational queries. The results come from the 1,000-query test split.
• The engine. For each query, the top five Google results were fetched and their full cleaned text handed to gpt-3.5-turbo, prompted to answer with an inline citation after every sentence. That prompt was reused from Liu, Zhang and Liang (2023). Five answers were sampled per query at temperature 0.7.
• The intervention. One of the five sources was picked at random and rewritten by GPT-3.5 using one of nine methods, one method at a time. The other four sources were left alone.
• The measurement. Visibility was scored two ways. Position-adjusted word count is the share of the answer's words in sentences citing the source, weighted toward the top of the answer. Subjective impression is seven facets — relevance, influence, uniqueness and others — scored by GPT-3.5 and rescaled to match the first metric. Results were averaged over five random seeds.

The nine methods were Authoritative, Statistics Addition, Keyword Stuffing, Cite Sources, Quotation Addition, Easy-to-Understand, Fluency Optimization, Unique Words and Technical Terms.

What it found

The main results are the paper's Table 1. On position-adjusted word count, with the unedited source at 19.3:

• Quotation Addition: 27.2 — the best result, and the source of the paper's headline 41% improvement on this metric.
• Statistics Addition: 25.2
• Fluency Optimization: 24.7
• Cite Sources: 24.6
• Technical Terms: 22.7
• Easy-to-Understand: 22.0
• Authoritative: 21.3
• Unique Words: 20.5
• Keyword Stuffing: 17.7 — below doing nothing.

The paper summarises this as Cite Sources, Quotation Addition and Statistics Addition achieving a relative improvement of 30–40% on position-adjusted word count and 15–30% on subjective impression, with fluency and readability edits gaining 15–30%. On subjective impression the best method improved on the baseline by 28%. A more persuasive, authoritative tone produced no significant improvement, and keyword stuffing, the classic SEO move, offered little to none.

Two details are easy to miss. First, Fluency Optimization (24.7) scored level with Cite Sources (24.6) on the main metric. The paper's Table 6, which repeats the results with deviations across seeds, reports 0.5 to 1.2 points per method and puts Cite Sources 0.9 ahead, so the two cannot be cleanly separated. Simply making the text read better did about as well as attributing it. Second, the "top three" grouping is the paper's own summary, not a gap in the table: Statistics Addition and Fluency Optimization sit only half a point apart.

What "up to 40%" measures

Every source's scores in an answer are normalised to the same total, and each answer drew on five sources, so a score near 20 is roughly an even fifth of the answer. Moving from 19.3 to 27.2 means the edited source went from about a fifth of the answer's weighted words to a little over a quarter. That is a real effect. It is also a relative gain on a share of one answer, from the best method on the more favourable of the two metrics — not 40% more traffic, and not 40% more citations.

It varied by subject

Table 3 lists the query categories where each method did best. Cite Sources helped most on statement and fact questions and on law and government; Quotation Addition on people and society, explanations and history; Statistics Addition on law and government, debate and opinion. The authors conclude that site owners should make domain-specific adjustments. They also note the category tags were assigned by GPT-4 and can be noisy.

It depended on where the source already ranked

Broken down by the source's original Google position (Table 2), the gains landed at the bottom. In the paper's words, Cite Sources "led to a substantial 115.1% increase in visibility for websites ranked fifth in SERP, while on average, the visibility of the top-ranked website decreased by 30.3%." Quotation and Statistics Addition show the same shape: large gains for fifth place, losses for first. The paper reads this as good news for smaller sites. It is also a warning for anyone whose page already leads: the same edit pointed the other way.

Combinations helped, measured on a subset

Pairing methods did better than any one alone. The best pair, Fluency Optimization with Statistics Addition, beat the best single method by more than 5.5%, and Cite Sources did markedly better in combination than on its own. Cost limits meant this was measured on 200 test queries, not the full 1,000.

The Perplexity test

The paper also ran the methods against Perplexity, a deployed engine. Quotation Addition again led on word share, with a 22% improvement; Statistics Addition gained 37% on subjective impression; keyword stuffing scored 10% below the baseline on word share. The paper's Table 7 holds two results its prose does not mention: keyword stuffing scored above the baseline on subjective impression (28.1 against 24.7), and Cite Sources scored below it (19.0 against 24.7).

The method matters more than the numbers. Perplexity does not let a user choose which URLs it reads, so the researchers uploaded the source text as files and ensured answers came only from those files. This test ran on 200 queries. It shows the edited text worked on a second model; it does not show anything about how Perplexity finds pages on the open web.

What it did not test

None of this is hidden in the paper. Most of it is simply left out when the paper is cited.

• Getting retrieved. Every edited page was already one of the five sources the engine read. Whether an edit helps a page get fetched, or ranked high enough to be among the five, was outside the experiment. The Limitations section says plainly that the authors did not evaluate how the methods affect search rankings.
• Whether the added material was true. The prompts in the authors' repository, which the paper points to, tell the model it "may invent these sources", to add quotes "even though fake and artificial", and to add statistics "even if hypothetical". The paper's own Table 4 example credits a figure to "The International Chocolate Consumption Research Group". The experiment measured how attributed, numeric text reads to a model, not the value of being right.
• Hyperlinks. Cite Sources meant attribution in prose — the prompt's example is "According to Google's latest report…" — not adding links. Nothing in the study tests whether a link affects a model's answer.
• Structure. Answer-first openings, question-shaped headings, structured data, llms.txt, robots.txt and freshness were not among the nine methods. The paper is silent on all of them, in either direction.
• Clicks, traffic and conversions. Both metrics describe the answer text. Neither observes a reader.
• Today's engines. The engine was GPT-3.5 in a two-step setup the authors say closely mimics commercial engines such as you.com and Perplexity at the time of writing. The authors write that the methods "may need to adapt over time" as engines evolve. Current systems retrieve, rank and cite differently.

The benchmark's own limits

GEO-bench is a reasonable research instrument with properties worth knowing before you generalise from it:

• The query mix is not a search log. A share of it is essay prompts, debate motions, creative tasks and GPT-4-generated questions. That is deliberate — the authors expected generative engines to handle harder questions than search does — but it means results are not weighted toward what people ask about any particular business.
• A model graded a model. GPT-3.5 rewrote the pages, GPT-3.5 wrote the answers, and GPT-3.5 scored the subjective metric. Nothing in the loop is a human reader.
• Being cited is not being represented faithfully. The word-share metric credits a source for every sentence that cites it. Liu, Zhang and Liang, whose prompt the study reused, audited four generative search engines by hand in 2023 and found only 74.5% of citations supported the sentence they were attached to.
• Five sources, whole text. Each answer drew on exactly five pages, passed in full, from one search. Deployed engines vary in how many sources they use, and Google says its AI features may issue several related searches per question, which it calls query fan-out.

What a site owner can act on

Strip out the fabrication and the experimental scaffolding, and the findings that survive are useful and cheap. Each one is something a good editor would ask for anyway:

1. Attribute your claims, in the sentence. Name where a fact comes from — "Google's canonicalization guide ranks redirects as the strongest signal" — and link the source so a reader can check it. The study tested the attribution; the link is what makes it verifiable.
2. Use the real number. Replace "most", "much faster" and "a lot" with the figure you already have. Statistics Addition was the second-strongest method on the main metric and the strongest on Perplexity's subjective score.
3. Quote real people, by name. Quotation Addition was the strongest single method on both engines' word share. A quotation from a named person with a link to where they said it is the version you can publish.
4. Edit for fluency and plain language. Fluency Optimization did about as well as citing sources, and it costs nothing but an editing pass.
5. Drop keyword stuffing and puffery. Both underperformed: stuffing fell below the unedited page on word share in both engines, and an authoritative tone made no significant difference.

The one thing not to copy is the study's shortcut. An invented quote or statistic is a false claim published under your name, and a reader, a journalist or a model comparing a second source can find it.

How Crawlune uses the paper, and where it does not

Crawlune's audit counts two of the three strongest signals directly: distinct external domains a page links to, and the figures it contains. It does not count quotations. Its other checks — the liftable answer block, question-shaped headings, a modified date and enough substance — are structural, and this paper did not test them. They follow from how retrieval-based engines assemble answers from passages, which is covered in how answer engines choose which sources to quote.

Nothing in the paper, and nothing in Crawlune, tells you whether a given engine will cite a given page. The audit tells you which of your pages lack the properties the study found models favour, so you can fix the ones that matter.

Reading GEO claims you see elsewhere

Google's guidance on generative AI search says that, from Google Search's perspective, optimising for generative AI search is still SEO, and it lists "GEO hacks" it considers unnecessary. That is not a contradiction of the paper, which measured something narrower than Google's ranking systems. But it is a reason to ask four questions of any claim that cites GEO:

• Which engine? The study used GPT-3.5 in 2023 and an upload-only Perplexity test.
• Which metric? Share of an answer's words is not citations, clicks or revenue.
• Retrieved or given? The study's pages were always in the engine's source set.
• Was it true? The study's added sources, quotes and statistics did not have to be.

A claim that survives all four is rare. A claim that ignores them is the paper's headline with the context removed.