Generative engine optimization in practice: a website AI search can cite

On 15 May 2026 Google published its guide to optimising for generative AI features on Google Search, announced in a Search Central post that day and updated on 10 July 2026. AI Overviews and AI Mode retrieve pages from the normal Search index and run several related searches at once (query fan-out). On AEO and GEO it is blunt: from Google's side, that work is "still SEO".

Its mythbusting section is the part I send clients. For Google Search you can ignore AI text files such as llms.txt, chunking content, rewriting in a special style for AI, chasing inauthentic mentions and special schema. A page for every fan-out query variation, made mainly to manipulate answers, breaks the scaled content abuse policy.

Google's AI features and your website page, updated 10 December 2025, adds that a page needs only to be indexed and eligible for a snippet. John Mueller's post on succeeding in AI search, from 21 May 2025, notes that nosnippet, max-snippet and noindex limit AI features too.

My reading: for Google there's no separate track to buy. But Google speaks only for Google, and the other engines run their own crawlers.

The term comes from GEO: Generative Engine Optimization by Pranjal Aggarwal and five co-authors, posted to arXiv on 16 November 2023 and accepted to KDD 2024. Its "up to 40%" now appears in a lot of marketing, so here is what they did.

Citing sources, adding quotations and adding statistics improved the main measure by 30 to 40% and the subjective one by 15 to 30%. Keyword stuffing gave little or no improvement. Retested on Perplexity, a live engine, gains reached 37%. Most interesting to me: cited sources raised visibility by 115.1% for the site ranked fifth, while the top-ranked site lost 30.3% on average.

The limits matter. A language model made the rewrites, so a "statistics added" page may hold figures nobody checked. The engine was GPT-3.5 reading five results, not today's systems. The paper didn't measure rankings, clicks or sales, and its authors say methods may need to adapt as engines change. I read it as evidence that sourced, specific writing gets reused more than vague writing, which is how I'd write anyway, not as a playbook.

This is the setting I check first, because a wrong line fails silently. Each company separates the bot that feeds its search answers from the one gathering training data, and often from a third that fetches pages when a user asks.

Each row comes from the owner's page: Google's common crawlers list (updated 14 July 2026), OpenAI's crawler overview, Perplexity's crawler page and Anthropic's help article on its crawlers (7 April 2026). Google says Google-Extended governs Gemini training and grounding and doesn't affect Search inclusion or ranking.

One trap: under Google's robots.txt specification, a crawler follows only the most specific group naming it, never combined with *. Add a group for OAI-SearchBot and you must repeat your private-path rules inside it.

Then check the layer in front. Google's AI features page asks that crawling be allowed "by any CDN or hosting infrastructure", and CDNs now ship AI bot controls: Cloudflare began asking every new domain on 1 July 2025 whether to allow AI crawlers. Letting training bots in is a licensing choice, not a design one; I make sure the client makes it on purpose.

llms.txt is a proposal by Jeremy Howard, published on 3 September 2024 and revised as version 2 on 10 August 2026: a Markdown file at a site's root with a short summary and links to clean Markdown versions of key pages, aimed at agents and AI tools.

Google's May 2026 guide says Search doesn't use it, and that keeping one "will neither harm nor help" there. Chrome differs: Lighthouse has an llms.txt audit in its experimental agentic browsing category (updated 5 May 2026), which marks a missing file not applicable and flags only server errors. I found nothing from OpenAI, Perplexity or Anthropic saying their search answers use it.

This is where I think most of the gain sits. Answer engines pull passages from pages, so each important page needs a passage worth pulling. Google's guide asks for "non-commodity" content with a first-hand view; Bing's post launching AI Performance (10 February 2026) says clear headings, tables and FAQ sections help AI systems find key facts.

On a B2B site:

• One question per page. Security, integrations, the pricing model and each main use case get their own URL and H1, worded the way buyers ask.
• The answer in the first two to four sentences, able to stand alone if quoted.
• Headings that answer follow-ups, so each section makes sense lifted out of the page.
• Tables as HTML text, never images, for anything compared or specified.
• Sources linked in the sentence, for any number or claim.

Google says you needn't rewrite or chunk anything for AI; Bing says structure helps. Both describe a page a person can scan, so that's what I design.

Google says structured data isn't required for AI features, though it still helps rich results and must match the visible text. So I add JSON-LD for what a page really is: Organization or Person for who you are, Article for guides, SoftwareApplication only where the facts exist, FAQPage only where the questions are visible.

Authorship matters more. Google's page on helpful, reliable, people-first content (updated 5 October 2026) asks whether it's self-evident who wrote a page and whether the byline leads to more about them, and calls AI-generated headshots and made-up names deception. Its Article structured data guidance asks for each author as a Person with a url or sameAs.

On a company site that means real team pages, bylines on guides, an About page that says plainly what the company does, and the same name, description and links everywhere. My reading, which none of these companies spells out: answer engines match entities across sources, so consistency is cheap and contradiction costly.

In Vercel and MERJ's study of AI crawlers, published 17 December 2024, none of the major AI crawlers it tracked, OAI-SearchBot, GPTBot, ClaudeBot and PerplexityBot among them, rendered JavaScript; Gemini, on Google's infrastructure, did. It's a 2024 snapshot, but I build as if it still holds: text, headings, tables and JSON-LD in the server's first response.

When I'm choosing Framer, Webflow or Next.js for a site, I check that pages are server-rendered or static, and in Next.js I keep content out of client-only components. It's part of how I plan search and speed before a site launches.

For speed, the Web Vitals targets are an LCP within 2.5 seconds, an INP of 200 milliseconds or less and a CLS of 0.1 or less, at the 75th percentile of visits. The guide also points to web.dev's guide to agent-friendly websites (updated 1 April 2026): agents read pages through screenshots, the HTML and the accessibility tree, so proper labels and stable layouts serve them too.

• Google Search Console. Generative AI performance reports for Search and Discover were announced on 3 June 2026 and reached all websites on 31 August 2026. The post lists impressions by page, country, device and date, but no clicks.
• Bing Webmaster Tools. AI Performance shows total citations, average cited pages per day, page-level citations and a sample of grounding queries across Copilot, Bing's AI summaries and some partners. Bing says the numbers don't indicate ranking or authority.

I found no equivalent from OpenAI, Perplexity or Anthropic; there, you have referral traffic and server logs filtered by the user agents above. Google warns that no third-party tool has access to its internal ranking or AI systems. "AI visibility" dashboards sample prompts; they don't see the system.

• Whether llms.txt affects any AI search answer. No search provider I read says it does.
• Whether allowing training bots changes how models describe you. Plausible, and unmeasured as far as I can find.
• Whether schema helps outside Google. Google says it isn't needed; the others haven't said.
• Whether the GEO paper's gains hold today. It tested GPT-3.5, and its authors expect methods to need adapting.
• How many people click through from AI answers. Google says these clicks are "higher quality"; it hasn't published click-through rates.

The order I work in:

1. Settle the crawler policy with the client: search bots in, training bots decided.
2. Write robots.txt with rules repeated per group, then check the CDN.
3. List the questions buyers ask and give each its own page or section.
4. Draft the answer-first paragraph for every key page before any layout.
5. Choose a platform that serves full HTML, and fetch pages with scripts off to confirm it.
6. Build components for tables, FAQs, bylines and sources that editors can use alone.
7. Add JSON-LD that matches the page, validated per template.
8. Set speed budgets per template against Core Web Vitals.
9. Verify the site in Search Console and Bing Webmaster Tools before launch.
10. Review the AI reports monthly and rework pages that are indexed but never appear.

If a rework changes URLs, my 26-check website redesign checklist protects what already ranks. AI search is one of several shifts in what I expect from web design in 2027.

If your site exists, start with an afternoon of checks, not a rebuild. Open robots.txt and your CDN's bot settings and see which bots you're turning away. Fetch your five key pages with JavaScript off and see what's left, then ask whether each first paragraph answers anything. Then verify the site in both webmaster tools.

Most of this is good website work done in the right order. If you'd like me to do it with you, that's the website design service I lead: a free 30 minute intro call, then a written scope and one fixed quote for the whole project, with no retainer. More about me is on my about page.

What is generative engine optimization?

Generative engine optimization (GEO) is the practice of making web pages more likely to be used and cited in AI answers, such as Google's AI Overviews, ChatGPT search or Perplexity. The term comes from a 2023 paper by Aggarwal and colleagues, presented at KDD 2024. In practice it overlaps heavily with SEO.

Is GEO different from SEO?

For Google, no. Its May 2026 guide says optimising for generative AI search is still SEO, because AI Overviews and AI Mode draw on the Search index and ranking systems. Other engines run their own crawlers, so the differences lie in crawler access, JavaScript rendering and measurement, not in a separate kind of content.

Do I need an llms.txt file for AI search?

Not for Google Search, which says it ignores the file. Chrome's Lighthouse checks for one in an experimental agentic audit but treats a missing file as not applicable. I found no statement from OpenAI, Perplexity or Anthropic that their search answers use it. It's cheap to add for agents, but don't expect citations from it.

Should I block GPTBot or ClaudeBot?

That's a licensing decision, not a visibility one. GPTBot and ClaudeBot gather training data; OAI-SearchBot and Claude-SearchBot feed search answers, and each is allowed or blocked separately. By their owners' documentation, blocking the training bots doesn't remove you from ChatGPT or Claude search. Blocking the search bots does.

Does structured data help you appear in AI Overviews?

Google says structured data isn't required for AI Overviews or AI Mode and no special schema exists for them. It still helps rich results and must match the visible page. I add accurate Organization, Person and Article markup with real author pages because it describes the page honestly, not because it guarantees a citation.

How do I know if AI search cites my website?

Google Search Console has generative AI performance reports, open to all sites since 31 August 2026, showing impressions by page, country and device. Bing Webmaster Tools has AI Performance, showing citations in Copilot and Bing's AI summaries. For ChatGPT, Perplexity and Claude, check referral traffic and crawler visits in your server logs.

Can anyone guarantee my site will be cited by ChatGPT or Google?

No. None of these companies publishes exactly how it picks sources, and no outside tool can see inside their systems. What you control is access, clarity, evidence and speed. Anyone promising guaranteed citations is promising something nobody outside those companies can deliver.

I read every source on 13 October 2026; dates in brackets are the publishers' own. Crawler facts come from each company's documentation; I didn't test crawlers myself. GEO figures are from the paper's version 3 (28 June 2024). The JavaScript finding is one 2024 study. Data is thin on how ChatGPT, Perplexity and Claude choose sources and on clicks from AI answers.

• Google Search Central, Optimizing your website for generative AI features (15 May 2026, updated 10 July 2026), read 13 October 2026: developers.google.com
• Google Search Central Blog, new generative AI resource, 15 May 2026, read 13 October 2026: developers.google.com
• Google Search Central, AI features and your website (updated 10 December 2025), read 13 October 2026: developers.google.com
• Google Search Central Blog, succeeding in AI search, 21 May 2025, read 13 October 2026: developers.google.com
• Google Search Central Blog, generative AI performance reports, 3 June 2026, read 13 October 2026: developers.google.com
• Google, common crawlers including Google-Extended (updated 14 July 2026), read 13 October 2026: developers.google.com
• Google, robots.txt specification (updated 31 August 2026), read 13 October 2026: developers.google.com
• Google Search Central, Article structured data (updated 8 September 2026), read 13 October 2026: developers.google.com
• Google Search Central, helpful, people-first content (updated 5 October 2026), read 13 October 2026: developers.google.com
• Aggarwal et al., GEO: Generative Engine Optimization, arXiv 2311.09735 v3, KDD 2024, read 13 October 2026: arxiv.org
• OpenAI, Overview of OpenAI crawlers, read 13 October 2026: developers.openai.com
• Perplexity, Perplexity crawlers, read 13 October 2026: docs.perplexity.ai
• Anthropic, help article on its crawlers (7 April 2026), read 13 October 2026: support.claude.com
• Jeremy Howard, The /llms.txt file, v2 (3 September 2024, modified 10 August 2026), read 13 October 2026: llmstxt.org
• Chrome for Developers, Lighthouse llms.txt audit (updated 5 May 2026), read 13 October 2026: developer.chrome.com
• web.dev, Build agent-friendly websites (updated 1 April 2026) and Web Vitals, read 13 October 2026: web.dev, web.dev
• Vercel and MERJ, The rise of the AI crawler, 17 December 2024, read 13 October 2026: vercel.com
• Bing Webmaster Blog, AI Performance public preview, 10 February 2026, read 13 October 2026: blogs.bing.com
• Cloudflare, press release on AI crawler permissions, 1 July 2025, read 13 October 2026: cloudflare.com