Crawlune / AI visibility
You can't buy a citation.
You can stop being skipped.
Answer engines increasingly end the search before anyone clicks. Crawlune measures four things about your own pages — crawler access, extractability, structured data, and a machine-readable summary — and reports what it finds as counts you can check. What an engine does with them is nobody's to promise.
- AI crawlers controlled per agent
- 6AI crawlers controlled per agent
- Extractability score per page
- 0–100Extractability score per page
- Audit, llms.txt & crawler controls
- FreeAudit, llms.txt & crawler controls
- Free and open source
- GPLFree and open source
Why Crawlune
The honest version of
answer-engine optimisation.
This is a field full of unfalsifiable claims. The way out is to only ever talk about what can be counted from your own site — and to say plainly where the measurement stops.
Training and retrieval are not the same thing
Blocking GPTBot is an editorial choice about model training. Blocking OAI-SearchBot restricts OpenAI’s search crawler. These are separate controls, so Crawlune records a purpose for every agent and flags blocked retrieval crawlers as a misconfiguration rather than a preference.
Every claim is a number you can check
How many pages have no liftable answer block. How many cite no external source and quote no statistic. Which crawlers you block. Whether you have an llms.txt. Those are countable from your own database, which is why they are what this product talks about.
Grounded in the published research
The signals are weighted toward what Aggarwal et al. found in GEO: Generative Engine Optimization (KDD 2024) — citing sources, adding quotations and adding statistics produced the largest visibility gains among the interventions they tested. That is a real finding on a specific benchmark, not a guarantee about your site.
Honest about the ceiling
Nothing here can tell you whether ChatGPT cites you, because that is not measurable from inside WordPress. What it can tell you is whether you have given an engine a reason not to — and that is the part you actually control.
What is built today
Everything that measures, and everything free
Each capability below is implemented and tested in the plugin as it stands. Hosted rewriting also works for activated licenses; public enrollment is not yet open.
Review AI rewrites before applying
Compare suggested answer blocks, headings and grounded FAQ results with your current content. Apply approved changes and undo a run from the journal. Hosted rewriting requires an activated license; public enrollment is not yet open.
Per-agent AI crawler control
Six crawlers across four vendors — OpenAI, Anthropic, Perplexity and Google — each recorded with whether blocking it affects training or live retrieval. Directives are emitted as denials only, so a per-agent rule never narrows a wildcard defined elsewhere in your robots.txt.
Extractability audit, per page
Scores each page 0–100 across six signals: whether it opens with a liftable answer, whether it cites external sources, whether it quotes statistics, whether headings ask questions, whether it exposes a modified date, and whether it is too thin to be worth quoting.
Site-wide visibility report
Median score, a count per weakness, and the weakest pages ranked — so the next thing to fix is the page a fix will move, not whichever one you happened to open.
llms.txt, generated from real content
Both llms.txt and llms-full.txt, built from what the site actually publishes rather than hand-maintained. Worth knowing: llms.txt is a convention for helping agents find readable content. Publishing it does not guarantee crawling, ranking or citations.
Score decomposition, not a number to trust
The weight of every signal is exposed, so the admin screen can break a score down instead of asking you to take it on faith. A score a reader cannot decompose is one they have to believe, and this product does not get to publish those.
Admin screen and REST surface
The visibility report has a screen in wp-admin and authenticated REST routes behind real capability checks, so the audit is scriptable as well as readable.
How it works
Audit, control access, review changes
The local audit and crawler controls require no license. Hosted rewriting requires an activated license, with review before apply.
- 01
Check who you're blocking
Crawlune reads your crawler access per agent and separates training from retrieval. If you are blocking a retrieval crawler, that alone can keep you out of live answers while everything else is right.
- 02
Audit what you publish
It scores each page for extractability — liftable answer, citations, statistics, question headings, freshness, substance — and rolls that into a site-wide report with the weakest pages ranked.
- 03
Publish a machine-readable summary
llms.txt and llms-full.txt are generated from your real content. A proposed convention rather than a proven ranking input, so treat it as cheap insurance.
- 04
Rewrite what the audit flagged
Turn a flagged opening paragraph into a liftable answer block. Every rewrite arrives as a word-level diff in a review queue — never applied silently — splices back without breaking block markup, and can be reverted from the journal.
Try it without installing anything
Free tools and guides
The checker runs the same analysis as the plugin's audit, against a public URL instead of your posts table. No signup.
Documentation
Every component of the extractability score and how each is measured, what llms.txt is and is not, what a rewrite may never change — and the one thing this product cannot tell you.
ExploreFree tools
Run the same extractability checks against any public URL, in the browser, with no signup and no plugin installed.
ExploreGuides
How answer engines pick sources, what the published research supports, and which of it you can actually act on.
ExploreBuild status
What ships, and what doesn't
Including the things this product will not do at all, which in a category this noisy is the more useful list.
Crawlune is available as a free plugin on WordPress.org. Crawler controls, the content audit, reports, llms.txt and structured data run locally. Hosted rewriting is enabled for activated licenses: answer blocks, headings and grounded FAQ results have been verified through generation, review, apply and undo. Citation tracking is not included.
Implemented and tested
- AI crawler access control, per agent, with training and retrieval separated
- Extractability audit across six signals, scored 0–100 per page
- Site-wide visibility report: median score, per-weakness counts, weakest pages
- llms.txt and llms-full.txt generated from real content
- Per-page structured data, where no SEO plugin already provides it
- Answer-block rewriting, reviewed as a word-level diff before anything is applied
- Heading rewriting toward the question a section answers, preserving every heading's level and link address
- Generated FAQ data, where every proposed answer has to be text already on the page
- Applying a rewrite without breaking block markup, and reverting a run
- Admin screens and REST routes
- Licensing, metered in items, on the shared WP Shelf platform
Not available
- Public licensing enrollment and published pricing are not yet available
- Citation tracking — whether you are actually quoted by a given engine
Why there are no prices here
The audit, llms.txt and the crawler controls are free on every tier including with no licence — they are the reason to install, and metering them would gate the thing that makes the rest worth doing. Only rewriting consumes quota; items cover answer blocks, headings and grounded FAQ results. There are no prices on this page because the ladder is not calibrated against measured per-item cost yet, and a quota published before it is measured is a guess with a number on it.
FAQ
Questions, answered
Crawlune is a WordPress plugin for AI visibility: making a site legible, accessible and quotable to answer engines. It does four things — controls AI crawler access per agent, audits whether pages are extractable, generates a machine-readable llms.txt summary, and reports which structured-data types a page already declares. All of that is implemented and tested. So is the metered half — answer-block rewriting, heading rewriting and generated FAQ data all select, dispatch, review, apply and revert — hosted rewriting is enabled for activated licenses. Public enrollment is not yet open. Citation tracking is not built. The free plugin is available on WordPress.org.
No plugin can promise that, and Crawlune does not. Whether a given engine quotes a given page is not measurable from inside WordPress, so any product claiming it is asking you to take an unfalsifiable claim on trust. What Crawlune does is subtractive: it finds the reasons an engine would skip you — a blocked retrieval crawler, a page with no liftable answer, no external citations, no statistics, no machine-readable summary — and reports each as a count you can verify yourself.
Yes. Install Crawlune – AI Crawler Controls from WordPress.org. The audit, crawler controls, reports and llms.txt run locally without an account or licence. Hosted rewriting is enabled for activated licenses; public enrollment is not yet open. You can also try the AI Visibility Checker on this site without installing.
GPTBot is OpenAI’s training crawler; disallowing it requests that crawled content not be used to train its foundation models. OAI-SearchBot is for ChatGPT search. OpenAI says opted-out sites are excluded from search answers but may still appear as navigational links. These controls are independent, and robots.txt is a directive to cooperating crawlers rather than an access barrier. Crawlune records a purpose for each of the six crawlers it manages and flags a blocked retrieval crawler as a misconfiguration.
llms.txt gives agents a curated guide to your site and links to readable content. It is a convention described at llmstxt.org, not a guarantee that a search or answer engine will crawl, rank or cite a page. Crawlune builds it from your published content so you can keep that guide current.
Aggarwal et al., GEO: Generative Engine Optimization (KDD 2024), which tested a range of content interventions and found that citing sources, adding quotations and adding statistics produced the largest visibility gains. Crawlune's audit weights those signals accordingly. It is a real result on a specific benchmark, not a guarantee that the same interventions will move your traffic.
The audit, llms.txt and the crawler controls are free on every tier, including with no licence — they are the reason to install it. Only rewriting is metered; items cover answer blocks, headings and grounded FAQ results. The prices are not published because the item counts have not been calibrated against measured per-item cost, and publishing a quota before measuring it is guesswork with a number attached.
No. It does no sitemaps, no redirects, no rank tracking and no keyword research, and it does not touch titles, meta descriptions or alt text — those belong to SnippetGrove, a separate WP Shelf plugin. Crawlune is about whether a machine reading your page can extract an answer from it and is allowed to fetch it at all.
WordPress 6.4 or later and PHP 8.1 or later. Crawlune is GPLv2-licensed.
Start with the free audit.
Install Crawlune from WordPress.org to audit your pages and manage crawler access. You can also check a public URL in the browser before installing.