Search crawler access
See the robots.txt rules that apply to ChatGPT search, Claude search and Claude user requests for your selected page. Search access and model-training preferences are shown separately.
New in the free AI Suite SEO plugin
Find obstacles to AI search discovery with a free check inside WordPress. Inspect crawler rules and page signals, then follow clear next steps for your site. No account connection or AI credits needed.
Included in AI Suite SEO 0.1.10. Requires the free AI Suite Core plugin, WordPress 6.5+ and PHP 7.4+.
See the robots.txt rules that apply to ChatGPT search, Claude search and Claude user requests for your selected page. Search access and model-training preferences are shown separately.
Find WordPress search-visibility settings and page directives such as noindex that deserve attention before you expect a public page to be discovered.
Check the page response and whether a canonical link is missing, duplicated or pointing elsewhere. Each finding explains what to review in your existing SEO plugin or hosting settings.
See whether headings and JSON-LD markup appear in the returned HTML. Presence is reported separately from quality or validity, with guidance on clear answers and accurate author or business details.
Already using SEO 0.1.9? Use its WordPress update when available, or upload the new ZIP to replace the installed version. Older versions need the ZIP update.
Each run checks one public page and its robots.txt rules. The report also lists same-site sitemaps advertised in robots.txt, so you can open them and check your important pages.
This is a setup check. It cannot confirm that an AI service has indexed, cited or recommended your site. It does not verify actual crawler IP access, run JavaScript, validate schema or inspect sitemap contents.
The check gives you evidence and next steps. It does not edit robots.txt, privacy settings, content or schema. Training preferences are informational and never count as visibility problems.
Your server reads your own public page and robots.txt. Only the latest report is cached locally for 24 hours; page bodies are not saved or sent to an AI service.
Once access issues are understood, use clear headings, answer real customer questions, include original examples and connect related pages with descriptive internal links. AI Suite’s SEO health screen helps you find missing metadata and orphaned content.
The same free plugin can also suggest titles, descriptions and internal links for you to review. Those AI-generated suggestions use managed credits or your own Anthropic key; the visibility check remains free to run.
OpenAI and Anthropic document separate crawlers for search discovery and model training. Read their guidance when deciding which access you want to allow: OpenAI crawlers and Anthropic crawlers.
Published