Quite possibly. One text file at yourclinic.com/robots.txt decides which crawlers may read your site, and a surprising number of med spa sites block AI crawlers without anyone ever choosing to. A security plugin toggle, a copied template, or an old developer decision can quietly opt your clinic out of every AI answer.
What robots.txt does
It is a plain text file of rules: which user agents may crawl which paths. Legitimate crawlers obey it completely. So one line, Disallow: /, aimed at the wrong bot, and that engine never reads another page of your site. No error, no warning. You just stop existing to it.
The bots that decide your AI visibility
- OAI-SearchBot and ChatGPT-User power ChatGPT search and live lookups.
- GPTBot feeds OpenAI training.
- ClaudeBot and Claude-SearchBot power Claude.
- PerplexityBot indexes for Perplexity.
- Google-Extended feeds Gemini and AI Overviews.
Block the search-facing ones and your clinic cannot be found or cited, no matter how good the site is.
How accidental blocks happen
- Security plugins with a "block bots" switch. Flipped on for spam, catches AI crawlers too.
- Copied robots.txt templates. Many popular templates shipped with AI-bot blocks during the 2023 panic and never got updated.
- CDN and firewall rules. Cloudflare bot-fight modes challenge AI crawlers even when robots.txt allows them.
- Old developer decisions. Someone blocked "scrapers" years ago. Nobody has looked since.
The fix
Explicitly allow the AI search crawlers in robots.txt, remove blanket disallows, and turn off bot-fight rules for verified AI user agents. It is minutes of work once you know it is there. Knowing it is there is the whole problem.
See what AI sees
Run your clinic through the free AI visibility check and get your score plus the exact fixes in about thirty seconds.
Check my clinic free →Straight answers
Should I block AI training bots on purpose?
That is a real choice, but make it knowingly. Blocking GPTBot keeps your content out of training data. Blocking OAI-SearchBot or ChatGPT-User keeps your clinic out of answers patients see today. For a business that wants patients, blocking search-facing bots costs bookings.
Does blocking bots protect my site?
Not meaningfully. Malicious scrapers ignore robots.txt entirely. The only bots that obey your blocks are the legitimate ones you probably want reading you.
What should a med spa robots.txt look like?
Allow the AI search crawlers by name, disallow only genuinely private paths, and reference your sitemap. Run the free check after any change to confirm all thirteen bots show green.