I added one tiny file to my website. Here's what happened in 14 days
Quick story.
I run a small tool. About 800 visits a month. Tiny. The kind of site nobody talks about because it's not a unicorn.
Earlier this year I noticed something weird. Half my Google traffic disappeared in two months. No penalty. No technical issue. Just slow bleeding.
Then I asked ChatGPT a question my own site answers really well.
It didn't mention me.
I tried Perplexity. Same. Claude. Same. Even Google's own AI Overviews — which had been crawling my site weekly for years — pulled answers from a competitor's site instead of mine.
That was the wake-up.
What I tried
I'd been hearing about llms.txt for a few months. It's a tiny text file you put at the root of your site that tells AI search engines what you're about and which pages they should look at first.
I put it off because I assumed it was a half-baked spec that wouldn't matter. I was wrong on both counts.
The format is solid (Anthropic, Mintlify, Cursor, the Pydantic folks all use it). And it definitely matters — the major AI labs confirmed last year that they read it.
So I made one. Took me 30 seconds because I used this free tool. Paste URL, wait, download, upload to site root. Done.
What changed in 14 days
I check my server logs every Sunday. Here's what showed up after I added llms.txt:
Day 1–3: nothing (waited for crawlers to discover the file)
Day 4: GPTBot showed up for the first time ever
Day 6: ClaudeBot crawled half the site
Day 8: Meta-ExternalAgent (Meta AI / WhatsApp) hit 12 pages
Day 11: PerplexityBot found me
Day 14: Amazonbot, CCBot also appeared
Five new crawlers in two weeks. I had literally never seen any of them in my logs before.
Did traffic come back?
Some. Not all. Honestly the truth is messier than the marketing version of this story would suggest:
Direct AI traffic: small but growing (10–20 visits a week from ChatGPT-style referrers)
Organic Google search: still down from peak, but stopped bleeding
Brand searches: slight uptick — possibly people seeing my site in AI answers and then googling
Total visits: now ~950/mo, up from ~800
Not a transformation. But it's the first month in six that the trend has reversed.
Should you do it?
If you have a website that depends on organic traffic, yes. There's no downside, no cost, no platform lock-in, no signup, no risk.
The only real argument against is "what if AI search doesn't matter as much as people think." Fair. But the file is 30 seconds of work and a free download, so the bet is asymmetric.
If you want to try it, the tool I used is https://llms-txt-generator.net. Free, no signup, no URL cap. There are other options too (sitespeak.ai, llmstxt.io) but I tried a few and this one had the cleanest output.
TL;DR
Free file. Takes a minute. AI crawlers find you. Maybe makes a difference. Almost definitely doesn't hurt.