I added one tiny file to my website. Here's what happened in 14 days
I run a small tool. About 800 visits a month. Tiny. The kind of site nobody talks about because it's not a unicorn.
Earlier this year I noticed something weird. Half my Google traffic disappeared in two months. No penalty. No technical issue. Just slow bleeding.
Then I asked ChatGPT a question my own site answers really well.
I tried Perplexity. Same. Claude. Same. Even Google's own AI Overviews — which had been crawling my site weekly for years — pulled answers from a competitor's site instead of mine.
I'd been hearing about llms.txt for a few months. It's a tiny text file you put at the root of your site that tells AI search engines what you're about and which pages they should look at first.
I put it off because I assumed it was a half-baked spec that wouldn't matter. I was wrong on both counts.
The format is solid (Anthropic, Mintlify, Cursor, the Pydantic folks all use it). And it definitely matters — the major AI labs confirmed last year that they read it.
So I made one. Took me 30 seconds because I used this free tool. Paste URL, wait, download, upload to site root. Done.
I check my server logs every Sunday. Here's what showed up after I added llms.txt:
Day 1–3: nothing (waited for crawlers to discover the file)
Day 4: GPTBot showed up for the first time ever
Day 6: ClaudeBot crawled half the site
Day 8: Meta-ExternalAgent (Meta AI / WhatsApp) hit 12 pages
Day 11: PerplexityBot found me
Day 14: Amazonbot, CCBot also appeared
Five new crawlers in two weeks. I had literally never seen any of them in my logs before.
Some. Not all. Honestly the truth is messier than the marketing version of this story would suggest:
Direct AI traffic: small but growing (10–20 visits a week from ChatGPT-style referrers)
Organic Google search: still down from peak, but stopped bleeding
Brand searches: slight uptick — possibly people seeing my site in AI answers and then googling
Total visits: now ~950/mo, up from ~800
Not a transformation. But it's the first month in six that the trend has reversed.
If you have a website that depends on organic traffic, yes. There's no downside, no cost, no platform lock-in, no signup, no risk.
The only real argument against is "what if AI search doesn't matter as much as people think." Fair. But the file is 30 seconds of work and a free download, so the bet is asymmetric.
If you want to try it, the tool I used is https://llms-txt-generator.net. Free, no signup, no URL cap. There are other options too (sitespeak.ai, llmstxt.io) but I tried a few and this one had the cleanest output.
Free file. Takes a minute. AI crawlers find you. Maybe makes a difference. Almost definitely doesn't hurt.
