Last month we built a toll booth. AI crawlers and data-harvesting bots that turned up looking for something to ingest got a slow text notice and an invoice for 0.00001 BTC instead of an expensive decoy. It was cheap to serve, it wasted their time, and it felt good.
This month we went back to see who was actually paying it.
MIRE is not a website. It sits behind twenty-eight of them, and it only ever sees what misses — anything the real sites return a 404 for gets handed to the maze instead. That makes it possible to ask a question you cannot normally ask: of everything arriving at these domains, how much is looking for something that was never there?
Twenty-nine days, 5 July to 2 August, both sets of logs complete:
| At the edge | Fell into the maze | ||
|---|---|---|---|
| Requests | 873'011 | 367'692 | 42.1% |
| Distinct IPs | 57'333 | 24'077 | 42.0% |
| Distinct user-agents | 7'434 | 2'215 | 29.8% |
Two out of every five requests to these domains is a miss. Not a typo, not a stale bookmark — a request for something that has never existed on any of them.
The user-agent line is the interesting one, because it moves differently. Only 30% of the strings we see ever end up in the maze, which means most of the variety on the internet is well-behaved and a small, determined minority generates the volume. 5'223 distinct user-agents — 70% of everything we saw — never missed once in 29 days.
If you rank user-agents by what proportion of their own traffic misses, you get a rather blunt document.
| Miss rate | Edge | Maze | Who |
|---|---|---|---|
| 94% | 20'088 | 18'811 | SemrushBot |
| 85% | 86'091 | 72'905 | AhrefsBot |
| 81% | 8'237 | 6'632 |
Mozilla/5.0 (Windows NT 10.0; Win64; x64)
— nothing else |
| 72% | 2'589 | 1'853 | l9scan / leakix |
| 67% | 18'291 | 12'318 | ClaudeBot |
| 67% | 2'760 | 1'836 | SERankingBacklinksBot |
| 63% | 67'681 | 42'567 | a generic Chrome string |
| 62% | 4'823 | 2'975 | Go-http-client/1.1 |
SemrushBot spends 94% of its time here asking for things that do not exist. It is not crawling these sites in any meaningful sense; it is enumerating a guess list against them. AhrefsBot, 85%. Between them that is 91'716 requests in a month, essentially all of it wrong on purpose.
Over the same 29 days, the July mechanism intercepted 105'065 of the 367'692 requests that reached the maze. Just over a quarter.
The other three quarters were not evading it. They simply were not the kind of thing it knew how to look for.
The single most common client hitting MIRE does not identify itself at all.
89'470 requests in 29 days — 24.3% of the maze — arrived with no User-Agent header. That is more than AhrefsBot. It came from just 331 addresses, which works out at 270 requests each.
The toll booth was a list of names. It matched GPTBotand ClaudeBotand SemrushBotand thirty-odd others, and it could not match the empty string, so a quarter of all traffic walked past it and collected a full, lovingly generated decoy on the way through.
That is now fixed, and it cost one line of code.
The subtler group had user-agents. Entirely ordinary ones.
| Requests | Distinct IPs | Requests per IP |
|---|---|---|
| 1'692 | 1'591 | 1.1 |
| 1'534 | 1'466 | 1.0 |
| 1'510 | 1'492 | 1.0 |
| 1'425 | 1'420 | 1.0 |
One request per address. Fifteen hundred addresses. One Mac Chrome string shared between all of them, then gone.
That is a residential proxy pool, spread deliberately thin so no single address ever accumulates enough history to look worth blocking. And the user-agent is not a clue — it is a real Chrome string that real people send every day. Any pattern sharp enough to catch it catches them too.
You cannot write a regex for this. The string is not the tell. The distribution is the tell, and no request carries its own distribution.
We widened the list anyway, because some of the gaps were embarrassing.
SERankingBacklinksBotis the same species as AhrefsBotand SemrushBot, both already on it. YisouSpider, DeepSeekBot, Google-CloudVertexBot, MistralAI-User,AI2Bot, Scrapy— all classes we already claimed to handle, merely spelled differently. Baidu, Yandex and Applebot joined them. Googlebot and bingbot deliberately did not; MIRE only works if scanners can still find it.
Total extra traffic captured by widening the list: 2'552 requests. 0.7%.
That is the uncomfortable finding of the month, and it is about our own July release. The mechanism we shipped as the headline feature turned out to be the least effective of this month's three changes, by a factor of thirty-five. The single line handling the empty string bought more than every name we added, several times over.
Maintaining a list of everyone who might lie to you is not a strategy. It is a subscription.
The thing we should have noticed sooner is in the shape of the traffic, not the strings.
Because MIRE only sees misses, the median IP address across 29 days made exactly one request. The 90th percentile made 19. And then:
4.46% of addresses account for 75.1% of everything.
Once that number is in front of you, the identification problem simply dissolves. You do not need to know whether the thing on the other end is a scraper, a scanner, a security vendor or somebody's broken cron job. You need to know it has been here 400 times and stopped learning anything new around request 50.
So MIRE now runs a decoy budget. The first fifty requests from an address get the full performance — the fake .env, the plausible admin panel, the SQL dump that takes eleven seconds to deliver. After that it drops to the same cheap notice the crawlers get. No classification, no list, no guessing about intent. A meter.
Replayed across the same 29 days:
| Requests | Share | |
|---|---|---|
| Before — named crawlers only | 105'065 | 28.6% |
| After — named crawlers | 107'617 | |
| — no user-agent | 89'470 | |
| — past decoy budget | 105'473 | |
| After — total cheap path | 302'560 | 82.3% |
| Still generating full decoys | 65'132 | 17.7% |
Four fifths of the maze now costs almost nothing to serve. The expensive fifth is the fifth that is still telling us something.
We widened the list anyway, because some of the gaps were significant.
SERankingBacklinksBotis the same species as AhrefsBotand SemrushBot, both already on it. YisouSpider, DeepSeekBot, Google-CloudVertexBot, MistralAI-User,AI2Bot, Scrapy— all classes we already claimed to handle, merely spelled differently. Baidu, Yandex and Applebot joined them. Googlebot and bingbot deliberately did not; MIRE only works if scanners can still find it.
Total extra traffic captured by widening the list: 2'552 requests. 0.7%.
That is the uncomfortable finding of the month, and it is about our own July release. The mechanism we shipped as the headline feature turned out to be the least effective of this month's three changes, by a factor of thirty-five. The single line handling the empty string bought more than every name we added, several times over.
Maintaining a list of everyone who might lie to you is not a strategy. It is a subscription.
The budget has one deliberate hole: POST requests are exempt.
A brute-force attempt against a fake login is the most valuable thing that happens here. It arrives as a POST, it carries a username and a password, and both go straight into the credential log — 4'241 pairs in these 29 days. Cutting someone off at fifty requests for being repetitive would be precisely the wrong call when the repetition is the payload. The meter only runs on GETs.
Everything else that makes MIRE worth operating still runs at full cost for that first fifty. Over the same 29 days: 131'817 canary tokens issued, 116'802 generic PHP traps, 1'940 fake logins, 1'429 upload handlers, 972 installers, 646 traversal decoys.
We did not get cheaper by doing less. We got cheaper by stopping doing it to audiences that had already seen the whole show.
Three changes went out this month, in ascending order of how much they actually mattered:
| # | Change | Captured | Share |
|---|---|---|---|
| 1 | Widened the named-crawler list | 2'552 | 0.7% |
| 2 | Requests with no user-agent take the cheap path | 89'470 | 24.3% |
| 3 | Per-IP decoy budget, fifty requests, GET only | 105'473 | 28.7% |
The one that took the most work returned the least. The one that took a single line returned thirty-five times more. That ordering is the release.
But everything in that table is a replay against historical logs, not an observation.
The change deployed at 11:10 today, which zeroed every per-IP counter, so the production record so far is 610 requests. In those: 198 no-user-agent interceptions, 29 named crawlers, and the decoy budget firing three times — first at 12:14:53 against an Azure address on ciso.li that had just reached its fifty-first request, then again on mire.cc an hour later. It works. It fires exactly where it was told to.
The live cheap path, though, is running at 37.7% — not the 82.3% the replay predicts — and that gap is the honest part. A budget only bites once addresses have accumulated history, and every counter on this box is three hours old. Of the two components, the one that needs no memory — the empty user-agent — is already carrying 32.5% on its own. The one that needs memory has produced three hits. Whether it converges on the replayed number over a full month is the thing to check, not something this post can claim.
Two other things are on the watch list.
The IP table holds 50'000 addresses. Inside a 29-day window the maze sees 24'077, comfortably under it — but across six months it has seen 92'988, so over long horizons eviction is real and counters do get reset. It works in our favour, since the cache drops least-recently-seen first and a repeat offender is by definition recently seen, but the budget is best-effort rather than a guarantee.
And 4.46% is this honeypot, on these domains, over these 29 days. It is not a law of nature.
July's conclusion was that the scanners keep telling us what to build and we were finally listening. That still holds. It turns out what they were saying in August is that we had built the right idea and put the wrong lock on it.
A toll booth that only charges the drivers who write their real names in the register is not a toll booth. It is an honour system with extra steps.
So it stopped asking for names. It counts axles now.
