With the new canary tokens, it’s the case that some visitors take a canary key, try it once and are never seen again. Others keep coming back.

I Just Can’t Get Enough

One of them has now visited malk.ch four times in a week, asked for more content each time, and spent one day trying the same key in three waves, 281 calls, before it expired. All tests bar the “who am I?” have, of course, failed.

Another turns up every other morning at half past six, takes a key and checks it within a second.

All times are Swiss local time.

What is “who am I?”

Every AWS key can make one call that needs no permissions at all: sts get-caller-identity (aka “who am I?”).

The response is the account number and the name of the user or role the key belongs to – the call does nothing else. This convincing lure makes the first check with a key seem authentic – which is the whole point.

MIRE’s canaries are real, live keys, so the attacker believes the key is alive and useful. A “yes” from “who am I?” is what convinces them that they have something worth coming back for – and the same call is what tells us what they are trying to achieve with their stolen key.

Field Report§019–16 OctoberKeys redacted
4
Visits to malk.ch from one block of addresses
11 → 53
Paths asked for, first visit to last
320
AWS calls with two keys
6
Calls that worked – all “who am I?”
“‘Who am I?’ kept saying yes, so they kept coming back.”

An old friend returns (again)

Regular readers have met this attacker already. In Three Keys, Three Hands it was “the shopping list”: someone hiding behind a VPN asked malk.ch for twelve things in four seconds on 10 October, took a key and, 28 minutes later, spent 111 seconds trying to access AI models, text messages, email and secrets. Nothing worked.

Investigating the logs, it’s clear that this was neither its first visit nor its last. Every visit came from the same small block of addresses at Datacamp, a hosting company whose ranges are rented out as VPN exits, with each claiming to be Chrome 124 on Windows. It has only ever visited malk.ch, and its list of paths grew each time (thank you, MIRE):

Field Report§02Four visitsKeys redacted
Evidence from MIRE and web logs for attacks on malk.ch
# when          requests  paths  new paths asked for             key handed out
 9 Oct 01:22        11     11   .env variants, config.js.php    fake (an old, synthetic key)
10 Oct 09:57        12     12   the home page                   key ████████  used: 39 AWS calls
10 Oct 16:18        42     21   .aws/credentials, SendGrid,     key ████████  never used
                                WooCommerce Stripe settings
15 Oct 14:18        53     53   ~25 Google Cloud key files,     key ████████  used: 281 AWS calls
                                .npmrc, appsettings.json,
                                settings.py

You can watch the attack and probing being developed. Each visit tried the previous paths and added new ones: first the payment-gateway settings of a WooCommerce shop, then every file name under which a Google Cloud service account might be saved – gcp.json, sa.json, firebase-adminsdk.json, vertex-credentials.json and about twenty more. Vertex is Google’s AI platform, so the list is now after AI models on two clouds.

The key collected at 16:18 on 10 October was never tried. I can’t say why, perhaps the morning key had been refused everything, perhaps the attacker was confused? Five days later they were back again – this time with a longer list and more patience.

Three waves

On 15 October at 14:18:06 a key was collected. Thirty seconds later it was in use, from the same address, by the same pair of Go programs that we saw on 10 October. Then the attacker went quiet for almost eight hours, before returning twice to test the key – this time from a different address in the same company, a VPN exit in Marseille rather than Madrid:

Field Report§03One key, 15 OctoberKey redacted
Alerts linked back to our attacker who loves cities beginning with “M”
# wave           from               calls  what was tried
14:18:34–14:20:26  VPN exit, Madrid     98   AI models ×64, text messages ×16,
                                          email quota ×6, start servers ×5,
                                          secrets, parameters
  … 7 h 41 min …
22:01:31–22:03:29  VPN exit, Marseille  74   AI models ×60, text messages ×6,
                                          email, secrets, who is this user?
  … 1 h 21 min …
23:24:31–23:26:19  VPN exit, Marseille 109   AI models ×56, text messages ×22,
                                          start servers ×10, email ×8, …
  … 02:18 the key expires …
281
Calls with one key
41
Attempts to run an AI model
15
Attempts to start a server
5
Calls that worked – all “who am I?”

Each wave follows the same script as the one on 10 October: check the key is alive, then ask Bedrock, Amazon’s AI service, which models it can use, and try to call one; then ask whether the account can send text messages and email; then go looking for stored secrets.

On 15 October two extra asks were seen where the script tried to start servers, usually for crypto mining or more scanning, and in the evening it asked IAM who the key belonged to and what the account contained.

The answers never changed. Every call except “who am I?” was refused, or rejected as malformed. The script doggedly returned and tried again. And again!

A person would have given up after the first wall of “AccessDenied” but the script doesn’t: the key is in a list of keys still marked “valid” and (brilliant) “who am I?” keeps giving a response so the automation keeps trying to use it.

Madrid in the afternoon and Marseille in the evening is probably one threat actor switching VPN exits, trying to hide. It does not appear to be a key sold on as the VPN is from the same provider, the attacker uses the same tools and same script. The waves stopped well before the key expired at 02:18, perhaps the three attempts finally caused confusion and frustration?

Ontbijt voor school?

A breakfast table at dawn: bread with chocolate sprinkles, a glass of milk, and a small brass key on a slice of bread, with a laptop glowing behind.
Same table, same time, every other morning.

The second returner is streamlined. A machine on a small hosting network in the Netherlands has been running the same routine on our sites since 22 August: load the home page, ask for five credential files – .env.local, .env.production, .env, credentials.json, service-account.json – then a few JavaScript bundles and .git/HEAD. Before 9 October it only ever got fakes.

Field Report§04Every other morningKeys redacted
What would you like for breakfast?
# when                 site      files  key                       checks
14 Oct 06:28:46  mire.cc      5   key ████████  5 × who am I?  +1 s
15 Oct           no visit
16 Oct 06:37:04  ciso.li      5   key ████████  5 × who am I?  +1 s

Five files, five copies of the same key, five checks, all from the same address with the AWS command-line tool, one second after the first file.

It doesn’t even notice that the five keys are one – could this be a script kiddie?

After “who am I?” says yes, it does nothing.

If the pattern holds it should be back on Sunday morning… I can’t wait!

This is the opposite of the Datacamp visitor’s behaviour: no snooping around, no second wave, just a collector that validates in the simplest manner. Somewhere else, presumably, is the list it feeds (or doesn’t).

This morning’s guests

While this post was being written, two more visitors dropped by.

This morning, while writing this post, another old friend from Three Keys, Three Hands came back: Omegatech, the small hosting network whose machine walked our .git folders in early October. The source IP is new as is the label (Chrome 109) and the target. Its scan included 64 requests over a five minute period – .env variants, .aws/credentials, SQL database dumps, Laravel config files and logs, Spring Boot’s actuator/env – and of course it consumed a key. One second later it was being tested from the same address:

Field Report§05Omegatech, 16 OctoberKey redacted
The Return of Omegatech
# what                                     calls  result
sts get-caller-identity                       1   success
list container clusters                      34   AccessDenied
ses identities, quota, sending enabled?      68   AccessDenied
lambda list functions                        17   AccessDenied
s3 list buckets                               1   AccessDenied
ses verify email identity                     1   AccessDenied
iam create user                               1   AccessDenied
iam create role                               2   AccessDenied

A different script from the Datacamp one, not targeting AI at all. First it checked for email capabilities, then containers and Lambda functions – places to run code – and then the three calls that matter: create a user, create a role, and try again. That is persistence. Had the key worked, Omegatech would now have its own login to the account, one that outlives the stolen key. Sadly, everything but “who am I?” doesn’t work…

Also during this morning, a machine at TECHOFF SRV LIMITED – the same company as our breakfast guest – asked barnettland.net for 57 paths in a minute.

It has been a regular there since July, with 18’084 requests so far, but this list was new: litellm.yaml, litellm/config.yaml, bedrock/config.yaml, proxy/config.yaml, gpt/config.json. LiteLLM is a gateway that sits in front of many AI providers, so its config file holds the keys for all of them. Hmmm, juicy free LLM access?

Field Report§06The LiteLLM hunterKey redacted
LiteLLM – must_have_AI_access
# time               from                    calls  what was tried
09:16:53–09:17:50  the collector              –   harvested (57 paths)
09:19:56–09:20:38  53 home lines             53   who am I? ×19, email ×17, SMS ×17
09:37:44           the collector               1   who am I?
10:43:58–10:44:09  30 home lines             31   list AI models ×14, run a model ×17
85
Calls with one key
84
Different addresses
28
Countries
20
Calls that worked – all “who am I?”
A city at night seen from above: scattered single lit windows across thousands of homes, each sending a thin thread of light up to one brass key hanging in the sky.
84 households, one key.

Almost every call came from a different address, and almost all of them were home broadband and mobile lines: Comcast, Bell Canada, Sky, TalkTalk, Viettel, Reliance Jio and more, in 28 countries. That is a residential proxy – traffic routed through other people’s home connections, often from devices whose owners have no idea, so that every request looks like a different ordinary household. Anything that counts per address sees one call each and shrugs. Tracebit saw all 85. On a SOC console that is 84 households making one call each – nothing to block, nothing to correlate. The only thread connecting them is the key itself.

This one is more “professional” it seems, wanting to cover tracks and then get in and get out – unnoticed. If all of the requests came from a single IP address, someone might notice – in this case, the canary told us the story.

And once again the end goal was AI: seventeen attempts to run a model using a stolen key – which had only one capability and that was to help tell this story.

One-Hit Wonders

Not everyone returns. On 13 October eleven addresses each took one key from one of our sites, and a separate machine checked all eleven in twelve seconds with TruffleHog, an open-source secret scanner.

None of those eleven IPs has been seen since. The checker has: on 15 October it cloned the fake .git folders on two of the sites the eleven had visited, three times in one day, until The MIRE’s decoy budget ran out. It never tried the keys again.

And one slow collector (let’s call it “The Snail”), from a Ukrainian IP, has been visiting on and off since July – 41 days so far. Since the 9th October update, it has taken 33 keys and none of them has been tried. Either the checker hasn’t run yet, or the keys are going to someone who hasn’t got round to them. Talk about feeling unloved…!!

What can we conclude?

Some stolen keys are not tried once but rather go into a queue, and the queue keeps trying the key as long as “who am I?” says yes. That’s why the canaries are temporary keys that expire after twelve hours: the alerts arrive with each call until the canary expires. That must be annoying.

The usage scope develops too. The Datacamp visitor grew from 11 paths to 53 in a week, and the new ones point the same way as its calls: AI models first, on any cloud that has them, and then onto functionality that can be leveraged for free to support other campaigns.

Since the 9th October update that brought seemingly valid tokens to life in The MIRE, 19 canary keys have been used, 4 of which were used by a pair of threat actors. The current numbers are on the statistics page and are updated hourly.