There Is No "Latest Chrome"

A few weeks ago I wrote that my TLS fingerprint benchmark was measuring keep-alive. Against one Cloudflare-fronted site, the winning fingerprint and plain Chrome tied once each attempt got its own connection, and my conclusion was that proxy-node luck, not the fingerprint, decided whether a request got through.

A week later I applied that lesson to a different site and it was wrong. This time the fingerprint was the problem, and it took me four fixes in about a day to find that out. One of them I had to roll back after it went 0 for 12 in production.

The fingerprint in question wasn't obscure. It was the default Chrome profile behind about 200 of my scrapers, and it had quietly fallen behind real Chrome. The fix for that then broke other sites, which changed how I manage fingerprints entirely.

TL;DR

• A luxury retailer's Akamai edge started returning 403 Access Denied to my Chrome-impersonating HTTP client on its category pages.
• Fix 1 (switch to a mobile Chrome profile) was based on a sweep from my laptop's single IP. It didn't work in production.
• Fix 2 assumed per-IP luck, because 1 of 6 identical production requests succeeded, and retried on fresh connections. It went 0/12 and I rolled it back.
• Fix 3 corrected stale headers in the shared Chrome profile but kept its Chrome 133 TLS handshake. 0/6 live.
• Fix 4 was a new profile built from a real Chrome 153 capture, with an up-to-date TLS handshake and HTTP/2 settings. 6/6 live.
• Updating the shared "latest" profiles in place to match real browsers broke other sites that worked with the old shapes. So every shipped profile is now immutable and versioned, and each scraper pins the versions that its own live tests chose.

Four fixes in about a day

The uptime monitor flagged two endpoints for the retailer: one that loads the site's navigation, and one that loads a product category page. Production logs showed them seconds apart, from the same egress path and the same client, with opposite results: navigation returned 200 with real data, and the category page returned a 403 from Akamai. That looked like a rule specific to category pages.

Fix 1: a different fingerprint, measured from one IP

I ran every profile in my HTTP client against the blocked category URL from my laptop. Almost everything got the 403 or a protocol error. The mobile Chrome profile got 200 with the full page, twice, on two different categories, while desktop Chrome failed both in the same interleaved pass.

Then my laptop's IP got throttled. After about a dozen requests in a few minutes, every request from that IP failed, including the homepage and the mobile profile itself. I noted that confound in the commit, kept the 2/2 vs 0/2 result as the evidence, and shipped a rotation with mobile Chrome first.

In production it didn't help. The category endpoint still failed, and now the navigation endpoint was failing too.

What I'd missed is simple. My laptop had no proxy, so it had exactly one IP. A sweep from one IP can only vary the fingerprint. It can't tell you anything about which exit a production request draws, and production draws from a rotating pool.

Fix 2: the lesson from my last post, applied to the wrong site

So I went to production and sent the exact same category request six times in a row. One of the six returned 200 with 390 products. The other five failed.

A stable fingerprint rule shouldn't vary from request to request against an unchanged client, so I concluded the problem was per-exit reputation. My note from that hour cites my own earlier lesson that a hard block is transient on a rotating pool and that "retrying is useless unless the retry gets a new connection." Both of the profiles I'd been rotating were pooled keep-alive clients, pinned to whichever exit they first drew. That explained everything neatly.

The fix was five attempts, each on a fresh connection, mixing three profiles, inside a 45-second budget. It measured 0 out of 12 against live production. That was worse than before any fix. I rolled production back to the last image from before the incident. The rollback commit says I was doing that "rather than shipping a third unverified guess."

Looking back, 1 in 6 was never strong evidence for per-exit luck. It's just as consistent with a block that's almost total, plus the odd exit or edge node that doesn't enforce it. I read it through the frame I'd most recently written about.

Fix 3: compare against a real browser

With both theories disproven by production measurements, I did what I should have done first: loaded the blocked page in a real, current browser and compared what it sent with what my client sent. The browser (Chrome 152) got the full page on the first try.

The diff found two problems. The retailer package overrode the accept header with a value thinner than real Chrome's. More importantly, the shared Chrome header map used by every Chrome profile in my client, about 200 scraper families, was pinned to Chrome 133. It still sent the obsolete pragma and cache-control headers, didn't send the priority header current Chrome sends, and claimed version 133 in the user agent.

I had fixed this exact bug before. In August a header diff against a real browser found the same stale defaults getting 403s on a secondhand-fashion site. I fixed it in the Brave profile's own header map and never touched the shared one.

Fix 3 added a new "Chrome latest" profile with corrected headers and a 152 user agent. I kept the Chrome 133 TLS handshake on purpose. The TLS library my client uses has no Chrome preset newer than 133, and a July analysis had concluded that the 133 handshake already carried every modern TLS extension that mattered, so it "is not meaningfully behind" newer Chrome at the wire level.

Live against production: 0 out of 6.

Fix 4: the handshake was part of it

That July conclusion was wrong. Testing the new profile the same afternoon, from one egress, with three attempts each and no retries:

The new profiles use a hand-built TLS ClientHello that matches a real Chrome 153 capture, and Chromium's current HTTP/2 settings. The old one differed in ways a bot manager can see before it reads a single header: Chrome 150+ added new post-quantum signature algorithms to the handshake, and the old profile's HTTP/2 SETTINGS frame advertised MAX_CONCURRENT_STREAMS=1000, which no recent Chrome sends. Fixing the headers alone changed nothing. The handshake was already enough to fail.

After the next deploy the live canaries went 6/6 on the retailer's endpoints.

Then fixing "latest" broke other sites

The obvious next step was to fix the shared profiles themselves: update the existing "latest" Chrome, Safari and Firefox profiles to match real current browsers, so all 200 families would benefit.

I tried that before versioning anything, and a live A/B showed it regressing other sites that worked with the old shapes:

• A UK broadcaster's news site (Akamai). The updated Safari profile got Akamai's 2 KB JavaScript interstitial. A bisect found only one thing mattered: accept-language. The real captured value, en-US, got the interstitial. The legacy profile's value, zh-CN, a leftover default from the upstream HTTP library, got the full page, because the request was leaving from a Hong Kong exit. The wrong value was working because it happened to match the exit's location.
• A UK newspaper (Akamai). Firefox failed only when HTTP/2 PRIORITY frames were missing, which one widely used reference implementation of modern Firefox gets wrong. The legacy Firefox and Safari profiles passed 3/3. The new Chrome 153 profile passed 1/3.
• A US grocery chain's search page (Akamai), through the production pool. The legacy Chrome 133 profiles cleared it 3/3 on the US pool, and nearly every new versioned profile got the JavaScript stub. That's the exact opposite of the retailer.

So there's no single correct Chrome. What one site's bot manager accepts, another's rejects, and "correct" (matching a real current browser) isn't the same as "accepted." A profile that worked on the grocery chain because of a quirk was just as valuable as one that matched Chrome 153 perfectly.

Versioned and frozen, like an API

So I stopped editing profiles. The rules now:

• Every shipped profile is immutable. A profile's name includes the browser, major version and platform, like browser_chrome_153_mac. Once shipped, its bytes never change. Legacy unversioned profiles are frozen too, byte-identical to before, and are not moving "latest" aliases.
• A new browser release is a new profile beside the old ones. Old ones stay until nothing uses them. 19 versioned profiles went in on the first day, across Chrome, Edge, Brave, Opera, Samsung Internet, Safari and Firefox.
• Each scraper pins exact versions, usually as a short ordered rotation chosen by its own live tests. The retailer now uses Chrome 153 macOS then Edge 153 Windows. The grocery chain's coupon endpoint rotates through legacy Safari profiles.
• Every profile is checked on the wire: a live test compares its actual TLS and HTTP/2 fingerprint against the capture it was built from.
• A weekly check flags new browser releases. Its first alert was a false positive. It read Chrome's version history, which lists short-lived early-stable builds, and reported Chrome 155 when the stable channel was actually serving 154. It now reads only the currently served release.

This also exposed why the Chrome 133 profile aged unnoticed. I already had a fingerprint drift monitor, but it only compares what the client sends against a committed baseline. It tells you if your profile changed. It can't tell you the profile has fallen behind the browser it claims to be. Those are different checks, and I only had the first.

The same week, a different mismatch

The grocery chain's coupon endpoint had its own version of this. Two days earlier my notes had concluded, after a production browser capture, that the failure was an upstream edge rejection and "not a missing profile." They also said: "Do not add guessed header values, fingerprints, or longer capture budgets."

Two things turned out to be wrong with that:

• The step that was being blocked wasn't needed. Watching the real page in a browser on my own connection showed the coupon call only needed a few static request headers, none tied to a session. The bootstrap request my service ran first, which was the call being rejected, could be skipped entirely.
• The service contradicted its own fingerprint. With those headers, a raw Safari-profile request through the US pool passed 8/8. Through the service it failed 5/5. The service's shared header helper forced a Chrome 146 user-agent onto every request, so a Safari TLS handshake arrived claiming to be Chrome. That mismatch is exactly what a bot manager looks for.

The fix shipped with the service passing 3 of 5 runs. The two misses were self-inflicted: those exits had just taken about 100 of my test requests and were serving Akamai's rate-limit challenge. The first live canaries after deploy were 1/3. A paced re-measure, with three seconds between requests, reordered the rotation to put iOS Safari first, and the next canaries were 3/3.

Per-exit luck is real here too. That paced re-measure had profiles ranging from 0/6 to 5/6 on the same pool. On the same day a pizza chain's menu endpoint cleared about half the time per request on every profile, and the fix there really was more attempts on fresh exits. My last post's conclusion wasn't wrong. It just wasn't universal, and I'd been treating it as if it were.

What I'd do differently

Measure against production, not your laptop. A sweep from one IP can only vary the fingerprint. For anything behind a rotating pool, run the sweep through the pool itself. For the grocery chain I finally did this by port-forwarding the production proxy service and pointing a probe binary at it. Three draws per profile only gives you a direction. Use eight or more interleaved draws before changing a service.

Check the body, not the status. Akamai's interstitial is a 200 with a 2 KB body. Several of my "passes" along the way were that page. Check the body size, or better, parse it.

Diff against a real browser first. Two wrong fixes came from reasoning about which layer was blocking. The comparison against a real browser took minutes and found the real problem immediately. The down-endpoint runbook I wrote after this week starts from a line I wish I'd written earlier: "A fix is a measurement, not a hypothesis." Another of its rules would have saved me the coupon detour: before tuning a step that keeps failing, ask whether the step is needed at all.

Don't carry a conclusion from one site to another. My keep-alive post was one Cloudflare site at n=15. It proved that fingerprint didn't matter there. I treated it as a rule about fingerprints, and the first time it didn't hold, it cost me a rollback. Which layer decides the outcome (fingerprint, headers, exit reputation, language consistency) is a property of the site and its bot manager, and it changes over time.

Version fingerprints like API contracts. A profile is an interface between your client and every site that trusts it. Changing it in place is a breaking change for every consumer at once, and you won't find out which ones broke until their monitors go red. Add new versions, keep the old ones, and let each consumer move when its own tests say so.