homeblogtags
  • Published on
    October 23, 2026 (Today)

    My Canaries Kept Passing and My Rollouts Kept Failing

    Web-ScrapingWeb-CrawlerGoKubernetesPostmortem
    A worker rollout passed an 80-minute production canary at 72.5-75.5% yield, then its standing pods tripped their circuit breakers at 9.5%. T...
  • Published on
    October 23, 2026 (Today)

    I Probed a Million Sites for llms.txt. Half the 200s Were Junk, and My Accept Header Invented More.

    Web-ScrapingData-StudyWeb-CrawlerGoKubernetesllms.txt
    I fetched /llms.txt from the Tranco top 1,000,000 domains. The run agreed with its own smoke test to about 99% and was still wrong in three ...
  • Published on
    October 16, 2026 (7d ago)

    There Is No "Latest Chrome"

    Web-ScrapingTLS-FingerprintingAkamaiHTTP/2DebuggingGo
    One Akamai 403 took four fixes in about a day. One was rolled back after 0 of 12 live requests passed, and one applied the lesson of my last...
  • Published on
    September 25, 2026 (28d ago)

    What Parallel Coding Agents Did to Go's Build Cache

    GoAI-AgentsBuild-SystemsDeveloper-ToolingGitDebugging
    I run several coding agents at once, each in its own git worktree of one Go monorepo, on one 8-core Mac. The model was never the bottleneck....
  • Published on
    September 18, 2026 (1mo ago)

    My TLS Fingerprint Benchmark Was Measuring Keep-Alive

    Web-ScrapingTLS-FingerprintingCloudflareProxiesBenchmarkingGo
    I benchmarked 18 browser TLS fingerprints against a Cloudflare-challenged site and one of them won at 94%. Twenty-four minutes later I retra...
  • Published on
    September 9, 2026 (1mo ago)

    Your Crawler Won't Crash. It'll Wedge.

    Distributed-SystemsWeb-CrawlerGoKubernetesDebuggingObservability
    Five real incidents from a distributed crawler fleet where nothing crashed, no alert fired, and the system quietly stopped doing its job. A ...
  • Published on
    September 8, 2026 (1mo ago)

    How Much of the Web Blocks Bots? I Got the Number Wrong Four Times First.

    Web-ScrapingAnti-botWeb-CrawlerGoKubernetesData-Study
    I scanned the Tranco top 1,000,000 domains to measure how much of the web runs managed bot defense. The final answer is 53.5% of reachable s...
© 2026 Built with 💖 by Tony Wang • With TypeScript, Next.js, Tailwind • Inspired by Leerob
Support me • Contact me
My projects: Crawlora • Namefy • BizHQ