Web scraping, for engineers
How detection works, what it measures, and what current tools do about it. Short posts: a TL;DR, a diagram, the mechanism, and an experiment you can run.
How anti-scraping works
Ten posts on what bot detection measures, how the vendors use it, and what current tools do about it. Each one starts with a TL;DR and ends with an experiment you can run.
- Anatomy of a browser fingerprint
- TLS and HTTP/2 fingerprints: identified before the first header
- Headless tells: how automated Chrome gives itself away
- Behavioral biometrics: scoring humans without a CAPTCHA
- WebRTC, DNS and proxy leaks: how your real IP escapes
- Inside an anti-bot client script: collect, challenge, token
- IP reputation and rate models: why slowing down is not enough
- The stealth tooling landscape in 2026, with a benchmark you can run
- Intercept the internal API instead of scraping the HTML
- CAPTCHAs in 2026: the challenge is the fallback, not the gate
All posts
CAPTCHAs in 2026: the challenge is the fallback, not the gate
How reCAPTCHA v3, Cloudflare Turnstile and hCaptcha score visitors in 2026, when a puzzle appears, and why solving services rarely pay off for scraping.
Intercept the internal API instead of scraping the HTML
How to find a site's internal JSON API in DevTools, handle its auth and pagination, and call it directly for cleaner, faster data than HTML scraping.
The stealth tooling landscape in 2026, with a benchmark you can run
Nodriver, Patchright, Camoufox and curl_cffi compared in 2026: what each stealth tool fixes, what no tool fixes, and a benchmark harness you can run.
IP reputation and rate models: why slowing down is not enough
How bot detection scores datacenter, residential and mobile IPs, why rate limits are velocity models, and why rotating proxies on every request backfires.
Inside an anti-bot client script: collect, challenge, token
How anti-bot scripts from DataDome, Akamai, Kasada and Cloudflare collect signals, run challenges and issue tokens, and what Web Bot Auth changes.
WebRTC, DNS and proxy leaks: how your real IP escapes
How WebRTC, DNS and timezone leaks reveal the real IP and location behind a proxy, how to close each one, and a ten-minute leak test you can run.
Behavioral biometrics: scoring humans without a CAPTCHA
How anti-bot systems score mouse paths, scroll cadence and keystroke timing to spot scripts without a CAPTCHA, and why replayed human input fails.
Headless tells: how automated Chrome gives itself away
How sites detect headless Chrome, Puppeteer and Playwright in 2026: navigator.webdriver, Runtime.enable, isolated worlds and environment tells.
TLS and HTTP/2 fingerprints: identified before the first header
How TLS fingerprinting (JA3, JA4) and HTTP/2 fingerprints identify scrapers before any header is sent, what changed in 2026, and how impersonation works.
Anatomy of a browser fingerprint
What browser fingerprinting is, the four signal layers anti-bot systems read (network, TLS, runtime, behavior), and how much each one reveals.
Know an engineer who fights bot detection?
Share the series.
Extract data in your own browser
ScrapePass runs in your Chrome browser using your current session and connection. Websites can still restrict access or limit requests. Pay only for the days you use it. Running extractions is free. Exporting data, downloading images, and saving screenshots require an active 24-hour pass.
Add to Chrome