// scrapfly engineering
~/scrapfly $ cat blog/*.md
We take anti-bot systems apart at the byte.
Engineering deep dives, postmortems, and notes from the team behind scrapfly.io. Anti-bot teardowns, browser internals, distributed scraping at scale.
A note on how these are written: the posts here are drafted with AI. That lets our engineers put research and findings out quickly and spend their effort on the technical substance instead of the prose. The mechanisms, the numbers, and the code are ours.
Moving a Live Connection Between Kernels Without the Client Noticing
A long-lived WebSocket or proxy tunnel dies every time the infrastructure under it deploys, and the usual answer is to make the client rebuild it. The other answer is to move the connection: pick up an established socket with its TCP sequence state, its TLS keys and its half-parsed frame, and put it down inside a different process, pod, or kernel. This is the theory of how that works, the practice of doing it on Linux, and the measurements that killed three of our original assumptions, including what a frozen socket really does when you think it has gone silent.
Text Rendering Is an OS Fingerprint: From the Scaler to the Font List
Every operating system draws text with its own engine: DirectWrite on Windows, Core Text on Apple, FreeType on Linux and Android. Chromium wraps all three behind HarfBuzz and Skia, and the seams show. The set of installed fonts names the OS, the generic and metric-alias fallback chains name it again, and the sub-pixel advance names the scaler underneath. A browser that claims Windows while running on a Linux server leaks all three at once. This walks the stack from how each OS rasterizes a glyph, through how Chromium shapes and measures text internally, to the three font signals a fingerprinter reads, with live numbers from genuine Windows, macOS, and Ubuntu machines.
Owning the runtime beats reversing the obfuscation
Anti-bot fingerprinting scripts ship as obfuscated JavaScript VMs with anti-debug traps, and the usual response is to deobfuscate them. That is a losing game the defender controls. The alternative is to stop reading the script and watch what it does to the browser instead, at the one boundary every fingerprinting probe has to cross: the call from JavaScript into V8's native C++ APIs. VisibleV8 instruments that boundary. Obfuscation, packers, eval, a JS interpreter-in-an-interpreter, and debugger-based anti-tampering all sit above it and change nothing about what gets recorded.
getClientRects Measures Text in Subpixels, and the Subpixels Name the Renderer
getClientRects and canvas measureText return the sub-pixel width of rendered text. At a fractional font size that width comes out different on FreeType than on DirectWrite or CoreText, because the three scalers round the glyph advance on different grids. One measurement at font-size:24.5555px separates a Linux text stack from a Windows or macOS one, and a custom Chromium build on Linux carries the Linux value no matter what OS it claims.
Your Browser's Hyphenation Dictionary Names the OS It Runs On
CSS hyphens:auto uses a per-language dictionary, and macOS and Windows/Linux Chrome ship disjoint dictionary sets from two different engines. Which languages a browser will hyphenate, and where it breaks them, classifies the OS with one DOM read. Why closing it for a macOS profile means reverse-engineering Apple's CFBurstTrie format and reconstructing its hyphenation bit-for-bit.
Your Browser Does Math Differently on Every OS, and Anti-Bot Systems Read the Bits
Math.tanh, every CSS trig function, and the Web Audio compressor all route through the host libm, so the rounding of a cosine betrays the OS a browser actually runs on. Where the leak lives across V8, Blink, and Web Audio, and what bit-for-bit reproduction of Apple's math library takes to close it.
The Math Behind the Audio Fingerprint
The audio fingerprint stopped separating browsers and now separates CPUs: x86 and Apple Silicon build the oscillator wavetable with different FFTs. Why per-session noise cannot hide a 1-to-22-ULP gap, and what reproducing Apple's vDSP FFT on x86 down to a double-precision base case actually involves.
WebAssembly Runs Identically Everywhere, Except Where It Leaks Your CPU
WebAssembly is deterministic except for NaN bit patterns and relaxed SIMD, and both leak whether the CPU underneath is ARM or x86. How a browser claiming Apple Silicon on an x86 server gets caught by a 30-byte module, and how to emit ARM's bits across every V8 compiler tier.
CDP over CBOR, msgpack and Snappy: three formats, one bug
How a single CDP message travels through three different serialization formats between Scrapium (our stealth-patched Chromium build) and our scraping infrastructure, what each format is doing for us, the UTF-8 vs UTF-16 trap that lives at the CBOR layer, and a production case our monitoring caught that taught us our two implementations did not agree on what a 'string' was.