Pgo on Bernát Gábor — Python packaging, tox, virtualenv & open source

Pgo on Bernát Gábor — Python packaging, tox, virtualenv & open sourcehttps://bernat.tech/tags/pgo/Engineering notes from Bernát Gábor — PyPA maintainer of tox, virtualenv, pipx, filelock, and platformdirs. Python packaging, type hints, and supply chain security.Hugo 0.163.3en-USThu, 18 Jun 2026 09:00:00 +0000Building a fast HTML toolkit in C for Pythonhttps://bernat.tech/posts/blazing-fast-html-parser/Thu, 18 Jun 2026 09:00:00 +0000Bernát Gáborhttps://bernat.tech/posts/blazing-fast-html-parser/<div class="alert alert-info"> TLDR: turbohtml does HTML escape, unescape, tokenize, query, serialize, and URL work in C, 3-22x faster than Python&#39;s standard library. The recurring trick is skipping work: <ul> <li>Scan in blocks, not characters. <a href="#the-swar-trick-checking-eight-bytes-with-one-subtraction">SWAR</a> clears eight bytes with a subtraction, <a href="#sixteen-bytes-with-one-shuffle">SIMD</a> sixteen with one shuffle; a clean block costs almost nothing.</li> <li>Measure, then write. <a href="#two-passes-measure-then-write">One pass sizes the output exactly</a> , so the second allocates once and bulk-copies the clean stretches.</li> <li>Keep text at its native width and copy it rarely. The tokenizer stamps its state machine <a href="#stamping-the-machine-once-per-width">once per width</a> and hands back <a href="#never-copy-text-you-dont-have-to">zero-copy slices</a> into the input.</li> <li>The same instincts across the toolkit. Tag names <a href="#interning-names-to-integers">interned to integers</a> , an <a href="#building-the-index-once-instead-of-every-time">id index built once</a> that turns an O(N²) path walk linear, and <a href="#recycling-the-wrapper-objects">wrapper objects recycled</a> on a free list.</li> <li>When the work is a standard, not a scan. Host encoding needs <a href="#when-the-work-is-a-standard-not-a-scan">Punycode, normalization, and Hangul by arithmetic</a> , with the Unicode tables generated at build time.</li> <li>Down to the build and the benchmark. <a href="#teaching-the-compiler-what-is-hot">LTO and PGO</a> squeeze the machine code; the CI gate <a href="#measuring-without-lying-to-yourself">counts instructions under Callgrind</a> so a regression cannot hide in the noise.</li> <li>Free-threaded. No shared mutable state, so it declares <code>Py_MOD_GIL_NOT_USED</code> and runs on the <a href="https://peps.python.org/pep-0703/" target="_blank" rel="noopener noreferrer">no-GIL build</a> without forcing the lock back on.</li> <li>Safe under hostile input. It <a href="#when-the-input-is-trying-to-hurt-you">caps nesting, dedups attributes in linear time, and overflow-checks every buffer</a> , and the sanitizer is validated against DOMPurify&rsquo;s own XSS corpus.</li> </ul> </div> turbohtml was built with Claude (Opus 4.8), not by hand, over a month and close to 300 iterations. I review the code and own its correctness; <a href="#how-this-was-built">more on how, and my thanks, at the end</a> .pythoncperformancesimdhtmlparsertokenizerunicodeturbohtmlidnapgoltobenchmarking