Bernát Gábor — Python packaging, tox, virtualenv & open source

How turbohtml makes Python HTML work 3-22x faster

Bernát Gábor — Thu, 18 Jun 2026 09:00:00 +0000

Python’s html.escape can scan one string five times. Moving those loops into C removes the interpreter from the hot path, which explains part of a 22x result on prose. Most prose contains nothing to escape. turbohtml gets the rest of the speedup by proving that sixteen bytes need no work, then moving past them in one step.

I began with a small accelerator for html.escape and html.unescape . The Python implementation of escape runs up to five str.replace passes. unescape runs a regular expression and calls back into Python for each match in the 2,231-entry HTML5 entity table. html.parser.HTMLParser calls unescape for each text run it encounters, so both functions sit on hot paths. Interpreter dispatch explains the standard-library gap; its authors made reasonable choices under different maintenance constraints.

Deterministic Multithreaded Testing in Python with blanket

Bernát Gábor — Thu, 28 May 2026 17:57:48 +0000

TLDR:

The problem : testing multithreaded code is hard because the OS scheduler decides which thread runs when, making race conditions all but impossible to reproduce in a test suite.
The solution : blanket wraps real threading primitives (Lock, Condition, Event, Barrier, Semaphore) and lets your test act as the scheduler, controlling which thread proceeds at each step.
Why now : free-threaded Python (no GIL ) shipped as an experimental build in 3.13, is officially supported in 3.14 , and keeps maturing in 3.15 . The GIL was hiding thread-safety bugs you did not know you had; without it, they surface.
How it works : every method call on a blanket primitive becomes a transaction that parks at a scheduler block. Your test unblocks transactions in whatever order you want, making execution 100% deterministic.
What makes it different : unlike stateless model checkers (Loom , Shuttle , CHESS ) that discover bugs by exploring interleavings, blanket lets you declare specific scenarios by hand, useful for regression tests of known bugs and for full coverage of rare code paths.

Most multithreaded Python codebases keep at least one test marked @pytest.mark.flaky(reruns=5): the one that fails once in a thousand because of a race condition you cannot reproduce on demand. The bug is a specific sequence of thread interactions. You do not get to pick the sequence; the OS scheduler does. You ship the retry and hope.

PyCon US 2026 Packaging Summit Recap

Bernát Gábor — Fri, 15 May 2026 23:00:00 +0000

The PyCon US 2026 Packaging Summit ran Friday May 15, 2026, from 1:45 PM to 5:45 PM in Room 201A of the Long Beach Convention Center. Three talks, nine lightning talks, six roundtable discussions. Organized by Pradyun Gedam , C.A.M. Gerlach , and Jannis Leidel . This recap is for anyone who could not be in the room.

TLDR:

Emma Smith’s revised PEP 777 (Wheel 2.0) is a minimal change with sub-PEPs layered on top; the first sub-PEP proposes Zstandard compression for ~25% smaller wheels.
Mike Fiedler brought three PyPI abuse vectors (persistent state, open-ended releases, PyPI as a CDN), framed by 3× growth against 1× resources (~24,000 new packages per month, up from ~8,000) and a single full-time PyPI safety and security engineer.
Mahe Iram Khan argued conda and pip are two parallel ecosystems, not two competing tools.
Lightning talks: PEP 772 Packaging Council approved (Barry Warsaw), mobile wheel pipeline live (Malcolm Smith), Coherent + Compile-to-Flit (Jason Coombs), wheel variants for AI accelerators and shared malware scanning (Joongi Kim), conda-pypi (Daniel Holth), Nebi (Dharhas Pothina).
Roundtables: PEP 803 abi3t , Wheel 2.0 hardening, cross-platform builds, CVE propagation, and nab (Damian Shaw’s pure-Python resolver aimed at pip swap-in).
Involved in Python packaging? Consider running for the Packaging Council this fall or becoming a voting PSF member to nominate and vote.

For background, I list the packaging-topic PEPs resolved in the year leading up to the summit (January 2025 through May 2026), newest first.

PyCon US 2026 Typing Summit Recap

Bernát Gábor — Thu, 14 May 2026 23:30:00 +0000

The PyCon US 2026 Typing Summit ran Thursday May 14, 2026, from 1 PM to 5 PM in Room 201A of the Long Beach Convention Center, the day before the main conference started. Eight talks plus a Typing Council Q&A, single track. This recap is for anyone who could not be in the room.

TLDR:

Guido van Rossum argued that PEP 484 ’s no-new-syntax rule is already broken in practice and that the field should weigh user pain over power features, citing the 2025 Python Typing Survey .
Jelle Zijlstra proposed adding intersection and restricted-negation types to the typing spec, with an inhabitation check as the load-bearing new rule.
Michael Sullivan presented PEP 827 (Vercel) for type manipulation, modelled on TypeScript’s conditional and mapped types.
Douglas Creager showed how ty represents generic-call constraints internally with ternary decision diagrams, and a third solver strategy that fixes a 9-line partial(choose, None) example every production checker today gets wrong.
Conner Nilsen presented a Pyrefly experiment with AI coding agents: type checking moves success on well-typed Meta code from 79.6% to 83.9% with 21% fewer steps; no measurable help on lightly-typed SWE-bench Verified.
Avik Chaudhuri demoed tensor-shape types in Pyrefly, blocked in practice by PEP 695 ’s eager evaluation of type parameters.
Jia Chen presented a Lean 4 formalization (Featherweight Python) with mechanized soundness and decidability proofs; AI assistants turned what used to take years into weeks.
The Typing Council panel (Carl Meyer, Jelle Zijlstra, Rebecca Chen on stage) opened the floor to attendee questions on governance, error-code consistency, metaprogramming, and the spec direction.

Experiments with AI agents and Pyrefly type errors — Conner Nilsen Link to heading

Conner (Meta, Pyrefly team) presented two questions: (1) does giving an AI coding agent a type checker help it finish tasks, and (2) does it prevent the agent from re-introducing old bugs while fixing new ones? His team ran two benchmarks with and without type-checker feedback and tracked three metrics: success rate, number of steps to completion, and wall-clock duration.

PyTexas 2026 Recap

Bernát Gábor — Sun, 19 Apr 2026 22:00:00 +0000

PyTexas is the annual Python conference held in Austin, Texas. The 2026 edition ran April 17–19 at the Austin Central Library in downtown Austin.

TLDR:

PyTexas 2026 ran April 17–19 in Austin. Friday was tutorials, Saturday and Sunday were talks with two keynotes and two lightning-talk blocks. A few themes kept coming back across unrelated talks:

Design deliberately. A thread across both keynotes. Hynek Schlawack: the domain model is “the precious” - design it first, translate at the edges. Dawn Wages: ownership over your stack as one of her three pillars for model and career specialization.
Agents should write code, not decide what to write. Peter Sobot’s Seven Stages of AI Grief ended on that line. Al Sweigart argued “agentic engineering” is vibe coding with better marketing, and that almost-right is worse than wrong. Maria Silvia Mielniczuk’s MCP talk built the same idea into an architecture: models suggest, only the server executes. Adam Gordon Bell’s running coach split deterministic work (plain Python) from interpretation (LLM). The Sunday opener framed it: when AI ships a bad PR, fix the process, not the model.
Code quality is an input to AI productivity. Miguel Vargas’s framing: AI agents produce cleaner, safer code in codebases that are already clean, safe, and typed, so Ruff, ty, and uv matter more now.
The supply chain is still the attack surface. Christopher Ariza on why pip install still runs arbitrary code, with .pth files, sitecustomize.py, and setup.py as the specific places to pay attention.
CPython itself is getting faster. Jacob Coffee on PEP 810 lazy imports for startup wins; Charlie Lin on the free-threaded build and what it takes to make an extension module safe under it.

This post details the takeaways from each talk I attended, in schedule order.

Defense in Depth: A Practical Guide to Python Supply Chain Security

Bernát Gábor — Tue, 10 Mar 2026 00:00:00 +0000

TLDR:

Layer your defenses and do not trust any single control. Use Ruff with security rules to catch bugs in your code before they ship. Pin all your dependencies with cryptographic hashes using uv lock or uv pip compile --generate-hashes so nobody can swap out packages on you. Run pip-audit in CI to catch known CVEs before they hit production. Generate SBOMs with CycloneDX so when the next Ultralytics-style compromise drops, you can answer “are we affected?” in minutes instead of days.

If you are publishing packages, ditch the long-lived API tokens and switch to Trusted Publishing with OIDC. This generates attestations automatically via Sigstore, linking your packages back to source repos. Organizations running internal mirrors can add a 7-day delay to let the community be your canary - but only if you have the infrastructure to maintain it.

Nothing here is perfect. Hash pinning stops tampering but will not save you from a malicious package you installed on day one. Scanning finds known CVEs but misses zero-days. Attestations prove where code came from, not whether it is safe. That is why you layer them - when one control fails, the others catch it. Start with linting and pinning for quick wins, add scanning and SBOMs next, then level up to advanced stuff as you mature.

Version numbers: how to use them?

Bernát Gábor — Sat, 16 May 2020 14:15:00 +0000

The DRY principle (an acronym for don’t repeat yourself) encourages software engineers to abstract code into a separate component and reuse it rather than write it over and over again. If this happens across the system, the best practice is to put it inside a package that lives on its own (a library) and then pull it in from the applications when required.

As most of us can’t think of every feature that the library might offer or what bugs it might contain, these packages tend to evolve. Therefore, we need some mechanism to encode these evolutions of the library, and most commonly, this is a version number.

Python packaging - Growing Pains

Bernát Gábor — Thu, 07 Feb 2019 13:41:03 +0000

In my previous two posts, I’ve gone over what package types python has , and how the package building works , especially with the introduction of the PEP-517/518. Although the changes were meant to make things more robust, we did run into a few issues while implementing and releasing them. The following are a few of those, serving as lessons learned for all of us and presenting some interesting problems to solve in the future.

Python packaging - Past, Present, Future

Bernát Gábor — Thu, 07 Feb 2019 13:40:59 +0000

Have you ever wondered what happens when you run pip install? This post gives a detailed overview of the steps involved in the past, and how it all changes with the adoption of PEP-517 and PEP-518.

In my previous post I’ve described how it’s possible to install three types of content: source tree, source distribution, and wheels. Only the last two types are uploaded to PyPI, the central Python repository. However, one could get its hands on a source tree (by feeding, for example, a git protocol for pip). The advantage of wheels over the others is that it does not require any build operation to happen on the user machine; it’s a download and extract.

The state of Python Packaging

Bernát Gábor — Thu, 07 Feb 2019 13:40:54 +0000

pip 19.0 has been released on 22nd January 2019. On the feature list, most notably, it now supports PEP-517, which by default is turned on when that the project has a pyproject.toml at the root folder. The PEP in question has been created in 2015 and accepted in 2017. Even though it took a while until pip implemented it, the release and the issues that followed confirmed that many people are not familiar with it. Read on if you want a picture of how the Python packaging ecosystem evolved today and where we hope to see it down the line. The introduction of this python enhancement proposal may cause some discomfort, but we will benefit from it in the long term.

The state of type hints in Python

Bernát Gábor — Wed, 30 May 2018 09:51:55 +0000

One of the main selling points for Python is that it is dynamically-typed. There is no plan to change this. Nevertheless, in September 2014 Guido van Rossum (Python BDFL ) created a python enhancement proposal (PEP-484 ) to add type hints to Python. It has been released for general usage a year later, in September 2015, as part of Python 3.5.0. Twenty-five years into its existence now there was a standard way to add type information to Python code. In this blog post, I’ll explore how the system matured, how you can use it, and what’s next for type hints.