Hedronite · Backend Dev · Rust · Fri 2026-10-23

One fast import a Rust hot path behind a Python module with PyO3 and maturin

Keep the core in plain Rust, release the interpreter lock around it, and ship one abi3 wheel.

Lesson Class: Dev (Rust backend · Python extension shell)
Tested with: pyo3 0.30.1 · maturin 1.15.0 · rustc 1.99.0 · CPython 3.13.15
Lab: crate/ · pytest (7) · bench.py · bench2.py
Source: PyO3 0.30.1 user guide · maturin docs
Core, Shell, Module
Plain Rust core, a thin #[pyfunction] shell that converts and maps errors, a #[pymodule] that registers it.
Detach for Threads
Four threads took 0.742 s holding the lock and 0.210 s detached, against 0.188 s for one call.
The Copy Is the Cost
Converting two million floats took 11.1 ms; the dot product on stored data took 0.50 ms.
Convert once.

The problem

Your Python service spends most of its time in one numeric loop. The profiler says so, and the loop is simple: multiply pairs of numbers, add them up, repeat a million times per call. Rewriting the service is out of scope, and the rest of it is fine as Python.

Threads do not fix this. CPython runs Python bytecode in only one thread at a time, because of a lock inside the interpreter called the GIL (global interpreter lock). If you start four threads that each run your pure-Python loop, they take turns. CPU-bound work gets no extra cores.

What you want is a single import, import hotpath, whose function is faster than the Python loop. While that function runs, the other Python threads should not be locked out. This lesson builds that with Rust, PyO3 0.30.1 (the Rust library that generates the Python bindings) and maturin 1.15.0 (the build tool that produces an installable wheel). The lab ran on an Apple Silicon Mac with rustc 1.99.0, Python 3.13.15 and uv 0.12.13.

The idea, in plain words

Keep the Python program as the caller. Move one function into a Rust crate and write its logic as ordinary Rust that knows nothing about Python. Wrap that core in a thin shell: a function whose signature PyO3 reads to generate the Python-facing binding, converting arguments on the way in and errors on the way out. Where Rust work touches no Python objects, release the interpreter lock so other Python threads can run in the meantime. For state that should live on the Rust side, add one class. Then let maturin compile the crate and package it as a wheel that pip can install.

A: The shape of a PyO3 module

A PyO3 module is a compiled library that Python imports like any other module. It has four parts, and the crate in this lab has all four. The crate type is cdylib, a dynamic library Python can load:

[lib]
name = "hotpath"
crate-type = ["cdylib"]

[dependencies]
pyo3 = { version = "0.30", features = ["abi3-py39"] }

1. A pure-Rust core. It takes slices and returns a Result with a String error. No Python type appears in it:

fn dot_core(a: &[f64], b: &[f64]) -> Result<f64, String> {
    if a.len() != b.len() {
        return Err(format!("length mismatch: {} vs {}", a.len(), b.len()));
    }
    Ok(a.iter().zip(b).map(|(x, y)| x * y).sum())
}

**2. A thin #[pyfunction] shell.** The attribute tells PyO3 to expose the function to Python:

#[pyfunction]
fn dot(a: Vec<f64>, b: Vec<f64>) -> PyResult<f64> {
    dot_core(&a, &b).map_err(PyValueError::new_err)
}

**3. A #[pyclass] with #[pymethods]** for state that lives on the Rust side. RunningMean holds a count and a mean in a plain Rust struct. #[new] is the constructor, push updates the state, #[getter] methods expose count and value, and __repr__ controls how Python prints it.

**4. A #[pymodule]** function that lists everything the module exports:

#[pymodule]
fn hotpath(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(dot, m)?)?;
    m.add_function(wrap_pyfunction!(dot_detached, m)?)?;
    m.add_function(wrap_pyfunction!(spin, m)?)?;
    m.add_class::<RunningMean>()?;
    Ok(())
}

Keep the core free of Python types so that you can test it as plain Rust. Python types also have to be held under the interpreter lock, and the core must be free of them so that it can run without that lock (section B). Otherwise every bug hunt would start inside the binding layer.

Conversion is automatic, and it copies. The parameter type Vec<f64> is the whole conversion instruction. When Python passes a list, PyO3 builds a fresh Vec<f64> from it, element by element, on every call. Ints are accepted too: the test hotpath.dot([1, 2], [3, 4]) returns 11.0. Errors go the other way. The shell turns the core's String into a ValueError, so hotpath.dot([1.0, 2.0], [1.0, 2.0, 3.0]) raises ValueError with the message length mismatch: 2 vs 3.

The same mapping applies to the class. Reading value on a fresh RunningMean raises ValueError: mean of zero samples. After pushing 2.0, 4.0 and 9.0, the lab printed this line from __repr__:

repr: RunningMean(n=3, mean=5)

Rust's {} formatting prints the float 5.0 as 5, which is why the repr shows mean=5. The struct fields are not exposed as attributes. Only the getters are, and a test confirms that rm.n = 5 raises AttributeError.

Pitfall: a string in the list.

B: Releasing the interpreter lock

**py.detach** is the PyO3 call that releases the GIL while a closure runs. The guide says it plainly: "the Python::detach method can be used to temporarily release the GIL, thus allowing other Python threads to run." The payoff is in the same section: "Now Python threads can use more than one CPU core, resolving the limitation which usually makes multi-threading in Python only good for IO-bound tasks."

Detaching is only safe for code that touches no Python objects. That is why dot_detached converts its arguments first and hands only plain Rust data to the closure:

#[pyfunction]
fn dot_detached(py: Python<'_>, a: Vec<f64>, b: Vec<f64>) -> PyResult<f64> {
    py.detach(|| dot_core(&a, &b)).map_err(PyValueError::new_err)
}

The conversion happens before the detach and the error mapping after it, so the closure sees no Python types. The lab also has a spin function, a loop of integer arithmetic with a detach flag, built to isolate the lock from everything else:

#[pyfunction]
fn spin(py: Python<'_>, n: u64, detach: bool) -> u64 {
    let work = || { /* wrapping multiply-add loop, no Python objects */ };
    if detach { py.detach(work) } else { work() }
}

The test starts four Python threads that each call spin(200_000_000, detach), and measures wall-clock time for all four:

one call 0.188s; 4 threads holding lock 0.742s; 4 threads detached 0.210s

One call takes 0.188 s. Four threads that keep the lock take 0.742 s, about four times as long, which is what four calls in a row would cost. Four detached threads take 0.210 s, close to a single call. The test asserts only freed < held. The 0.022 s gap between 0.210 s and one call is measured, but the lab did not look into its cause.

**Why #[pyclass] must be Sync.** Detaching is one half of the threading story, and the class is the other. The guide says: "Python objects may be accessed by multiple Python threads simultaneously; therefore #[pyclass] objects must be Sync." A RunningMean holds a u64 and an f64, which satisfy that. The guide adds that runtime borrow checking is on by default and that a class can opt out by declaring itself frozen. RunningMean keeps the default, because push takes &mut self and needs that check. The Pair class in the next section is declared #[pyclass(frozen)]: it is built once and only read afterwards, so it has nothing to borrow mutably.

Pitfall: threads that take 4x as long.

C: Building and shipping with maturin

maturin is the build tool that compiles the crate and packages it for Python. maturin develop builds the crate and installs it into the current virtualenv. maturin build writes wheels into target/wheels. A wheel is the file format pip installs. Your pyproject.toml hands the build to maturin:

[build-system]
requires = ["maturin>=1.9,<2.0"]
build-backend = "maturin"

[project]
name = "hotpath"
version = "0.1.0"
requires-python = ">=3.9"

For the edit loop, run maturin develop --release inside a uv-created virtualenv. The --release flag matters because an unoptimised Rust build would make any timing meaningless. The lab's compile took 2.67 s and ended with these lines:

📦 Built wheel for abi3 Python ≥ 3.9 to /var/folders/.../hotpath-0.1.0-cp39-abi3-macosx_11_0_arm64.whl
✏️ Setting installed package as editable
🛠 Installed hotpath-0.1.0

abi3 is the choice that gave the cp39-abi3 in that file name. The abi3-py39 feature in Cargo.toml turns on a limited Python API. The guide describes the trade: "Package vendors can opt-in to the "abi3" limited Python API which allows their wheels to be used on multiple Python versions, reducing the number of wheels they need to compile, but restricts the functionality they can use." Without it, the default is: "By default, Python extension modules can only be used with the same Python version they were compiled against." With it, one wheel serves CPython 3.9 and every later version.

There is one limit to know before you promise anything: "The free-threaded build of CPython cannot load abi3 wheels but both builds can load abi3t wheels." If your deployment includes free-threaded interpreters, an abi3 wheel will not load there. The lab built only the abi3 wheel and did not try abi3t.

One more detail from the guide: "Historically PyO3 used an extension-module feature to perform the same function now done by the PYO3_BUILD_EXTENSION_MODULE env var." The lab ran both ways. The first build listed extension-module in Cargo.toml and passed all seven tests; the second dropped it, maturin recompiled pyo3-build-config, and the same seven tests passed in the same 1.21 s. On PyO3 0.30 with maturin, leave the feature out.

**Pitfall: cargo test will not link on macOS.**

Putting it together

Seven pytest tests exercise the whole crate from the Python side:

TestWhat it proved
test_dotdot([1,2,3], [4,5,6]) returns 32.0
test_dot_mismatch_is_valueerrorunequal lengths raise ValueError with length mismatch: 2 vs 3
test_dot_rejects_stra "a" element raises TypeError before Rust runs
test_ints_are_accepteddot([1, 2], [3, 4]) returns 11.0
test_running_meanempty value raises ValueError; after 2, 4, 9 the count is 3 and the value is 5.0
test_no_new_attributesassigning rm.n = 5 raises AttributeError
test_detach_lets_threads_overlapfour detached threads finish faster than four that hold the lock

How to run it

uv venv
source .venv/bin/activate
uv pip install maturin pytest
maturin develop --release
python -m pytest -q -s tests
python bench.py
python bench2.py

The lab's last pytest line was 7 passed in 1.21s. To produce a wheel you can ship:

maturin build --release

That wrote hotpath-0.1.0-cp39-abi3-macosx_11_0_arm64.whl into target/wheels. The cp39-abi3 part is the Python side: any CPython 3.9 or later. The macosx_11_0_arm64 part is the platform, which matches maturin's default MACOSX_DEPLOYMENT_TARGET=11.0 for aarch64-apple-darwin. abi3 saves you one wheel per Python version, not one per operating system or CPU.

What the numbers say

The benchmark takes the best of five runs on a million-element dot product:

python sum/zip   best of 5: 19.5 ms
hotpath.dot      best of 5: 11.9 ms
abs diff: 7.887138053774834e-09

That is a speedup of about 1.6x, not the order-of-magnitude jump people often expect. Every call to hotpath.dot copies two Python lists, a million elements each, into Vec<f64>, so the question is how the 11.9 ms divides between copying and arithmetic. The lab answered it with a second class that converts the lists once, in its constructor, and keeps them on the Rust side:

#[pyclass(frozen)]
struct Pair {
    a: Vec<f64>,
    b: Vec<f64>,
}

#[pymethods]
impl Pair {
    #[new]
    fn new(a: Vec<f64>, b: Vec<f64>) -> PyResult<Self> { /* same length check, then store */ }

    fn dot(&self) -> f64 {
        dot_core(&self.a, &self.b).unwrap()
    }
}

Timing the constructor and the method separately, on the same million-element lists:

hotpath.dot(a, b)      best of 5: 11.9 ms
hotpath.Pair(a, b)     best of 5: 11.1 ms
p.dot() stored data    best of 5: 0.50 ms
same result: True

A second run gave 12.3, 11.7 and 0.50 ms. Building the Pair, which is nothing but the conversion, costs about as much as the whole dot call. The multiply-and-add over data already in Rust takes half a millisecond, roughly 39 times faster than the 19.5 ms Python loop. The copy is where the time goes. If your Python code calls the hot function many times on the same data, convert once and keep the data on the Rust side; if every call brings fresh data in a Python list, expect a small speedup, because the conversion cannot be skipped.

The two results differ by 7.887e-9. The two sums disagree in the last digits because they add the products in different ways. The lab claims nothing more than that. For real data, compare with a tolerance instead of ==.

The threading result adds to that. A pure-Python loop gets nothing from four threads, while the detached function finished four calls in 0.210 s against 0.188 s for one.

Check yourself

  1. **What happens when you call hotpath.dot(["a"], [1.0]), and why does your Rust code never run?** PyO3 converts each element to f64 when it builds the Vec<f64> arguments, and "a" fails that conversion. Python raises TypeError: must be real number, not str before the function body starts.
  1. **Why does dot_detached convert its arguments before calling py.detach, and what may the closure touch?** The closure runs without the interpreter lock, so it may touch only plain Rust data and no Python objects. The Vec<f64> values are already converted, and dot_core has no Python types in it.
  1. **Four threads calling spin without detaching took 0.742 s, and one call took 0.188 s. Explain the number and give the fix.** Each thread keeps the lock for its entire run, so the four calls run one after another (about 4 × 0.188 s). Wrap the pure-Rust work in py.detach; the detached run took 0.210 s.
  1. **What does the abi3-py39 feature buy you, and what can it not do?** One cp39-abi3 wheel works across CPython 3.9 and later, so you build fewer wheels, at the cost of a restricted API. The free-threaded build of CPython cannot load abi3 wheels.
  1. **The Rust dot was 11.9 ms against 19.5 ms for Python. How much of the 11.9 ms is copying, and how do you know?** Most of it. Converting the two lists alone (building a Pair) took 11.1 ms, and the dot product over the stored vectors took 0.50 ms. Keep the data on the Rust side when you can reuse it.

Sources