Hedronite · Dev Lesson · Polyglot-Dev / Rust · Fri 2026-09-25 · v2 (Rust)

Rust chunk_by group runs over sorted keys called from Python via PyO3 and Maturin

Sort by the key, split where the key changes, hand Python owned lists. The grouping lives in Rust; Python only calls it.

Lesson Class: Dev (Rust depth · PyO3 bridge)
Language Idiom: stable sort_by + slice::chunk_by · #[pyfunction] / #[pymodule]
Paired Ops: Rust cargo script StackSet instance status census
Paired Cert: AWS DOP StackSets permission models / operation preferences
Grounding: TRPL Ch.13 iterators · PyO3 user guide · Maturin user guide
Runs
chunk_by splits consecutive equal keys only.
Owned
collect returns owned Vec groups; no shared-iterator trap.
Built
maturin develop --uv + python call.py, real output.
Rust owns the rule. Python owns the call site.

<!-- hal:authoritative:yaml -->

Sort by the key, split where the key changes, hand Python owned lists. The grouping lives in Rust; Python only calls it.

§I — Frame

This lesson replaces the Python v1, which taught itertools.groupby. The contract carries over unchanged: grouping works on consecutive runs, so the input must be ordered by the grouping key first. What moves is where the logic lives. The grouping primitive becomes a small Rust crate. PyO3 exposes it as a Python function, and Maturin builds and installs it into a virtual environment. Python stays as the calling shell for a notebook or pipeline that already speaks Python.

The rows are the Ops census rows: a (stack set, status) key and an account/region member.

§II — Language idiom: stable sort, then chunk_by

Two std facts define the tool.

  1. **slice::sort_by is stable.** Equal keys keep their input order, so members inside a group stay in the order the census produced them.
  2. **slice::chunk_by splits on a predicate over neighbours.** It yields sub-slices where every adjacent pair satisfies the closure. With |a, b| a.0 == b.0, each sub-slice is one run of equal keys. Like Python's groupby, it never looks back: unsorted input gives one run per change.

TRPL chapter 13 frames the rest: map and collect are iterator adaptors, and collect consumes the iterator into an owned Vec. Each group comes back owned, so the Python shared-iterator trap from v1 cannot occur. A group stays valid after the next one is produced.

On the boundary, the PyO3 type-mapping table does the work. A Python list of tuples extracts into Vec<((String, String), String)>, and the returned Vec<(Key, Vec<String>)> converts back to a list of tuples holding lists. Extraction copies, so the Rust sort happens on Rust-owned data and never mutates the caller's list.

§III — Worked example against the census row

Cargo.toml:

[package]
name = "sorted_groups"
version = "0.1.0"
edition = "2021"

[lib]
name = "sorted_groups"
crate-type = ["cdylib"]

[dependencies]
pyo3 = "0.27"

pyproject.toml (Maturin reads [tool.maturin] and turns on pyo3/extension-module):

[build-system]
requires = ["maturin>=1.9,<2.0"]
build-backend = "maturin"

[project]
name = "sorted_groups"
version = "0.1.0"
requires-python = ">=3.10"

[tool.maturin]
features = ["pyo3/extension-module"]

src/lib.rs:

use pyo3::prelude::*;

type Key = (String, String);

pub fn group_runs_by<K: Ord + Clone, V: Clone>(rows: &mut [(K, V)], sort: bool) -> Vec<(K, Vec<V>)> {
    if sort {
        rows.sort_by(|a, b| a.0.cmp(&b.0));
    }
    rows.chunk_by(|a, b| a.0 == b.0)
        .map(|run| (run[0].0.clone(), run.iter().map(|(_, v)| v.clone()).collect()))
        .collect()
}

#[pyfunction]
#[pyo3(signature = (rows, sort = true))]
fn group_runs(mut rows: Vec<(Key, String)>, sort: bool) -> Vec<(Key, Vec<String>)> {
    group_runs_by(&mut rows, sort)
}

#[pymodule]
fn sorted_groups(m: &Bound<'_, PyModule>) -> PyResult<()> {
    m.add_function(wrap_pyfunction!(group_runs, m)?)?;
    Ok(())
}

call.py:

from sorted_groups import group_runs

rows = [
    (("baseline-iam", "CURRENT"), "111111111111/us-east-1"),
    (("config-rules", "OUTDATED"), "222222222222/eu-west-1"),
    (("baseline-iam", "OUTDATED"), "222222222222/us-east-1"),
    (("config-rules", "CURRENT"), "111111111111/us-east-1"),
    (("baseline-iam", "OUTDATED"), "333333333333/us-east-1"),
]

print([key for key, _ in group_runs(rows, sort=False)])

grouped = group_runs(rows)
for (stack_set, status), members in grouped:
    print(stack_set, status, len(members), members)

print(type(grouped).__name__, type(grouped[0]).__name__, type(grouped[0][1]).__name__)
print(rows[0])

Build and run, on the lab Mac, in a scratch venv under /tmp (removed afterwards):

uv venv .venv --python python3
. .venv/bin/activate
uvx --from 'maturin>=1.9,<2.0' maturin develop --uv
python call.py

Real output (CPython 3.12.13, maturin 1.15.0, pyo3 0.27.2):

[('baseline-iam', 'CURRENT'), ('config-rules', 'OUTDATED'), ('baseline-iam', 'OUTDATED'), ('config-rules', 'CURRENT'), ('baseline-iam', 'OUTDATED')]
baseline-iam CURRENT 1 ['111111111111/us-east-1']
baseline-iam OUTDATED 2 ['222222222222/us-east-1', '333333333333/us-east-1']
config-rules CURRENT 1 ['111111111111/us-east-1']
config-rules OUTDATED 1 ['222222222222/eu-west-1']
list tuple list
(('baseline-iam', 'CURRENT'), '111111111111/us-east-1')

Read the four print sites in order.

Print one is the unsorted trap. With sort=False, five rows give five runs, and baseline-iam appears three times. Same contract as itertools.groupby.

Print two is the idiom. Stable sort, then chunk_by: one line per (stack set, status), members in input order.

Print three is the boundary. list tuple list: PyO3 turned the Rust Vec of tuples of Vec into native Python types.

Print four is ownership. rows[0] is still the first input row. Extraction copied the list, so the Rust sort never touched the caller's data.

§IV — What not to do

Do not group unsorted rows and expect one group per key. Do not sort by one key and chunk by another unless the chunk key is a prefix of the sort key. Do not reach for sort_unstable_by here; it may reorder members within a group. Do not keep the grouping logic in Python and call Rust only for I/O; the crate owns the rule, Python owns the call site. Do not ship the extension-module feature in a crate you also want to cargo test as a normal binary without checking the PyO3 guidance on linking.

§V — Close instruction

Build the crate with maturin develop --uv in a throwaway venv. Feed it the Ops census rows with sort=False, count the repeated keys, then rerun with the default and confirm one group per (stack set, status). Pair: Ops streams these rows from aws-sdk-cloudformation in a cargo script; Cert opens StackSets permission models and operation preferences for DOP.

Related