tpt-lexregion

Compiler-assisted lexical memory regions & zero-jitter allocation for Rust.

Deterministic memory management for HFT, real-time audio, game engines, and embedded systems. Open a lexical region, allocate into it through standard collections (Vec, HashMap, Box, …) via the nightly Allocator API, and get the whole pool back in O(1) when the block ends:

• No free-lists. No fragmentation. One contiguous slab per region; individual deallocations are no-ops by design.
• Zero allocator jitter at steady state. Heap slabs are recycled through a thread-local cache (std builds), so a loop that opens a region per frame stops touching the global allocator after warm-up.
• Escape-proof by construction. The region's brand lifetime 'r is an invariant, higher-ranked parameter: rustc itself rejects any attempt to smuggle references, handles, or whole collections out of the block.
• Multi-threaded variant. SharedRegion uses a lock-free CAS bump cursor; its handles are Send + Sync.
• no_std friendly. Caller-supplied-buffer entry points work without any heap at all.

License: dual MIT OR Apache-2.0 — © TPT Solutions

Status

Crate	Version	Notes
tpt-lexregion	0.1.0	core crate (public API)
tpt-lexregion-macros	0.1.0	region! macro + best-effort #[region_attr] lint
tpt-lexregion-bench	internal	criterion suites, not published

Requires nightly Rust (allocator_api, plus btreemap_alloc / closure_lifetime_binder in dependent crates where used). Pinned via rust-toolchain.toml.

The spec's example, working today

#![feature(allocator_api)]
use std::collections::HashMap;
use tpt_lexregion::prelude::*;

fn process_massive_dataset() -> u64 {
    // Open a lexical region; everything allocated inside is bound to it.
    region!(capacity: 64 * 1024 * 1024, {
        // Standard collections allocate directly into the region.
        let mut map: HashMap<u64, Vec<u8>, std::collections::hash_map::RandomState, _> =
            HashMap::new_in(region.allocator());

        for i in 0..100_000u64 {
            map.insert(i, vec![0u8; 64]); // zero global-allocator jitter
        }

        map.values().map(|v| v[0] as u64).sum::<u64>()
        // <-- Region drops here: the pool is freed in O(1). Zero fragmentation.
    })
}

(Deviations from the design doc are recorded in docs/adr-001-escape-analysis-spike.md: the macro takes the body as a trailing block argument, because region!(...) { .. } is not valid macro-invocation grammar.)

Entry points

Function / macro	Pool backing	Threads
with_region / region!	heap slab (thread-cached under std)	single
with_region_in_buffer	caller-supplied buffer	single
with_shared_region	heap slab, lock-free cursor	many (Send + Sync)
with_shared_region_in_buffer	caller-supplied buffer	many
with_growable_region	chained slabs, grows on rollover (capped)	single

Domain-shaped runnable examples live in tpt-lexregion/examples/: hft_tick_processing.rs, audio_callback_arena.rs, game_frame_allocator.rs, embedded_sensor_ring.rs (plus the basic/collections/threads/embedded introductions).

Comparison with other arena crates

Verified against current docs.rs pages (Aug 2026). A note on names first: the crates.io crate literally named region is an OS virtual-memory API (mprotect/mlock/VirtualQuery wrappers) — it is not an arena allocator and is unrelated despite the name collision.

	tpt-lexregion	bumpalo	typed-arena	generativity / ghost-cell
What it is	Lexical scope-owned regions + nightly Allocator handles	General-purpose growable bump arena	Single-type arena	Branding / proof-token toolkits (no allocation)
Escape-safety mechanism	Invariant brand 'r created by an HRTB closure: rustc rejects every escape, including whole collections	Ordinary borrows: values can't outlive &Bump, but the arena itself lives wherever you put it	Same-lifetime borrows; into_vec can move contents out wholesale	make_guard! invariant brands (fn(&'a()) -> &'a () — the same idiom we build on) / proven-safe GhostToken interior mutability
Pool growth	Fixed pools fail cleanly (Err(AllocError)); opt-in GrowableRegion chains slabs under a mandatory hard cap	Grows by chaining fresh chunks automatically; try_* variants available	Grows through its backing store	n/a
Failure policy	Raw paths never panic; std collections still follow their own OOM convention	Global-allocator semantics on growth; fallible variants exist	Global-allocator semantics	n/a
Per-value destructors	Opt-in & explicit: alloc_with_drop / RegionCell (LIFO, panic-safe)	Not run in bulk; boxed::Box<T> wrapper drops T individually	Not run in bulk; into_vec() recovers ownership instead	n/a
Threading	SharedRegion: lock-free CAS cursor, Send + Sync handles	!Sync; bumpalo-herd pools arenas for threads	Not a focus	n/a
no_std	Yes (buffer-backed APIs need no heap at all)	Yes by default	Std-oriented	Yes
Toolchain	Nightly (allocator_api) required	Stable (MSRV 1.71.1); nightly/stable allocator-API adapters optional	Stable	Stable

Honest positioning: if you want a battle-tested general arena on stable Rust, bumpalo is the incumbent and you should probably use it. This crate is for the narrower niche where the lexical scope is the lifetime: the compiler-enforced block boundary (nothing escapes, including collections), fixed-capacity determinism with clean failure paths, thread-local slab reuse for zero steady-state jitter, buffer-backed no_std operation, and opt-in destructors that cannot be forgotten.

Getting started in 5 minutes

Generate a ready-to-build project from the bundled starter template:

cargo install cargo-generate
cargo generate --path template --name my-realtime-app
cd my-realtime-app && cargo run

The generated project pins nightly via its own rust-toolchain.toml, depends on this crate, and contains one runnable region-per-frame example. (Answering "yes" to the use_local_path prompt links a sibling checkout of this repository instead — handy for hacking on the crate itself. CI runs this whole flow as the template-generate job.)

Documentation

• Safety model — what the compiler enforces vs. what is best-effort.
• Compiler-errors cookbook — every escape-attempt diagnostic translated into plain English, with fixes.
• Growable regions — chained slabs, the never-span invariant, and the honest jitter cost of growth.
• ADR-002: nested regions — why children carve sub-slabs instead of rewinding the cursor, and why shared variants are out of scope.
• ADR-001: escape-analysis spike — why AST-level escape analysis was rejected as load-bearing, and how the §3.1/§4 macro-form inconsistency was resolved.
• Concurrency design — lock-free reclamation strategy and Send/Sync bounds; loom model-checking setup.
• Embedded usage — buffer-backed regions on no_std.
• Benchmarks & methodology — HFT burst, audio jitter (p50/p99/p99.9/max/stddev), bulk dataset; how to reproduce.

Development

cargo test                      # unit + integration + doctests
cargo test --release -p tpt-lexregion --test datasets   # 1M-insert run
RUSTFLAGS="" cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check
cargo build -p tpt-lexregion --no-default-features \
    --target thumbv7em-none-eabihf                       # embedded check
cargo test -p tpt-lexregion --features loom --release --test loom_model
cargo +nightly miri test -p tpt-lexregion --lib --tests
cargo bench                     # criterion suites (see docs/benchmarks.md)

CI runs all of the above on every push (see .github/workflows/ci.yml).