tpt-local-ai

The missing "plumbing" for Rust developers building local, privacy-first AI applications. Python has huggingface_hub; Rust needs the TPT equivalent.

tpt-local-ai is a Cargo workspace of five small, dependency-conscious crates that cover the unglamorous but essential steps of running models locally:

Crate	Purpose	Highlights
tpt-hf-hub	Async Hugging Face model downloader & cache manager	Resumable Range downloads, SHA256 verification, atomic writes, pluggable progress UI
tpt-jinja-chat	Pure-Rust Jinja2 subset for chat templates	Zero dependencies, hand-rolled parser, for/if/set/filters/expressions
tpt-safetensors-io	Memory-mapped safetensors reader/writer	Zero-copy memmap2, streaming builder with correct header alignment
tpt-tokenizer-core	Pure-Rust BPE + WordPiece tokenizer	no_std + alloc friendly, byte-level BPE, no C++ bindings
tpt-lora-merge	CPU-based LoRA weight merging	ndarray math, multi-adapter weighted sum, ships a CLI

Quickstart: the whole pipeline

The cookbook example chains all five crates end-to-end — resolve a Hub cache (and optionally download real files), write a base checkpoint plus a LoRA adapter, merge them, render a chat template, and tokenize the result:

# Runs fully offline by default:
cargo run -p tpt-cookbook

# Exercise a real Hub download too:
TPT_COOKBOOK_HUB_REPO=gpt2 TPT_COOKBOOK_HUB_FILE=config.json \
    cargo run -p tpt-cookbook

It prints each stage of the pipeline:

[1/5] tpt-hf-hub            -> cache dir + optional download
[2/5] tpt-safetensors-io    -> write base + adapter checkpoints
[3/5] tpt-lora-merge        -> base + scale * (B @ A)
[4/5] tpt-jinja-chat        -> render chat template into a prompt
[5/5] tpt-tokenizer-core    -> tokenize (and round-trip) the prompt

See crates/cookbook/src/main.rs for the fully commented source. Each crate also ships its own focused examples/.

Browser playground (WASM)

Because tpt-jinja-chat and tpt-tokenizer-core are pure Rust with zero/minimal dependencies, they compile straight to WebAssembly. The crates/wasm-demo crate is a small in-browser playground for rendering chat templates and tokenizing text entirely client-side — a "zero dependency" proof point. Build it with wasm-pack build crates/wasm-demo --target web (see its README).

How it compares

These crates deliberately trade breadth for a small, auditable, pure-/minimal-Rust footprint. If you need the full feature set of the mainstream libraries, use them; if you want lean plumbing you can read end-to-end, use these.

Need	Mainstream option	tpt-local-ai differentiator
Hub downloads	hf-hub	rustls-only (no OpenSSL), pluggable progress trait, explicit offline mode, path-traversal hardening, retry/backoff, concurrency knob
Chat templates	minijinja	zero dependencies, hand-rolled parser scoped to the Jinja subset real chat templates use (tojson, raise_exception, namespace(), ...), with a fuzz target
Tokenizers	tokenizers (HF)	100% pure Rust, no_std + alloc, no C++/ONNX build step, byte-level BPE + WordPiece
safetensors I/O	safetensors	mmap-backed zero-copy reads, streaming builder, adversarial-header hardening + fuzz target, overflow-checks on offset math
LoRA merging	(mostly Python / PEFT)	CPU-only Rust library and CLI, multi-adapter weighted sum, adapter_config.json auto-scale, dtype preservation, dry-run

MSRV

Rust 1.80.0. Enforced in CI.

Contributing

Code is developed and maintained by the project maintainer, so pull requests with code changes aren't required — but issues are very welcome. Please report bugs, request features, and ask questions at the issue tracker. See CONTRIBUTING.md for details and local dev setup.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.