tpt-ai-agents
RustRust workspace for building LLM-powered agents: unified streaming clients for OpenAI/Anthropic/Ollama, RAG pipeline (chunking, embedding, context management), vector store traits (Qdrant/Milvus/pgvector), tool-use macros, prompt templating, tokenization, agent memory, eval harness, and a mock server.
Languages
tpt-ai-agents
Workspace of independent, composable Rust crates for LLM/agent infrastructure.
See GETTING_STARTED.md for a quickstart and docs/ARCHITECTURE.md for how the crates compose.
Crates
| Crate | Version | Maturity | Description | Tier |
|---|---|---|---|---|
tpt-llm-client-core |  | Production | Unified streaming HTTP/SSE client for OpenAI, Anthropic, Ollama with automatic retry/backoff | 0 |
tpt-vector-store-traits |  | Beta | Standardized async traits for Qdrant, Milvus, pgvector | 0 |
tpt-vector-store-qdrant |  | Beta | Real VectorStore adapter backed by Qdrant | 1 |
tpt-rag-pipeline |  | Beta | Chunking, embedding batching, context-window management | 1 |
tpt-tool-use-macros |  | Production | Proc macros exposing Rust functions as LLM tools | 0 |
tpt-prompt-template |  | Production | Safe, typed templating for prompts | 0 |
tpt-onnx-runtime-utils |  | Beta | High-level wrappers for local ONNX inference | 0 |
tpt-tokenizers-fast |  | Production | Lightweight, no_std-compatible BPE/SentencePiece tokenization | 0 |
tpt-agent-memory |  | Production | In-memory and file-persistent graph memory with keyword and embedding-based semantic search for multi-turn state | 1 |
tpt-eval-harness |  | Beta | Tools for running LLM eval datasets deterministically | 1 |
tpt-ai-mock-server |  | Production | Mock server for testing LLM integrations | 0 |
tpt-vector-store-pgvector |  | Beta | Real VectorStore adapter backed by pgvector (PostgreSQL) | 1 |
tpt-embeddings-client |  | Beta | Unified async client for OpenAI, Cohere, and custom embedding APIs | 0 |
tpt-agent-graph |  | Beta | Thin state-machine/DAG orchestration for LLM agent turns | 0 |
Maturity levels:
- Production — full implementation, comprehensive tests, real-world usage expected
- Beta — functional implementation, may have API adjustments before 1.0
Publish Order
Publish by tier, waiting for crates.io propagation between tiers:
Tier 0 (no internal deps): tpt-tokenizers-fast, tpt-llm-client-core, tpt-vector-store-traits, tpt-prompt-template, tpt-tool-use-macros, tpt-onnx-runtime-utils, tpt-ai-mock-server, tpt-embeddings-client, tpt-agent-graph
Tier 1 (depend on Tier 0): tpt-rag-pipeline, tpt-agent-memory, tpt-eval-harness, tpt-vector-store-qdrant, tpt-vector-store-pgvector
Verified via cargo publish --dry-run for every crate (2026-07-23): all of
Tier 0 packages and dry-run-uploads cleanly on its own now that
tpt-llm-client-core's only internal dependency (tpt-prompt-template,
via the now-removed tool-use feature) is gone. Tier 1 crates correctly
fail dry-run today since their Tier 0 dependencies aren't on crates.io yet
— expected, not a bug.
MSRV
Rust 1.75 — enforced via rust-toolchain.toml and CI.
License
Dual-licensed under MIT and Apache-2.0.