Python and Rust View on GitHub

Native boundary

Python and Rust

litGraph uses PyO3 0.22 and maturin to ship a Rust runtime behind a Python-first package. The boundary is intentionally narrow: Python handles application composition, while Rust owns performance-sensitive and concurrency-heavy work.

Package shape

python/litgraph/__init__.py
        │ imports and augments
        ▼
litgraph.litgraph               native extension
        │ built from
        ▼
crates/litgraph-py              PyO3 conversion + registration
        │ delegates to
        ▼
litgraph-core + graph + agents + adapters

Maturin configuration lives in pyproject.toml:

  • manifest-path points to crates/litgraph-py/Cargo.toml.
  • module-name installs the extension as litgraph.litgraph.
  • python-source packages the surrounding python/litgraph code.
  • the pyo3/extension-module feature configures native linking.

The pure-Python package can still expose helpers in a development checkout where the native module has not been built, but production installs include the extension.

Native module registration

crates/litgraph-py/src/lib.rs registers focused submodules such as providers, graph, tools, agents, retrieval, observability, cache, memory, MCP, middleware, deep-agent, and serve.

Submodules are inserted into sys.modules in addition to being attached to the parent module. This is what makes both forms work:

import litgraph
from litgraph.graph import StateGraph

Keep registration declarative and local. Adding a new Python surface normally means adding or extending one module file and registering its classes or functions from that module’s register function.

Interpreter discipline

The safe call shape is:

Python values→validate / convert→release Python→native I/O or compute→convert result

Do not hold the GIL across provider requests, database work, graph execution, or other blocking native operations. PyO3 0.22 bindings use the version-appropriate thread-release API; when the project upgrades PyO3, preserve the behavior with the corresponding detached execution API.

Python callbacks are the exception: a Python graph node or tool callable must reacquire Python long enough to invoke it and convert its return value. Drop the acquisition again before awaiting more native work.

Shared async runtime

The binding owns one lazily initialized Tokio runtime for the process. Provider methods, graph scheduling, and async storage use it rather than creating a runtime per call.

Do not call a fresh Runtime::new() inside each PyO3 method. Beyond startup overhead, nested runtimes make cancellation and spawned task ownership difficult to reason about.

Add a native API

  1. Define the Rust contract. Put shared types or traits in litgraph-core; put implementation in the focused crate.
  2. Test it in Rust. Cover success, error, cancellation, and serialization behavior before adding Python conversion.
  3. Add the binding. Convert arguments while Python is held, release it around native work, and map domain errors to useful Python exceptions.
  4. Register the surface. Extend the relevant register function and, for a new submodule, add it in lib.rs.
  5. Add Python ergonomics only when needed. Decorators and type-aware wrappers belong under python/litgraph.
  6. Update PEP 561 stubs. Keep signatures, defaults, return types, and module placement synchronized.
  7. Add one Python test file per public surface. Test the rebuilt extension inside the project environment.

Rebuild and verify

pixi run develop
pixi run test-python
pixi run test-stubs

For focused Rust work:

cargo check --workspace
cargo test --workspace --lib
cargo clippy --workspace --all-targets

Always run Python tests through the Pixi environment or the project .venv that maturin updated. A separate Homebrew or system interpreter can have an older installed wheel and create convincing false failures.

abi3 compatibility

The wheel targets Python’s stable ABI starting at 3.9. Avoid using an API that requires a newer CPython-specific ABI unless the minimum is intentionally changed. Free-threaded Python also makes interpreter-release discipline and thread-safe shared state essential.

The repository’s free-threading notes document the current audit and opt-in path.

Error design

Native errors should reach Python with:

  • a stable exception category;
  • enough context to identify the provider, node, tool, or backend;
  • a corrective action when one is known;
  • no secrets, raw credentials, or unnecessarily large payloads.

Avoid returning sentinel dictionaries for errors that callers cannot safely ignore. Conversely, do not translate ordinary model refusals or empty retrieval results into runtime exceptions when they are valid domain outcomes.

Keep the boundary thin

If a PyO3 method starts owning retries, scheduling, storage policy, or provider logic, move that behavior into a pure Rust crate and leave only argument/result conversion in the binding. This keeps Rust reuse, testing, and concurrency properties intact.