Rust

Rust Testing Strategy #

This doc outlines how we test our Rust libraries and apps.

  • Unit tests: fast, pure Rust tests for code that doesn’t need middleware.
  • Integration tests: run real middleware (RabbitMQ, Redis, S3 via LocalStack, Postgres/Yugabyte) through mise rust-test. Use mocks for external services like Xero, QBO, etc.

Tooling we use #

  • Rust test stack: tokio (async tests), anyhow (ergonomic errors)
  • Containers: docker-compose tools/tests/rust/docker-compose.yml, owned by the Bun wrapper behind mise rust-test
  • Mock: wiremock = "0.6".
  • Feature flags (per crate):
    • <crate>-integration-test, e.g. bonsai-mq-integration-test, bonsai-cache-integration-test
      • Workspace alias: integration-test-all -> enables all integration tests across crates.

Why shared Docker infra? It boots real services in Docker before tests run, so we validate against actual middleware without dev/staging cloud infra. One shared stack is reused across crates.

For mise rust-test, we start one shared Yugabyte/Redis/RabbitMQ/LocalStack stack using docker-compose from tools/tests/rust/docker-compose.yml and pass its ports through TOFU_TEST_*_PORT env vars, and the harnesses reuse those services instead of starting new containers. Rust integration tests are expected to run through mise rust-test; direct cargo test or cargo nextest for integration tests is unsupported because the wrapper owns the shared infra and Yugabyte cache lifecycle.

For performance reasons database is cached between runs, and can be pulled from s3 storage if there is no available version locally. Because of that mise rust-test pulls aws credentials from doppler. If you want to avoid that run mise rust-test – –no-doppler

For the Yugabyte cache path, Linux requires host GNU tar as tar, cp, and zstd; macOS requires host GNU tar installed as gtar plus zstd. On macOS, mise rust-test uses the disk-backed runtime cache path rather than /dev/shm. Linux may still use /dev/shm when it is enabled and the estimated source cache size still fits with the configured runtime margin. The default rust-test S3 cache artifact is test-source.tar.zst.

Shared Rust integration helpers live in libs/rust/tofu-test-utils. Rust tests do not start containers; they only read the TOFU_TEST_*_PORT env vars injected by mise rust-test.

What goes where #

  • Unit tests (src/**, #[cfg(test)])
    • Pure logic, helpers, serializers, formatters, retry math, etc.
      • No network, no containers.
  • Integration tests (tests/**)
    • Exercise public APIs that talk to middleware (publish/subscribe, upload/read/delete, cache ops, DB repos, etc.)
      • Each service gets its own test folder/binary (e.g., tests/rabbitmq, tests/redis, tests/s3)

Feature flags (keep integration tests opt-in) #

Each crate:

# Cargo.toml
[features]
bonsai-cache-integration-test = [] # or bonsai-mq-integration-test, etc.
integration-test-all = ["bonsai-cache-integration-test"]

Crates without integration tests still define:

[features]
integration-test-all = []

Workspace oneliner:

mise rust-test

mise rust-test forwards extra arguments to cargo nextest run -F integration-test-all. That means forwarded args should use nextest syntax, not cargo/libtest-only trailing args like -- --exact.

Examples:

mise rust-test-cache-pull-s3
mise rust-test-cache-pull-s3 -- --no-doppler
mise rust-test -- -p bonsai-database --test yugabyte sqlx_entity_repository::entity::test_entity
mise rust-test -- --no-doppler -- -p bonsai-database --test yugabyte sqlx_entity_repository::entity::test_entity

If RUST_TEST_THREADS is set, the wrapper maps it to cargo nextest run -j <value> unless you already passed an explicit -j or --test-threads.

In Coder workspaces, mise rust-test also defaults CARGO_BUILD_JOBS=2 unless you already set CARGO_BUILD_JOBS yourself. That keeps Rust compilation from saturating the workspace while still allowing higher nextest execution concurrency.

Minimal Integration Pattern #

For middleware-backed Rust tests in this repo:

  1. mise rust-test starts shared infra once and injects TOFU_TEST_*_PORT.
  2. Rust tests use helpers from tofu-test-utils.
  3. Tests stay focused on behavior, not container setup.

Use these shared helpers:

  • DB: tofu_test_utils::yugabyte::{get_test_database, get_test_database_with_details}
  • DB names: generated constants in tofu_test_utils::database_names
  • Redis: tofu_test_utils::redis::new_test_redis_client
  • RabbitMQ: tofu_test_utils::rabbitmq::{new_test_mq_client, raw_channel, purge_queue}
  • LocalStack/S3: tofu_test_utils::localstack::{create_bucket, new_test_s3_client}
  • App-style helpers: tofu_test_utils::app_stack::{new_cache_client, new_mq_client, new_storage_client}

Crate-specific setup, such as bonsapi auth env vars, should live in tests/integration/common/utils.rs as setup_test(). Do not add containers.rs, test_env.rs, app-stack wrappers, or Rust-side container lifecycle code.

Example:

#![cfg(feature = "bonsai-cache-integration-test")]

use anyhow::Result;
use tofu_test_utils::redis::new_test_redis_client;

#[tokio::test]
async fn can_set_and_get() -> Result<()> {
    let client = new_test_redis_client()?;
    // assert the behavior you care about against real Redis
    Ok(())
}

When adding new middleware:

  1. Add the service to tools/tests/rust/docker-compose.yml if it should participate in shared mise rust-test.
  2. Update tools/tests/rust/src/index.ts to export the matching TOFU_TEST_*_PORT and wait for readiness.
  3. Add or extend the corresponding env-driven helper in libs/rust/tofu-test-utils.

Unit test pattern (no containers) #

  • Co-locate in the module:
#[cfg(test)]
mod tests {
    use super::*;
    #[test]
    fn formats_headers() {
        assert_eq!(format_key("abcd1234"), "****1234");
    }
}
  • Use pure data + deterministic assertions
  • For functions that normally hit middleware, factor the logic so the pure parts are testable without I/O (e.g., payload building, routing key selection, TTL bucketing, key naming, retry/backoff math).

Running tests #

  • Everything:
mise rust-test
  • Just one crate + its integration tests through the wrapper:
mise rust-test -- -p bonsai-mq
  • Unittests only
mise rust-test-unit

Practical notes #

  • Use short, unique resource names per test (bucket names, queues/keys).
  • Keep integration tests small and focused: one behavior per test (publish-then-consume, put-then-get-then-delete, set-then-get, simple query).
  • If an operation is flaky on initial readiness, add a tiny retry/backoff in the test (not in prod code).