Billing System

Billing System Documentation #

This document describes the billing and subscription management system in BonsAI, which integrates Stripe for payment processing and Clerk for free trial management.

Architecture Overview #

The billing system consists of several components working together:

  1. Stripe Integration: Handles payment processing, subscription management, and billing events
  2. Clerk Integration: Manages organizations and user authentication, also free trial limitations
  3. Billing API: Provides endpoints for retrieving billing information
  4. Cache Layer: Redis-based caching for performance optimization
  5. Webhook Handlers: Process real-time updates from Stripe and Clerk

Billing Flow Diagram # 

graph TB
    subgraph "User Actions"
        A[User Creates Organization]
        B[User Manages Subscription]
    end

    subgraph "Clerk"
        C[Organization Created Event]
        D[Organization Metadata]
    end

    subgraph "BonsAPI"
        E[Clerk Webhook Handler]
        F[Stripe Webhook Handler]
        G[Billing Service]
        H[Organization Repository]
        I[Billing API Endpoints]
    end

    subgraph "Stripe"
        J[Customer Created]
        K[Subscription Events]
        L[Customer Portal]
    end

    subgraph "Cache Layer"
        M[Redis Cache]
    end

    subgraph "Frontend"
        N[Billing Page]
        O[Billing Alerts]
    end

    A --> C
    C --> E
    E --> J
    E --> D
    J --> K
    K --> F
    F --> M
    B --> L
    I --> G
    G --> H
    G --> M
    G --> J
    N --> I
    O --> I
    M --> I

    style A fill:#e1f5fe
    style B fill:#e1f5fe
    style N fill:#e1f5fe
    style O fill:#e1f5fe
    style M fill:#fff3e0
    style J fill:#e8f5e9
    style K fill:#e8f5e9
    style L fill:#e8f5e9

Core Components #

1. Stripe Client (libs/rust/bonsai-integration/src/billing/stripe/) #

The Stripe client provides methods for:

  • Creating customers
  • Managing subscriptions
  • Handling webhooks
  • Retrieving billing information

2. Billing Service (apps/bonsapi/src/service/billing/) #

The billing service layer handles:

  • Reading billing information
  • Managing trial periods
  • Checking subscription status
  • Calculating usage limits

3. Organization Management #

Organizations are the billable entities in BonsAI:

  • Each organization has a stripe_customer_id field
  • Organizations are created via Clerk webhooks
  • Stripe customers are automatically created for new organizations

Coupons #

Stripe coupon ids are random, so a coupon is identified by its metadata.category (welcome, referrer) plus percent_off.

  • PricingService.ListCoupons (bonsapi-internal gRPC, cluster-only): every coupon with its category (OTHER when unrecognised), filterable by category and only_valid. Uncached, so times_redeemed is live. Open to every internal caller.
  • MCP list_coupons (tofu-external-mcp): served to the sales agent only, for the webapp onboarding chat. Returns uncategorised coupons only, so welcome and referrer coupons never reach sales.
  • PricingReadService::find_coupon(category, percent): bonsapi’s own lookup, from a 5-minute cache. Returns the oldest redeemable match, or logs discount_coupon_missing. percent is a whole number.

Onboarding offer #

The Welcome Discount offer, its window experiment (ONBOARDING_OFFER_WINDOWS in Doppler), decline and the logs are documented under Onboarding Discounts.

Checkout. POST /internal/api/v1/billings/checkout (cluster-only) is what the webapp’s checkout route calls for every checkout, the member offer and the billing page alike: it refuses a price the organization is not offered (400) or one listed in another currency than the page showed (409), and returns the catalog, the session currency and, while onboarding, the Welcome coupon. The request names the surface that listed the price: onboarding sells the self-serve catalog to an organization that is still onboarding, billing (also the default) never sells a price that is not is_in_app, onboarding or not.

Onboarding checkout. POST /api/v1/onboarding/checkout (tofu-web, behind ONBOARDING_V2_CHECKOUT) creates the Checkout Session in bonsapi instead of the webapp’s route, with the same price guard and coupon choice. It is documented under Onboarding Discounts.

Listing. GET /api/v1/billings/plans lists what the organization may buy by the same rules (sales’ billing_plans overrides included).

Database Schema #

Organization Table Extension # 

ALTER TABLE organization
ADD COLUMN stripe_customer_id VARCHAR(255);

CREATE INDEX organization_stripe_customer_id_idx
ON organization(stripe_customer_id);

Webhook Handling #

Stripe Webhooks #

Endpoint: POST /webhook/stripe

Handled Events:

  • customer.subscription.created - Handles cache invalidation and immediate charging
  • customer.subscription.deleted - Invalidates billing cache
  • customer.subscription.resumed - Invalidates billing cache
  • customer.subscription.updated - Invalidates billing cache
  • payment_intent.payment_failed - Invalidates billing cache, notifies the organization’s billing managers, and starts the payment-failure grace period
  • invoice.paid / invoice.payment_succeeded - Invalidates billing cache and cancels any pending payment-failure notification

Security:

  • Webhook signatures are verified using STRIPE_WEBHOOK_SIGNING_SECRET
  • Uses stripe::Webhook::construct_event() for cryptographic verification
  • Falls back to manual parsing for API version mismatches (with signature verification)
  • Rejects any unsigned or tampered requests

Processing Flow:

  1. Verify webhook signature
  2. Extract customer ID from subscription event
  3. Look up organization by stripe_customer_id
  4. Invalidate billing cache for the organization

Payment Failure Notifications #

A failed collection attempt is notified in two stages, so an organization that resells tofu to its own clients is not exposed to them by a billing problem it has not had a chance to fix (ENG-7801).

  1. Immediately - payment_intent.payment_failed enqueues a PurchaseFailed notification addressed to BillingManagers, and writes a billing_failure_notification row recording notify_members_after = now + grace period.
  2. After the grace period - bonsai-housekeeping (every 10 minutes) picks up rows whose notify_members_after has passed and whose members_notified_at is still null, and enqueues a PurchaseFailed notification addressed to RemainingMembers: every organization member who is not a billing manager. Soft-deleted organizations are skipped.

A payment settled inside the window deletes the row, so the second stage never fires. A repeat failure - Stripe emits payment_intent.payment_failed on every dunning retry - does not move notify_members_after, otherwise the escalation would be postponed for as long as Stripe kept retrying.

Configuration: BILLING_FAILURE_GRACE_PERIOD_IN_MILLISECS on bonsapi. Unset defaults to 5 days (DEFAULT_BILLING_FAILURE_GRACE_PERIOD_IN_MILLISECS).

Recipient scoping: “billing managers” are the org roles that can actually settle a payment - org:system_owner, org:system_admin and org:billing (“Can manage billing”). org:contributor and org:collaborator cannot, and are notified only by stage 2.

Organization roles live only in Clerk, so bonsai-notification needs CLERK_SECRET_KEY to resolve them. Without it no role resolves and the notification fails closed - nothing is sent, rather than falling back to every member.

Recipient privacy: every templated email is sent as a Mailgun batch (recipient-variables), so each recipient receives their own message and no recipient can see any other recipient’s address. See basic_form_fields in libs/rust/bonsai-email/src/mailgun/mod.rs.

Clerk Webhooks #

Endpoint: POST /webhook/clerk

Handled Events:

  • organization.created: Creates Stripe customer and sets trial metadata
  • organization.updated: Upserts Stripe customer and invalidates cache

Trial System #

New organizations receive a 3-day free trial with:

  • Trial Duration: 3 days from organization creation
  • Trial Limits:
    • Max entities: 10
    • Max invoices per month: 300

Trial metadata is stored in Clerk organization metadata:

{
  "trial_expiry_date": "2024-01-15",
  "trial_max_entities": 2,
  "trial_max_invoices_per_month": 100
}

Failed reads. Dispatch reserves a read’s usage when it queues the job. The worker reports a failed read by sending the extraction back to PENDING with an EXTRACTION error (PATCH /internal/api/v1/extractions/{id}/status or POST /internal/api/v1/extractions/batch/status/pending). The first such report gives back what dispatch reserved, by the same rule (a bank statement its primary pages, anything else one, nothing for a re-read of something the AI already read), so a reported failure never uses up the allowance. Concurrent reports of the same failure refund it once. This applies to every organization, paid plans included.

Two cases get no refund. A read that times out is failed by the housekeeping sweep straight in the database (mark_stale_extractions_as_failed, mark_stale_awaiting_extractions_as_failed), so a later report finds it failed already; it stops counting when the usage cache is next rebuilt from the database, which counts only reads that produced data (the cache key lives an hour for monthly limits, six hours for annual ones). A read last touched before the current period began was reserved against the previous period, so the new one is left alone.

Immediate Charging for Mid-Month Subscriptions #

When a subscription is created after the 1st of the month, BonsAI handles immediate charging to ensure customers pay for the current month’s usage:

How It Works #

  1. Billing Cycle Anchor: All subscriptions are anchored to the 1st of each month
  2. Immediate Charge: When subscribing after the 1st, an immediate invoice is created for the current month
  3. Metadata Requirements: Subscriptions must include:
    • charge_current_month: "true" - Enables immediate charging
    • price_id: "price_xxx" - The price to charge for the current month

Processing Flow # 

graph LR
    A[Subscription Created] --> B{Is it the 1st?}
    B -->|Yes| C[No Immediate Charge]
    B -->|No| D{charge_current_month = true?}
    D -->|No| E[No Immediate Charge]
    D -->|Yes| F[Create Immediate Invoice]
    F --> G[Charge Customer]
    G --> H[Regular Billing Continues on 1st]

Implementation Details #

  • Handled in customer.subscription.created webhook
  • Creates a one-time invoice for the current month
  • Charges immediately using the default payment method
  • Regular subscription billing continues on the 1st of each month

Error Handling #

  • If immediate charge fails, the invoice remains open for collection
  • Customer receives standard Stripe payment retry emails
  • Subscription remains active during retry period

Caching Strategy #

The billing system uses Redis caching to minimize API calls to Stripe:

Cache Keys #

  • Pattern: billing:plan:{organization_id}
  • TTL: 5 minutes
  • Structure:
{
  "subscription_status": "active",
  "current_plan": "starter",
  "billing_cycle_end": "2024-02-01T00:00:00Z",
  "max_invoices_per_month": 100,
  "max_entities": 10
}

Cache Invalidation #

Cache is invalidated when:

  • Stripe subscription events are received
  • Organization is updated via Clerk
  • After 5 minutes

Frontend Integration #

The webapp displays billing information and alerts:

Billing Alert Component #

Shows warnings when:

  • Free trial is expiring (< 3 days)
  • Subscription is inactive
  • Usage limits are approached

Billing Page #

Located at /settings/organization/billing:

  • Shows current plan details
  • Provides link to Stripe customer portal
  • Displays usage statistics
  • Shows trial information

State Management #

Billing state is managed via:

  • useBilling hook for fetching billing data
  • useOrganization hook for organization details
  • Automatic refetch on focus and authentication changes

Environment Variables #

Required environment variables:

# Stripe Configuration
STRIPE_SECRET_KEY=sk_test_...
STRIPE_WEBHOOK_SIGNING_SECRET=whsec_...

# Clerk Configuration
CLERK_SECRET_KEY=sk_test_...
CLERK_WEBHOOK_SIGNING_SECRET=whsec_...

# API Configuration
BONSAPI_EXTERNAL_HOST=https://api.bonsai.com

Error Handling #

Common Error Scenarios #

  1. No Stripe Customer: Organization exists but no Stripe customer

    • Automatically creates customer on first billing request
    • Logs warning for monitoring

  2. Cache Miss: Billing information not in cache

    • Falls back to Stripe API
    • Repopulates cache with fresh data

  3. Webhook Failures: Failed to process webhook

    • Logs error with details
    • Returns 200 OK to prevent retries (for non-critical events)
    • Critical events may return error for retry

Monitoring and Debugging #

Key Log Points #

  • Webhook receipt and processing
  • Cache hits/misses
  • Stripe API calls
  • Customer creation events
  • Subscription changes

Metrics to Monitor #

  • Webhook processing time
  • Cache hit rate
  • Stripe API latency
  • Failed webhook attempts
  • Trial conversion rate

Development and Testing #

Local Development #

  1. Spin up ngrok
  2. Set up Stripe test account
  3. Configure webhook endpoint using ngrok’s host
  4. Use update BONSAPI_EXTERNAL_HOST and STRIPE_WEBHOOK_SIGNING_SECRET in /.env

Security Considerations #

  1. Webhook Verification: All webhooks are cryptographically verified
  2. Authorization: Billing endpoints require valid JWT tokens
  3. Rate Limiting: Consider implementing rate limits for billing endpoints
  4. Audit Logging: All billing changes are logged for compliance
  5. PCI Compliance: No payment information is stored locally

Future Enhancements #

  1. Usage Tracking: Real-time usage monitoring
  2. Billing Alerts: Email notifications for billing events
  3. Multiple Plans: Support for different subscription tiers
  4. Metered Billing: Usage-based pricing options
  5. Invoice Management: Direct invoice access and management