Extraction v2 Overview #

Classification: Internal-Only

Extraction v2 is a Rust extraction pipeline that runs beside the current hinoki pipeline, not instead of it. A router worker sits in front of 100% of extractions and assigns each one to an arm; the v1 arm is the pipeline that runs today, unchanged. Only AP bill is in scope.

The PoC (ENG-7054, ENG-7095, ENG-7100, ENG-7101) established the approach offline. Nothing from it reached the monorepo except two client libraries — one for Extend, one for the Bedrock models. This is a greenfield build informed by the PoC findings, not a port of PoC code.

Why #

The current pipeline is vision-only: every field prediction is an image-input call to an expensive model, and contact matching compares names. That gives us two problems, and the targets follow from them:

Contact matching is called out separately because a wrong contact is the most expensive single error. Contact knowledge is an input to every field stage — header, lines and accounting alike — so one bad match degrades the whole document, not one field.

v2 attacks both: Extend AI supplies a text layer (with bounding boxes) so the field stages can run on text with a cheap Bedrock model instead of images with an expensive one, and contact matching gets signals beyond name similarity.

Flow #

bonsapi enqueues extraction
        │
        ▼
      router                      ── arm assignment, sticky, kill switch
        │
        ├── v1 arm ──▶ bonsai-invoice + hinoki   (unchanged)
        │
        └── v2 arm ──▶ the Rust extraction worker
                            │
                            ▼
                       ┌─────────┐
                       │  prep   │  page type, multi-doc split, rotation, complexity
                       └─────────┘
                            │
                            ▼
                       ┌─────────┐
                       │ Extend  │  parse → markdown text + bbox layer
                       └─────────┘
                            │
                            ▼
                     ┌───────────────┐
                     │ contact match │  multi-signal
                     └───────────────┘
                            │
                            │  contact knowledge
              ┌─────────────┴─────────────┐
              ▼                           ▼
      ┌────────────────┐          ┌────────────────┐
      │ header fields  │          │   line items   │   (parallel)
      └────────────────┘          └────────────────┘
              │                           │
              └─────────────┬─────────────┘
                            ▼
                    ┌────────────────┐
                    │   accounting   │  needs both
                    └────────────────┘
                            │
                            ▼
                  aggregation + bbox attach
                            │
                            ▼
                  validation (shared calculation engine)
                     │                    │
                   pass                  fail
                     │                    │
                     │                    ▼
                     │          re-enqueue to Extend at higher
                     │          effort (capped) ──▶ terminal
                     │                             needs-review
                     ▼
              write back via bonsapi

What each stage does, costs and does on failure is in Stages. The router, arm assignment and shadow slice are in Routing & A/B.

Stage boundaries #

The stage graph above is the contract. What matters for parallel implementation is what each stage is allowed to assume about the ones before it.

Priority fields are not a stage. A priority field is one the entity asked us to be extra careful about, and it is handled inside whichever stage already owns that field, so it inherits that stage’s position in the graph — no separate node, no separate queue, no extra join. See the priority-field pass.

Three rules hold across every stage:

1. Only the write-back step writes to bonsapi. No stage persists partial results. This is a hard requirement for the A/B, not tidiness: if v2 writes intermediate snapshots that v1 does not, the accuracy views see a different number of them and the comparison stops meaning anything.
2. Stage outputs are typed values, not loose blobs. Two people building two stages should fail to compile rather than disagree at runtime.
3. A stage owns its priority-field pass. The pass may not widen the stage’s output or read another stage’s — if it needs something a sibling produced, the field belongs to the sibling.

Two workers #

The v2 arm is a Rust worker consuming its own queue. The router is a separate worker in front of it rather than a branch inside the Python one, because it sits ahead of 100% of extractions: it has to fail closed to the v1 arm and carry a kill switch without a hinoki deploy.

The stage logic is pure and separable from the worker that drives it, which is what lets the accuracy harness run one stage at a time against recorded inputs. That property is load-bearing for measurement, not a matter of taste.

What v2 replaces in hinoki #

For the v2 arm on AP bill only, these hinoki components are not called:

Moving prep into the v2 arm is a deliberate choice and the one with real risk. It makes the arm self-contained — no Python hop in the middle — but it needs image input in the LLM client, which is text-only today, and it means split decisions are made by a different model than v1 makes them with. A different split decision is more visible to a user than a wrong field: they see the wrong number of documents. Prep is therefore measured against v1’s decisions on the same document rather than against ground truth, which does not exist for a split.

What stays #

Everything not listed above, specifically:

• The entire v1 arm, untouched. It is the control group; changing it invalidates the comparison.
• Every other extraction type. AR invoice, direct expense, bank statement and credit note stay on hinoki in full.
• Knowledge. Contact knowledge, entity knowledge, knowledge conflicts and the knowledge workers are consumed by v2, not reimplemented.
• The bonsapi service layer. v2 reaches bonsapi over gRPC rather than the internal REST scope the Python worker uses, following the precedent tofu-external-mcp set. The transport differs; the code underneath does not — the gRPC handlers delegate to the same services the internal REST controllers call, which is what keeps v1 and v2 writing identically. See Talking to bonsapi.
• tofu-snippet as the gate in front of extraction, and document conversion — both upstream of the router.

Write-path parity is the biggest risk here, above extraction quality. If v2 does not produce the same extracted data, confidences and bbox through the same store path, the webapp, the exports and the accuracy views all diverge — and then the A/B measures our plumbing rather than the pipeline.

Cost budget #

The 3¢ per page ceiling splits in two: 1.5¢ for Extend, 1.5¢ for everything else. The allocation below is the design target; measured per-stage cost replaces it once the pipeline runs on real traffic.

Two things this table makes visible:

• Prep landed inside “the rest”. The original split was 1.5¢ Extend and 1.5¢ for the field stages; prep’s vision calls now sit in that same 1.5¢ alongside contact matching, the three field stages and their priority-field passes. Prep is the first place to look if the budget is missed.
• Escalation is charged separately. A validation failure re-parses at the expensive tier, which costs more per page than the cheap one. That spend is tracked against an escalation budget with an attempt cap, not against the per-page line above — otherwise a small share of hard documents silently consumes the headroom for all of them. Escalation outcomes are recorded so that “did the higher-effort re-run actually fix it” is answerable from data rather than from memory.

Scope #

In scope: AP bill, on production traffic, behind an A/B.

Out of scope: every other extraction type. Rolling out to AR invoice, direct expense, bank statement and credit note follows the AP bill readout, gated on it. Bank statement in particular will strain the shared stage shape — no contact, transactions rather than lines — and is planned for, not designed for.