tofu-web Chat Runtime Spike #

Classification: Internal-Only

Recommendation superseded. §1 below recommends assistant-ui. Platform Decisions §8 retires both candidates, for a better reason than either of mine: CopilotKit’s AnthropicAdapter never calls its injected client, so SigV4 cannot work, and its BedrockAdapter takes no sessionToken, which rules out STS credentials and therefore IRSA on EKS. The tool runtime stays @tanstack/ai. Everything measured here still stands — the SSE numbers in §2 are why §8 also retires dropping Start’s server.

Status: complete. This answers the two things Platform Decisions §12 said had to be done before committing — “spike CopilotKit’s runtime on bun against tofu-external-mcp, and confirm the SSE path holds under the same conditions that broke it in Start” — and §11.2, “CopilotKit or assistant-ui”.

Everything below was measured on this machine against the running dev stack, on 2026-09-08. Numbers are from the spike, not from documentation. Where a claim in the revision turned out to be wrong, that is called out.

1. Recommendation (superseded — see the note above) #

Adopt assistant-ui, not CopilotKit. Not on features — CopilotKit has more of them, and its runtime is genuinely good. On stack fit, which is the criterion the question was asked under, and where the gap is not close:

We are on zod 4.4.3, zustand 5.0.15, radix-ui 1.4.3 and lucide-react 0.545.0. assistant-ui is on the same three libraries at the same majors. CopilotKit is on the other side of a major on zod and brings a second copy of the icon set.

Two of those are not cosmetic:

zod 3 is a hard dependency, not a peer. @copilotkit/react-core’s peer is zod: >=3.25, which zod 4 satisfies — but @copilotkit/core depends on zod: ^3.25.75 and @copilotkit/shared on ^3.23.3, so a zod 3 lands in the bundle whatever we declare. This app’s tool story rests on zod 4 specifically: ai/tools.ts hands the command’s own schema to the runtime as-is, because on zod 4 the schema is the JSON Schema, which is what makes “a tool whose arguments have drifted from the command’s” impossible rather than merely unlikely. Crossing into a zod-3 runtime puts zod-to-json-schema back in the middle of that guarantee.

The duplicate lucide-react is a bug we have already had. @copilotkit/react-core pins lucide-react ^0.525.0; we are on ^0.545.0 in both apps/tofu-web and @bonsai/ui. Two copies of lucide in one tree is exactly what produced the SSR hydration failure earlier in this branch — the server and the client drew different Building2 glyphs and React tore the tree down. It was fixed by aligning the versions, which is not available to us if a dependency pins its own.

The cost of choosing assistant-ui is real and should be stated plainly: it ships no server runtime. CopilotKit gives you a service; assistant-ui gives you a client and expects a backend. That is more work — but it is work we have already done once (server/ai/orchestrator.ts, server/ai/mcp-client.ts) and it is the half of the problem that is ours anyway, because the tool allowlist, the permission set and the taint rules are not something either library models.

2. The SSE question, answered #

§3 of the revision cites three open issues as the case against Start. Two of them were tested.

bun#27479 — quiet SSE streams die at ~10s #

Reproduced, and it is already solved in this monorepo. Two identical bun + Hono servers, one with default settings and one with idleTimeout: 120, each holding an SSE stream that sends one frame, stays quiet for 25 seconds, then sends a second frame:

apps/tofu-external-mcp/src/index.ts already sets exactly this, with a comment saying why:

// Downstream bonsapi calls can be slow under load; bump Bun's default 10s
// idle timeout so the MCP transport doesn't drop the connection mid-response.
export default { port, idleTimeout: 120, fetch: app.fetch };

This is not an argument for leaving Start. It is a bun issue, so it would hit the proposed apps/tofu-web-ai identically, and it is fixed by one server option we already use in production. It belongs on the deployment checklist, not in the framework decision.

router#7991 — an abandoned request becomes a 500 and a console.error #

Did not reproduce on our pinned version. Ten aborted SSR requests against the running tofu-web dev server (Start 1.167/1.169, nitro pinned) produced zero log lines and left the server healthy.

That is a narrower test than the issue describes — SSR page requests in dev, not server functions and not a production build — so it is evidence, not a refutation. But the revision presents this as “an alert flood” for an app built on SSE streams, and on the version we actually run, against the request shape we actually serve, there was no flood and no error.

The same test, on the two candidate runtimes #

Five client aborts mid-stream, each on its own thread:

An aborted CopilotKit thread also accepts a new run afterwards, so the abort releases its lock.

3. CopilotKit’s runtime on bun — it works #

@copilotkit/runtime v2 booted on bun + Hono via createCopilotEndpoint and served a real turn against Anthropic, with a client-provided tool:

RUN_STARTED       → threadId, runId, messages, tools, context, forwardedProps
TOOL_CALL_START   → go_to_route
TOOL_CALL_ARGS    → {"routeId": "home"}   (streamed in two deltas)
TOOL_CALL_END
RUN_FINISHED      → finishReason: "tool-calls", 947 in / 57 out

The whole turn completed in ~16 ms of wall clock after the model responded. GET /info advertises mode: "sse", tools.clientProvided: true, humanInTheLoop.interrupts: true and thread endpoints for list and inspect. The run body is { threadId, runId, messages, tools, context, forwardedProps, state } — close enough to what our @tanstack/ai chat route already sends that the migration would be mechanical.

Two things worth knowing before anyone runs it anywhere real:

• Telemetry is on by default. The runtime prints a notice and phones home unless COPILOTKIT_TELEMETRY_DISABLED=true. For a service handling customer accounting data that is a deployment-config item, not a footnote.
• Concurrent runs on one thread throw. Two runs on the same threadId produce Error: Thread already running with a full stack trace on every rejected run, unless onConcurrentRun: 'supersede' is set. A user reloading a tab mid-stream and reconnecting on the same thread is exactly this shape.

Also worth recording, because it cuts the other way from the recommendation: CopilotKit v2 exports convertInputToTanStackAI and convertTanStackStream. It has first-class TanStack AI interop, which is more than assistant-ui offers and would have made the transition from @tanstack/ai shorter.

4. MCP reach #

A bun process reaches tofu-external-mcp at http://localhost:3003/ (the transport is mounted at /, not /mcp) and gets 401 {"error":"invalid_token"} without a Clerk bearer, which is the server behaving correctly rather than a failure. apps/tofu-web/src/server/ai/mcp-client.ts already speaks this protocol with a forwarded token; nothing about either candidate changes that, because in both designs the MCP client stays ours.

5. Where the revision was wrong about assistant-ui #

§2 says assistant-ui’s strengths are behind unstable_ APIs. Half right, and the half that is wrong matters: 27 of its 159 exports are unstable_, and they are the interactables and WebMCP ones — unstable_useInteractable, unstable_Interactables, unstable_useWebMcpProvider, unstable_defaultWebMcpFilter.

The approval seam is stable: hitlTool, humanTool, toolApprovalAcceptsText. So is tool registration (tool, useAssistantTool, makeAssistantTool), MCP toolkits (defineMcpToolkit, defineToolkit), thread persistence (RemoteThreadList, useRemoteThreadListRuntime, InMemoryThreadList) and useExternalStoreRuntime, which lets our command registry stay the source of truth rather than being mirrored into the library’s.

That last one is the deciding feature for this codebase. The revision’s §4 asks CopilotKit to own “registration, serialisation, the mount lifecycle, transport, cancellation, the confirmation round trip and the chat UI”. We do not want it to own registration: the registry is where the reversibility tier, the permission, the caller and the audit record live, and those are the parts no library models. useExternalStoreRuntime is the seam for keeping them.

6. What this changes in the revision #

• §1 #1 / §3, the case for leaving Start loses two of its three legs. bun#27479 is a bun issue we already fix with one line, and router#7991 did not reproduce here. The third — server functions 404ing in bun production builds (router#5328) — was not tested and is the one to check before any production cutover. On the evidence so far, dropping SSR and standing up a second deployable is not justified by the bugs cited for it.
• §1 #2 and #3 / §11.2 resolve to assistant-ui, on stack fit.
• §11.1, the chat bridge, is settled the other way. The owner’s reason is one this document did not consider: an OAuth round trip to an accounting provider returns to a different tab, and the originating tab has to hear about it. Neither library’s thread model covers cross-tab awareness, because it is not a conversation-continuity problem. The bridge stays, and the transcript store it was built beside gets wired.

7. Reproducing this #

The spike lives outside the repo, under the session scratchpad. It is four files: two bun + Hono SSE servers (default and tuned), one CopilotKit v2 runtime, one assistant-stream endpoint. Rebuilding it is bun add hono @copilotkit/runtime and bun add @assistant-ui/react assistant-stream hono plus about sixty lines. Nothing in it is worth keeping in the repository; the numbers above are the output.