Webapp API Request Logging #

The webapp automatically logs all API requests to provide better observability and debugging capabilities. Logs are sent to Datadog and follow the same format as the backend API.

Log Format #

All API requests are logged in the format:

METHOD PATH STATUS LATENCYms

Example:

GET /api/healthcheck 200 0.024692ms
POST /api/dd-log 200 15.342156ms
GET /api/encryption/key 200 2.156789ms

Trace ID Propagation #

Every API request includes an X-TOFU-TRACE-ID header for distributed tracing:

1. Incoming requests: Trace ID is extracted from the request header or generated in middleware if not present
2. Outgoing requests: Trace ID is generated by fetch-instance.ts or axios-instance.ts when calling external APIs

This enables end-to-end request tracing across the entire system:

Browser → Webapp API → BonsAPI → External Services
  ↓          ↓           ↓            ↓
trace-1   trace-1    trace-2      trace-3

Implementing Logged API Routes #

All new API routes should use the withLogging wrapper:

import { NextResponse } from "next/server";
import type { NextRequest } from "next/server";
import { withLogging } from "@/shared/lib/api/with-logging";

export const POST = withLogging(async (request: NextRequest) => {
  // Your route logic here
  const data = await request.json();

  // Process...

  return NextResponse.json({ success: true });
});

Important: Avoiding Cyclic Logging #

Do NOT wrap logging endpoints with withLogging to avoid cyclic dependencies:

// ❌ BAD - Creates cyclic logging
// /api/dd-log/route.ts (forwards client logs to Datadog)
export const POST = withLogging(async (request: NextRequest) => {
  logger.info({ message: "Log received" }); // This uses the same logger!
});

// ✅ GOOD - Exclude logging endpoints from withLogging
// /api/dd-log/route.ts
export async function POST(request: NextRequest) {
  logger.info({ message: "Log received" });
}

Endpoints to exclude from withLogging:

• /api/dd-log - Client-side log forwarding endpoint
• Any endpoint that uses logger from @/shared/lib/logger/server

Benefits #

• Automatic logging: No need to manually log each request
• Consistent format: Matches backend logging pattern
• Error handling: Errors are logged with full context
• Performance tracking: Request latency is automatically calculated
• Trace ID injection: Response includes trace ID header

Viewing Logs in Datadog #

Logs are sent to Datadog with the following structure:

Fields:

• message: METHOD PATH STATUS LATENCYms
• label: http_request
• data:
  • trace_id: Unique request identifier (UUID)
  • method: HTTP method (GET, POST, etc.)
  • path: Request path
  • status: HTTP status code
  • latency_ms: Request duration in milliseconds

Query examples:

# All webapp API requests
service:bonsai-webapp label:http_request

# Failed requests (4xx or 5xx)
service:bonsai-webapp label:http_request @data.status:>=400

# Slow requests (>1 second)
service:bonsai-webapp label:http_request @data.latency_ms:>1000

# Specific trace ID
service:bonsai-webapp @data.trace_id:"abc-123-def-456"

# Specific endpoint
service:bonsai-webapp @data.path:"/api/dd-log"

How It Works #

1. Middleware Layer #

The webapp middleware (apps/webapp/src/middleware.ts) handles trace ID generation for incoming API requests:

// Extract or generate trace ID
const traceId = req.headers.get("x-tofu-trace-id") || uuidv4();

// Clone request headers and add trace ID
const requestHeaders = new Headers(req.headers);
requestHeaders.set("x-tofu-trace-id", traceId);

// Pass modified headers to route handler
return NextResponse.next({
  request: { headers: requestHeaders },
});

This follows the Next.js middleware pattern for modifying request headers.

2. Route Handler Wrapper #

The withLogging wrapper (apps/webapp/src/shared/lib/api/with-logging.ts) captures request/response metadata:

1. Extracts trace ID from request headers (set by middleware)
2. Records start time
3. Executes the route handler
4. Captures status code and errors
5. Calculates latency
6. Logs to Datadog asynchronously (non-blocking)
7. Adds trace ID to response headers

3. Logging Backend #

Logs are sent to Datadog via Winston logger:

• Winston HTTP transport sends to Datadog intake API
• Logs are structured as JSON for easy parsing
• Datadog retains logs for 15 days

Migration Guide #

Converting Existing Routes #

Before:

export async function GET() {
  return NextResponse.json({ status: "healthy" });
}

After:

import { withLogging } from "@/shared/lib/api/with-logging";

export const GET = withLogging(async () => {
  return NextResponse.json({ status: "healthy" });
});

Converting Routes with Parameters #

Before:

export async function POST(request: NextRequest) {
  const body = await request.json();
  // ... logic
  return NextResponse.json({ success: true });
}

After:

import { withLogging } from "@/shared/lib/api/with-logging";

export const POST = withLogging(async (request: NextRequest) => {
  const body = await request.json();
  // ... logic
  return NextResponse.json({ success: true });
});

Converting Routes with Context #

Some Next.js routes receive a context parameter (e.g., dynamic routes):

import { withLogging } from "@/shared/lib/api/with-logging";

export const GET = withLogging(
  async (request: NextRequest, context: { params: { id: string } }) => {
    const { id } = context.params;
    // ... logic
    return NextResponse.json({ id, data: {} });
  }
);

Performance Considerations #

Logging Overhead #

The withLogging wrapper adds minimal overhead:

• Synchronous: ~0.1-0.5ms (trace ID extraction, timestamp)
• Asynchronous: Logging happens in setImmediate() and doesn’t block the response

Best Practices #

1. Keep route handlers fast: The wrapper measures total latency
2. Don’t log sensitive data: Request/response bodies are not logged by default
3. Use appropriate log levels:
   • info for 2xx responses
   • warn for 4xx responses
   • error for 5xx responses

Troubleshooting #

Logs Not Appearing in Datadog #

1. Check environment: Ensure DATADOG_API_KEY is set in Doppler
2. Verify service name: Logs should have service:bonsai-webapp
3. Check retention: Logs are retained for 15 days
4. Network issues: Check if Winston can reach Datadog intake API

Missing Trace IDs #

1. Check middleware: Ensure middleware is generating/extracting trace IDs
2. Verify headers: Check if x-tofu-trace-id is in request/response
3. Client-side: Ensure fetch-instance/axios-instance are generating IDs for outbound calls

Performance Issues #

1. Check latency: Query Datadog for high latency_ms values
2. Identify bottlenecks: Use trace IDs to correlate frontend/backend logs
3. Monitor resources: Check if logging is causing memory/CPU issues

Related Documentation #

• Log Management Runbook - Searching and analyzing logs
• Monitoring & Alerting - Setting up alerts
• Backend API Logging - How the backend logs requests

Future Enhancements #

• Distributed tracing: Propagate parent trace ID from webapp → backend
• OpenTelemetry: Standardized observability with automatic instrumentation
• Request/response body logging: With PII filtering for debugging
• Performance metrics: Percentile-based latency tracking