Legend State Indexeddb Sync

Query Sync with Legend State and IndexedDB #

This document outlines the process for implementing synchronization between Legend State observables and IndexedDB for persistent data storage in the webapp.

Overview #

Our webapp uses Legend State for reactive state management combined with IndexedDB for client-side persistence. This setup allows data to persist across browser sessions and provides optimistic updates with backend synchronization.

Architecture #

The sync system consists of three main components:

  1. Table Configuration (persist.ts) - Defines table names and database versioning
  2. Observable Creation - Creates synced observables using syncedDataStateQueryObservable
  3. Initialization (use-init-sync.ts) - Ensures all observables are properly initialized

Adding a New Table #

When creating a new data table with Legend State and IndexedDB persistence, follow these steps:

Step 1: Define Table Configuration #

In apps/webapp/src/shared/utils/persist.ts:

  1. Increment the database version:

    // Increment the DB_VERSION when adding a new table
const DB_VERSION = 12; // Previously was 11

  2. Add the table name constant:

    export const TABLE_YOUR_FEATURE = "yourFeature";

  3. Add the table name to the tableNames array:

    export const getPersistPlugin = () => {
  if (!persistPluginInstance) {
    persistPluginInstance = observablePersistIndexedDB({
      databaseName: DEFAULT_PERSIST_DB_NAME,
      version: DB_VERSION,
      tableNames: [
        TABLE_ENTITIES,
        TABLE_DOCUMENTS,
        TABLE_INVOICES,
        // ... existing tables
        TABLE_YOUR_FEATURE, // Add your new table here
      ],
    });
  }
  return persistPluginInstance;
};

Step 2: Create the Observable #

Create a new file for your feature (e.g., apps/webapp/src/shared/sync/your-feature.ts):

import type { Observable } from "@legendapp/state";
import { syncedDataStateQueryObservable } from "@/shared/utils/query";
import type { DbState } from "@/shared/sync/types";
import { TABLE_YOUR_FEATURE } from "@/shared/utils/persist";
import { auth$ } from "@/shared/sync/auth";
import { doesCurrentEntityExist$, ui$ } from "@/shared/sync/ui";

// Define your data interface
interface YourFeatureData {
  id: string;
  name: string;
  // ... other properties
}

// Define the state interface extending DbState
interface YourFeatureState extends DbState<YourFeatureData[]> {
  // Add any custom methods here
  getById: (id: string) => YourFeatureData | undefined;
}

// Create the observable
const yourFeature$: Observable<YourFeatureState> =
  syncedDataStateQueryObservable(
    TABLE_YOUR_FEATURE,
    () => ({
      list: getYourFeatureQueryOptions({
        entity_id: ui$.currentEntityId.get() || "",
      }),
      transform: {
        list: (data) => data?.data ?? [],
      },
      persist: true,
      enabledQueryObserver: true,
      staleTime: 5000,
      refetchInterval: 10000,
      mode: "set",
    }),
    {
      dependencies: [ui$.currentEntityId],
      waitFor: [auth$.isAuthenticated, doesCurrentEntityExist$],
      extensions: {
        getById: (data$) => (id: string) => {
          return data$.get().find((item) => item.id === id);
        },
      },
    }
  );

export default yourFeature$;

Step 3: Register in the State Tree #

In apps/webapp/src/shared/sync/index.ts:

import yourFeature$ from "@/shared/sync/your-feature";

export const state$ = observable({
  auth: auth$,
  db: {
    entities: entities$,
    documents: documents$,
    invoices: invoices$,
    yourFeature: yourFeature$, // Add your observable here
    // ... other tables
  },
  ui: ui$,
});

Step 4: Initialize the Observable #

In apps/webapp/src/shared/sync/hooks/use-init-sync.ts, add your observable to the initialization:

const initSync = async (): Promise<void> => {
  // ... existing initialization code

  try {
    // Initialize by directly accessing the data$ properties
    state$.db.entities.data$.get();
    state$.db.documents.data$.get();
    state$.db.invoices.data$.get();
    state$.db.yourFeature.data$.get(); // Add this line

    // ... rest of initialization
  } catch (err) {
    // ... error handling
  }
};

Updating Existing Tables #

When modifying an existing table structure (adding/removing fields, changing indexes, etc.):

Step 1: Update Database Version #

In apps/webapp/src/shared/utils/persist.ts:

// Increment the DB_VERSION
const DB_VERSION = 12; // Previously was 11

Step 2: Update Initialization #

In apps/webapp/src/shared/sync/hooks/use-init-sync.ts, ensure your observable’s data$.get() is called in the initialization:

const initSync = async (): Promise<void> => {
  try {
    // Make sure to include your updated observable
    state$.db.yourUpdatedFeature.data$.get();

    // ... other initializations
  } catch (err) {
    // ... error handling
  }
};

Real-World Examples #

Example 1: Entities Table # 

const entities$: Observable<EntitiesState> = syncedDataStateQueryObservable(
  TABLE_ENTITIES,
  {
    list: getGetEntitiesQueryOptions(),
    transform: {
      list: (data) => data?.data || [],
    },
    persist: true,
    enabledQueryObserver: true,
    staleTime: 5000,
    refetchInterval: 10000,
    mode: 'set',
  },
  {
    extensions: {
      getById: (data$) => (id: string) => {
        return data$.get().find((entity) => entity.id === id);
      },
    },
  }
);

Example 2: Accounting Data with Dependencies # 

const accountingAccounts$: Observable<AccountingAccountsState> =
  syncedDataStateQueryObservable(
    TABLE_ACCOUNTING_ACCOUNTS,
    () => ({
      list: getAccountingAccountsQueryOptions(
        accountingIntegrations$.data$?.get()?.[0]?.id || '',
        {
          entity_id: ui$.currentEntityId.get() || '',
        }
      ),
      transform: {
        list: (data) => data?.data ?? [],
      },
      persist: true,
      enabledQueryObserver: true,
      staleTime: 3000,
      refetchInterval: 3000,
      mode: 'set',
    }),
    {
      dependencies: [ui$.integrationId, ui$.currentEntityId],
      waitFor: [auth$.isAuthenticated, doesCurrentEntityExist$],
    }
  );

Important Notes #

Database Versioning #

  • Always increment DB_VERSION when adding new tables or updating existing table structures
  • This applies to both creating new tables and modifying existing ones
  • This ensures proper IndexedDB schema migration

Development Environment #

  • When developing locally, close the browser tab before making table changes
  • This prevents version conflicts during development

Naming Conventions #

  • Table name constants should be in SCREAMING_SNAKE_CASE
  • Observable names should end with $ (e.g., entities$)
  • The table name should match the first parameter of syncedDataStateQueryObservable

Dependencies and Wait Conditions #

  • Use dependencies array for observables that should trigger re-queries when changed
  • Use waitFor array to ensure required conditions are met before starting queries
  • Common wait conditions include authentication and entity existence

Error Handling #

  • Always include proper error handling in your observables
  • Use the logger utility for consistent error reporting
  • Consider showing user-friendly error messages with toast notifications

Troubleshooting #

Common Issues #

  1. Table not found errors: Ensure the table name is added to both the constant and the tableNames array
  2. Data not persisting: Check that persist: true is set in the observable configuration
  3. Initialization errors: Verify that the observable’s data$.get() is called in use-init-sync.ts
  4. Version conflicts: Make sure to increment DB_VERSION when making schema changes

Performance Considerations #

  • Set appropriate staleTime and refetchInterval values based on data update frequency
  • Use waitFor conditions to prevent unnecessary queries
  • Consider using extensions for computed values to avoid recalculation
  • Always use batch() when updating multiple observables (see Performance Optimization section below)

Performance Optimization #

Critical: Use batch() for Multiple State Updates #

The most important performance optimization for Legend State is using batch() to combine multiple state updates into a single render cycle.

Why Batching Matters #

Without batching, each .set(), .push(), .assign(), or .delete() call triggers a separate re-render of all subscribed components. With batching, multiple updates = 1 render instead of N renders.

import { batch } from '@legendapp/state';

// ❌ BAD - 3 separate renders
function updateUserProfile(name: string, email: string, avatar: string) {
  state$.user.name.set(name);     // Render 1
  state$.user.email.set(email);   // Render 2
  state$.user.avatar.set(avatar); // Render 3
}

// ✅ GOOD - Single render
function updateUserProfile(name: string, email: string, avatar: string) {
  batch(() => {
    state$.user.name.set(name);
    state$.user.email.set(email);
    state$.user.avatar.set(avatar);
  }); // Renders once after batch completes
}

When to Use batch() #

Use batch() whenever you update 2 or more observables in sequence:

  1. Selection Functions - Updating selected item + related state
  2. Reset/Clear Operations - Clearing multiple filters or resetting state
  3. Mutation Handlers - After API calls that update multiple observables
  4. Bulk Operations - Processing arrays or multiple items
  5. Entity Switching - Changing entity + related dependent state

Batching in syncedQuery Mutations #

Always batch state updates in mutation handlers:

import { batch } from '@legendapp/state';
import { syncedDataStateQueryObservable } from '@/shared/utils/query';

const items$: Observable<ItemsState> = syncedDataStateQueryObservable(
  TABLE_ITEMS,
  {
    list: getItemsQueryOptions(),
    transform: { list: (data) => data?.data ?? [] },
    persist: true,
    enabledQueryObserver: true,
    mode: 'set',
  },
  {
    mutations: {
      create: {
        mutationFn: async (newItem: ItemInput) => {
          const result = await createItem(newItem);

          // ✅ Batch all post-mutation state updates
          batch(() => {
            state$.db.items.data$.push(result);
            state$.ui.selectedItemId.set(result.id);
            state$.ui.isLoading.set(false);
          });

          return result;
        },
      },
    },
  }
);

Batching with Async Operations #

Important: batch() only works with synchronous code. Batch before and after async operations:

// ❌ WRONG - batch doesn't work across async boundaries
batch(async () => {
  const result = await api.call(); // This breaks the batch
  state$.data.set(result);
});

// ✅ CORRECT - batch synchronous updates separately
async function processData() {
  // Batch: Set loading states
  batch(() => {
    state$.isLoading.set(true);
    state$.error.set(null);
  });

  // Async operation (not batched)
  const result = await api.call();

  // Batch: Update with results
  batch(() => {
    state$.data.set(result);
    state$.isLoading.set(false);
    state$.lastUpdated.set(Date.now());
  });
}

Selection Functions Pattern #

When implementing selection functions that update multiple related observables, always use batch():

import { batch } from '@legendapp/state';

// ✅ GOOD - Batched selection (6 renders → 1 render)
export function selectItemById(itemId: string) {
  batch(() => {
    const item = state$.db.items.data$.get().find(i => i.id === itemId);

    if (!item) {
      state$.ui.selectedItemId.set(null);
      state$.ui.selectedItemPageId.set(null);
      return;
    }

    state$.ui.selectedItemId.set(itemId);
    state$.ui.selectedItemPageId.set(item.pages?.[0]?.id ?? null);
    state$.ui.activeDetailPanel.set('info');
    state$.ui.error.set(null);
  });
}

Reset/Clear Operations Pattern #

Clearing multiple filters or resetting state should always be batched:

import { batch } from '@legendapp/state';

// ✅ GOOD - Batched reset (8 renders → 1 render)
export function resetFilters() {
  batch(() => {
    state$.ui.filters.status.set([]);
    state$.ui.filters.dateRange.set(null);
    state$.ui.filters.searchTerm.set('');
    state$.ui.filters.tags.set([]);
    state$.ui.filters.category.set(null);
    state$.ui.filters.sortBy.set('createdAt');
    state$.ui.filters.sortOrder.set('desc');
    state$.ui.filters.pageSize.set(25);
  });
}

Bulk Array Operations #

When pushing multiple items to an array, batch the operations:

import { batch } from '@legendapp/state';

// ❌ BAD - N renders for N items
function addItems(newItems: Item[]) {
  newItems.forEach(item => {
    state$.items.push(item); // Renders N times
  });
}

// ✅ GOOD - Single render
function addItems(newItems: Item[]) {
  batch(() => {
    newItems.forEach(item => {
      state$.items.push(item);
    });
  }); // Renders once
}

When to Use .get() vs .peek() #

Use .get() when #

  • Inside computed observables (required for dependency tracking)
  • In React components (to trigger re-renders on changes)
  • When you need reactive updates
// ✅ Reactive read in computed observable
const filtered$ = computed(() => {
  const items = state$.db.items.data$.get(); // Tracks dependency
  const searchTerm = state$.ui.searchTerm.get(); // Tracks dependency
  return items.filter(item => item.name.includes(searchTerm));
});

Use .peek() when #

  • Reading values in event handlers (one-time reads)
  • Initialization code (doesn’t need reactivity)
  • Logging/debugging (no subscription needed)
  • Inside batch() when reading values for comparison
  • When you explicitly want to avoid creating a reactive subscription
// ✅ Non-reactive read in event handler
function handleClick() {
  const currentValue = state$.count.peek(); // No subscription
  if (currentValue > 10) {
    doSomething();
  }
}

// ✅ Non-reactive read in batch
function updateIfChanged(newValue: string) {
  batch(() => {
    const current = state$.value.peek(); // No subscription
    if (current !== newValue) {
      state$.value.set(newValue);
      state$.lastModified.set(Date.now());
    }
  });
}

Nested Batching #

Nested batch() calls are safe and will be coalesced into a single update:

function outerFunction() {
  batch(() => {
    state$.a.set(1);
    innerFunction(); // Also uses batch internally
    state$.b.set(2);
  });
}

function innerFunction() {
  batch(() => {
    state$.c.set(3);
    state$.d.set(4);
  });
}
// All 4 updates trigger a single render

Common Pitfalls to Avoid #

  1. Don’t batch single updates - adds unnecessary overhead:
// ❌ Unnecessary
batch(() => state$.count.set(5));

// ✅ Just use set directly
state$.count.set(5);
  1. Don’t forget to batch in mutation handlers - this is where most gains are:
// ❌ Multiple renders after API call
const result = await api.create(data);
state$.items.push(result);
state$.selectedId.set(result.id);

// ✅ Single render
const result = await api.create(data);
batch(() => {
  state$.items.push(result);
  state$.selectedId.set(result.id);
});
  1. Batch at the right level - don’t create multiple small batches when you can create one large batch:
// ❌ Multiple small batches
items.forEach(item => {
  batch(() => {
    state$.items.push(item);
    state$.count.set(prev => prev + 1);
  });
});

// ✅ One large batch
batch(() => {
  items.forEach(item => {
    state$.items.push(item);
    state$.count.set(prev => prev + 1);
  });
});

Performance Impact #

Proper use of batch() can provide:

  • 30-50% reduction in unnecessary re-renders on hot paths
  • 10x faster bulk array operations (100 items: 100 renders → 1 render)
  • 75-90% reduction in renders for complex operations (selection, filtering, resets)

Further Reading #

For more Legend State best practices and patterns, see:

This documentation should be updated as the sync system evolves and new patterns emerge.