#Testing Patterns Observed in OpenClaw

This document captures observations about OpenClaw's testing framework, file organization, and test execution patterns based on analysis of the codebase. These patterns ensure tests integrate seamlessly with CI.

#Testing Framework: Vitest

OpenClaw uses Vitest as its testing framework with the V8 coverage provider.

#Why Vitest

• Fast test execution with native ESM support
• Compatible with Vite ecosystem
• Built-in coverage with V8 (no Istanbul overhead)
• Native TypeScript support

#Vitest Configuration

Configuration is in vitest.config.ts at the repository root.

#Key Configuration Settings

{
  test: {
    pool: "forks",           // Use forks, not threads
    poolOptions: {
      forks: {
        maxForks: process.env.CI ? 2 : Math.min(os.cpus().length, 16),
        minForks: process.env.CI ? 2 : 1
      }
    },
    testTimeout: 120000,     // 120s for tests
    hookTimeout: process.env.CI ? 180000 : 120000,
    unstubEnvs: true,        // Don't stub environment variables
    unstubGlobals: true      // Don't stub globals
  }
}

#Pool Configuration: Forks, Not Threads

• pool: "forks" - Each test file runs in a separate process
• Max workers: CI uses 2-3 workers, local development caps at 16 workers
• Why forks? - Better isolation, avoids shared memory issues with native modules

#Timeouts

• Test timeout: 120 seconds (2 minutes)
• Hook timeout: 120 seconds local, 180 seconds CI
• Long timeouts accommodate integration tests and gateway operations

#Environment Stubbing

• unstubEnvs: true - Tests see real environment variables
• unstubGlobals: true - Tests see real global objects
• Ensures tests run in realistic conditions

#Coverage Configuration

#Coverage Thresholds

{
  coverage: {
    provider: "v8",
    thresholds: {
      lines: 70,
      functions: 70,
      branches: 55,
      statements: 70
    }
  }
}

#Coverage Requirements

• 70% lines, functions, and statements - Must cover most code paths
• 55% branches - Lower threshold for conditional logic
• V8 provider - Native coverage, no instrumentation overhead

#Test File Patterns

#Colocated Tests (Standard Pattern)

Test files live alongside source files, not in a separate test/ directory.

Pattern: src/**/*.test.ts

Example structure:

src/
  gateway/
    gateway.ts
    gateway.test.ts          # Colocated test
  skills/
    skill-executor.ts
    skill-executor.test.ts   # Colocated test

Why colocated? - Easier to find related tests, better encapsulation

#E2E Tests

Pattern: *.e2e.test.ts

E2E tests verify end-to-end workflows (e.g., full gateway request cycle).

Example: src/gateway/gateway.e2e.test.ts

#Live Tests

Pattern: *.live.test.ts

Live tests require external services (e.g., real Anthropic API).

Running: LIVE=1 pnpm test or pnpm test:live

Example: src/agents/anthropic.live.test.ts

#Test Structure

#Standard Imports

import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";

All test utilities from Vitest - No need for separate assertion library

#Hoisted Mocks

Use vi.hoisted() for mocks that need to be defined before imports.

Pattern observed:

const mockLogger = vi.hoisted(() => ({
  info: vi.fn(),
  error: vi.fn(),
  warn: vi.fn()
}));

vi.mock("../utils/logger.js", () => mockLogger);

#Module Mocks

Use vi.mock() to replace entire modules.

Pattern observed:

vi.mock("../config/loader.js", () => ({
  loadConfig: vi.fn().mockResolvedValue({ tenantId: "test-tenant" })
}));

#Test Harnesses and Helpers

Complex setups use dedicated harness files.

Patterns observed:

• *.test-harness.ts - Complex test fixtures and setup
• *.test-helpers.ts - Reusable test utilities

Example: src/gateway/gateway.test-harness.ts provides mock gateway context

#Setup File

Global test setup is in test/setup.ts.

#Key Setup Function: withIsolatedTestHome()

Purpose: Creates isolated home directory for each test

Pattern observed:

beforeEach(() => {
  withIsolatedTestHome();
});

Why? - Prevents test interference via shared config files

#Running Tests

#Standard Commands

# Run all tests
pnpm test

# Run with coverage
pnpm test:coverage

# Run live tests (requires LIVE=1)
pnpm test:live

# Run specific test file
pnpm test src/gateway/gateway.test.ts

#Low-Memory Testing

For CI or resource-constrained environments:

OPENCLAW_TEST_PROFILE=low OPENCLAW_TEST_SERIAL_GATEWAY=1 pnpm test

What this does:

• OPENCLAW_TEST_PROFILE=low - Reduces parallel workers
• OPENCLAW_TEST_SERIAL_GATEWAY=1 - Runs gateway tests serially (not parallel)

#Common Test Patterns

#Describe Blocks for Organization

describe("Gateway", () => {
  describe("request handling", () => {
    it("should handle valid requests", () => {
      // test
    });

    it("should reject invalid requests", () => {
      // test
    });
  });
});

#Async Tests with await

it("should execute skill", async () => {
  const result = await executeSkill("test-skill", {});
  expect(result).toBeDefined();
});

#Cleanup with afterEach

afterEach(() => {
  vi.clearAllMocks();
});

Why? - Prevents mock state from leaking between tests

#Test Isolation Best Practices

#Always Reset Mocks

Use vi.clearAllMocks() or vi.resetAllMocks() in afterEach blocks.

#Use Isolated Test Home

Always call withIsolatedTestHome() in beforeEach for tests that touch config files.

#Avoid Shared State

Don't rely on test execution order. Each test should be independent.

#CI Considerations

#Parallel Execution

Tests run in parallel by default. Use OPENCLAW_TEST_SERIAL_GATEWAY=1 if needed.

#Coverage Enforcement

CI fails if coverage thresholds are not met (70% lines, 55% branches).

#Timeout Failures

If tests timeout in CI but pass locally, check:

• Hook timeout settings (180s in CI vs 120s locally)
• Network latency for integration tests
• Resource contention (use low-memory profile)

#Why This Matters

Following these patterns ensures:

1. CI passes - Tests run reliably in CI environment
2. Isolation - Tests don't interfere with each other
3. Coverage - Code meets quality thresholds
4. Maintainability - Colocated tests are easy to find and update
5. Performance - Fork pool and V8 coverage provide fast execution

#Cross-References

• Code Style Conventions: TypeScript and linting rules for test files
• Test Failures Troubleshooting (planned): Common test issues and solutions
• Gateway Architecture: Context for gateway integration tests