TradingAgents/docs/testing/README.md

Testing Overview

TradingAgents uses a comprehensive testing strategy to ensure code quality and reliability.

Testing Philosophy

Our testing approach combines:

• Unit Tests: Fast, isolated tests for individual components
• Integration Tests: Tests for component interactions
• End-to-End Tests: Full workflow validation
• Regression Tests: Prevent fixed bugs from returning

Test Structure

tests/
├── __init__.py                  # Package initialization
├── conftest.py                  # Root-level fixtures and configuration
├── unit/                        # Unit tests (fast, isolated)
│   ├── __init__.py
│   ├── conftest.py              # Unit test specific fixtures
│   ├── test_conftest_hierarchy.py
│   ├── test_documentation_structure.py
│   ├── test_exceptions.py
│   ├── test_logging_config.py
│   └── test_report_exporter.py
├── integration/                 # Integration tests (medium speed)
│   ├── __init__.py
│   ├── conftest.py              # Integration test specific fixtures
│   ├── test_akshare.py
│   ├── test_cli_error_handling.py
│   └── test_openrouter.py
├── e2e/                         # End-to-end tests (slow, complete workflows)
│   ├── __init__.py
│   ├── conftest.py              # E2E-specific fixtures
│   ├── README.md                # E2E testing guidelines
│   └── test_deepseek.py
└── CHROMADB_COLLECTION_TESTS.md # ChromaDB test documentation

Running Tests

All Tests

pytest tests/

Specific Test Categories

# Unit tests only
pytest tests/unit/

# Integration tests only
pytest tests/integration/

# End-to-end tests only
pytest tests/e2e/ -m e2e

With Coverage

pytest tests/ --cov=tradingagents --cov-report=html

Specific Test File

pytest tests/unit/test_analysts.py -v

Specific Test Function

pytest tests/unit/test_analysts.py::test_market_analyst_initialization -v

Test Categories

Unit Tests

Purpose: Test individual functions and classes in isolation

Characteristics:

• Fast (<1 second per test)
• No external dependencies
• Use mocks for LLMs and data vendors
• High coverage target (90%+)

Example:

def test_analyst_initialization():
    """Test analyst can be initialized."""
    llm = Mock()
    tools = []

    analyst = MarketAnalyst(llm, tools)

    assert analyst.name == "market"
    assert analyst.llm == llm

Integration Tests

Purpose: Test component interactions

Characteristics:

• Medium speed (1-30 seconds)
• May use test APIs or mocks
• Validate workflows
• Coverage target (70%+)

Example:

def test_data_vendor_integration():
    """Test data vendor can provide data."""
    interface = DataInterface()
    data = interface.get_stock_data("NVDA", "2024-01-01", "2024-01-10")

    assert "close" in data
    assert len(data["close"]) > 0

End-to-End Tests

Purpose: Test complete workflows from a user's perspective

Characteristics:

• Slow (multiple seconds to minutes)
• Use real or test APIs with realistic data
• Validate complete system integration
• Focus on critical user journeys
• Minimal count (most expensive tests to run)

Location: tests/e2e/

Marker: @pytest.mark.e2e

Example:

import pytest

pytestmark = pytest.mark.e2e

def test_complete_data_workflow(e2e_environment):
    """
    Test complete workflow: data ingestion → analysis → report.

    This test validates the entire user journey from fetching market data
    to generating a trading report.
    """
    # Arrange: Set up data source
    # Act: Execute complete workflow
    # Assert: Validate final report output
    pass

See E2E Testing Guide for detailed guidelines and examples.

Test Fixtures and conftest.py Hierarchy

TradingAgents uses a hierarchical conftest.py structure to organize fixtures by test scope:

Fixture Organization

tests/
├── conftest.py              # Root fixtures - accessible to all tests
│   ├── Environment fixtures (mock_env_openrouter, mock_env_openai, etc.)
│   ├── LangChain mocking (mock_langchain_classes)
│   ├── ChromaDB mocking (mock_chromadb)
│   ├── Memory mocking (mock_memory)
│   ├── Configuration fixtures (sample_config, openrouter_config)
│   └── Temporary directory fixtures (temp_output_dir)
├── unit/conftest.py         # Unit test specific fixtures
│   ├── Data vendor mocking (mock_akshare, mock_yfinance)
│   ├── Sample data (sample_dataframe)
│   ├── Time mocking (mock_time_sleep)
│   ├── HTTP mocking (mock_requests)
│   └── Subprocess mocking (mock_subprocess)
├── integration/conftest.py  # Integration test specific fixtures
│   ├── Live ChromaDB (live_chromadb)
│   └── Integration temp directory (integration_temp_dir)
└── e2e/conftest.py          # End-to-end test specific fixtures
    └── E2E environment setup (e2e_environment)

Root-Level Fixtures (tests/conftest.py)

Available to all tests in any subdirectory:

Environment Variable Fixtures:

• mock_env_openrouter - Sets OPENROUTER_API_KEY, clears others
• mock_env_openai - Sets OPENAI_API_KEY, clears others
• mock_env_anthropic - Sets ANTHROPIC_API_KEY, clears others
• mock_env_google - Sets GOOGLE_API_KEY, clears others
• mock_env_empty - Clears all API keys (for error testing)

Mocking Fixtures:

• mock_langchain_classes - Mocks ChatOpenAI, ChatAnthropic, ChatGoogleGenerativeAI
• mock_chromadb - Mocks ChromaDB Client with get_or_create_collection()
• mock_memory - Mocks FinancialSituationMemory
• mock_openai_client - Mocks OpenAI client with embeddings

Configuration Fixtures:

• sample_config - Default configuration for testing
• openrouter_config - OpenRouter-specific configuration

Utility Fixtures:

• temp_output_dir - Temporary directory for test artifacts

Unit Test Fixtures (tests/unit/conftest.py)

Only available in tests/unit/ directory:

• mock_akshare - Mocks akshare data vendor
• mock_yfinance - Mocks yfinance data vendor
• sample_dataframe - Sample stock data DataFrame
• mock_time_sleep - Mocks time.sleep for retry tests
• mock_requests - Mocks HTTP requests module
• mock_subprocess - Mocks subprocess module

Integration Test Fixtures (tests/integration/conftest.py)

Only available in tests/integration/ directory:

• live_chromadb - Live ChromaDB instance (session-scoped)
• integration_temp_dir - Temporary directory with cleanup

End-to-End Test Fixtures (tests/e2e/conftest.py)

Only available in tests/e2e/ directory:

• e2e_environment - Complete environment setup for end-to-end testing with all dependencies initialized

Using Fixtures

# Root-level fixture available to all tests
def test_openrouter_env(mock_env_openrouter):
    """Test using environment fixture."""
    import os
    assert os.getenv("OPENROUTER_API_KEY") is not None

# Unit-specific fixture only available in tests/unit/
def test_akshare_mock(mock_akshare):
    """Test data vendor mocking."""
    mock_akshare.stock_us_hist.return_value = pd.DataFrame(...)
    # Use the mock

# Integration-specific fixture only available in tests/integration/
def test_chromadb_integration(live_chromadb):
    """Test with real ChromaDB instance."""
    collection = live_chromadb.get_or_create_collection("test")
    assert collection is not None

Fixture Scope and Lifetime

• function (default) - Created fresh for each test
• session - Created once for entire test session (only live_chromadb)
• module - Created once per test file

Environment fixtures use patch.dict() to automatically restore environment after each test.

Writing Tests

See Writing Tests Guide for detailed patterns and examples.

Coverage Goals

• Overall: 80%+
• Unit Tests: 90%+
• Integration Tests: 70%+
• Critical Paths: 100%

Continuous Integration

Tests run automatically on:

• Pull requests
• Pushes to main branch
• Pre-commit hooks (optional)

Best Practices

1. Write Tests First: TDD approach when possible
2. One Assertion: Focus tests on single behaviors
3. Clear Names: test_<function>_<scenario>_<expected>
4. Use Fixtures: DRY principle for setup
5. Mock External Calls: Don't hit real APIs in unit tests
6. Fast Tests: Keep unit tests under 1 second
7. Isolation: Tests should not depend on each other
8. Documentation: Add docstrings to complex tests

See Also

• Running Tests
• Writing Tests
• Test Organization Best Practices