5.1 KiB
5.1 KiB
Running Tests
Complete guide for running the TradingAgents test suite.
Prerequisites
Install test dependencies:
pip install -r requirements.txt
pytest --version # Verify pytest is installed
Basic Usage
Run All Tests
pytest tests/
Run with Verbose Output
pytest tests/ -v
Run with Coverage
pytest tests/ --cov=tradingagents --cov-report=html
View coverage report:
open htmlcov/index.html # macOS
xdg-open htmlcov/index.html # Linux
start htmlcov/index.html # Windows
Test Selection
By Directory
# Unit tests only
pytest tests/unit/
# Integration tests only
pytest tests/integration/
# Regression tests only
pytest tests/regression/
By File
pytest tests/unit/test_analysts.py
By Test Function
pytest tests/unit/test_analysts.py::test_market_analyst_initialization
By Test Class
pytest tests/unit/test_analysts.py::TestMarketAnalyst
By Pattern
# Run all tests matching pattern
pytest -k "analyst"
# Run tests NOT matching pattern
pytest -k "not integration"
# Multiple patterns
pytest -k "analyst and not integration"
By Markers
# Smoke tests (critical path)
pytest -m smoke
# Integration tests
pytest -m integration
# Skip slow tests
pytest -m "not slow"
Output Options
Minimal Output
pytest tests/ -q
Show All Output
pytest tests/ -v -s
Show Only Failures
pytest tests/ --tb=short
Show Failed Tests First
pytest tests/ --failed-first
Stop on First Failure
pytest tests/ -x
Stop After N Failures
pytest tests/ --maxfail=3
Parallel Execution
Run tests in parallel for faster execution:
# Install pytest-xdist
pip install pytest-xdist
# Run with 4 workers
pytest tests/ -n 4
# Run with auto-detect workers
pytest tests/ -n auto
Test Coverage
Generate Coverage Report
pytest tests/ --cov=tradingagents --cov-report=term-missing
Coverage with HTML Report
pytest tests/ --cov=tradingagents --cov-report=html
Coverage for Specific Module
pytest tests/ --cov=tradingagents.agents --cov-report=term
Minimum Coverage Threshold
pytest tests/ --cov=tradingagents --cov-fail-under=80
Debugging Tests
Run with Python Debugger
pytest tests/ --pdb
Drop into Debugger on Failure
pytest tests/ --pdb -x
Show Local Variables on Failure
pytest tests/ -l
Show Print Statements
pytest tests/ -s
Environment Setup
Set Environment Variables
# For single test run
OPENAI_API_KEY=test_key pytest tests/
# Or create .env.test
cat > .env.test <<EOF
OPENAI_API_KEY=test_key
ALPHA_VANTAGE_API_KEY=test_key
EOF
# Load in tests
pytest tests/
Skip Tests Requiring API Keys
# Skip integration tests
pytest tests/ -m "not integration"
Continuous Integration
Pre-Commit Testing
Create .git/hooks/pre-commit:
#!/bin/bash
pytest tests/regression/smoke/ --maxfail=1 || exit 1
Make executable:
chmod +x .git/hooks/pre-commit
GitHub Actions
Tests run automatically on push/PR via .github/workflows/tests.yml
Test Filtering
By Test Tier
# Smoke tests only (fastest)
pytest -m smoke
# Regression tests
pytest -m regression
# Unit tests
pytest tests/unit/
Skip Slow Tests
pytest -m "not slow"
Skip External API Tests
pytest -m "not api"
Common Workflows
Quick Check
# Fast smoke tests
pytest -m smoke -v
Pre-Commit
# Critical path tests
pytest tests/regression/smoke/ -v
Full Validation
# All tests with coverage
pytest tests/ --cov=tradingagents --cov-report=html -v
Debugging Failure
# Run specific test with debugger
pytest tests/unit/test_analysts.py::test_failing_test --pdb -s
Performance
Show Slowest Tests
pytest tests/ --durations=10
Show All Test Durations
pytest tests/ --durations=0
Output Formats
JUnit XML (for CI)
pytest tests/ --junit-xml=test-results.xml
JSON Report
pip install pytest-json-report
pytest tests/ --json-report --json-report-file=report.json
Troubleshooting
Tests Not Found
Issue: ERROR: file or directory not found: tests/
Solution: Run from project root
cd /path/to/TradingAgents
pytest tests/
Import Errors
Issue: ModuleNotFoundError: No module named 'tradingagents'
Solution: Install package in editable mode
pip install -e .
Fixture Not Found
Issue: fixture 'mock_llm' not found
Solution: Check conftest.py is in test directory
Slow Tests
Issue: Tests taking too long
Solution: Run specific categories or use parallel execution
pytest tests/unit/ -n auto