Commit Graph

132 Commits

Author SHA1 Message Date
Youssef Aitousarrah 6f202f88f2 docs: add volume analysis enhancements design document
Complete design for transforming get_unusual_volume into sophisticated
multi-signal analysis tool with:
- Volume pattern analysis (accumulation, compression, distribution)
- Sector-relative comparison (percentile ranking vs peers)
- Price-volume divergence detection (bullish/bearish signals)

Includes architecture, implementation details, testing strategy,
and performance considerations. Estimated 30-40% signal quality
improvement with 2-3x execution time trade-off.

Phased implementation approach:
Phase 1: Pattern analysis (3-4h)
Phase 2: Divergence detection (4-5h)
Phase 3: Sector comparison (5-6h)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 08:38:06 -08:00
Youssef Aitousarrah f1178b4a57 refactor: organize discovery config into dedicated filter/enrichment sections
- Created nested "filters" section for all filter-stage settings
  (min_average_volume, same-day movers, recent movers, etc.)
- Created nested "enrichment" section for batch news settings
- Updated CandidateFilter to read from new nested structure
- Added backward compatibility fallback for old flat config
- Improved config organization and clarity

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 08:22:39 -08:00
Youssef Aitousarrah 41e91e72d1 fix: load scanners in __init__ to survive linter auto-fix
Final fix for scanner registration issue. Previous attempts
to add scanner import at module level were removed by the
pre-commit hook's ruff --fix auto-formatter.

Solution:
- Import scanners inside DiscoveryGraph.__init__() method
- Use the import (assign to _) so it's not "unused"
- Linter won't remove imports that are actually used

This ensures scanners always load when DiscoveryGraph is instantiated.

Verified: 8 scanners now properly registered

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:47:26 -08:00
Youssef Aitousarrah f6943e1615 fix: add noqa comment to prevent linter from removing scanner import
The scanner import needs # noqa: F401 to prevent linters from
removing it as "unused". The import is required for side effects
(triggering scanner registration).

Without this:
- Pre-commit hook removes the import
- Scanners don't register
- Discovery returns 0 candidates

Fix:
- Added # noqa: F401 comment to scanner import
- Linter will now preserve this import
- Verified 8 scanners properly registered

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:46:27 -08:00
Youssef Aitousarrah 1d52211383 fix: restore missing scanner import causing 0 recommendations
Critical bugfix: Scanner modules weren't being imported, causing
SCANNER_REGISTRY to remain empty and discovery to return 0 candidates.

Root Cause:
- Import line "from tradingagents.dataflows.discovery import scanners"
  was accidentally removed during concurrent execution refactoring
- Without this import, scanner @register() decorators never execute
- Result: SCANNER_REGISTRY.get_all_scanners() returns empty list

Fix:
- Restored scanner import in discovery_graph.py line 6
- Scanners now properly register on module import
- Verified 8 scanners now registered and working

Impact:
- Before: 0 candidates, 0 recommendations
- After: 60-70 candidates, 15 recommendations (normal operation)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:39:20 -08:00
Youssef Aitousarrah 369f8c444b feat: discovery system code quality improvements and concurrent execution
Implement comprehensive code quality improvements and performance optimizations
for the discovery pipeline based on code review findings.

## Key Improvements

### 1. Common Utilities (DRY Principle)
- Created `tradingagents/dataflows/discovery/common_utils.py`
- Extracted ticker parsing logic (eliminates 40+ lines of duplication)
- Centralized stopwords list (71 common non-ticker words)
- Added ReDoS protection (100KB text length limit)
- Provides `validate_candidate_structure()` for output validation

### 2. Scanner Output Validation
- Two-layer validation approach:
  - Registration-time: Check scanner class structure
  - Runtime: Validate each candidate dictionary
- Added `scan_with_validation()` wrapper in BaseScanner
- Validates required keys: ticker, source, context, priority
- Graceful error handling with structured logging

### 3. Configuration-Driven Design
- Moved magic numbers to `default_config.py`:
  - `ticker_universe`: Top 20 liquid options tickers
  - `min_volume`: 1000 (options flow threshold)
  - `min_transaction_value`: $25,000 (insider buying filter)
- Fixed hardcoded absolute paths to relative paths
- Improved portability across development environments

### 4. Concurrent Scanner Execution (37% Performance Gain)
- Implemented ThreadPoolExecutor for parallel scanner execution
- Configuration: `scanner_execution.concurrent`, `max_workers`, `timeout_seconds`
- Performance: 42s vs 67s (37% faster with 8 scanners)
- Thread-safe state management (each scanner gets copy)
- Per-scanner timeout with graceful degradation
- Error isolation (one failure doesn't stop others)

### 5. Error Handling Improvements
- Changed bare `except:` to `except Exception:` (avoid catching KeyboardInterrupt)
- Added structured logging with `exc_info=True` and extra fields
- Implemented graceful degradation throughout pipeline

## Files Changed

### Core Implementation
- `tradingagents/__init__.py` (NEW) - Package initialization
- `tradingagents/default_config.py` - Scanner execution config, magic numbers
- `tradingagents/graph/discovery_graph.py` - Concurrent execution logic
- `tradingagents/dataflows/discovery/common_utils.py` (NEW) - Shared utilities
- `tradingagents/dataflows/discovery/scanner_registry.py` - Validation wrapper
- `tradingagents/dataflows/discovery/scanners/*.py` - Use common utilities

### Testing & Documentation
- `tests/test_concurrent_scanners.py` (NEW) - Comprehensive test suite
- `verify_concurrent_execution.py` (NEW) - Performance verification
- `CONCURRENT_EXECUTION.md` (NEW) - Implementation documentation

## Test Results

All tests passing (exit code 0):
-  Concurrent execution: 42s, 66-69 candidates
-  Sequential fallback: 56-67s, 65-68 candidates
-  Timeout handling: Graceful degradation with 1s timeout
-  Error isolation: Individual failures don't cascade

## Performance Impact

- Scanner execution: 37% faster (42s vs 67s)
- Time saved: ~25 seconds per discovery run
- At scale: 4+ minutes saved daily in production
- Same candidate quality (65-69 tickers in both modes)

## Breaking Changes

None. Concurrent execution is opt-in via config flag.
Sequential mode remains available as fallback.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:27:01 -08:00
Youssef Aitousarrah 2376fc74a1 Update 2025-12-11 00:23:28 -08:00
Youssef Aitousarrah ea4ee9176b Update 2025-12-09 23:16:53 -08:00
Aitous 8f5e2f6e5e
Merge pull request #1 from Aitous/task/ya/multi-vendors-execution
Task/ya/multi vendors execution
2025-12-06 15:44:36 -08:00
Youssef Aitousarrah ccc78c694b Update 2025-12-06 15:39:49 -08:00
Youssef Aitousarrah 5cf57e5d97 Update 2025-12-02 20:49:42 -08:00
Youssef Aitousarrah 5128f1ad00 Remove irrelevant test 2025-11-23 17:25:27 -08:00
Youssef Aitousarrah 9080dbc0c0 Add pytest tests for multi-vendor routing logic
Tests verify:
- Single vendor stops after first success
- Multi-vendor stops after all primaries (even if they fail)
- Fallback vendors are not attempted when primaries are configured
- Tool-level config overrides category-level config

Tests use pytest with fixtures and mocked vendors, can run without API keys in CI/CD.

Run with: pytest tests/test_multi_vendor_routing.py -v
2025-11-23 17:20:20 -08:00
Youssef Aitousarrah 9ee66746a5 Fix: Stop after all primary vendors in multi-vendor config
When multiple primary vendors are configured (e.g., 'reddit,alpha_vantage'),
the system now correctly stops after attempting all primary vendors instead
of continuing through all fallback vendors.

Changes:
- Track which primary vendors have been attempted in a list
- Add stopping condition when all primary vendors are attempted
- Preserve existing single-vendor behavior (stop after first success)

This prevents unnecessary API calls and ensures predictable behavior.
2025-11-23 17:09:15 -08:00
Edward Sun 13b826a31d
Merge pull request #245 from TauricResearch/feat/tooloptim
Y Finance Tools Optimizations
2025-10-09 00:34:10 -07:00
Edward Sun b2ef960da7 updated readme 2025-10-09 00:32:04 -07:00
Edward Sun a5dcc7da45 update readme 2025-10-06 20:33:12 -07:00
Edward Sun 7bb2941b07 optimized yfin fetching to be much faster 2025-10-06 19:58:01 -07:00
Yijia Xiao 32be17c606
Merge pull request #235 from luohy15/data_vendor
Add Alpha Vantage API Integration and Refactor Data Provider Architecture
2025-10-05 16:01:30 -07:00
Edward Sun c07dcf026b added fallbacks for tools 2025-10-03 22:40:09 -07:00
luohy15 d23fb539e9 minor fix 2025-09-30 13:27:48 +08:00
luohy15 b01051b9f4 Switch default data vendor
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 12:43:27 +08:00
luohy15 8fdbbcca3d alpha vantage api key url 2025-09-29 18:22:31 +08:00
luohy15 86bc0e793f minor fix 2025-09-27 00:04:59 +08:00
luohy15 7fc9c28a94 Add environment variable configuration support
- Add .env.example file with API key placeholders
- Update README.md with .env file setup instructions
- Add dotenv loading in main.py for environment variables

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:58:51 +08:00
luohy15 7bcc2cbd8a Update configuration documentation for Alpha Vantage data vendor
Add data vendor configuration examples in README and main.py showing how to configure Alpha Vantage as the primary data provider. Update documentation to reflect the current default behavior of using Alpha Vantage for real-time market data access.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:52:26 +08:00
luohy15 6211b1132a Improve Alpha Vantage indicator column parsing with robust mapping
- Replace hardcoded column indices with column name lookup
- Add mapping for all supported indicators to their expected CSV column names
- Handle missing columns gracefully with descriptive error messages
- Strip whitespace from header parsing for reliability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:36:36 +08:00
luohy15 8b04ec307f minor fix 2025-09-26 23:25:33 +08:00
luohy15 0ab323c2c6 Add Alpha Vantage API integration as primary data provider
- Replace FinnHub with Alpha Vantage API in README documentation
- Implement comprehensive Alpha Vantage modules:
  - Stock data (daily OHLCV with date filtering)
  - Technical indicators (SMA, EMA, MACD, RSI, Bollinger Bands, ATR)
  - Fundamental data (overview, balance sheet, cashflow, income statement)
  - News and sentiment data with insider transactions
- Update news analyst tools to use ticker-based news search
- Integrate Alpha Vantage vendor methods into interface routing
- Maintain backward compatibility with existing vendor system

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 22:57:50 +08:00
luohy15 a6734d71bc WIP 2025-09-26 16:17:50 +08:00
Yijia Xiao a438acdbbd
Merge pull request #89 from Mirza-Samad-Ahmed-Baig/fixes
Enhancement: agent reflection, logging improvement
2025-07-03 10:15:39 -04:00
Yijia Xiao c73e374e7c
Update main.py 2025-07-03 10:14:06 -04:00
mirza-samad-ahmed-baig f704828f89 Fix: Prevent infinite loops, enable reflection, and improve logging 2025-07-03 17:43:40 +05:00
Edward Sun fda4f664e8
Merge pull request #49 from Zhongyi-Lu/a
Exclude `.env` from Git.
2025-07-01 09:17:46 -07:00
Yijia Xiao 718df34932
Merge pull request #29 from ZeroAct/save_results
Save results
2025-06-26 00:28:30 -04:00
Max Wong 43aa9c5d09
Local Ollama (#53)
- Fix typo 'Start' 'End'
- Add llama3.1 selection
- Use 'quick_think_llm' model instead of hard-coding GPT
2025-06-26 00:27:01 -04:00
Yijia Xiao 26c5ba5a78
Revert "Docker support and Ollama support (#47)" (#57)
This reverts commit 78ea029a0b.
2025-06-26 00:07:58 -04:00
Geeta Chauhan 78ea029a0b
Docker support and Ollama support (#47)
- Added support for running CLI and Ollama server via Docker
- Introduced tests for local embeddings model and standalone Docker setup
- Enabled conditional Ollama server launch via LLM_PROVIDER
2025-06-25 23:57:05 -04:00
Huijae Lee ee3d499894
Merge branch 'TauricResearch:main' into save_results 2025-06-25 08:43:19 +09:00
Yijia Xiao 7abff0f354
Merge pull request #46 from AtharvSabde/patch-2
Updated requirements.txt based on latest commit
2025-06-23 20:40:58 -04:00
Yijia Xiao b575bd0941
Merge pull request #52 from TauricResearch/dev
Merge dev into main. Add support for Anthropic and OpenRouter.
2025-06-23 20:38:14 -04:00
Zhongyi Lu b8f712b170 Exclude `.env` from Git 2025-06-21 23:29:26 -07:00
Edward Sun 52284ce13c fixed anthropic support. Anthropic has different format of response when it has tool calls. Explicit handling added 2025-06-21 12:51:34 -07:00
Atharv Sabde 11804f88ff
Updated requirements.txt based on latest commit
PULL REQUEST: Add support for other backends, such as OpenRouter and Ollama

it had two requirments missing. added those
2025-06-20 15:58:22 +05:30
Yijia Xiao 1e86e74314
Merge pull request #40 from RealMyth21/main
Updated README.md: Swap Trader and Management order.
2025-06-19 15:10:36 -04:00
Yijia Xiao c2f897fc67
Merge pull request #43 from AtharvSabde/patch-1
fundamentals_analyst.py (spelling mistake in instruction: Makrdown -> Markdown)
2025-06-19 15:05:08 -04:00
Yijia Xiao ed32081f57
Merge pull request #44 from TauricResearch/dev
Merge dev into main branch
2025-06-19 15:00:07 -04:00
Atharv Sabde 2af7ef3d79
fundamentals_analyst.py(spelling mistake.markdown) 2025-06-19 21:48:16 +05:30
Mithil Srungarapu 383deb72aa
Updated README.md
The diagrams were switched, so I fixed it.
2025-06-18 19:08:10 -07:00
Edward Sun 7eaf4d995f update clear msg bc anthropic needs at least 1 msg in chat call 2025-06-15 23:14:47 -07:00