369f8c444b
feat: discovery system code quality improvements and concurrent execution
...
Implement comprehensive code quality improvements and performance optimizations
for the discovery pipeline based on code review findings.
## Key Improvements
### 1. Common Utilities (DRY Principle)
- Created `tradingagents/dataflows/discovery/common_utils.py`
- Extracted ticker parsing logic (eliminates 40+ lines of duplication)
- Centralized stopwords list (71 common non-ticker words)
- Added ReDoS protection (100KB text length limit)
- Provides `validate_candidate_structure()` for output validation
### 2. Scanner Output Validation
- Two-layer validation approach:
- Registration-time: Check scanner class structure
- Runtime: Validate each candidate dictionary
- Added `scan_with_validation()` wrapper in BaseScanner
- Validates required keys: ticker, source, context, priority
- Graceful error handling with structured logging
### 3. Configuration-Driven Design
- Moved magic numbers to `default_config.py`:
- `ticker_universe`: Top 20 liquid options tickers
- `min_volume`: 1000 (options flow threshold)
- `min_transaction_value`: $25,000 (insider buying filter)
- Fixed hardcoded absolute paths to relative paths
- Improved portability across development environments
### 4. Concurrent Scanner Execution (37% Performance Gain)
- Implemented ThreadPoolExecutor for parallel scanner execution
- Configuration: `scanner_execution.concurrent`, `max_workers`, `timeout_seconds`
- Performance: 42s vs 67s (37% faster with 8 scanners)
- Thread-safe state management (each scanner gets copy)
- Per-scanner timeout with graceful degradation
- Error isolation (one failure doesn't stop others)
### 5. Error Handling Improvements
- Changed bare `except:` to `except Exception:` (avoid catching KeyboardInterrupt)
- Added structured logging with `exc_info=True` and extra fields
- Implemented graceful degradation throughout pipeline
## Files Changed
### Core Implementation
- `tradingagents/__init__.py` (NEW) - Package initialization
- `tradingagents/default_config.py` - Scanner execution config, magic numbers
- `tradingagents/graph/discovery_graph.py` - Concurrent execution logic
- `tradingagents/dataflows/discovery/common_utils.py` (NEW) - Shared utilities
- `tradingagents/dataflows/discovery/scanner_registry.py` - Validation wrapper
- `tradingagents/dataflows/discovery/scanners/*.py` - Use common utilities
### Testing & Documentation
- `tests/test_concurrent_scanners.py` (NEW) - Comprehensive test suite
- `verify_concurrent_execution.py` (NEW) - Performance verification
- `CONCURRENT_EXECUTION.md` (NEW) - Implementation documentation
## Test Results
All tests passing (exit code 0):
- ✅ Concurrent execution: 42s, 66-69 candidates
- ✅ Sequential fallback: 56-67s, 65-68 candidates
- ✅ Timeout handling: Graceful degradation with 1s timeout
- ✅ Error isolation: Individual failures don't cascade
## Performance Impact
- Scanner execution: 37% faster (42s vs 67s)
- Time saved: ~25 seconds per discovery run
- At scale: 4+ minutes saved daily in production
- Same candidate quality (65-69 tickers in both modes)
## Breaking Changes
None. Concurrent execution is opt-in via config flag.
Sequential mode remains available as fallback.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:27:01 -08:00
2376fc74a1
Update
2025-12-11 00:23:28 -08:00
ea4ee9176b
Update
2025-12-09 23:16:53 -08:00
8f5e2f6e5e
Merge pull request #1 from Aitous/task/ya/multi-vendors-execution
...
Task/ya/multi vendors execution
2025-12-06 15:44:36 -08:00
ccc78c694b
Update
2025-12-06 15:39:49 -08:00
5cf57e5d97
Update
2025-12-02 20:49:42 -08:00
5128f1ad00
Remove irrelevant test
2025-11-23 17:25:27 -08:00
9080dbc0c0
Add pytest tests for multi-vendor routing logic
...
Tests verify:
- Single vendor stops after first success
- Multi-vendor stops after all primaries (even if they fail)
- Fallback vendors are not attempted when primaries are configured
- Tool-level config overrides category-level config
Tests use pytest with fixtures and mocked vendors, can run without API keys in CI/CD.
Run with: pytest tests/test_multi_vendor_routing.py -v
2025-11-23 17:20:20 -08:00
9ee66746a5
Fix: Stop after all primary vendors in multi-vendor config
...
When multiple primary vendors are configured (e.g., 'reddit,alpha_vantage'),
the system now correctly stops after attempting all primary vendors instead
of continuing through all fallback vendors.
Changes:
- Track which primary vendors have been attempted in a list
- Add stopping condition when all primary vendors are attempted
- Preserve existing single-vendor behavior (stop after first success)
This prevents unnecessary API calls and ensures predictable behavior.
2025-11-23 17:09:15 -08:00
13b826a31d
Merge pull request #245 from TauricResearch/feat/tooloptim
...
Y Finance Tools Optimizations
2025-10-09 00:34:10 -07:00
b2ef960da7
updated readme
2025-10-09 00:32:04 -07:00
a5dcc7da45
update readme
2025-10-06 20:33:12 -07:00
7bb2941b07
optimized yfin fetching to be much faster
2025-10-06 19:58:01 -07:00
32be17c606
Merge pull request #235 from luohy15/data_vendor
...
Add Alpha Vantage API Integration and Refactor Data Provider Architecture
2025-10-05 16:01:30 -07:00
c07dcf026b
added fallbacks for tools
2025-10-03 22:40:09 -07:00
d23fb539e9
minor fix
2025-09-30 13:27:48 +08:00
b01051b9f4
Switch default data vendor
...
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 12:43:27 +08:00
8fdbbcca3d
alpha vantage api key url
2025-09-29 18:22:31 +08:00
86bc0e793f
minor fix
2025-09-27 00:04:59 +08:00
7fc9c28a94
Add environment variable configuration support
...
- Add .env.example file with API key placeholders
- Update README.md with .env file setup instructions
- Add dotenv loading in main.py for environment variables
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:58:51 +08:00
7bcc2cbd8a
Update configuration documentation for Alpha Vantage data vendor
...
Add data vendor configuration examples in README and main.py showing how to configure Alpha Vantage as the primary data provider. Update documentation to reflect the current default behavior of using Alpha Vantage for real-time market data access.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:52:26 +08:00
6211b1132a
Improve Alpha Vantage indicator column parsing with robust mapping
...
- Replace hardcoded column indices with column name lookup
- Add mapping for all supported indicators to their expected CSV column names
- Handle missing columns gracefully with descriptive error messages
- Strip whitespace from header parsing for reliability
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:36:36 +08:00
8b04ec307f
minor fix
2025-09-26 23:25:33 +08:00
0ab323c2c6
Add Alpha Vantage API integration as primary data provider
...
- Replace FinnHub with Alpha Vantage API in README documentation
- Implement comprehensive Alpha Vantage modules:
- Stock data (daily OHLCV with date filtering)
- Technical indicators (SMA, EMA, MACD, RSI, Bollinger Bands, ATR)
- Fundamental data (overview, balance sheet, cashflow, income statement)
- News and sentiment data with insider transactions
- Update news analyst tools to use ticker-based news search
- Integrate Alpha Vantage vendor methods into interface routing
- Maintain backward compatibility with existing vendor system
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 22:57:50 +08:00
a6734d71bc
WIP
2025-09-26 16:17:50 +08:00
a438acdbbd
Merge pull request #89 from Mirza-Samad-Ahmed-Baig/fixes
...
Enhancement: agent reflection, logging improvement
2025-07-03 10:15:39 -04:00
c73e374e7c
Update main.py
2025-07-03 10:14:06 -04:00
f704828f89
Fix: Prevent infinite loops, enable reflection, and improve logging
2025-07-03 17:43:40 +05:00
fda4f664e8
Merge pull request #49 from Zhongyi-Lu/a
...
Exclude `.env` from Git.
2025-07-01 09:17:46 -07:00
718df34932
Merge pull request #29 from ZeroAct/save_results
...
Save results
2025-06-26 00:28:30 -04:00
43aa9c5d09
Local Ollama ( #53 )
...
- Fix typo 'Start' 'End'
- Add llama3.1 selection
- Use 'quick_think_llm' model instead of hard-coding GPT
2025-06-26 00:27:01 -04:00
26c5ba5a78
Revert "Docker support and Ollama support ( #47 )" ( #57 )
...
This reverts commit 78ea029a0b
2025-06-26 00:07:58 -04:00
78ea029a0b
Docker support and Ollama support ( #47 )
...
- Added support for running CLI and Ollama server via Docker
- Introduced tests for local embeddings model and standalone Docker setup
- Enabled conditional Ollama server launch via LLM_PROVIDER
2025-06-25 23:57:05 -04:00
ee3d499894
Merge branch 'TauricResearch:main' into save_results
2025-06-25 08:43:19 +09:00
7abff0f354
Merge pull request #46 from AtharvSabde/patch-2
...
Updated requirements.txt based on latest commit
2025-06-23 20:40:58 -04:00
b575bd0941
Merge pull request #52 from TauricResearch/dev
...
Merge dev into main. Add support for Anthropic and OpenRouter.
2025-06-23 20:38:14 -04:00
b8f712b170
Exclude `.env` from Git
2025-06-21 23:29:26 -07:00
52284ce13c
fixed anthropic support. Anthropic has different format of response when it has tool calls. Explicit handling added
2025-06-21 12:51:34 -07:00
11804f88ff
Updated requirements.txt based on latest commit
...
PULL REQUEST: Add support for other backends, such as OpenRouter and Ollama
it had two requirments missing. added those
2025-06-20 15:58:22 +05:30
1e86e74314
Merge pull request #40 from RealMyth21/main
...
Updated README.md: Swap Trader and Management order.
2025-06-19 15:10:36 -04:00
c2f897fc67
Merge pull request #43 from AtharvSabde/patch-1
...
fundamentals_analyst.py (spelling mistake in instruction: Makrdown -> Markdown)
2025-06-19 15:05:08 -04:00
ed32081f57
Merge pull request #44 from TauricResearch/dev
...
Merge dev into main branch
2025-06-19 15:00:07 -04:00
2af7ef3d79
fundamentals_analyst.py(spelling mistake.markdown)
2025-06-19 21:48:16 +05:30
383deb72aa
Updated README.md
...
The diagrams were switched, so I fixed it.
2025-06-18 19:08:10 -07:00
7eaf4d995f
update clear msg bc anthropic needs at least 1 msg in chat call
2025-06-15 23:14:47 -07:00
da84ef43aa
main works, cli bugs
2025-06-15 22:20:59 -07:00
90b23e72f5
Merge pull request #25 from maxer137/main
...
Add support for other backends, such as OpenRouter and Ollama
2025-06-15 16:06:20 -07:00
417b09712c
refactor
2025-06-12 13:53:28 +09:00
570644d939
Fix ticker hardcoding in prompt ( #28 )
2025-06-11 19:43:39 -07:00
9647359246
save reports & logs under results_dir
2025-06-12 11:25:07 +09:00
Youssef Aitousarrah
Aitous
Edward Sun
Edward Sun
Yijia Xiao
luohy15
mirza-samad-ahmed-baig
Max Wong
Geeta Chauhan
Huijae Lee
Zhongyi Lu
Atharv Sabde
Mithil Srungarapu
saksham0161