Commit Graph

51 Commits

Author SHA1 Message Date
Youssef Aitousarrah ec8309a34e Update 2026-02-20 08:38:15 -08:00
Youssef Aitousarrah 1c20dc8c90 feat: improve all 9 scanners and add 3 new scanners
Phase 1 - Fix existing scanners:
- Options flow: apply min_premium filter, scan 3 expirations
- Volume accumulation: distinguish accumulation vs distribution
- Reddit DD: use LLM quality score for priority (skip <60)
- Reddit trending: add mention counts, scale priority by volume
- Semantic news: include headlines, add catalyst classification
- Earnings calendar: add pre-earnings accumulation + EPS estimates
- Market movers: add price ($5) and volume (500K) validation
- ML signal: raise min_win_prob from 35% to 50%

Phase 2 - New scanners:
- Analyst upgrades: monitors rating changes via Alpha Vantage
- Technical breakout: volume-confirmed breakouts above 20d high
- Sector rotation: finds laggards in accelerating sectors

All 12 scanners register with valid Strategy enum values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 08:36:18 -08:00
Youssef Aitousarrah 573b756b4b fix(insider-buying): preserve transaction details, add cluster detection and smart priority
- Call get_finviz_insider_buying with return_structured=True and deduplicate=False
  to get all raw transaction dicts instead of parsing markdown
- Group transactions by ticker for cluster detection (2+ unique insiders = CRITICAL)
- Smart priority: CEO/CFO + >$100K = CRITICAL, director + >$50K = HIGH, etc.
- Preserve insider_name, insider_title, transaction_value, num_insiders_buying in output
- Rich context strings: "CEO John Smith purchased $250K of AAPL shares"
- Update finviz_scraper alias to pass through return_structured and deduplicate params

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 08:36:18 -08:00
Youssef Aitousarrah 74e8703d60 Add daily price movement to charts and fix intraday data rendering
- Add daily price movement display with color coding (green/red)
- Add 1D (intraday) and 7D chart options with granular data:
  - 1D: 5-minute interval for detailed intraday view
  - 7D: hourly interval for smooth 7-day chart
- Fix discontinuous chart rendering by plotting against sequential index for intraday data
- Eliminate overnight/weekend gaps in hourly charts
- Add timezone normalization for consistent date handling between daily and intraday data
- Improve fallback logic when data is sparse
- Better handling of yfinance column names (Datetime vs Date)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-02-17 12:55:12 -08:00
Youssef Aitousarrah 22433d6b7d Update 2026-02-17 12:12:20 -08:00
Youssef Aitousarrah fd951be8bc Update 2026-02-17 12:07:07 -08:00
Youssef Aitousarrah 457d650e42 Update 2026-02-17 10:27:13 -08:00
Youssef Aitousarrah aef83cd6e0 Update the price chart 2026-02-16 22:41:54 -08:00
Youssef Aitousarrah 6831339b78 Remore unused code and improve the UI 2026-02-16 14:17:43 -08:00
Youssef Aitousarrah 8d3205043e Update 2026-02-16 14:17:41 -08:00
Youssef Aitousarrah f4aceef857 feat: add daily discovery workflow, recommendation history, and scanner improvements
- Add GitHub Actions workflow for daily discovery (8:30 AM ET, weekdays)
- Add headless run_daily_discovery.py script for scheduling
- Expand options_flow scanner to use tickers.txt with parallel execution
- Add recommendation history section to Performance page with filters and charts
- Fix strategy name normalization (momentum/Momentum/Momentum-Hype → momentum)
- Fix strategy metrics to count all recs, not just evaluated ones
- Add error handling to Streamlit page rendering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 22:07:02 -08:00
Youssef Aitousarrah 1ead4d9638 feat: add theme module and fix Streamlit Cloud deployment
- Add tradingagents/ui/theme.py (design system: colors, CSS, Plotly templates)
- Add .streamlit/config.toml for dark theme configuration
- Fix Plotly duplicate keyword args in performance.py and todays_picks.py
- Replace deprecated use_container_width with width="stretch" (Streamlit 1.54+)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 22:40:08 -08:00
Youssef Aitousarrah 8ebb42114d Add recommendations folder so that the UI can display it 4 2026-02-10 22:28:52 -08:00
Youssef Aitousarrah cb5ae49501 chore: linter formatting + ML scanner logging, prompt control, ranker reasoning
- Add ML signal scanner results table logging
- Add log_prompts_console config flag for prompt visibility control
- Expand ranker investment thesis to 4-6 sentence structured reasoning
- Linter auto-formatting across modified files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 23:04:38 -08:00
Youssef Aitousarrah 43bdd6de11 feat: discovery pipeline enhancements with ML signal scanner
Major additions:
- ML win probability scanner: scans ticker universe using trained
  LightGBM/TabPFN model, surfaces candidates with P(WIN) above threshold
- 30-feature engineering pipeline (20 base + 10 interaction features)
  computed from OHLCV data via stockstats + pandas
- Triple-barrier labeling for training data generation
- Dataset builder and training script with calibration analysis
- Discovery enrichment: confluence scoring, short interest extraction,
  earnings estimates, options signal normalization, quant pre-score
- Configurable prompt logging (log_prompts_console flag)
- Enhanced ranker investment thesis (4-6 sentence reasoning)
- Typed DiscoveryConfig dataclass for all discovery settings
- Console price charts for visual ticker analysis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 22:53:42 -08:00
Youssef Aitousarrah f1178b4a57 refactor: organize discovery config into dedicated filter/enrichment sections
- Created nested "filters" section for all filter-stage settings
  (min_average_volume, same-day movers, recent movers, etc.)
- Created nested "enrichment" section for batch news settings
- Updated CandidateFilter to read from new nested structure
- Added backward compatibility fallback for old flat config
- Improved config organization and clarity

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-06 08:22:39 -08:00
Youssef Aitousarrah 41e91e72d1 fix: load scanners in __init__ to survive linter auto-fix
Final fix for scanner registration issue. Previous attempts
to add scanner import at module level were removed by the
pre-commit hook's ruff --fix auto-formatter.

Solution:
- Import scanners inside DiscoveryGraph.__init__() method
- Use the import (assign to _) so it's not "unused"
- Linter won't remove imports that are actually used

This ensures scanners always load when DiscoveryGraph is instantiated.

Verified: 8 scanners now properly registered

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:47:26 -08:00
Youssef Aitousarrah f6943e1615 fix: add noqa comment to prevent linter from removing scanner import
The scanner import needs # noqa: F401 to prevent linters from
removing it as "unused". The import is required for side effects
(triggering scanner registration).

Without this:
- Pre-commit hook removes the import
- Scanners don't register
- Discovery returns 0 candidates

Fix:
- Added # noqa: F401 comment to scanner import
- Linter will now preserve this import
- Verified 8 scanners properly registered

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:46:27 -08:00
Youssef Aitousarrah 1d52211383 fix: restore missing scanner import causing 0 recommendations
Critical bugfix: Scanner modules weren't being imported, causing
SCANNER_REGISTRY to remain empty and discovery to return 0 candidates.

Root Cause:
- Import line "from tradingagents.dataflows.discovery import scanners"
  was accidentally removed during concurrent execution refactoring
- Without this import, scanner @register() decorators never execute
- Result: SCANNER_REGISTRY.get_all_scanners() returns empty list

Fix:
- Restored scanner import in discovery_graph.py line 6
- Scanners now properly register on module import
- Verified 8 scanners now registered and working

Impact:
- Before: 0 candidates, 0 recommendations
- After: 60-70 candidates, 15 recommendations (normal operation)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:39:20 -08:00
Youssef Aitousarrah 369f8c444b feat: discovery system code quality improvements and concurrent execution
Implement comprehensive code quality improvements and performance optimizations
for the discovery pipeline based on code review findings.

## Key Improvements

### 1. Common Utilities (DRY Principle)
- Created `tradingagents/dataflows/discovery/common_utils.py`
- Extracted ticker parsing logic (eliminates 40+ lines of duplication)
- Centralized stopwords list (71 common non-ticker words)
- Added ReDoS protection (100KB text length limit)
- Provides `validate_candidate_structure()` for output validation

### 2. Scanner Output Validation
- Two-layer validation approach:
  - Registration-time: Check scanner class structure
  - Runtime: Validate each candidate dictionary
- Added `scan_with_validation()` wrapper in BaseScanner
- Validates required keys: ticker, source, context, priority
- Graceful error handling with structured logging

### 3. Configuration-Driven Design
- Moved magic numbers to `default_config.py`:
  - `ticker_universe`: Top 20 liquid options tickers
  - `min_volume`: 1000 (options flow threshold)
  - `min_transaction_value`: $25,000 (insider buying filter)
- Fixed hardcoded absolute paths to relative paths
- Improved portability across development environments

### 4. Concurrent Scanner Execution (37% Performance Gain)
- Implemented ThreadPoolExecutor for parallel scanner execution
- Configuration: `scanner_execution.concurrent`, `max_workers`, `timeout_seconds`
- Performance: 42s vs 67s (37% faster with 8 scanners)
- Thread-safe state management (each scanner gets copy)
- Per-scanner timeout with graceful degradation
- Error isolation (one failure doesn't stop others)

### 5. Error Handling Improvements
- Changed bare `except:` to `except Exception:` (avoid catching KeyboardInterrupt)
- Added structured logging with `exc_info=True` and extra fields
- Implemented graceful degradation throughout pipeline

## Files Changed

### Core Implementation
- `tradingagents/__init__.py` (NEW) - Package initialization
- `tradingagents/default_config.py` - Scanner execution config, magic numbers
- `tradingagents/graph/discovery_graph.py` - Concurrent execution logic
- `tradingagents/dataflows/discovery/common_utils.py` (NEW) - Shared utilities
- `tradingagents/dataflows/discovery/scanner_registry.py` - Validation wrapper
- `tradingagents/dataflows/discovery/scanners/*.py` - Use common utilities

### Testing & Documentation
- `tests/test_concurrent_scanners.py` (NEW) - Comprehensive test suite
- `verify_concurrent_execution.py` (NEW) - Performance verification
- `CONCURRENT_EXECUTION.md` (NEW) - Implementation documentation

## Test Results

All tests passing (exit code 0):
-  Concurrent execution: 42s, 66-69 candidates
-  Sequential fallback: 56-67s, 65-68 candidates
-  Timeout handling: Graceful degradation with 1s timeout
-  Error isolation: Individual failures don't cascade

## Performance Impact

- Scanner execution: 37% faster (42s vs 67s)
- Time saved: ~25 seconds per discovery run
- At scale: 4+ minutes saved daily in production
- Same candidate quality (65-69 tickers in both modes)

## Breaking Changes

None. Concurrent execution is opt-in via config flag.
Sequential mode remains available as fallback.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-02-05 23:27:01 -08:00
Youssef Aitousarrah 2376fc74a1 Update 2025-12-11 00:23:28 -08:00
Youssef Aitousarrah ea4ee9176b Update 2025-12-09 23:16:53 -08:00
Youssef Aitousarrah ccc78c694b Update 2025-12-06 15:39:49 -08:00
Youssef Aitousarrah 5cf57e5d97 Update 2025-12-02 20:49:42 -08:00
Youssef Aitousarrah 9ee66746a5 Fix: Stop after all primary vendors in multi-vendor config
When multiple primary vendors are configured (e.g., 'reddit,alpha_vantage'),
the system now correctly stops after attempting all primary vendors instead
of continuing through all fallback vendors.

Changes:
- Track which primary vendors have been attempted in a list
- Add stopping condition when all primary vendors are attempted
- Preserve existing single-vendor behavior (stop after first success)

This prevents unnecessary API calls and ensures predictable behavior.
2025-11-23 17:09:15 -08:00
Edward Sun 7bb2941b07 optimized yfin fetching to be much faster 2025-10-06 19:58:01 -07:00
Edward Sun c07dcf026b added fallbacks for tools 2025-10-03 22:40:09 -07:00
luohy15 d23fb539e9 minor fix 2025-09-30 13:27:48 +08:00
luohy15 b01051b9f4 Switch default data vendor
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-30 12:43:27 +08:00
luohy15 86bc0e793f minor fix 2025-09-27 00:04:59 +08:00
luohy15 6211b1132a Improve Alpha Vantage indicator column parsing with robust mapping
- Replace hardcoded column indices with column name lookup
- Add mapping for all supported indicators to their expected CSV column names
- Handle missing columns gracefully with descriptive error messages
- Strip whitespace from header parsing for reliability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 23:36:36 +08:00
luohy15 8b04ec307f minor fix 2025-09-26 23:25:33 +08:00
luohy15 0ab323c2c6 Add Alpha Vantage API integration as primary data provider
- Replace FinnHub with Alpha Vantage API in README documentation
- Implement comprehensive Alpha Vantage modules:
  - Stock data (daily OHLCV with date filtering)
  - Technical indicators (SMA, EMA, MACD, RSI, Bollinger Bands, ATR)
  - Fundamental data (overview, balance sheet, cashflow, income statement)
  - News and sentiment data with insider transactions
- Update news analyst tools to use ticker-based news search
- Integrate Alpha Vantage vendor methods into interface routing
- Maintain backward compatibility with existing vendor system

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-26 22:57:50 +08:00
luohy15 a6734d71bc WIP 2025-09-26 16:17:50 +08:00
mirza-samad-ahmed-baig f704828f89 Fix: Prevent infinite loops, enable reflection, and improve logging 2025-07-03 17:43:40 +05:00
Yijia Xiao 718df34932
Merge pull request #29 from ZeroAct/save_results
Save results
2025-06-26 00:28:30 -04:00
Max Wong 43aa9c5d09
Local Ollama (#53)
- Fix typo 'Start' 'End'
- Add llama3.1 selection
- Use 'quick_think_llm' model instead of hard-coding GPT
2025-06-26 00:27:01 -04:00
Yijia Xiao 26c5ba5a78
Revert "Docker support and Ollama support (#47)" (#57)
This reverts commit 78ea029a0b.
2025-06-26 00:07:58 -04:00
Geeta Chauhan 78ea029a0b
Docker support and Ollama support (#47)
- Added support for running CLI and Ollama server via Docker
- Introduced tests for local embeddings model and standalone Docker setup
- Enabled conditional Ollama server launch via LLM_PROVIDER
2025-06-25 23:57:05 -04:00
Huijae Lee ee3d499894
Merge branch 'TauricResearch:main' into save_results 2025-06-25 08:43:19 +09:00
Yijia Xiao b575bd0941
Merge pull request #52 from TauricResearch/dev
Merge dev into main. Add support for Anthropic and OpenRouter.
2025-06-23 20:38:14 -04:00
Edward Sun 52284ce13c fixed anthropic support. Anthropic has different format of response when it has tool calls. Explicit handling added 2025-06-21 12:51:34 -07:00
Yijia Xiao c2f897fc67
Merge pull request #43 from AtharvSabde/patch-1
fundamentals_analyst.py (spelling mistake in instruction: Makrdown -> Markdown)
2025-06-19 15:05:08 -04:00
Atharv Sabde 2af7ef3d79
fundamentals_analyst.py(spelling mistake.markdown) 2025-06-19 21:48:16 +05:30
Edward Sun 7eaf4d995f update clear msg bc anthropic needs at least 1 msg in chat call 2025-06-15 23:14:47 -07:00
Edward Sun da84ef43aa main works, cli bugs 2025-06-15 22:20:59 -07:00
Edward Sun 90b23e72f5
Merge pull request #25 from maxer137/main
Add support for other backends, such as OpenRouter and Ollama
2025-06-15 16:06:20 -07:00
saksham0161 570644d939
Fix ticker hardcoding in prompt (#28) 2025-06-11 19:43:39 -07:00
ZeroAct 9647359246 save reports & logs under results_dir 2025-06-12 11:25:07 +09:00
maxer137 99789f9cd1 Add support for other backends, such as OpenRouter and olama
This aims to offer alternative OpenAI capable api's.
This offers people to experiment with running the application locally
2025-06-11 14:19:25 +02:00