TradingAgents/tradingagents/dataflows
Youssef Aitousarrah c792b17ab6 fix(discovery): fix three scanner hang/validation bugs found in ranker_debug.log
1. executor.shutdown(wait=True) still blocked after global timeout (critical)
   The previous fix added timeout= to as_completed() but used `with
   ThreadPoolExecutor() as executor`, whose __exit__ calls shutdown(wait=True).
   This meant the process still hung waiting for stuck threads (ml_signal) even
   after the TimeoutError was caught.  Fixed by creating the executor explicitly
   and calling shutdown(wait=False) in a finally block.

2. ml_signal hangs on every run — "Batch-downloading 592 tickers (1y)..." never
   completes. Root cause: a single yfinance request for 592 tickers × 1 year of
   daily OHLCV is a very large payload that regularly times out at the network
   layer. Fixed by:
   - Reducing default lookback from "1y" to "6mo" (halves download size)
   - Splitting downloads into 150-ticker chunks so a slow chunk doesn't kill
     the whole scan (partial results are still returned)

3. C (Citigroup) and other single-letter NYSE tickers rejected as invalid.
   validate_ticker_format used ^[A-Z]{2,5}$ requiring at least 2 letters.
   Real tickers like C, A, F, T, X, M are 1 letter. Fixed to ^[A-Z]{1,5}$.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 22:35:42 -08:00
..
discovery fix(discovery): fix three scanner hang/validation bugs found in ranker_debug.log 2026-02-20 22:35:42 -08:00
__init__.py WIP 2025-09-26 16:17:50 +08:00
alpha_vantage.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_analysts.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_common.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_fundamentals.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_indicator.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_news.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_stock.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
alpha_vantage_volume.py chore: linter formatting + ML scanner logging, prompt control, ranker reasoning 2026-02-09 23:04:38 -08:00
config.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
delisted_cache.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
finnhub_api.py chore: linter formatting + ML scanner logging, prompt control, ranker reasoning 2026-02-09 23:04:38 -08:00
finviz_scraper.py Update 2026-02-20 08:38:15 -08:00
fmp_api.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
google.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
googlenews_utils.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
interface.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
local.py chore: linter formatting + ML scanner logging, prompt control, ranker reasoning 2026-02-09 23:04:38 -08:00
market_data_utils.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
news_semantic_scanner.py Update 2026-02-20 08:39:37 -08:00
openai.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
reddit_api.py chore: linter formatting + ML scanner logging, prompt control, ranker reasoning 2026-02-09 23:04:38 -08:00
reddit_utils.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
semantic_discovery.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
stockstats_utils.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
technical_analyst.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
ticker_semantic_db.py Remore unused code and improve the UI 2026-02-16 14:17:43 -08:00
tradier_api.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
twitter_data.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
utils.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00
y_finance.py feat: discovery pipeline enhancements with ML signal scanner 2026-02-09 22:53:42 -08:00