TradingAgents/tradingagents/dataflows
Ahmet Guzererler 2201afd33a fix: harden dataflows layer against silent failures and data corruption
The pipeline was emitting hundreds of errors:
  'Invalid number of return arguments after parsing column name: Date'

Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.

- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
  df['Date'] after wrap()

- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
  '2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
  and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
  dropping rows that would distort indicator calculations

- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
  surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
  AlphaVantageError and FinnhubError — failures appear in observability
  telemetry and can trigger vendor fallback

- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
  with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible

- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common

- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
  (dynamic cache filename, corruption re-fetch, YFinanceError propagation,
  explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
  yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
2026-03-22 07:23:14 +01:00
..
__init__.py WIP 2025-09-26 16:17:50 +08:00
alpha_vantage.py Refactor interface.py to remove unused imports in alpha_vantage.py 2026-03-21 08:27:10 +00:00
alpha_vantage_common.py fix: harden dataflows layer against silent failures and data corruption 2026-03-22 07:23:14 +01:00
alpha_vantage_fundamentals.py Add Alpha Vantage API integration as primary data provider 2025-09-26 22:57:50 +08:00
alpha_vantage_indicator.py Refactor `get_indicator` function in `alpha_vantage_indicator.py` 2026-03-21 08:26:02 +00:00
alpha_vantage_news.py fix: improve data vendor implementations and tool signatures 2026-02-03 22:27:20 +00:00
alpha_vantage_scanner.py feat: Complete 3-phase LLM scanner pipeline with inline tool execution 2026-03-17 08:41:40 +01:00
alpha_vantage_stock.py Add Alpha Vantage API integration as primary data provider 2025-09-26 22:57:50 +08:00
config.py feat: add multi-provider LLM support with thinking configurations 2026-02-03 22:27:20 +00:00
finnhub.py feat: Finnhub integration layer, 141 tests, and vendor evaluation report (#16) 2026-03-18 11:28:43 +01:00
finnhub_common.py Resolve merge conflicts after PR #16 merge into main (#17) 2026-03-18 11:38:44 +01:00
finnhub_fundamentals.py Resolve merge conflicts after PR #16 merge into main (#17) 2026-03-18 11:38:44 +01:00
finnhub_indicators.py 🧹 Remove unused `timedelta` import in finnhub_indicators.py 2026-03-21 01:54:05 +00:00
finnhub_news.py Fix Finnhub API error handling and add coverage 2026-03-21 14:51:29 +00:00
finnhub_scanner.py 🧹 remove unused import _make_api_request from finnhub_scanner.py 2026-03-21 08:22:49 +00:00
finnhub_stock.py Remove unused pandas import and clean up finnhub_stock.py 2026-03-21 08:21:43 +00:00
interface.py fix: harden dataflows layer against silent failures and data corruption 2026-03-22 07:23:14 +01:00
macro_regime.py Merge pull request #54 from aguzererler/test-macro-regime-fmt-pct-18208719821293052000 2026-03-21 17:36:20 +01:00
peer_comparison.py feat: medium-term positioning upgrade (debate rounds, TTM, peer comparison, macro regime) (#14) 2026-03-17 22:27:40 +01:00
stockstats_utils.py fix: harden dataflows layer against silent failures and data corruption 2026-03-22 07:23:14 +01:00
ttm_analysis.py Add financial tools analysis doc and fix YoY revenue growth bug in TTM analysis 2026-03-21 16:57:21 +00:00
utils.py chore: remove unused imports `os` and `json` from `tradingagents/dataflows/utils.py` 2026-03-21 01:34:15 +00:00
y_finance.py fix: harden dataflows layer against silent failures and data corruption 2026-03-22 07:23:14 +01:00
yfinance_news.py fix: improve data vendor implementations and tool signatures 2026-02-03 22:27:20 +00:00
yfinance_scanner.py feat: improve Industry Deep Dive report quality with enriched data, sector routing, and tool-call nudge 2026-03-17 20:10:45 +01:00