TradingAgents/tradingagents
Ahmet Guzererler eafdce3121 fix: harden dataflows layer against silent failures and data corruption
## Problem (Incident Post-mortem)

The pipeline was emitting hundreds of errors:
  'Invalid number of return arguments after parsing column name: Date'

Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.

## Fixes

### 1. Fix df['Date'] stockstats bug (already shipped in prior commit)
- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
  df['Date'] after wrap()

### 2. Extract _load_or_fetch_ohlcv() — single OHLCV authority
- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
  '2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
  and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
  dropping rows that would distort indicator calculations

### 3. YFinanceError typed exception
- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
  surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
  AlphaVantageError and FinnhubError — failures appear in observability
  telemetry and can trigger vendor fallback

### 4. Explicit date column discovery in alpha_vantage_common
- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
  with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible

### 5. Structured logging
- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common

## Tests
- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
  (dynamic cache filename, corruption re-fetch, YFinanceError propagation,
  explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
  yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
2026-03-22 00:07:32 +01:00
..
agents Merge pull request #83 from aguzererler/fix-unused-import-time-6733591745228634997 2026-03-21 23:25:01 +01:00
dataflows fix: harden dataflows layer against silent failures and data corruption 2026-03-22 00:07:32 +01:00
graph Merge pull request #61 from aguzererler/fix/agent-utils-unused-imports-948671579039994874 2026-03-21 17:35:19 +01:00
llm_clients Sync fork with upstream TauricResearch/TradingAgents (v0.2.1) (#12) 2026-03-18 07:00:37 +01:00
pipeline Optimize synchronous API execution inside async loop in macro_bridge.py 2026-03-21 19:56:23 +00:00
portfolio refactor: centralize env loading in default_config.py; fix .env.example 2026-03-21 22:33:06 +00:00
__init__.py Sync fork with upstream TauricResearch/TradingAgents (v0.2.1) (#12) 2026-03-18 07:00:37 +01:00
api_usage.py Address code review: add debug logging for unknown method resolution 2026-03-21 17:27:38 +00:00
daily_digest.py feat: daily digest consolidation and Google NotebookLM sync (#23) 2026-03-19 12:21:03 +01:00
default_config.py refactor: centralize env loading in default_config.py; fix .env.example 2026-03-21 22:33:06 +00:00
notebook_sync.py feat: NotebookLM sync with date-specific sources and consolidation (#28) 2026-03-19 15:39:25 +01:00
observability.py Add API consumption estimation module and CLI command 2026-03-21 17:25:26 +00:00
report_paths.py feat: daily digest consolidation and Google NotebookLM sync (#23) 2026-03-19 12:21:03 +01:00