## Problem (Incident Post-mortem)
The pipeline was emitting hundreds of errors:
'Invalid number of return arguments after parsing column name: Date'
Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.
## Fixes
### 1. Fix df['Date'] stockstats bug (already shipped in prior commit)
- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
df['Date'] after wrap()
### 2. Extract _load_or_fetch_ohlcv() — single OHLCV authority
- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
'2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
dropping rows that would distort indicator calculations
### 3. YFinanceError typed exception
- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
AlphaVantageError and FinnhubError — failures appear in observability
telemetry and can trigger vendor fallback
### 4. Explicit date column discovery in alpha_vantage_common
- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible
### 5. Structured logging
- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common
## Tests
- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
(dynamic cache filename, corruption re-fetch, YFinanceError propagation,
explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
- Removed unused `import time` from `tradingagents/agents/analysts/news_analyst.py`
- Verified file syntax with `py_compile`
- Confirmed that the import was successfully removed
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Replaces the O(N) database operations in the `TradeExecutor`'s
`execute_decisions` SELL loop with a single `batch_remove_holdings`
call to the repository. The new repository method calculates updates
in memory, resolves duplicate operations on the same ticker, and issues
the updates via newly implemented `psycopg2.extras.execute_batch`
routines on the `SupabaseClient`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Optimize the _percentile calculation to use heapq.nsmallest or heapq.nlargest when requesting small extreme percentiles (like 5% VaR) from large lists, falling back to sorted() only when necessary. This avoids fully sorting the entire array.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
- Create docs/FINANCIAL_TOOLS_ANALYSIS.md with comprehensive 4-point analysis:
1. Implementation accuracy review for all indicators and metrics
2. Library assessment (stockstats vs TA-Lib vs pandas-ta)
3. Alpha Vantage debate (local calc vs API-fetched)
4. Data flow & API mapping for every financial tool
- Fix off-by-one in ttm_analysis.py: YoY revenue used quarterly[-4]
(3 quarters back) instead of quarterly[-5] (4 quarters = 1 year back)
- Add test_revenue_yoy_is_four_quarters_back test to validate the fix
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
- Added edge case test for `_find_col` in `tests/unit/test_ttm_analysis.py` (from PR #56).
- Enhanced `_clean_dataframe` in `tradingagents/dataflows/stockstats_utils.py` to parse dates, drop invalid rows, fill price gaps, and lowercase columns (combining PRs #58 and #60).
- Expanded the test suite in `tests/unit/test_stockstats_utils.py` to cover the new `_clean_dataframe` functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Extracted the API request logic in `finnhub_news.py` to a private `_fetch_company_news_data` helper to properly catch `Exception` and return an empty list without violating the `str` return type of the main `get_company_news` function. Explicitly allows `ThirdPartyTimeoutError` to propagate to preserve timeout behavior.
Added corresponding tests to mock generic API exceptions and invalid response types. Retained the test verifying fallback behavior for invalid numeric values within `get_insider_transactions`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Fixed `_signal_vix_trend` to correctly return neutral for insufficient history (`< 21`).
- Added `test_short_history_is_neutral` to `TestSignalVixTrend`.
- Extended coverage for short history and edge cases in `TestSignalCreditSpread`, `TestSignalYieldCurve`, `TestSignalMarketBreadth`, and `TestSignalSectorRotation`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
This commit optimizes `PortfolioSnapshot` instantiation when loaded from the DB or JSON dictionaries by removing the immediate `json.loads(holdings_snapshot)` parsing inside the `from_dict` constructor. Instead, it overrides `__getattribute__` to implement a lazy-loading pattern, where the `holdings_snapshot` string is only parsed into a Python dictionary the very first time the field is accessed.
This optimization ensures that parsing large JSON payloads doesn't block the instantiation of snapshots, which is particularly beneficial in workflows that load a large historical series of snapshots but never access their `holdings_snapshot` field.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added tests to verify the dataframe cleaning logic in stockstats_utils.
Tests cover lowercasing of columns, handling non-string columns, and ensuring original dataframe is not mutated.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed re-exported tool imports from `tradingagents/agents/utils/agent_utils.py` to declutter the file and prevent unnecessary dependency loading. Updated all downstream modules (tests, analysts, scanners, and the trading graph) to import the required tools directly from their respective source files in `tradingagents/agents/utils/`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
The file `tradingagents/dataflows/alpha_vantage.py` previously acted as an aggregator but did not use the imports it defined, leading to linter warnings (ruff). The dependencies in `tradingagents/dataflows/interface.py` were refactored to import directly from the specialized modules instead. This safely allowed removing all unused imports from `alpha_vantage.py`, improving code health and module clarity without affecting functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
This change refactors the large monolithic `get_indicator` function by extracting large mapping dictionaries out of the function scope as constants and separating API fetching logic and CSV parsing logic into private helper functions. This drastically improves the maintainability and readability of `alpha_vantage_indicator.py` without introducing any behavioral changes.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added `TestFmtPct` class to `tests/unit/test_macro_regime.py` to test the `_fmt_pct` function from `tradingagents.dataflows.macro_regime`.
The test covers `None`, positive, negative, and zero values.
Also updated `_fmt_pct` implementation to match the requested `+.1f` formatting.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed unused `import json` and `import time` from `tradingagents/agents/managers/risk_manager.py` to improve code health and cleanliness. Also removed unused variable `company_name`. Tested to ensure no functionality is affected.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed the unused `import json` line from `tradingagents/agents/managers/research_manager.py` to improve code health and maintainability.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Ahmet Guzererler
copilot-swe-agent[bot]
google-labs-jules[bot]