Before this change, the pipeline showed a generic 'Analyzing...' spinner
for the entire multi-ticker run with no way to know which ticker was
processing or whether anything was actually working.
Changes:
- macro_bridge.py:
- run_ticker_analysis: logs '▶ Starting', '✓ complete in Xs', '✗ FAILED'
with elapsed time per ticker using logger.info/logger.error
- run_all_tickers: replaced asyncio.gather (swallows all progress) with
asyncio.as_completed + optional on_ticker_done(result, done, total)
callback; uses asyncio.Semaphore for max_concurrent control
- Added time and Callable imports
- cli/main.py run_pipeline:
- Replaced Live(Spinner) with Rich Progress bar (spinner + bar + counter
+ elapsed time)
- Prints '▷ Queued: TICKER' before analysis starts for each ticker
- on_ticker_done callback prints '✓ TICKER (N/M, Xs elapsed) → decision'
or '✗ TICKER failed ...' immediately as each ticker finishes
- Prints total elapsed time when all tickers complete
The pipeline was emitting hundreds of errors:
'Invalid number of return arguments after parsing column name: Date'
Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.
- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
df['Date'] after wrap()
- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
'2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
dropping rows that would distort indicator calculations
- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
AlphaVantageError and FinnhubError — failures appear in observability
telemetry and can trigger vendor fallback
- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible
- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common
- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
(dynamic cache filename, corruption re-fetch, YFinanceError propagation,
explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
- Removed unused `import time` from `tradingagents/agents/analysts/news_analyst.py`
- Verified file syntax with `py_compile`
- Confirmed that the import was successfully removed
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Replaced the inefficient `iterrows()` loop with vectorized operations:
`df.set_index("Date")[indicator].fillna("N/A").astype(str).to_dict()`
This change improves performance while maintaining the exact original behavior for NaN values and string conversion.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed the unused `json` import in `tradingagents/agents/analysts/market_analyst.py` to reduce visual clutter and namespace pollution.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Use `--file` instead of `--text` for adding sources to avoid ARG_MAX issues and argument injection.
- Add `--` separator to prevent positional arguments from being misinterpreted as flags.
- Reorder arguments to place options before positional ones.
- Update unit tests and add new security-focused tests.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Replaces the O(N) database operations in the `TradeExecutor`'s
`execute_decisions` SELL loop with a single `batch_remove_holdings`
call to the repository. The new repository method calculates updates
in memory, resolves duplicate operations on the same ticker, and issues
the updates via newly implemented `psycopg2.extras.execute_batch`
routines on the `SupabaseClient`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Optimize the _percentile calculation to use heapq.nsmallest or heapq.nlargest when requesting small extreme percentiles (like 5% VaR) from large lists, falling back to sorted() only when necessary. This avoids fully sorting the entire array.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
- Create docs/FINANCIAL_TOOLS_ANALYSIS.md with comprehensive 4-point analysis:
1. Implementation accuracy review for all indicators and metrics
2. Library assessment (stockstats vs TA-Lib vs pandas-ta)
3. Alpha Vantage debate (local calc vs API-fetched)
4. Data flow & API mapping for every financial tool
- Fix off-by-one in ttm_analysis.py: YoY revenue used quarterly[-4]
(3 quarters back) instead of quarterly[-5] (4 quarters = 1 year back)
- Add test_revenue_yoy_is_four_quarters_back test to validate the fix
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
- Added edge case test for `_find_col` in `tests/unit/test_ttm_analysis.py` (from PR #56).
- Enhanced `_clean_dataframe` in `tradingagents/dataflows/stockstats_utils.py` to parse dates, drop invalid rows, fill price gaps, and lowercase columns (combining PRs #58 and #60).
- Expanded the test suite in `tests/unit/test_stockstats_utils.py` to cover the new `_clean_dataframe` functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Extracted the API request logic in `finnhub_news.py` to a private `_fetch_company_news_data` helper to properly catch `Exception` and return an empty list without violating the `str` return type of the main `get_company_news` function. Explicitly allows `ThirdPartyTimeoutError` to propagate to preserve timeout behavior.
Added corresponding tests to mock generic API exceptions and invalid response types. Retained the test verifying fallback behavior for invalid numeric values within `get_insider_transactions`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Fixed `_signal_vix_trend` to correctly return neutral for insufficient history (`< 21`).
- Added `test_short_history_is_neutral` to `TestSignalVixTrend`.
- Extended coverage for short history and edge cases in `TestSignalCreditSpread`, `TestSignalYieldCurve`, `TestSignalMarketBreadth`, and `TestSignalSectorRotation`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
This commit optimizes `PortfolioSnapshot` instantiation when loaded from the DB or JSON dictionaries by removing the immediate `json.loads(holdings_snapshot)` parsing inside the `from_dict` constructor. Instead, it overrides `__getattribute__` to implement a lazy-loading pattern, where the `holdings_snapshot` string is only parsed into a Python dictionary the very first time the field is accessed.
This optimization ensures that parsing large JSON payloads doesn't block the instantiation of snapshots, which is particularly beneficial in workflows that load a large historical series of snapshots but never access their `holdings_snapshot` field.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added tests to verify the dataframe cleaning logic in stockstats_utils.
Tests cover lowercasing of columns, handling non-string columns, and ensuring original dataframe is not mutated.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed re-exported tool imports from `tradingagents/agents/utils/agent_utils.py` to declutter the file and prevent unnecessary dependency loading. Updated all downstream modules (tests, analysts, scanners, and the trading graph) to import the required tools directly from their respective source files in `tradingagents/agents/utils/`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
The file `tradingagents/dataflows/alpha_vantage.py` previously acted as an aggregator but did not use the imports it defined, leading to linter warnings (ruff). The dependencies in `tradingagents/dataflows/interface.py` were refactored to import directly from the specialized modules instead. This safely allowed removing all unused imports from `alpha_vantage.py`, improving code health and module clarity without affecting functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
This change refactors the large monolithic `get_indicator` function by extracting large mapping dictionaries out of the function scope as constants and separating API fetching logic and CSV parsing logic into private helper functions. This drastically improves the maintainability and readability of `alpha_vantage_indicator.py` without introducing any behavioral changes.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added `TestFmtPct` class to `tests/unit/test_macro_regime.py` to test the `_fmt_pct` function from `tradingagents.dataflows.macro_regime`.
The test covers `None`, positive, negative, and zero values.
Also updated `_fmt_pct` implementation to match the requested `+.1f` formatting.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Ahmet Guzererler
copilot-swe-agent[bot]
google-labs-jules[bot]