- Wrap each event-type branch (LLM start/end, tool start/end) in try/except
to prevent a single unexpected object shape from crashing the streaming loop
- Add _safe_dict() helper to guard response_metadata and usage_metadata
access — some providers return non-dict types (bound methods, etc.)
- Fix potential_text extraction: check for None AND callable before using
- Ensure all event IDs use .get() with fallback to prevent KeyError
- Fix test file: remove hardcoded /Users/Ahmet/ path, add edge-case tests
for non-dict metadata, tool events, and unknown event types
- All 725 unit tests pass, TypeScript compiles clean
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/fe6575b5-c03b-4037-bd98-a94303ae8313
In langchain_core >=1.0 plain Generation no longer stores a .message
attribute - that only exists on ChatGeneration. Tests were constructing
Generation(message=AIMessage(...)) which silently dropped the message,
making hasattr(generation, "message") return False and skipping the
token-counting path (all usage assertions failed with 0).
- Replace Generation(message=...) with ChatGeneration(message=AIMessage(...))
in test_stats_handler_on_llm_end_with_usage and thread_safety test
- Use UsageMetadata(input_tokens=N, output_tokens=N, total_tokens=N)
instead of bare dict (total_tokens is required in langchain_core 1.2+)
- Pass usage_metadata via AIMessage constructor instead of post-init
attribute assignment (avoids pydantic validation bypass)
- Keep Generation(text=...) in test_stats_handler_on_llm_end_no_usage
(correctly tests the "no usage" branch — plain Generation has no .message)
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/ce079791-08ef-4f2e-9f31-a1ae6a26b4cb
## Problem (Incident Post-mortem)
The pipeline was emitting hundreds of errors:
'Invalid number of return arguments after parsing column name: Date'
Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.
## Fixes
### 1. Fix df['Date'] stockstats bug (already shipped in prior commit)
- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
df['Date'] after wrap()
### 2. Extract _load_or_fetch_ohlcv() — single OHLCV authority
- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
'2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
dropping rows that would distort indicator calculations
### 3. YFinanceError typed exception
- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
AlphaVantageError and FinnhubError — failures appear in observability
telemetry and can trigger vendor fallback
### 4. Explicit date column discovery in alpha_vantage_common
- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible
### 5. Structured logging
- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common
## Tests
- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
(dynamic cache filename, corruption re-fetch, YFinanceError propagation,
explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
Replaces the O(N) database operations in the `TradeExecutor`'s
`execute_decisions` SELL loop with a single `batch_remove_holdings`
call to the repository. The new repository method calculates updates
in memory, resolves duplicate operations on the same ticker, and issues
the updates via newly implemented `psycopg2.extras.execute_batch`
routines on the `SupabaseClient`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
- Create docs/FINANCIAL_TOOLS_ANALYSIS.md with comprehensive 4-point analysis:
1. Implementation accuracy review for all indicators and metrics
2. Library assessment (stockstats vs TA-Lib vs pandas-ta)
3. Alpha Vantage debate (local calc vs API-fetched)
4. Data flow & API mapping for every financial tool
- Fix off-by-one in ttm_analysis.py: YoY revenue used quarterly[-4]
(3 quarters back) instead of quarterly[-5] (4 quarters = 1 year back)
- Add test_revenue_yoy_is_four_quarters_back test to validate the fix
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
- Added edge case test for `_find_col` in `tests/unit/test_ttm_analysis.py` (from PR #56).
- Enhanced `_clean_dataframe` in `tradingagents/dataflows/stockstats_utils.py` to parse dates, drop invalid rows, fill price gaps, and lowercase columns (combining PRs #58 and #60).
- Expanded the test suite in `tests/unit/test_stockstats_utils.py` to cover the new `_clean_dataframe` functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Replaced potentially unsafe or missing tool call parsing logic with `ast.literal_eval` in `cli/main.py`.
- Created a new `parse_tool_call` helper to handle fallback parsing for LLM tool calls formatted as strings.
- Added comprehensive unit tests in `tests/unit/test_cli_main_tools.py` verifying behavior for valid strings, `ValueError`, `SyntaxError`, dicts, and objects.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Extracted the API request logic in `finnhub_news.py` to a private `_fetch_company_news_data` helper to properly catch `Exception` and return an empty list without violating the `str` return type of the main `get_company_news` function. Explicitly allows `ThirdPartyTimeoutError` to propagate to preserve timeout behavior.
Added corresponding tests to mock generic API exceptions and invalid response types. Retained the test verifying fallback behavior for invalid numeric values within `get_insider_transactions`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Fixed `_signal_vix_trend` to correctly return neutral for insufficient history (`< 21`).
- Added `test_short_history_is_neutral` to `TestSignalVixTrend`.
- Extended coverage for short history and edge cases in `TestSignalCreditSpread`, `TestSignalYieldCurve`, `TestSignalMarketBreadth`, and `TestSignalSectorRotation`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added tests to verify the dataframe cleaning logic in stockstats_utils.
Tests cover lowercasing of columns, handling non-string columns, and ensuring original dataframe is not mutated.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed re-exported tool imports from `tradingagents/agents/utils/agent_utils.py` to declutter the file and prevent unnecessary dependency loading. Updated all downstream modules (tests, analysts, scanners, and the trading graph) to import the required tools directly from their respective source files in `tradingagents/agents/utils/`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added `TestFmtPct` class to `tests/unit/test_macro_regime.py` to test the `_fmt_pct` function from `tradingagents.dataflows.macro_regime`.
The test covers `None`, positive, negative, and zero values.
Also updated `_fmt_pct` implementation to match the requested `+.1f` formatting.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Adds an edge case test in `test_config.py` to verify that `min_cash_pct` + `max_position_pct` can be exactly `1.0` without raising a `ValueError`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Implemented `tests/portfolio/test_config.py` to test the `validate_config` function in `tradingagents/portfolio/config.py`.
Key improvements:
- Added `test_validate_config_max_positions_invalid` to verify `max_positions` boundary conditions (0, -1).
- Added happy path test `test_validate_config_valid`.
- Added tests for `max_position_pct`, `max_sector_pct`, `min_cash_pct`, and `default_budget` validation.
- Verified that tests correctly catch bugs by temporarily disabling validation logic.
These tests ensure the portfolio manager configuration is robustly validated before use.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- tradingagents/portfolio/risk_metrics.py: pure-Python computation of
Sharpe, Sortino, VaR, max drawdown, beta, sector concentration from
PortfolioSnapshot NAV history — no LLM, no external dependencies
- tradingagents/portfolio/__init__.py: export compute_risk_metrics
- tradingagents/agents/utils/portfolio_tools.py: 4 LangChain tools
wrapping Holding.enrich, Portfolio.enrich, ReportStore APIs, and
compute_risk_metrics so agents can access portfolio data without
reimplementing computations
- tests/portfolio/test_risk_metrics.py: 48 tests for risk metrics
- tests/unit/test_portfolio_tools.py: 19 tests for portfolio tools
- tests/portfolio/test_repository.py: fix pre-existing import error
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Replace supabase-py stubs with working psycopg2 implementation using
Supabase pooler connection string. Implement full business logic in
repository (avg cost basis, cash accounting, trade recording, snapshots).
Add 12 unit tests + 4 integration tests (51 total portfolio tests pass).
Fix cash_pct bug in models.py, update docs for psycopg2 + pooler pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implement integration tests for scanner vendor routing, ensuring correct routing to Alpha Vantage and fallback to yfinance.
- Create comprehensive unit tests for TTM analysis, covering metrics computation and report formatting.
- Introduce fail-fast vendor routing tests to verify immediate failure for methods not in FALLBACK_ALLOWED.
- Develop extensive integration tests for the yfinance data layer, mocking external calls to validate functionality across various financial data retrieval methods.
* gitignore
* feat: unify report paths under reports/daily/{date}/ hierarchy
All generated artifacts now land under a single reports/ tree:
- reports/daily/{date}/market/ for scan results (was results/macro_scan/)
- reports/daily/{date}/{TICKER}/ for per-ticker analysis (was reports/{TICKER}_{timestamp}/)
- reports/daily/{date}/{TICKER}/eval/ for eval logs (was eval_results/{TICKER}/...)
Adds tradingagents/report_paths.py with centralized path helpers used by
CLI commands, trading graph, and pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: structured observability logging for LLM, tool, and vendor calls
Add RunLogger (tradingagents/observability.py) that emits JSON-lines events
for every LLM call (model, agent, tokens in/out, latency), tool invocation
(tool name, args, success, latency), data vendor call (method, vendor,
success/failure, latency), and report save.
Integration points:
- route_to_vendor: log_vendor_call() on every try/catch
- run_tool_loop: log_tool_call() on every tool invoke
- ScannerGraph: new callbacks param, passes RunLogger.callback to all LLM tiers
- pipeline/macro_bridge: picks up RunLogger from thread-local, passes to TradingAgentsGraph
- cli/main.py: one RunLogger per command (analyze/scan/pipeline), write_log()
at end, summary line printed to console
Log files co-located with reports:
reports/daily/{date}/{TICKER}/run_log.jsonl (analyze)
reports/daily/{date}/market/run_log.jsonl (scan)
reports/daily/{date}/run_log.jsonl (pipeline)
Also fix test_long_response_no_nudge: update "A"*600 → "A"*2100 to match
MIN_REPORT_LENGTH=2000 threshold set in an earlier commit.
Update memory system context files (ARCHITECTURE, COMPONENTS, CONVENTIONS,
GLOSSARY, CURRENT_STATE) to document observability and report path systems.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add extract_json() utility for robust LLM JSON parsing
Handles DeepSeek R1 <think> blocks, markdown code fences, and
preamble/postamble text that LLMs wrap around JSON output.
Applied to macro_synthesis, macro_bridge, and CLI scan output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: opt-in vendor fallback — fail-fast by default (ADR 011)
Silent cross-vendor fallback corrupts signal quality when data contracts
differ (e.g., AV news has sentiment scores yfinance lacks). Only methods
with fungible data contracts (OHLCV, indices, sector/industry perf,
market movers) now get fallback. All others raise immediately.
- Add FALLBACK_ALLOWED whitelist to interface.py
- Rewrite route_to_vendor() with fail-fast/fallback branching
- Improve error messages with method name, vendors tried, and exception chaining
- Add 11 new tests in test_vendor_failfast.py
- Update ADRs 002 (superseded), 008, 010; create ADR 011
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
copilot-swe-agent[bot]
Ahmet Guzererler
google-labs-jules[bot]