In dataflow API wrappers, generic `except Exception` handlers were swallowing
timeout exceptions (like `requests.exceptions.Timeout` and `ThirdPartyTimeoutError`),
preventing upstream retry logic and proper error handling from functioning correctly.
This patch:
- Adds explicit catching of `requests.exceptions.Timeout` to raise as `ThirdPartyTimeoutError` in `yfinance` wrapper modules (`y_finance.py`, `yfinance_news.py`, `yfinance_scanner.py`).
- Adds explicit catching and re-raising of `ThirdPartyTimeoutError` before generic exception handlers across the dataflows.
- Retains existing exception inheritance logic to avoid backward compatibility breaks.
- Updates `TestEnsureIndexesInInit` to use the explicit `ensure_indexes()` call per lazy loading changes.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
* chore: initial commit for feat/fast-reject-short-circuits branch
* test: fix fast-reject unit tests (25/25 passing)
- Fix test_normal_flow_without_abort: expected 'Bull Researcher' not
'Bear Researcher' — when current_response is empty the conditional
logic routes to Bull first (conditional_logic.py line 74)
- Analyst abort-instruction tests: replace closure-attribute hacks
(market_analyst.llm = ...) with patch() on prefetch_tools_parallel
and run_tool_loop so the tests run without network access
- Fix co_consts substring check: system_message is compiled into one
large string constant, so use any(...in str(c)...) instead of direct
tuple membership
- PM tests: create mock_llm before create_portfolio_manager so the
closure captures it and mock_llm.invoke.called assertions work; add
missing company_of_interest key to all state dicts
- Integration tests: merge analyst/PM node outputs back into state
({**state, **result}) instead of replacing it, preserving keys like
fundamentals_report and investment_debate_state; patch network calls;
fix normal-flow routing assertions to != 'Portfolio Manager' since
the exact next node depends on transient debate state
* feat: introduce flow_id with timestamp-based report versioning
Replace run_id with flow_id as the primary grouping concept (one flow =
one user analysis intent spanning scan + pipeline + portfolio). Reports
are now written as {timestamp}_{name}.json so load methods always return
the latest version by lexicographic sort, eliminating the latest.json
pointer pattern for new flows.
Key changes:
- report_paths.py: add generate_flow_id(), ts_now() (ms precision),
flow_id kwarg on all path helpers; keep run_id / pointer helpers for
backward compatibility
- ReportStore: dual-mode save/load — flow_id uses timestamped layout,
run_id uses legacy runs/{id}/ layout with latest.json
- MongoReportStore: add flow_id field and index; run_id stays for compat
- DualReportStore: expose flow_id property
- store_factory: accept flow_id as primary param, run_id as alias
- runs.py / langgraph_engine.py: generate and thread flow_id through all
trigger endpoints and run methods
- Tests: add flow_id coverage for all layers; 905 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: PM brain upgrade — macro/micro summary agents, memory split, forensic dashboard
Replaces the PM's raw-JSON context (~6,800 tokens on deep_think) with a
MAP-REDUCE compression layer using two parallel mid_think summary agents,
achieving ~70% cost reduction at the PM tier.
Architecture:
- MacroMemory: new regime-level memory class (MongoDB/JSON, separate from
per-ticker reflexion memory) with record_macro_state/build_macro_context
- ReflexionMemory: extended with collection_name param to isolate
micro_reflexion from the pipeline reflexion collection (with distinct
local JSON fallback path to prevent file collision)
- Macro_Summary_Agent (mid_think): compresses scan_summary into a 1-page
regime brief with memory injection; sentinel guard prevents LLM call on
empty/error scan data ("NO DATA AVAILABLE - ABORT MACRO")
- Micro_Summary_Agent (mid_think): compresses holding_reviews + candidates
into a markdown table brief with per-ticker memory injection
- Portfolio graph: parallel fan-out (prioritize_candidates → macro_summary
‖ micro_summary → make_pm_decision) using _last_value reducers for safe
concurrent state writes (ADR-005 pattern)
- PM refactor: Pydantic PMDecisionSchema enforces Forensic Execution
Dashboard output (macro_regime, forensic_report, per-trade
macro_alignment/memory_note/position_sizing_logic); with_structured_output
as primary path, extract_json fallback for non-conforming providers
- PM sentinel handling: "NO DATA AVAILABLE" in macro_brief substituted
with actionable conservative guidance before LLM sees it
62 new unit tests across 4 test files covering all new components.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: address code review — relaxed error guard, ticker_analyses, PM memory wiring
1. macro_summary_agent: relaxed error guard to only abort when scan_summary's
sole key is "error" (partial failures with real data are now processed)
2. micro_summary_agent: now reads ticker_analyses from state and enriches
the per-ticker table with trading graph analysis data
3. portfolio_graph: wires macro_memory and micro_memory to PM factory call
4. test_empty_state: updated test for new partial-failure behavior
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: introduce flow_id with timestamp-based report versioning
Replace run_id with flow_id as the primary grouping concept (one flow =
one user analysis intent spanning scan + pipeline + portfolio). Reports
are now written as {timestamp}_{name}.json so load methods always return
the latest version by lexicographic sort, eliminating the latest.json
pointer pattern for new flows.
Key changes:
- report_paths.py: add generate_flow_id(), ts_now() (ms precision),
flow_id kwarg on all path helpers; keep run_id / pointer helpers for
backward compatibility
- ReportStore: dual-mode save/load — flow_id uses timestamped layout,
run_id uses legacy runs/{id}/ layout with latest.json
- MongoReportStore: add flow_id field and index; run_id stays for compat
- DualReportStore: expose flow_id property
- store_factory: accept flow_id as primary param, run_id as alias
- runs.py / langgraph_engine.py: generate and thread flow_id through all
trigger endpoints and run methods
- Tests: add flow_id coverage for all layers; 905 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: load flow_id in FE to resume runs and fix max_tickers cap on continuation
- Add flow_id to RunParams interface and initial state
- loadRun() now restores flow_id + max_auto_tickers from history so the next
run continues in the same flow directory (Phase 1 scan skipped, already-done
tickers skipped via skip-if-exists logic)
- startRun() spreads flow_id into the request body when set, letting the backend
reuse the existing flow directory instead of generating a fresh flow_id
- After each run, params.flow_id is updated from the response so subsequent
runs automatically continue from the same flow
- max_auto_tickers restored from run.params.max_tickers ensures the ticker cap
matches the original run; scan_tickers[:max_t] on the backend then limits
the Phase 2 queue to the user's setting even when the existing scan has more
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(mongo): fast-fail timeout + lazy ensure_indexes to avoid 30s block on fallback
MongoClient previously used pymongo's 30-second serverSelectionTimeoutMS default,
causing store_factory to hang for 30s before falling back to the filesystem when
Atlas is unreachable. Also, ensure_indexes() was called eagerly in __init__,
making every store construction attempt block on a live network call.
- Set serverSelectionTimeoutMS=5_000 so fallback is triggered in ≤5s
- Move ensure_indexes() call out of __init__ — indexes are now created lazily
on the first _save() call via a guarded self._indexes_ensured flag
- ensure_indexes() is still idempotent and safe to call explicitly in tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(store): wrap all DualReportStore mongo calls in _try_mongo() for graceful degradation
Any MongoDB exception (SSL error, ServerSelectionTimeout, auth failure) was
propagating uncaught through DualReportStore and crashing the run. Reads
would return an error instead of falling back to local, and writes would
abort mid-run without saving anything.
Introduce a single _try_mongo(fn, default) helper that:
- Executes the Mongo callable
- Catches *any* exception, logs it as WARNING with type + message
- Returns the default value so the caller continues with local-only data
Pattern per method:
writes → try mongo (fire-and-forget); always return local result
reads → try mongo first; fall back to local on None or exception
lists → try mongo; fall back to local on empty/None
Runs now complete successfully even when Atlas is unreachable or returns SSL
errors. MongoDB sync resumes automatically once connectivity is restored.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(observability): non-blocking MongoDB inserts + 5s timeout in RunLogger
Every LLM and tool callback called _append() which synchronously called
insert_one() against MongoDB. When Atlas was unreachable this blocked the
entire LangGraph run for pymongo's 30-second default timeout per event,
effectively serializing all agent work behind MongoDB retries.
Two fixes:
1. serverSelectionTimeoutMS=5_000 on the RunLogger's MongoClient — consistent
with the same fix applied to MongoReportStore.
2. MongoDB inserts are now fire-and-forget via daemon threads — _append() spawns
a Thread(target=_insert, daemon=True) and returns immediately. LLM callbacks
and tool events are never delayed by MongoDB connectivity issues.
Failures are still reported via WARNING log from the background thread.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* revert(observability): restore synchronous MongoDB inserts in RunLogger
Root cause was an IP whitelist issue on Atlas causing SSL failures, not
insert volume. The background-thread approach added unnecessary complexity.
The 5s serverSelectionTimeoutMS is retained as a defensive safeguard.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Initial plan
* feat: include portfolio holdings in auto mode pipeline analysis
In run_auto (both AgentOS and CLI), Phase 2 now loads current portfolio
holdings and merges their tickers with scan candidates before running
the per-ticker pipeline. This ensures the portfolio manager has fresh
analysis for both new opportunities and existing positions.
Key changes:
- macro_bridge.py: add candidates_from_holdings() factory
- langgraph_engine.py run_auto: merge holding tickers with scan tickers
- cli/main.py auto: load holdings, create StockCandidates, pass to run_pipeline
- cli/main.py run_pipeline: accept optional holdings_candidates parameter
- 9 new unit tests covering holdings inclusion, dedup, and graceful fallback
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/53065a07-d9f8-47be-9956-0eb4ee8c87da
* fix: normalize ticker case in dedup and clarify count display
Address code review feedback:
- Use .upper() for case-insensitive ticker comparison in run_pipeline
- Display accurate filtered scan count instead of raw candidate count
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/53065a07-d9f8-47be-9956-0eb4ee8c87da
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
## Summary
- Adds `smart_money_scanner` as a new Phase 1b node that runs sequentially after `sector_scanner`, surfacing institutional footprints via Finviz screeners
- Introduces the **Golden Overlap** strategy in `macro_synthesis`: stocks confirmed by both top-down macro themes and bottom-up Finviz signals are labelled high-conviction
- Fixes model-name badge overflow in AgentGraph (long model IDs like OpenRouter paths were visually spilling into adjacent nodes)
- Completes all documentation: ADR-014, dataflow, architecture, components, glossary, current-state
## Key Decisions (see ADR-014)
- 3 zero-parameter tools (`get_insider_buying_stocks`, `get_unusual_volume_stocks`, `get_breakout_accumulation_stocks`) instead of 1 parameterised tool — prevents LLM hallucinations on string args
- Sequential after `sector_scanner` (not parallel fan-out) — gives access to `sector_performance_report` context and avoids `MAX_TOOL_ROUNDS=5` truncation in market_movers_scanner
- Graceful fallback: `_run_finviz_screen()` catches all exceptions and returns an error string — pipeline never hard-fails on web-scraper failure
- `breakout_accumulation` (52-wk high + 2x vol = O'Neil CAN SLIM institutional signal) replaces `oversold_bounces` (RSI<30 = retail contrarian, not smart money)
## Test Plan
- [x] 6 new mocked tests in `tests/unit/test_scanner_mocked.py` (happy path, empty DF, exception, sort order)
- [x] Fixed `tests/unit/test_scanner_graph.py` — added `smart_money_scanner` mock to compilation test
- [x] 2 pre-existing test failures excluded (verified baseline before changes)
- [x] AgentGraph badge: visually verified truncation with long OpenRouter model identifiers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Added comprehensive unit tests for `fundamentals_analyst`, `market_analyst`,
`social_media_analyst`, and `news_analyst` to verify that they correctly
handle recursive tool calling via `run_tool_loop`. A MockLLM was created
to simulate a two-turn conversation (tool call request followed by a final
report generation) to ensure the `.invoke()` bug does not regress. Added
missing `build_instrument_context` imports to those agents to prevent
NameErrors.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Wrap each event-type branch (LLM start/end, tool start/end) in try/except
to prevent a single unexpected object shape from crashing the streaming loop
- Add _safe_dict() helper to guard response_metadata and usage_metadata
access — some providers return non-dict types (bound methods, etc.)
- Fix potential_text extraction: check for None AND callable before using
- Ensure all event IDs use .get() with fallback to prevent KeyError
- Fix test file: remove hardcoded /Users/Ahmet/ path, add edge-case tests
for non-dict metadata, tool events, and unknown event types
- All 725 unit tests pass, TypeScript compiles clean
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/fe6575b5-c03b-4037-bd98-a94303ae8313
The pipeline was emitting hundreds of errors:
'Invalid number of return arguments after parsing column name: Date'
Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.
- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
df['Date'] after wrap()
- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
'2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
dropping rows that would distort indicator calculations
- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
AlphaVantageError and FinnhubError — failures appear in observability
telemetry and can trigger vendor fallback
- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible
- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common
- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
(dynamic cache filename, corruption re-fetch, YFinanceError propagation,
explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
Implement comprehensive unit tests for the `_safe_fmt` utility function
to ensure robust handling of numeric formatting and edge cases.
- Test None values and custom fallbacks
- Test various numeric types (int, float, string)
- Test custom format strings
- Test error handling for invalid types
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Use `--file` instead of `--text` for adding sources to avoid ARG_MAX issues and argument injection.
- Add `--` separator to prevent positional arguments from being misinterpreted as flags.
- Reorder arguments to place options before positional ones.
- Update unit tests and add new security-focused tests.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
- Create docs/FINANCIAL_TOOLS_ANALYSIS.md with comprehensive 4-point analysis:
1. Implementation accuracy review for all indicators and metrics
2. Library assessment (stockstats vs TA-Lib vs pandas-ta)
3. Alpha Vantage debate (local calc vs API-fetched)
4. Data flow & API mapping for every financial tool
- Fix off-by-one in ttm_analysis.py: YoY revenue used quarterly[-4]
(3 quarters back) instead of quarterly[-5] (4 quarters = 1 year back)
- Add test_revenue_yoy_is_four_quarters_back test to validate the fix
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
- Added edge case test for `_find_col` in `tests/unit/test_ttm_analysis.py` (from PR #56).
- Enhanced `_clean_dataframe` in `tradingagents/dataflows/stockstats_utils.py` to parse dates, drop invalid rows, fill price gaps, and lowercase columns (combining PRs #58 and #60).
- Expanded the test suite in `tests/unit/test_stockstats_utils.py` to cover the new `_clean_dataframe` functionality.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Replaced potentially unsafe or missing tool call parsing logic with `ast.literal_eval` in `cli/main.py`.
- Created a new `parse_tool_call` helper to handle fallback parsing for LLM tool calls formatted as strings.
- Added comprehensive unit tests in `tests/unit/test_cli_main_tools.py` verifying behavior for valid strings, `ValueError`, `SyntaxError`, dicts, and objects.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Extracted the API request logic in `finnhub_news.py` to a private `_fetch_company_news_data` helper to properly catch `Exception` and return an empty list without violating the `str` return type of the main `get_company_news` function. Explicitly allows `ThirdPartyTimeoutError` to propagate to preserve timeout behavior.
Added corresponding tests to mock generic API exceptions and invalid response types. Retained the test verifying fallback behavior for invalid numeric values within `get_insider_transactions`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Fixed `_signal_vix_trend` to correctly return neutral for insufficient history (`< 21`).
- Added `test_short_history_is_neutral` to `TestSignalVixTrend`.
- Extended coverage for short history and edge cases in `TestSignalCreditSpread`, `TestSignalYieldCurve`, `TestSignalMarketBreadth`, and `TestSignalSectorRotation`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added tests to verify the dataframe cleaning logic in stockstats_utils.
Tests cover lowercasing of columns, handling non-string columns, and ensuring original dataframe is not mutated.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Removed re-exported tool imports from `tradingagents/agents/utils/agent_utils.py` to declutter the file and prevent unnecessary dependency loading. Updated all downstream modules (tests, analysts, scanners, and the trading graph) to import the required tools directly from their respective source files in `tradingagents/agents/utils/`.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Added `TestFmtPct` class to `tests/unit/test_macro_regime.py` to test the `_fmt_pct` function from `tradingagents.dataflows.macro_regime`.
The test covers `None`, positive, negative, and zero values.
Also updated `_fmt_pct` implementation to match the requested `+.1f` formatting.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- tradingagents/portfolio/risk_metrics.py: pure-Python computation of
Sharpe, Sortino, VaR, max drawdown, beta, sector concentration from
PortfolioSnapshot NAV history — no LLM, no external dependencies
- tradingagents/portfolio/__init__.py: export compute_risk_metrics
- tradingagents/agents/utils/portfolio_tools.py: 4 LangChain tools
wrapping Holding.enrich, Portfolio.enrich, ReportStore APIs, and
compute_risk_metrics so agents can access portfolio data without
reimplementing computations
- tests/portfolio/test_risk_metrics.py: 48 tests for risk metrics
- tests/unit/test_portfolio_tools.py: 19 tests for portfolio tools
- tests/portfolio/test_repository.py: fix pre-existing import error
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
- Implement integration tests for scanner vendor routing, ensuring correct routing to Alpha Vantage and fallback to yfinance.
- Create comprehensive unit tests for TTM analysis, covering metrics computation and report formatting.
- Introduce fail-fast vendor routing tests to verify immediate failure for methods not in FALLBACK_ALLOWED.
- Develop extensive integration tests for the yfinance data layer, mocking external calls to validate functionality across various financial data retrieval methods.
ahmet guzererler
Copilot
google-labs-jules[bot]