Commit Graph

41 Commits

Author SHA1 Message Date
ahmet guzererler 4c14080d73
feat(scanner): Finviz smart money scanner + Golden Overlap strategy
## Summary
- Adds `smart_money_scanner` as a new Phase 1b node that runs sequentially after `sector_scanner`, surfacing institutional footprints via Finviz screeners
- Introduces the **Golden Overlap** strategy in `macro_synthesis`: stocks confirmed by both top-down macro themes and bottom-up Finviz signals are labelled high-conviction
- Fixes model-name badge overflow in AgentGraph (long model IDs like OpenRouter paths were visually spilling into adjacent nodes)
- Completes all documentation: ADR-014, dataflow, architecture, components, glossary, current-state

## Key Decisions (see ADR-014)
- 3 zero-parameter tools (`get_insider_buying_stocks`, `get_unusual_volume_stocks`, `get_breakout_accumulation_stocks`) instead of 1 parameterised tool — prevents LLM hallucinations on string args
- Sequential after `sector_scanner` (not parallel fan-out) — gives access to `sector_performance_report` context and avoids `MAX_TOOL_ROUNDS=5` truncation in market_movers_scanner
- Graceful fallback: `_run_finviz_screen()` catches all exceptions and returns an error string — pipeline never hard-fails on web-scraper failure
- `breakout_accumulation` (52-wk high + 2x vol = O'Neil CAN SLIM institutional signal) replaces `oversold_bounces` (RSI<30 = retail contrarian, not smart money)

## Test Plan
- [x] 6 new mocked tests in `tests/unit/test_scanner_mocked.py` (happy path, empty DF, exception, sort order)
- [x] Fixed `tests/unit/test_scanner_graph.py` — added `smart_money_scanner` mock to compilation test
- [x] 2 pre-existing test failures excluded (verified baseline before changes)
- [x] AgentGraph badge: visually verified truncation with long OpenRouter model identifiers

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-03-24 16:03:17 +01:00
copilot-swe-agent[bot] 9c014a8038 Parallel pre-fetch for analyst agents to reduce LLM round-trips
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/a8c6b73e-9694-4e34-8767-36e475deb12f
2026-03-23 21:08:15 +00:00
copilot-swe-agent[bot] 69a2d72e60 Add tests for LangGraphEngine run modes (run_scan, run_pipeline, run_portfolio, run_auto)
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/ba209f7b-e985-42bb-bdce-95e73bc67bfc
2026-03-23 20:06:27 +00:00
copilot-swe-agent[bot] 9187a69ce2 Fix report saving, event storing, auto tickers, portfolio report loading, and state propagation
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/9fff2f5e-b384-4b0a-8e23-ede31af12fe2
2026-03-23 18:33:11 +00:00
google-labs-jules[bot] 3721aab110 test: add unit tests for analyst agents tool looping
Added comprehensive unit tests for `fundamentals_analyst`, `market_analyst`,
`social_media_analyst`, and `news_analyst` to verify that they correctly
handle recursive tool calling via `run_tool_loop`. A MockLLM was created
to simulate a two-turn conversation (tool call request followed by a final
report generation) to ensure the `.invoke()` bug does not regress. Added
missing `build_instrument_context` imports to those agents to prevent
NameErrors.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-23 17:22:28 +00:00
copilot-swe-agent[bot] 4c186a55e8 merge: sync with upstream TauricResearch/TradingAgents v0.2.2
Merges upstream/main into our fork, bringing in:
- Security: Remove chainlit (CVE-2026-22218), patch LangGrinch vulnerability
- Bug fixes: debate state init, UTF-8 encoding, stock data parsing, debate round config
- Features: OpenAI Responses API, five-tier rating scale, Anthropic effort support,
  exchange-qualified tickers, yfinance retry with exponential backoff
- Refactor: risk_manager renamed to portfolio_manager

Conflicts resolved preserving our fork's:
- Per-tier LLM config (quick/mid/deep think with provider overrides)
- Separate tool files (fundamental_data_tools, core_stock_tools, etc.)
- Custom tools (get_ttm_analysis, get_peer_comparison, get_sector_relative, get_macro_regime)
- Dynamic Ollama model fetching
- Enhanced fundamentals analyst prompt with TTM analysis
- Hardened stockstats/yfinance data pipeline (_load_or_fetch_ohlcv)
- AgentOS observability layer, scanner pipeline, portfolio management

Tests: 727 passed, 14 skipped
2026-03-23 12:17:25 +00:00
copilot-swe-agent[bot] a77e3f1264 fix: make _map_langgraph_event crash-proof with try/except and _safe_dict
- Wrap each event-type branch (LLM start/end, tool start/end) in try/except
  to prevent a single unexpected object shape from crashing the streaming loop
- Add _safe_dict() helper to guard response_metadata and usage_metadata
  access — some providers return non-dict types (bound methods, etc.)
- Fix potential_text extraction: check for None AND callable before using
- Ensure all event IDs use .get() with fallback to prevent KeyError
- Fix test file: remove hardcoded /Users/Ahmet/ path, add edge-case tests
  for non-dict metadata, tool events, and unknown event types
- All 725 unit tests pass, TypeScript compiles clean

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/fe6575b5-c03b-4037-bd98-a94303ae8313
2026-03-23 09:17:19 +00:00
Ahmet Guzererler 319168c74f feat: Resolve "main_portfolio" alias in portfolio routes and improve LangGraph content extraction robustness with new unit tests. 2026-03-23 10:02:52 +01:00
Ahmet Guzererler 2201afd33a fix: harden dataflows layer against silent failures and data corruption
The pipeline was emitting hundreds of errors:
  'Invalid number of return arguments after parsing column name: Date'

Root cause: after _clean_dataframe() lowercases all columns, stockstats.wrap()
promotes 'date' to the DataFrame index. Subsequent df['Date'] access caused
stockstats to try parsing 'Date' as a technical indicator name.

- stockstats_utils.py + y_finance.py: use df.index.strftime() instead of
  df['Date'] after wrap()

- Eliminates duplicated 30-line download+cache boilerplate in two places
- Cache filename is always derived from today's date — hardcoded stale date
  '2015-01-01-2025-03-25' in local mode is gone
- Corruption/truncation detection: files <50 rows or unparseable are deleted
  and re-fetched rather than silently returning bad data
- Drops on_bad_lines='skip' — malformed CSVs now raise instead of silently
  dropping rows that would distort indicator calculations

- Defined in stockstats_utils.py; raised instead of print()+return ''
- get_stockstats_indicator now raises YFinanceError on failure so errors
  surface to callers rather than delivering empty strings to LLM agents
- interface.py route_to_vendor now catches YFinanceError alongside
  AlphaVantageError and FinnhubError — failures appear in observability
  telemetry and can trigger vendor fallback

- _filter_csv_by_date_range: replaced df.columns[0] positional assumption
  with explicit search for 'time'/'timestamp'/'date' column
- ValueError re-raised (not swallowed) so bad API response shape is visible

- Replaced all print() calls in changed files with logging.getLogger()
- Added logging import + logger to alpha_vantage_common

- tests/unit/test_incident_fixes.py: 12 new unit tests covering all fixes
  (dynamic cache filename, corruption re-fetch, YFinanceError propagation,
  explicit column lookup, empty download raises)
- tests/integration/test_stockstats_live.py: 11 live tests against real
  yfinance API (all major indicators, weekend N/A, regression guard)
- All 70 tests pass (59 unit + 11 live integration)
2026-03-22 07:23:14 +01:00
ahmet guzererler 26bbe853c1
Merge pull request #77 from aguzererler/security-fix-nlm-subprocess-injection-14014624082945194366 2026-03-22 06:48:34 +01:00
ahmet guzererler 918ba6e9cd
Merge pull request #82 from aguzererler/testing-improvement-safe-fmt-16856446375371467929 2026-03-22 06:47:51 +01:00
google-labs-jules[bot] 6b6e813928 🧪 Add unit tests for _safe_fmt in finnhub_scanner
Implement comprehensive unit tests for the `_safe_fmt` utility function
to ensure robust handling of numeric formatting and edge cases.
- Test None values and custom fallbacks
- Test various numeric types (int, float, string)
- Test custom format strings
- Test error handling for invalid types

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 22:20:19 +00:00
google-labs-jules[bot] 98f51a4b16 Fix security vulnerability in NotebookLM sync subprocess calls
- Use `--file` instead of `--text` for adding sources to avoid ARG_MAX issues and argument injection.
- Add `--` separator to prevent positional arguments from being misinterpreted as flags.
- Reorder arguments to place options before positional ones.
- Update unit tests and add new security-focused tests.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 22:17:55 +00:00
ahmet guzererler 442b38dff4
Merge pull request #69 from aguzererler/copilot/review-financial-tools-implementation
Financial tools analysis doc and fix YoY revenue growth off-by-one
2026-03-21 22:41:55 +01:00
copilot-swe-agent[bot] 92ebc13ce4 Add API consumption estimation module and CLI command
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
  for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
  assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
  breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
  now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
2026-03-21 17:25:26 +00:00
copilot-swe-agent[bot] a8ccff31a0 Add YoY edge case tests for 5 and 4 quarter scenarios
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
2026-03-21 16:59:03 +00:00
copilot-swe-agent[bot] 3d76acf17d Add financial tools analysis doc and fix YoY revenue growth bug in TTM analysis
- Create docs/FINANCIAL_TOOLS_ANALYSIS.md with comprehensive 4-point analysis:
  1. Implementation accuracy review for all indicators and metrics
  2. Library assessment (stockstats vs TA-Lib vs pandas-ta)
  3. Alpha Vantage debate (local calc vs API-fetched)
  4. Data flow & API mapping for every financial tool
- Fix off-by-one in ttm_analysis.py: YoY revenue used quarterly[-4]
  (3 quarters back) instead of quarterly[-5] (4 quarters = 1 year back)
- Add test_revenue_yoy_is_four_quarters_back test to validate the fix

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/b594017b-ed84-4786-9b81-200a78eb5d76
2026-03-21 16:57:21 +00:00
google-labs-jules[bot] c8465af163 🧪 Resolve PRs #56, #58, and #60
- Added edge case test for `_find_col` in `tests/unit/test_ttm_analysis.py` (from PR #56).
- Enhanced `_clean_dataframe` in `tradingagents/dataflows/stockstats_utils.py` to parse dates, drop invalid rows, fill price gaps, and lowercase columns (combining PRs #58 and #60).
- Expanded the test suite in `tests/unit/test_stockstats_utils.py` to cover the new `_clean_dataframe` functionality.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 16:51:49 +00:00
ahmet guzererler 2ec8a17216
Merge pull request #54 from aguzererler/test-macro-regime-fmt-pct-18208719821293052000
🧪 Add tests for `_fmt_pct` in macro regime
2026-03-21 17:36:20 +01:00
ahmet guzererler 7b9510a99b
Merge pull request #61 from aguzererler/fix/agent-utils-unused-imports-948671579039994874
🧹 fix: remove unused imports from agent_utils.py
2026-03-21 17:35:19 +01:00
ahmet guzererler 86a0f5d9b7
Merge pull request #59 from aguzererler/testing/industry-deep-dive-parsing-12762969703991793840
🧪 test: add tests and parsing logic for text formats in _extract_top_sectors
2026-03-21 17:34:47 +01:00
ahmet guzererler cce5755b30
Merge pull request #62 from aguzererler/test-stockstats-utils-15966200471023157106
🧪 [testing improvement] Add unit tests for _clean_dataframe in stockstats_utils
2026-03-21 17:29:41 +01:00
ahmet guzererler b0424a36d7
Merge pull request #66 from aguzererler/jules-17548469684748509551-99819dec
🧪 [Fix Finnhub API error handling and add coverage]
2026-03-21 17:27:49 +01:00
ahmet guzererler 5c91aa8bb3
Merge pull request #50 from aguzererler/test-ttm-analysis-find-col-8347777247783205733
🧪 Add unit tests for _find_col in ttm_analysis
2026-03-21 17:26:22 +01:00
ahmet guzererler e5dcfb0e25
Merge pull request #49 from aguzererler/test-extract-article-data-17131555993247333235
🧪 Improve test coverage for `_extract_article_data`
2026-03-21 17:25:19 +01:00
ahmet guzererler 319feac087
Merge pull request #65 from aguzererler/testing-macro-regime-edge-cases-7674628334263228754
🧪 Fix VIX trend logic and add extensive tests for macro regime short history edge cases
2026-03-21 17:23:38 +01:00
google-labs-jules[bot] 6f052fffea 🧪 Implement robust tool call parsing and unit tests
- Replaced potentially unsafe or missing tool call parsing logic with `ast.literal_eval` in `cli/main.py`.
- Created a new `parse_tool_call` helper to handle fallback parsing for LLM tool calls formatted as strings.
- Added comprehensive unit tests in `tests/unit/test_cli_main_tools.py` verifying behavior for valid strings, `ValueError`, `SyntaxError`, dicts, and objects.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 14:56:12 +00:00
google-labs-jules[bot] bdaf188b33 Fix Finnhub API error handling and add coverage
Extracted the API request logic in `finnhub_news.py` to a private `_fetch_company_news_data` helper to properly catch `Exception` and return an empty list without violating the `str` return type of the main `get_company_news` function. Explicitly allows `ThirdPartyTimeoutError` to propagate to preserve timeout behavior.

Added corresponding tests to mock generic API exceptions and invalid response types. Retained the test verifying fallback behavior for invalid numeric values within `get_insider_transactions`.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 14:51:29 +00:00
google-labs-jules[bot] 177d35ede5 🧪 Fix VIX trend logic and add extensive tests for macro regime short history edge cases
- Fixed `_signal_vix_trend` to correctly return neutral for insufficient history (`< 21`).
- Added `test_short_history_is_neutral` to `TestSignalVixTrend`.
- Extended coverage for short history and edge cases in `TestSignalCreditSpread`, `TestSignalYieldCurve`, `TestSignalMarketBreadth`, and `TestSignalSectorRotation`.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 14:48:20 +00:00
google-labs-jules[bot] a7f5f67f94 🧪 [testing improvement] Add unit tests for _clean_dataframe in stockstats_utils
Added tests to verify the dataframe cleaning logic in stockstats_utils.
Tests cover lowercasing of columns, handling non-string columns, and ensuring original dataframe is not mutated.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:30:52 +00:00
google-labs-jules[bot] 3bed14e5c1 🧹 fix: remove unused imports from agent_utils.py
Removed re-exported tool imports from `tradingagents/agents/utils/agent_utils.py` to declutter the file and prevent unnecessary dependency loading. Updated all downstream modules (tests, analysts, scanners, and the trading graph) to import the required tools directly from their respective source files in `tradingagents/agents/utils/`.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:30:43 +00:00
google-labs-jules[bot] 644ce57b9c test: add tests and parsing logic for text formats in _extract_top_sectors
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:28:35 +00:00
google-labs-jules[bot] ab86ccb3a7 test: add tests for _fmt_pct in macro regime
Added `TestFmtPct` class to `tests/unit/test_macro_regime.py` to test the `_fmt_pct` function from `tradingagents.dataflows.macro_regime`.
The test covers `None`, positive, negative, and zero values.
Also updated `_fmt_pct` implementation to match the requested `+.1f` formatting.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:24:45 +00:00
google-labs-jules[bot] 9a6e8c5c7c Add edge case tests for _find_col in ttm_analysis
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:23:16 +00:00
google-labs-jules[bot] bb7d22d3a3 🧪 Add tests for _extract_article_data in yfinance_news.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 08:23:12 +00:00
google-labs-jules[bot] 3b9eef380d test: add unit tests for _tokenize in FinancialSituationMemory
Adds comprehensive tests for `_tokenize` method inside
`tradingagents/agents/utils/memory.py` focusing on regex behavior
`\b\w+\b` with case mixing, punctuation, edges (empty string),
numbers, underscores, quoting, and hyphenation strings.

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-21 01:54:23 +00:00
copilot-swe-agent[bot] 066460a501 feat: add portfolio risk metrics module and LangChain agent tools
- tradingagents/portfolio/risk_metrics.py: pure-Python computation of
  Sharpe, Sortino, VaR, max drawdown, beta, sector concentration from
  PortfolioSnapshot NAV history — no LLM, no external dependencies
- tradingagents/portfolio/__init__.py: export compute_risk_metrics
- tradingagents/agents/utils/portfolio_tools.py: 4 LangChain tools
  wrapping Holding.enrich, Portfolio.enrich, ReportStore APIs, and
  compute_risk_metrics so agents can access portfolio data without
  reimplementing computations
- tests/portfolio/test_risk_metrics.py: 48 tests for risk metrics
- tests/unit/test_portfolio_tools.py: 19 tests for portfolio tools
- tests/portfolio/test_repository.py: fix pre-existing import error

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-20 14:42:43 +00:00
Copilot 179e55f264
fix(tests): complete PR #26 — enforce socket isolation in unit tier and add test suite reference doc (#30)
* Initial plan

* fix(tests): complete PR #26 — move integration tests to tests/integration/ and fix 4 failing unit tests

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>

* docs: create docs/testing.md — comprehensive test suite reference with 5 Mermaid flowcharts

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-19 17:41:25 +01:00
ahmet guzererler d92fd9cab1
feat: NotebookLM sync with date-specific sources and consolidation (#28) 2026-03-19 15:39:25 +01:00
Ahmet Guzererler 57c9b3ecaa refactor(tests): consolidate live-API tests into integration/, move mocked tests to unit/
- Replaced 7 duplicate live-API test files in tests/ root with:
  - tests/integration/test_scanner_live.py — consolidated scanner + yfinance + vendor fallback tests (marked @pytest.mark.integration, excluded from default run)
  - tests/e2e/test_llm_e2e.py — LLM e2e test extracted from test_scanner_comprehensive
  - tests/unit/test_industry_deep_dive.py — mocked _extract_top_sectors + run_tool_loop tests
  - tests/unit/test_scanner_fallback.py — mocked AV failover tests
- Removed record_mode hardcode from integration/conftest.py (VCR.py cannot intercept curl_cffi used by yfinance; integration tests run live against real APIs on demand)

Deleted: test_scanner_tools, test_scanner_end_to_end, test_scanner_complete_e2e,
         test_scanner_final, test_scanner_comprehensive, test_scanner_fallback,
         test_industry_deep_dive

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 14:24:52 +01:00
Ahmet Guzererler 8c9183cf10 Add integration and unit tests for scanner routing, TTM analysis, vendor fail-fast, and yfinance data layer
- Implement integration tests for scanner vendor routing, ensuring correct routing to Alpha Vantage and fallback to yfinance.
- Create comprehensive unit tests for TTM analysis, covering metrics computation and report formatting.
- Introduce fail-fast vendor routing tests to verify immediate failure for methods not in FALLBACK_ALLOWED.
- Develop extensive integration tests for the yfinance data layer, mocking external calls to validate functionality across various financial data retrieval methods.
2026-03-19 13:51:57 +01:00