Remove the potential DoS and code-execution vulnerability by replacing `ast.literal_eval(tool_call)` with `json.loads` and `extract_json` in `cli/main.py`. Ensures strict JSON parsing without breaking tests or relying on unsafe structures.
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
* Initial plan
* feat: include portfolio holdings in auto mode pipeline analysis
In run_auto (both AgentOS and CLI), Phase 2 now loads current portfolio
holdings and merges their tickers with scan candidates before running
the per-ticker pipeline. This ensures the portfolio manager has fresh
analysis for both new opportunities and existing positions.
Key changes:
- macro_bridge.py: add candidates_from_holdings() factory
- langgraph_engine.py run_auto: merge holding tickers with scan tickers
- cli/main.py auto: load holdings, create StockCandidates, pass to run_pipeline
- cli/main.py run_pipeline: accept optional holdings_candidates parameter
- 9 new unit tests covering holdings inclusion, dedup, and graceful fallback
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/53065a07-d9f8-47be-9956-0eb4ee8c87da
* fix: normalize ticker case in dedup and clarify count display
Address code review feedback:
- Use .upper() for case-insensitive ticker comparison in run_pipeline
- Display accurate filtered scan count instead of raw candidate count
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/53065a07-d9f8-47be-9956-0eb4ee8c87da
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Add effort parameter (high/medium/low) for Claude 4.5+ and 4.6 models,
consistent with OpenAI reasoning_effort and Google thinking_level.
Also add content normalization for Anthropic responses.
- Point requirements.txt to pyproject.toml as single source of truth
- Resolve welcome.txt path relative to module for CLI portability
- Include cli/static files in package build
- Extract shared normalize_content for OpenAI Responses API and
Gemini 3 list-format responses into base_client.py
- Update README install and CLI usage instructions
- y_finance.py: replace print() with logger.warning() in bulk-stats fallback
- macro_bridge.py: add elapsed_seconds field to TickerResult, populate in
run_ticker_analysis (success + error paths)
- cli/main.py: move inline 'import time as _time' and rich.progress imports
to module level; use result.elapsed_seconds for accurate per-ticker timing
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/68fcf34c-8d55-4436-b743-f79fff68713f
Before this change, the pipeline showed a generic 'Analyzing...' spinner
for the entire multi-ticker run with no way to know which ticker was
processing or whether anything was actually working.
Changes:
- macro_bridge.py:
- run_ticker_analysis: logs '▶ Starting', '✓ complete in Xs', '✗ FAILED'
with elapsed time per ticker using logger.info/logger.error
- run_all_tickers: replaced asyncio.gather (swallows all progress) with
asyncio.as_completed + optional on_ticker_done(result, done, total)
callback; uses asyncio.Semaphore for max_concurrent control
- Added time and Callable imports
- cli/main.py run_pipeline:
- Replaced Live(Spinner) with Rich Progress bar (spinner + bar + counter
+ elapsed time)
- Prints '▷ Queued: TICKER' before analysis starts for each ticker
- on_ticker_done callback prints '✓ TICKER (N/M, Xs elapsed) → decision'
or '✗ TICKER failed ...' immediately as each ticker finishes
- Prints total elapsed time when all tickers complete
- New tradingagents/api_usage.py: Pre-run estimation of API calls per vendor
for analyze, scan, and pipeline commands. Includes Alpha Vantage tier
assessment (free: 25/day vs premium: 75/min).
- New CLI command: `estimate-api [analyze|scan|pipeline|all]`
- Enhanced observability: RunLogger.summary() now includes vendor_methods
breakdown (vendor → method → call count)
- Enhanced CLI output: All 3 command summaries (analyze, scan, pipeline)
now show per-vendor breakdown and Alpha Vantage assessment after runs
- 32 new tests in tests/unit/test_api_usage.py
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
Agent-Logs-Url: https://github.com/aguzererler/TradingAgents/sessions/bb80e772-3e03-420e-bb0e-76cfdde14a04
- Replaced potentially unsafe or missing tool call parsing logic with `ast.literal_eval` in `cli/main.py`.
- Created a new `parse_tool_call` helper to handle fallback parsing for LLM tool calls formatted as strings.
- Added comprehensive unit tests in `tests/unit/test_cli_main_tools.py` verifying behavior for valid strings, `ValueError`, `SyntaxError`, dicts, and objects.
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
* feat: daily digest consolidation and NotebookLM sync
- Add tradingagents/daily_digest.py: appends timestamped entries from
analyze and scan runs into a single reports/daily/{date}/daily_digest.md
- Add tradingagents/notebook_sync.py: uploads digest to Google NotebookLM
via nlm CLI, deleting the previous version before uploading (opt-in,
skips silently if NOTEBOOK_ID is not set)
- Add get_digest_path() helper to report_paths.py
- Hook both analyze and scan CLI commands to append + sync after each run
- Add NOTEBOOK_ID to .env.example
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: update agent memory for daily digest + NotebookLM sync
Update CURRENT_STATE, ARCHITECTURE, and COMPONENTS context files to
reflect the feat/daily-digest-notebooklm implementation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: correct nlm CLI commands and env var name for NotebookLM sync
- Use nlm note list/create/update instead of source list/add/delete
- Parse notes from {"notes": [...]} response structure
- Rename NOTEBOOK_ID -> NOTEBOOKLM_ID in both code and .env.example
- Auto-discover nlm at ~/.local/bin/nlm when not in PATH
- Tested: create on first run, update on subsequent runs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* gitignore
* feat: unify report paths under reports/daily/{date}/ hierarchy
All generated artifacts now land under a single reports/ tree:
- reports/daily/{date}/market/ for scan results (was results/macro_scan/)
- reports/daily/{date}/{TICKER}/ for per-ticker analysis (was reports/{TICKER}_{timestamp}/)
- reports/daily/{date}/{TICKER}/eval/ for eval logs (was eval_results/{TICKER}/...)
Adds tradingagents/report_paths.py with centralized path helpers used by
CLI commands, trading graph, and pipeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: structured observability logging for LLM, tool, and vendor calls
Add RunLogger (tradingagents/observability.py) that emits JSON-lines events
for every LLM call (model, agent, tokens in/out, latency), tool invocation
(tool name, args, success, latency), data vendor call (method, vendor,
success/failure, latency), and report save.
Integration points:
- route_to_vendor: log_vendor_call() on every try/catch
- run_tool_loop: log_tool_call() on every tool invoke
- ScannerGraph: new callbacks param, passes RunLogger.callback to all LLM tiers
- pipeline/macro_bridge: picks up RunLogger from thread-local, passes to TradingAgentsGraph
- cli/main.py: one RunLogger per command (analyze/scan/pipeline), write_log()
at end, summary line printed to console
Log files co-located with reports:
reports/daily/{date}/{TICKER}/run_log.jsonl (analyze)
reports/daily/{date}/market/run_log.jsonl (scan)
reports/daily/{date}/run_log.jsonl (pipeline)
Also fix test_long_response_no_nudge: update "A"*600 → "A"*2100 to match
MIN_REPORT_LENGTH=2000 threshold set in an earlier commit.
Update memory system context files (ARCHITECTURE, COMPONENTS, CONVENTIONS,
GLOSSARY, CURRENT_STATE) to document observability and report path systems.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add extract_json() utility for robust LLM JSON parsing
Handles DeepSeek R1 <think> blocks, markdown code fences, and
preamble/postamble text that LLMs wrap around JSON output.
Applied to macro_synthesis, macro_bridge, and CLI scan output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: opt-in vendor fallback — fail-fast by default (ADR 011)
Silent cross-vendor fallback corrupts signal quality when data contracts
differ (e.g., AV news has sentiment scores yfinance lacks). Only methods
with fungible data contracts (OHLCV, indices, sector/industry perf,
market movers) now get fallback. All others raise immediately.
- Add FALLBACK_ALLOWED whitelist to interface.py
- Rewrite route_to_vendor() with fail-fast/fallback branching
- Improve error messages with method name, vendors tried, and exception chaining
- Add 11 new tests in test_vendor_failfast.py
- Update ADRs 002 (superseded), 008, 010; create ADR 011
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The scanner pipeline now runs end-to-end: Phase 1 (geopolitical, market
movers, sector scanners in parallel via Ollama), Phase 2 (industry deep
dive), Phase 3 (macro synthesis via OpenRouter/DeepSeek R1).
Key changes:
- Add tool_runner.py with run_tool_loop() for inline tool execution in
scanner agents (scanner graph has no ToolNode, unlike trading graph)
- Fix vendor fallback: catch AlphaVantageError base class, raise on
total failure instead of embedding errors in return values
- Rewrite yfinance sector perf to use SPDR ETF proxies (Sector.overview
has no performance data)
- Fix Ollama remote host support in openai_client.py
- Add LangGraph state reducers for parallel fan-out writes
- Add --date CLI flag for non-interactive scanner invocation
- Fix .env loading to find keys from both CWD and project root
- Add hybrid LLM config (per-tier provider/backend_url)
- Add project tracking: DECISIONS.md, PROGRESS.md, MISTAKES.md
- Add 9 new test files covering exceptions, fallback, and routing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents UnicodeEncodeError on Windows where the default encoding
(cp1252/gbk) cannot handle Unicode characters in LLM output.
Closes#77, closes#114, closes#126, closes#215, closes#332
- Created new files for industry performance, market indices, market movers, sector performance, and topic news.
- Implemented end-to-end tests for scanner functionality, ensuring all tools return expected data formats and can save results to files.
- Added integration tests to verify scanner tools work seamlessly with the CLI scan command.
- Enhanced test coverage for individual scanner tools, validating output structure and content.
## Summary
The changes refactor the scanner tool invocation to use LangChain's StructuredTool `.invoke()` method consistently across the codebase. This includes updating the CLI scan command, rewriting tests to use the new invocation pattern, and correcting yfinance screener key mappings. The changes also add comprehensive end-to-end test suites for scanner functionality.
## Issues Found
| Severity | File:Line | Issue |
|----------|-----------|-------|
| WARNING | cli/main.py:1193-1218 | Inconsistent error handling - some tools check for "Error" prefix while others check for "No data" prefix, but the actual error messages from yfinance_scanner.py use different formats |
| WARNING | tradingagents/dataflows/yfinance_scanner.py:34 | The condition `if not data or 'quotes' not in data:` may not catch all error cases - yfinance screener can return empty data structures that evaluate to False but don't contain 'quotes' key |
| SUGGESTION | tests/test_scanner_tools.py:38-46 | Test could be more robust by checking for actual data content rather than just headers |
| SUGGESTION | cli/main.py:1193-1218 | Consider extracting the scanner tool invocation pattern into a helper function to reduce duplication |
## Detailed Findings
### File: cli/main.py:1193-1218
- **Confidence:** 85%
- **Problem:** The error handling checks for different prefixes ("Error" vs "No data") but the actual functions in yfinance_scanner.py return error messages with different formats (e.g., "Error fetching market movers for..."). This inconsistency could lead to improper error handling where error results are still saved to files.
- **Suggestion:** Standardize error checking by creating a helper function that checks if a result indicates an error, or modify the yfinance_scanner functions to return consistent error prefixes.
### File: tradingagents/dataflows/yfinance_scanner.py:34
- **Confidence:** 80%
- **Problem:** The condition `if not data or 'quotes' not in data:` assumes that if data exists, it will contain a 'quotes' key. However, yfinance screener might return data in different formats or empty objects that don't contain this key, leading to potential KeyError exceptions.
- **Suggestion:** Add more robust checking: `if not data or not isinstance(data, dict) or 'quotes' not in data:` to prevent attribute errors.
### File: tests/test_scanner_tools.py:38-46
- **Confidence:** 75%
- **Problem:** The test for market movers only checks that the result contains the expected header but doesn't verify that actual financial data is present in the table rows.
- **Suggestion:** Enhance the test to verify that data rows are present (e.g., check for table rows with actual data, not just headers).
### File: cli/main.py:1193-1218
- **Confidence:** 70%
- **Problem:** The scanner tool invocation pattern is repeated 5 times with only minor variations in arguments, violating the DRY principle.
- **Suggestion:** Extract this pattern into a helper function like `invoke_scanner_tool(tool, args, filename)` to reduce code duplication and improve maintainability.
## Recommendation
**APPROVE WITH SUGGESTIONS**
The changes are fundamentally sound and improve code consistency by standardizing on the StructuredTool `.invoke()` interface. The added test coverage is excellent. Addressing the minor issues noted above would further improve robustness and maintainability.
- Added market-wide analysis capabilities (movers, indices, sectors, industries, topic news)
- Implemented yfinance and Alpha Vantage data fetching modules
- Added LangChain tools for scanner functions
- Created scanner state definitions and graph components
- Integrated scan command into CLI
- Added configuration for scanner_data vendor routing
- Included test files for scanner components
This implements a new feature for global macro scanning to identify market-wide trends and opportunities.
- Add save prompt after analysis with organized subfolder structure
- Fix report truncation by using sequential panels instead of Columns
- Add optional full report display prompt
- Add update_analyst_statuses() for unified status logic (pending/in_progress/completed)
- Normalize analyst selection to predefined ANALYST_ORDER for consistent execution
- Add message deduplication to prevent duplicates from stream_mode=values
- Restructure streaming loop so state handlers run on every chunk
- Add StatsCallbackHandler for tracking LLM calls, tool calls, and tokens
- Integrate callbacks into TradingAgentsGraph and all LLM clients
- Dynamic agent/report counts based on selected analysts
- Fix report completion counting (tied to agent completion)
ahmet guzererler
Copilot
Yijia-Xiao
Yijia Xiao
google-labs-jules[bot]
CadeYu
luohy15
Huijae Lee
Edward Sun
maxer137