Commit Graph

19 Commits

Author SHA1 Message Date
copilot-swe-agent[bot] 96a2c79cb3 Implement Portfolio Manager Phases 2-5: risk evaluation, candidate prioritization, holding reviewer agent, PM decision agent, trade executor, and portfolio graph orchestration
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-20 14:44:22 +00:00
Ahmet Guzererler a17e5f3707 feat: complete portfolio data foundation — psycopg2 client, repository, tests
Replace supabase-py stubs with working psycopg2 implementation using
Supabase pooler connection string. Implement full business logic in
repository (avg cost basis, cash accounting, trade recording, snapshots).
Add 12 unit tests + 4 integration tests (51 total portfolio tests pass).
Fix cash_pct bug in models.py, update docs for psycopg2 + pooler pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 14:06:50 +01:00
copilot-swe-agent[bot] 7ea9866d1d docs: ADR-012 — raw supabase-py over Prisma/SQLAlchemy for portfolio data layer
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-20 10:48:40 +00:00
copilot-swe-agent[bot] f1cabe7a4a feat: portfolio manager data foundation — docs, SQL migration, and module scaffolding
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-20 10:40:48 +00:00
Copilot 179e55f264
fix(tests): complete PR #26 — enforce socket isolation in unit tier and add test suite reference doc (#30)
* Initial plan

* fix(tests): complete PR #26 — move integration tests to tests/integration/ and fix 4 failing unit tests

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>

* docs: create docs/testing.md — comprehensive test suite reference with 5 Mermaid flowcharts

Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: aguzererler <6199053+aguzererler@users.noreply.github.com>
2026-03-19 17:41:25 +01:00
ahmet guzererler d92fd9cab1
feat: NotebookLM sync with date-specific sources and consolidation (#28) 2026-03-19 15:39:25 +01:00
ahmet guzererler d4fefc804e
feat: daily digest consolidation and Google NotebookLM sync (#23)
* feat: daily digest consolidation and NotebookLM sync

- Add tradingagents/daily_digest.py: appends timestamped entries from
  analyze and scan runs into a single reports/daily/{date}/daily_digest.md
- Add tradingagents/notebook_sync.py: uploads digest to Google NotebookLM
  via nlm CLI, deleting the previous version before uploading (opt-in,
  skips silently if NOTEBOOK_ID is not set)
- Add get_digest_path() helper to report_paths.py
- Hook both analyze and scan CLI commands to append + sync after each run
- Add NOTEBOOK_ID to .env.example

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: update agent memory for daily digest + NotebookLM sync

Update CURRENT_STATE, ARCHITECTURE, and COMPONENTS context files to
reflect the feat/daily-digest-notebooklm implementation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: correct nlm CLI commands and env var name for NotebookLM sync

- Use nlm note list/create/update instead of source list/add/delete
- Parse notes from {"notes": [...]} response structure
- Rename NOTEBOOK_ID -> NOTEBOOKLM_ID in both code and .env.example
- Auto-discover nlm at ~/.local/bin/nlm when not in PATH
- Tested: create on first run, update on subsequent runs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 12:21:03 +01:00
ahmet guzererler a90f14c086
feat: unified report paths, structured observability logging, and memory system update (#22)
* gitignore

* feat: unify report paths under reports/daily/{date}/ hierarchy

All generated artifacts now land under a single reports/ tree:
- reports/daily/{date}/market/ for scan results (was results/macro_scan/)
- reports/daily/{date}/{TICKER}/ for per-ticker analysis (was reports/{TICKER}_{timestamp}/)
- reports/daily/{date}/{TICKER}/eval/ for eval logs (was eval_results/{TICKER}/...)

Adds tradingagents/report_paths.py with centralized path helpers used by
CLI commands, trading graph, and pipeline.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: structured observability logging for LLM, tool, and vendor calls

Add RunLogger (tradingagents/observability.py) that emits JSON-lines events
for every LLM call (model, agent, tokens in/out, latency), tool invocation
(tool name, args, success, latency), data vendor call (method, vendor,
success/failure, latency), and report save.

Integration points:
- route_to_vendor: log_vendor_call() on every try/catch
- run_tool_loop: log_tool_call() on every tool invoke
- ScannerGraph: new callbacks param, passes RunLogger.callback to all LLM tiers
- pipeline/macro_bridge: picks up RunLogger from thread-local, passes to TradingAgentsGraph
- cli/main.py: one RunLogger per command (analyze/scan/pipeline), write_log()
  at end, summary line printed to console

Log files co-located with reports:
  reports/daily/{date}/{TICKER}/run_log.jsonl   (analyze)
  reports/daily/{date}/market/run_log.jsonl     (scan)
  reports/daily/{date}/run_log.jsonl            (pipeline)

Also fix test_long_response_no_nudge: update "A"*600 → "A"*2100 to match
MIN_REPORT_LENGTH=2000 threshold set in an earlier commit.

Update memory system context files (ARCHITECTURE, COMPONENTS, CONVENTIONS,
GLOSSARY, CURRENT_STATE) to document observability and report path systems.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 09:06:40 +01:00
ahmet guzererler 8fcd58ad0f
feat: memory system v2 — skills, context files, post-commit hook (#21)
* gitignore

* feat: memory system v2 — builder/reader skills, context files, post-commit hook

Replace architecture-coordinator skill with two focused skills:
- memory-builder: extracts repo knowledge into 4-layer memory system with
  mandatory verification steps (deps, model names, agent count, ADR cross-refs)
- memory-reader: loads memory at session start, enforces ADR constraints,
  detects staleness

Generate 5 verified context files under docs/agent/context/:
- ARCHITECTURE.md (105 lines) — system design, pipelines, LLM tiers
- COMPONENTS.md (188 lines) — 17 agent factories, extension guides, test inventory
- CONVENTIONS.md (88 lines) — coding rules with source citations
- TECH_STACK.md (75 lines) — 22 deps from pyproject.toml with versions
- GLOSSARY.md (92 lines) — 50+ terms across 9 domains

Add PostToolUse hook in .claude/settings.json to remind agent to update
CURRENT_STATE.md after git commits.

All 30 factual claims audited and verified against source code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 18:38:03 +01:00
ahmet guzererler fa8a0d56fb
feat: opt-in vendor fallback — fail-fast by default (#18)
* feat: add extract_json() utility for robust LLM JSON parsing

Handles DeepSeek R1 <think> blocks, markdown code fences, and
preamble/postamble text that LLMs wrap around JSON output.
Applied to macro_synthesis, macro_bridge, and CLI scan output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: opt-in vendor fallback — fail-fast by default (ADR 011)

Silent cross-vendor fallback corrupts signal quality when data contracts
differ (e.g., AV news has sentiment scores yfinance lacks). Only methods
with fungible data contracts (OHLCV, indices, sector/industry perf,
market movers) now get fallback. All others raise immediately.

- Add FALLBACK_ALLOWED whitelist to interface.py
- Rewrite route_to_vendor() with fail-fast/fallback branching
- Improve error messages with method name, vendors tried, and exception chaining
- Add 11 new tests in test_vendor_failfast.py
- Update ADRs 002 (superseded), 008, 010; create ADR 011

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-18 14:25:38 +01:00
Ahmet Guzererler b50bc30217 Merge branch 'claude/objective-galileo': test fixes and plans
- fix: resolve 12 pre-existing test failures across 5 test files
  (callable() check, env var isolation, integration markers, stale assertions)
- docs: add implementation plans 011 (opt-in fallback) and 012 (test fixes)
- chore: unblock docs/agent/plans/ from .gitignore

Full offline suite: 388 passed, 70 deselected, 0 failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 12:16:02 +01:00
Ahmet Guzererler ff3e2817bf docs: add implementation plans for opt-in fallback (011) and test fixes (012)
Also unblock docs/agent/plans/ from .gitignore — the plans/ rule was
intended for generated scan-result plans, not design documents.

- 011-opt-in-vendor-fallback.md: fail-fast-by-default with FALLBACK_ALLOWED
  whitelist for fungible data tools (OHLCV, indices, sector/industry perf,
  market movers); 4-phase plan covering interface.py change, new tests,
  ADR docs, and verification
- 012-fix-preexisting-test-failures.md: root-cause analysis and exact fixes
  for all 12 pre-existing failures across 5 test files (now implemented)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 12:12:07 +01:00
ahmet guzererler 26cd4c8b78
feat: Finnhub integration layer, 141 tests, and vendor evaluation report (#16)
* feat: add Finnhub integration layer, tests, and evaluation report

Adds a complete Finnhub data vendor integration as a supplementary
source alongside Alpha Vantage — zero changes to existing functionality.

New dataflow modules:
- finnhub_common.py: exception hierarchy, thread-safe rate limiter (60/min), _make_api_request
- finnhub_stock.py: get_stock_candles, get_quote
- finnhub_fundamentals.py: get_company_profile, get_financial_statements, get_basic_financials
- finnhub_news.py: get_company_news, get_market_news, get_insider_transactions
- finnhub_scanner.py: market movers (S&P 500 basket workaround), indices, sectors, topic news
- finnhub_indicators.py: SMA, EMA, MACD, RSI, BBANDS, ATR via /indicator endpoint
- finnhub.py: facade re-exporting all public functions

New tests:
- test_finnhub_integration.py: 100 offline (mocked HTTP) tests — all passing
- test_finnhub_live_integration.py: 41 live integration tests — skip gracefully when FINNHUB_API_KEY unset

Evaluation report (docs/finnhub_evaluation.md):
- Full coverage matrix vs Alpha Vantage across 5 data categories
- Free tier viability analysis (60 calls/min)
- Unique capabilities: earnings calendar, economic calendar, XBRL as-filed filings
- Recommendation: add as supplementary vendor for calendar data only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: mark paid-tier Finnhub endpoints; update evaluation with live results

Live testing with free-tier key confirmed:
- /quote, /stock/profile2, /stock/metric, /company-news, /news,
  /stock/insider-transactions → all free tier (27 live tests PASS)
- /stock/candle, /financials-reported, /indicator → paid tier HTTP 403
  (14 tests now properly skipped with @pytest.mark.paid_tier)

Also:
- Register 'integration' and 'paid_tier' markers in pyproject.toml
- Update docs/finnhub_evaluation.md with confirmed endpoint availability table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: wire Finnhub into routing layer — insider txns, calendars, fallback

Changes:
- interface.py: Finnhub added as third vendor (alongside yfinance + AV)
  - get_insider_transactions: Finnhub primary (free, + MSPR bonus signal)
  - get_market_indices/sector_performance/topic_news: Finnhub added as option
  - Fallback catch extended: (AlphaVantageError, FinnhubError, ConnectionError, TimeoutError)
  - New calendar_data category with get_earnings_calendar + get_economic_calendar
- finnhub_scanner.py: added get_earnings_calendar_finnhub, get_economic_calendar_finnhub
  (FOMC/CPI/NFP/GDP events + earnings beats — unique, not in AV at any tier)
- finnhub.py: re-exports new calendar functions
- scanner_tools.py: @tool wrappers for get_earnings_calendar, get_economic_calendar
- default_config.py: tool_vendors["get_insider_transactions"]="finnhub",
  calendar_data vendor category defaulting to "finnhub"
- .env.example: FINNHUB_API_KEY documented
- docs/agent/decisions/010-finnhub-vendor-integration.md: ADR for this decision

All 173 offline tests pass. ADR 002 constraints respected throughout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 11:28:43 +01:00
Ahmet Guzererler cf636232aa fix: resolve 12 pre-existing test failures across 5 test files
Root causes fixed:
- test_config_wiring.py: `callable()` returns False on LangChain @tool
  objects — replaced with `hasattr(x, "invoke")` check
- test_env_override.py: `load_dotenv()` in default_config.py re-reads
  .env on importlib.reload(), leaking user's TRADINGAGENTS_* env vars
  into isolation tests — mock env vars before reload
- test_scanner_comprehensive.py: LLM-calling test was not marked
  @pytest.mark.integration — added marker so offline runs skip it
- test_scanner_fallback.py: assertions used stale `_output_files` list
  from a previous run when output dir already existed — clear dir in
  setUp; also fixed tool-availability check using hasattr(x, "invoke")
- test_scanner_graph.py: output-file path assertions used hardcoded
  date string instead of fixture date; graph node assertions checked
  for removed node names

Full offline suite: 388 passed, 70 deselected, 0 failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 11:11:00 +01:00
Ahmet Guzererler 04b7efdb68 feat: wire Finnhub into routing layer — insider txns, calendars, fallback
Changes:
- interface.py: Finnhub added as third vendor (alongside yfinance + AV)
  - get_insider_transactions: Finnhub primary (free, + MSPR bonus signal)
  - get_market_indices/sector_performance/topic_news: Finnhub added as option
  - Fallback catch extended: (AlphaVantageError, FinnhubError, ConnectionError, TimeoutError)
  - New calendar_data category with get_earnings_calendar + get_economic_calendar
- finnhub_scanner.py: added get_earnings_calendar_finnhub, get_economic_calendar_finnhub
  (FOMC/CPI/NFP/GDP events + earnings beats — unique, not in AV at any tier)
- finnhub.py: re-exports new calendar functions
- scanner_tools.py: @tool wrappers for get_earnings_calendar, get_economic_calendar
- default_config.py: tool_vendors["get_insider_transactions"]="finnhub",
  calendar_data vendor category defaulting to "finnhub"
- .env.example: FINNHUB_API_KEY documented
- docs/agent/decisions/010-finnhub-vendor-integration.md: ADR for this decision

All 173 offline tests pass. ADR 002 constraints respected throughout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 09:41:41 +01:00
Ahmet Guzererler 5b6d3a0c3f test: mark paid-tier Finnhub endpoints; update evaluation with live results
Live testing with free-tier key confirmed:
- /quote, /stock/profile2, /stock/metric, /company-news, /news,
  /stock/insider-transactions → all free tier (27 live tests PASS)
- /stock/candle, /financials-reported, /indicator → paid tier HTTP 403
  (14 tests now properly skipped with @pytest.mark.paid_tier)

Also:
- Register 'integration' and 'paid_tier' markers in pyproject.toml
- Update docs/finnhub_evaluation.md with confirmed endpoint availability table

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 08:57:00 +01:00
Ahmet Guzererler 25067a56f7 feat: add Finnhub integration layer, tests, and evaluation report
Adds a complete Finnhub data vendor integration as a supplementary
source alongside Alpha Vantage — zero changes to existing functionality.

New dataflow modules:
- finnhub_common.py: exception hierarchy, thread-safe rate limiter (60/min), _make_api_request
- finnhub_stock.py: get_stock_candles, get_quote
- finnhub_fundamentals.py: get_company_profile, get_financial_statements, get_basic_financials
- finnhub_news.py: get_company_news, get_market_news, get_insider_transactions
- finnhub_scanner.py: market movers (S&P 500 basket workaround), indices, sectors, topic news
- finnhub_indicators.py: SMA, EMA, MACD, RSI, BBANDS, ATR via /indicator endpoint
- finnhub.py: facade re-exporting all public functions

New tests:
- test_finnhub_integration.py: 100 offline (mocked HTTP) tests — all passing
- test_finnhub_live_integration.py: 41 live integration tests — skip gracefully when FINNHUB_API_KEY unset

Evaluation report (docs/finnhub_evaluation.md):
- Full coverage matrix vs Alpha Vantage across 5 data categories
- Free tier viability analysis (60 calls/min)
- Unique capabilities: earnings calendar, economic calendar, XBRL as-filed filings
- Recommendation: add as supplementary vendor for calendar data only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 07:54:21 +01:00
Ahmet Guzererler 10484684bc docs: update memory files after PR #13 (Industry Deep Dive quality fix)
- CURRENT_STATE.md: remove Industry Deep Dive blocker (resolved), update
  test count 38 → 53, add PR #13 to Recent Progress, update milestone focus
- decisions/009-industry-deep-dive-quality.md: new ADR documenting the
  three-pronged fix (enriched data, explicit sector routing, tool-call nudge)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 20:20:39 +01:00
Ahmet Guzererler 0b4260a4e2 feat: add agentic memory scaffold and migrate tracking files to docs/agent/
Migrate DECISIONS.md, MISTAKES.md, PROGRESS.md, agents/, plans/, and
tradingagents/llm_clients/TODO.md into a structured docs/agent/ scaffold
with ADR-style decisions, plans, templates, and a live state tracker.

This gives agent workflows a standard memory structure for decisions,
plans, logs, and session continuity via CURRENT_STATE.md.

Agent-Ref: docs/agent/plans/global-macro-scanner.md
State-Updated: Yes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 17:14:11 +01:00