diff --git a/docs/LangGraphFlow.png b/docs/LangGraphFlow.png new file mode 100644 index 00000000..ac2c8269 Binary files /dev/null and b/docs/LangGraphFlow.png differ diff --git a/docs/TradeDog_Roadmap.md b/docs/TradeDog_Roadmap.md index 64712a89..1e40f136 100644 --- a/docs/TradeDog_Roadmap.md +++ b/docs/TradeDog_Roadmap.md @@ -1,671 +1,493 @@ # TradeDog — Solo Developer Roadmap -### From Research Framework to Autonomous Trading Platform + +**From Research Framework to Autonomous Trading Platform** *Built on TauricResearch/TradingAgents + LangGraph | NYSE + NASDAQ | Long-Only* +For code snippets, schemas, and architecture patterns see [design_reference.md](design_reference.md). + --- ## What You Already Have -Your fork already gives you the core intelligence layer: -| Agent | Role | Status | -|---|---|---| -| Fundamentals Analyst | Financials, earnings, insider data | ✅ Built | -| Sentiment Analyst | Reddit/Twitter mood scoring | ✅ Built | -| News Analyst | Macro/event impact | ✅ Built | -| Technical Analyst | Indicators, patterns | ✅ Built | -| Bull/Bear Researcher | Debate-based conviction | ✅ Built | -| Trader Agent | Decision synthesis | ✅ Built | -| Risk Manager | Exposure checks | ✅ Built | -| Fund Manager | Final approval | ✅ Built | +| Agent | Role | Status | +| -------------------- | ---------------------------------- | ------ | +| Fundamentals Analyst | Financials, earnings, insider data | Done | +| Sentiment Analyst | Reddit/Twitter mood scoring | Done | +| News Analyst | Macro/event impact | Done | +| Technical Analyst | Indicators, patterns | Done | +| Bull/Bear Researcher | Debate-based conviction | Done | +| Trader Agent | Decision synthesis | Done | +| Risk Manager | Exposure checks | Done | +| Fund Manager | Final approval | Done | -**What's missing:** Production-grade execution layer, auto-buy logic, exit/monitoring loop, position tracking, conviction scoring, and a dashboard to observe all of it. ---- - -## The Full Architecture Target - -``` -[Watchlist Scanner] - ↓ -[Research Pipeline] ← Fundamentals / Sentiment / News / Technical - ↓ -[Bull vs Bear Debate] - ↓ -[Trader → Risk Manager → Fund Manager] - ↓ - Conviction Score - ↓ -[Auto-Buy Engine] ←→ [Broker API: Alpaca / IBKR] - ↓ -[Position Monitor — runs every N minutes] - ├── Profit target hit → SELL - ├── Trailing stop triggered → SELL - ├── Reversal signal detected → SELL - └── Time-based exit (optional) - ↓ -[Trade Logger + Dashboard] -``` +**What's missing:** Execution layer, auto-buy logic, exit/monitoring loop, position tracking, conviction scoring, dashboard. --- ## Phase Overview -| Phase | Focus | Duration | Deliverable | -|---|---|---|---| -| **0** | Codebase audit & cleanup | 1–2 weeks | Clean, documented fork | -| **1** | Data layer hardening + watchlist | 1–2 weeks | Reliable data, fallbacks, liquidity filters | -| **2** | Paper trading execution layer | 3–4 weeks | Auto-buy/sell in simulation | -| **3** | Conviction scoring + signal control | 2–3 weeks | Buy only when score crosses threshold | -| **4** | Position monitoring & auto-exit | 3–4 weeks | Trailing stops, profit targets, reversals | -| **5** | Portfolio-level risk controls | 2–3 weeks | Max positions, sector exposure, drawdown limits | -| **6** | Dashboard & observability | 2–3 weeks | Web UI showing live state | -| **7** | Live trading (gradual) | Ongoing | Real money, small size, scaled carefully | -**Total realistic timeline: 5–7 months** for a solo developer going at a sustainable pace. +| Phase | Focus | Duration | Milestone | +| ----- | ----------------------------------- | --------- | --------- | +| **0** | Codebase audit and cleanup | 1-2 weeks | v0.1 | +| **1** | Data layer hardening + watchlist | 1-2 weeks | v0.1 | +| **2** | Paper trading execution layer | 3-4 weeks | v0.1 | +| **3** | Conviction scoring + signal control | 2-3 weeks | v0.2 | +| **4** | Position monitoring and auto-exit | 3-4 weeks | v0.2 | +| **5** | Portfolio-level risk controls | 2-3 weeks | v0.3 | +| **6** | Dashboard and observability | 2-3 weeks | v0.3 | +| **7** | Live trading (gradual) | Ongoing | v1.0 | + + +**Total realistic timeline: 5-7 months** at a sustainable pace. --- -## Phase 0 — Codebase Audit & Foundation -**Duration: 1–2 weeks** +# Milestone v0.1 — Minimal End-to-End Loop -### Goals -Understand every file before adding anything. Establish a clean base. +*Manual trigger, single ticker analysis through to a paper trade execution.* -### Tasks +--- -**Week 1 — Read and map everything** -- [ ] Read all files under `tradingagents/` top to bottom -- [ ] Draw a flow diagram showing how `TradingAgentsGraph.propagate()` calls each agent -- [ ] Document what each agent returns (format, fields, meaning) -- [ ] Identify where `FinnHub` API is called and what endpoints are used -- [ ] Identify what config options exist in `default_config.py` -- [ ] Run `main.py` and `test.py` and make sure they work from scratch in your environment -- [ ] Set up a `.env` file with all required keys (FinnHub, OpenAI, etc.) +## Phase 0 — Codebase Audit and Foundation -**Week 2 — Clean and prepare** -- [ ] Add Python type hints and docstrings to any function that doesn't have them -- [ ] Create a `docs/architecture.md` with your flow diagram -- [ ] Create a `docs/agent_contracts.md` documenting each agent's input/output schema -- [ ] Set up `pytest` and write one smoke test per agent -- [ ] Set up a `dev` branch — all new work goes to `dev`, only tested code merges to `main` -- [ ] Add `logging` (not print) throughout using Python's `logging` module -- [ ] Pin all dependency versions in `requirements.txt` +**Goal:** Understand every file before adding anything. Establish a clean, documented, testable base. -### Key Files to Study -``` -tradingagents/ -├── graph/trading_graph.py ← Main orchestrator, start here -├── agents/ ← One file per agent role -├── dataflows/ ← Data fetching layer -└── default_config.py ← All tunable knobs -``` +**Duration:** 1-2 weeks -### Decision Points -- Confirm which LLM provider you'll use for production (Claude Sonnet is cost-effective for the quick agents; use it for analysts, use a reasoning model for Trader/Risk Manager) -- Confirm your broker choice now — **Alpaca** is strongly recommended for paper trading (free paper API, full NYSE/NASDAQ coverage). IBKR is a solid alternative for live trading later. +### Week 1 — Read and Map + + +| # | Task | ~Hours | Files | +| --- | ---------------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------- | +| 0.1 | Read all files under `tradingagents/` top to bottom | 4h | `tradingagents/`** | +| 0.2 | Draw a flow diagram of how `TradingAgentsGraph.propagate()` calls each agent | 2h | `tradingagents/graph/trading_graph.py`, `tradingagents/graph/propagation.py` | +| 0.3 | Document what each agent returns (format, fields, meaning) | 3h | `tradingagents/agents/`** | +| 0.4 | mm | 2h | `tradingagents/dataflows/**` | +| 0.5 | Review all config options in `default_config.py` | 1h | `tradingagents/default_config.py` | +| 0.6 | Run `main.py` and `test.py` end-to-end in your environment | 2h | `main.py`, `test.py` | +| 0.7 | Set up `.env` with all required API keys | 1h | `.env`, `.env.example` | + + +- 0.1 — Read all source files under `tradingagents/` +- 0.2 — Draw propagation flow diagram +- 0.3 — Document each agent's input/output contract +- 0.4 — Map all data API calls and endpoints +- 0.5 — Review all config options +- 0.6 — Run `main.py` and `test.py` successfully +- 0.7 — Set up `.env` with all keys + +### Week 2 — Clean and Prepare + + +| # | Task | ~Hours | Files | +| ---- | -------------------------------------------------------------------------------- | ------ | ------------------------------------------- | +| 0.8 | Add type hints and docstrings to functions missing them | 4h | `tradingagents/**` | +| 0.9 | Create `docs/agent_contracts.md` documenting each agent's I/O schema | 2h | `docs/agent_contracts.md` | +| 0.10 | Set up `pytest` with a conftest and one smoke test per agent | 3h | `tests/conftest.py`, `tests/test_agents.py` | +| 0.11 | Create `dev` branch — all new work goes there, only tested code merges to `main` | 0.5h | git | +| 0.12 | Replace all `print()` with Python `logging` module calls | 3h | `tradingagents/**` | +| 0.13 | Pin all dependency versions in `requirements.txt` | 1h | `requirements.txt` | +| 0.14 | Update `docs/architecture.md` with your flow diagram from 0.2 | 1h | `docs/architecture.md` | + + +- 0.8 — Add type hints and docstrings +- 0.9 — Create `docs/agent_contracts.md` +- 0.10 — Set up pytest with smoke tests +- 0.11 — Create `dev` branch +- 0.12 — Replace print statements with logging +- 0.13 — Pin dependency versions +- 0.14 — Update architecture doc with flow diagram + +### Decision Points (resolve before moving on) + +- Choose LLM provider for production (recommendation: Claude Sonnet for analysts, reasoning model for Trader/Risk Manager) +- Choose broker for paper trading (recommendation: Alpaca — free paper API, full NYSE/NASDAQ) + +### Definition of Done + +- You can run `main.py` cleanly and get a trading decision for any ticker +- `pytest` passes with at least one test per agent +- `docs/architecture.md` and `docs/agent_contracts.md` exist and are accurate +- All dependencies are pinned +- Logging works (no raw print statements) --- ## Phase 1 — Data Layer Hardening + Watchlist -**Duration: 1–2 weeks** *(shorter than originally planned — FinnHub already covers NYSE/NASDAQ well)* -### Goals -The framework already targets US stocks, so this phase is about making the data layer robust and production-grade rather than adding a new market. You want reliable, clean OHLCV and fundamental data before any money is on the line. +**Goal:** Make the data layer robust and production-grade. Reliable, clean OHLCV and fundamental data before any money touches the system. -### Data Source Strategy +**Duration:** 1-2 weeks -You're already in great shape here. Both FinnHub and yfinance have excellent US coverage. +**Prereqs:** Phase 0 complete -| Source | Use Case | Cost | Notes | -|---|---|---|---| -| FinnHub | Real-time quotes, news, insider trades | Free tier | Already wired in | -| `yfinance` | OHLCV history, fundamentals fallback | Free | Add as secondary/fallback | -| Polygon.io | Higher-quality tick data if needed later | Paid | Skip for now | -| Alpha Vantage | Alternative fundamentals | Free tier | Keep as backup | -**Implementation Plan** -- [ ] Add `yfinance` as a fallback in `dataflows/` — if FinnHub returns empty or errors, fall through to yfinance -- [ ] Add rate limit handling and retry logic for FinnHub (it drops requests under load) -- [ ] Standardize the OHLCV return format into a `MarketData` dataclass used by all agents -- [ ] Add a data validation step — reject and log any ticker that returns incomplete data -- [ ] Cache responses to disk (pickle or SQLite) so a re-run doesn't re-hit the API +| # | Task | ~Hours | Files | +| ---- | ------------------------------------------------------------------------------------------- | ------ | ----------------------------------------- | +| 1.1 | Add yfinance as a fallback in `dataflows/` — if primary source errors, fall through | 3h | `tradingagents/dataflows/interface.py` | +| 1.2 | Add rate limit handling and retry logic for API calls | 2h | `tradingagents/dataflows/interface.py` | +| 1.3 | Create a `MarketData` dataclass — standardized OHLCV format used by all agents | 2h | `tradingagents/dataflows/models.py` (new) | +| 1.4 | Add data validation — reject and log any ticker returning incomplete data | 2h | `tradingagents/dataflows/interface.py` | +| 1.5 | Add disk caching for API responses (pickle or SQLite) so re-runs don't re-hit APIs | 3h | `tradingagents/dataflows/` | +| 1.6 | Create `watchlist/watchlist.json` with starter tickers (~36 across sectors) | 1h | `watchlist/watchlist.json` (new) | +| 1.7 | Implement liquidity filter (min volume + min market cap) | 2h | `watchlist/filters.py` (new) | +| 1.8 | Test: run `propagate()` on 10 tickers, verify clean data with no empty fields or NaN prices | 2h | `tests/test_data_layer.py` (new) | +| 1.9 | Test: simulate API failure and verify fallback activates | 1h | `tests/test_data_layer.py` | +| 1.10 | Log API call counts per run to estimate monthly costs | 1h | `tradingagents/dataflows/` | -**Watchlist Setup** -Create a curated starting watchlist. Don't try to scan the whole NYSE — start focused: -```json -{ - "large_cap": ["AAPL", "MSFT", "NVDA", "GOOGL", "AMZN", "META", "JPM", "UNH", "V", "MA"], - "growth": ["CRWD", "SNOW", "NET", "DDOG", "SMCI", "ARM", "PLTR"], - "value": ["BRK-B", "JNJ", "PG", "KO", "WMT", "HD", "MCD"], - "financials": ["GS", "MS", "BAC", "C", "WFC"], - "energy": ["XOM", "CVX", "COP", "SLB"] -} -``` -This gives you ~36 quality tickers across sectors. Plenty to start. -**Market Hours & Scheduling** -- NYSE/NASDAQ: 9:30 AM – 4:00 PM ET -- Pre-market analysis run: 8:00–9:15 AM ET (agents analyze, build signals) -- Market open: execution window 9:30–10:30 AM ET (buy signals fire here) -- Monitoring loop: runs every 5 min during market hours -- After-hours: position review, log summary, prep next day's watchlist +- 1.1 — yfinance fallback in dataflows +- 1.2 — Rate limit handling and retry logic +- 1.3 — `MarketData` dataclass +- 1.4 — Data validation step +- 1.5 — Disk caching for API responses +- 1.6 — Create watchlist JSON +- 1.7 — Liquidity filter +- 1.8 — Test: propagate on 10 tickers, verify clean data +- 1.9 — Test: API failure fallback +- 1.10 — Log API call counts per run -**Liquidity Filter** -Only analyze stocks with sufficient liquidity to avoid slippage: -```python -MIN_AVG_DAILY_VOLUME = 1_000_000 # 1M shares/day minimum -MIN_MARKET_CAP = 2_000_000_000 # $2B market cap minimum -``` +See [design_reference.md — Watchlist Design](design_reference.md#watchlist-design) and [Data Source Strategy](design_reference.md#data-source-strategy) for details. -**Testing** -- [ ] Run full `propagate()` on 10 tickers across different sectors -- [ ] Verify clean data returns for all 10 — no empty fields, no NaN prices -- [ ] Simulate a FinnHub API failure and verify yfinance fallback activates -- [ ] Log API call counts per run so you can estimate monthly costs +### Definition of Done + +- `propagate()` works on 10+ tickers with no data errors +- API failure gracefully falls back to yfinance +- Watchlist JSON exists with ~36 tickers +- API calls are cached so a second run is instant --- ## Phase 2 — Paper Trading Execution Layer -**Duration: 3–4 weeks** -### Goals -Connect the agent decision to an actual order. Use paper trading only. No real money yet. This is the most critical phase — get it right before going live. +**Goal:** Connect the agent decision to an actual order. Paper trading only, no real money. This is the most critical phase. -### Broker Setup: Alpaca (Recommended Start) -Alpaca offers a free paper trading API with full NYSE and NASDAQ coverage. It's the best starting point for paper trading — no account minimums, clean REST API, and a Python SDK. +**Duration:** 3-4 weeks -**Alpaca Setup** -```bash -pip install alpaca-trade-api -# or the newer: -pip install alpaca-py -``` +**Prereqs:** Phase 1 complete -**Architecture** -Create a new module: `tradingagents/execution/` +| # | Task | ~Hours | Files | +| ---- | ------------------------------------------------------------------------------- | ------ | -------------------------------------------------------------------------- | +| 2.1 | Create `database/schema.sql` and `database/db.py` with SQLite setup | 3h | `database/schema.sql`, `database/db.py` (new) | +| 2.2 | Create `database/models.py` with `Position`, `Order`, `AccountInfo` dataclasses | 2h | `database/models.py` (new) | +| 2.3 | Define `BrokerInterface` abstract base class | 2h | `execution/broker_interface.py` (new) | +| 2.4 | Build `PaperBroker` implementing `BrokerInterface` with SQLite backend | 4h | `execution/paper_broker.py` (new) | +| 2.5 | Wire Fund Manager agent approval to `BrokerInterface.place_market_buy()` | 3h | `tradingagents/graph/trading_graph.py`, `execution/order_manager.py` (new) | +| 2.6 | Test: run `propagate()` on AAPL and NVDA, confirm position records are created | 2h | `tests/test_execution.py` (new) | +| 2.7 | Add position sizing logic (use formula from design ref) | 2h | `portfolio/position_sizer.py` (new) | +| 2.8 | Build `AlpacaBroker` implementing `BrokerInterface` | 4h | `execution/alpaca_broker.py` (new) | +| 2.9 | Add Alpaca paper credentials to `.env` and config | 1h | `.env`, `tradingagents/default_config.py` | +| 2.10 | Switch config to use `AlpacaBroker` with paper mode | 1h | `tradingagents/default_config.py` | +| 2.11 | Run 10 paper trades end-to-end, inspect results in DB | 3h | manual | -``` -execution/ -├── broker_interface.py ← Abstract base class -├── alpaca_broker.py ← Alpaca implementation -├── ibkr_broker.py ← IBKR implementation (Phase 7) -├── paper_broker.py ← Local simulation (no API needed) -└── order_manager.py ← Order lifecycle tracking -``` -**The Broker Interface (define this first)** -```python -class BrokerInterface: - def place_market_buy(self, ticker: str, qty: int) -> Order: ... - def place_market_sell(self, ticker: str, qty: int) -> Order: ... - def get_positions(self) -> list[Position]: ... - def get_account(self) -> AccountInfo: ... - def cancel_order(self, order_id: str) -> bool: ... -``` +- 2.1 — SQLite database setup +- 2.2 — Data models (Position, Order, AccountInfo) +- 2.3 — BrokerInterface abstract class +- 2.4 — PaperBroker with SQLite backend +- 2.5 — Wire Fund Manager to broker execution +- 2.6 — Test: propagate creates position records +- 2.7 — Position sizing logic +- 2.8 — AlpacaBroker implementation +- 2.9 — Alpaca paper credentials in config +- 2.10 — Switch to AlpacaBroker in paper mode +- 2.11 — Run 10 paper trades, inspect results -Start with `PaperBroker` — a pure Python simulation that tracks positions in a local SQLite database. This lets you test the full loop without any API. +See [design_reference.md — Execution Layer Architecture](design_reference.md#execution-layer-architecture), [Broker Interface](design_reference.md#broker-interface), [Position Sizing Formula](design_reference.md#position-sizing-formula), and [Database Schema](design_reference.md#database-schema) for implementation details. -**Tasks** -- [ ] Build `PaperBroker` with SQLite backend first -- [ ] Create `Position` and `Order` dataclasses -- [ ] Wire the Fund Manager agent's approval → `BrokerInterface.place_market_buy()` -- [ ] Test: run `propagate()` on AAPL and NVDA, confirm they create position records -- [ ] Add position sizing logic (see Phase 3) -- [ ] Build `AlpacaBroker` implementing the same interface -- [ ] Switch config to use `AlpacaBroker` with paper credentials -- [ ] Run 10 paper trades end-to-end, inspect results +### Definition of Done -**Position Size Formula (start simple)** -```python -def calculate_position_size(account_value, conviction_score, price, max_position_pct=0.05): - # Never risk more than 5% of account on one trade - max_dollars = account_value * max_position_pct - # Scale by conviction (0.0 to 1.0) - dollars_to_invest = max_dollars * conviction_score - shares = int(dollars_to_invest / price) - return max(1, shares) -``` +- Running `propagate()` on a ticker results in a paper trade being recorded in SQLite +- `PaperBroker` and `AlpacaBroker` both pass the same test suite +- 10 paper trades executed end-to-end with no errors +- Position records visible in the database --- -## Phase 3 — Conviction Scoring & Auto-Buy Control -**Duration: 2–3 weeks** +# Milestone v0.2 — Conviction + Profit Guardian -### Goals -Not every agent decision should trigger a buy. Add a conviction score system so the platform only buys when multiple agents agree strongly. - -### Conviction Score Design - -The agents currently produce a BUY/SELL/HOLD decision with rationale. Extend this to also produce a **conviction score (0–100)**. - -**Scoring Approach — Weight Each Agent's Input** -```python -CONVICTION_WEIGHTS = { - "technical": 0.25, # RSI, MACD, moving average signals - "fundamental": 0.25, # P/E, growth, financial health - "sentiment": 0.20, # Social/news mood - "bull_bear": 0.30, # Debate outcome (most decisive) -} - -def calculate_conviction(agent_signals: dict) -> float: - score = 0 - for agent, weight in CONVICTION_WEIGHTS.items(): - # Each agent returns -1.0 (strong sell) to +1.0 (strong buy) - score += agent_signals[agent] * weight - return score # -1.0 to +1.0 -``` - -**Auto-Buy Rules** -```python -BUY_THRESHOLD = 0.65 # Must have >65% conviction to buy -MIN_AGENTS_AGREE = 3 # At least 3 of 4 analyst agents must agree direction -MAX_POSITIONS = 10 # Never hold more than 10 stocks at once -COOLDOWN_HOURS = 24 # Don't re-analyze same stock within 24h of last trade -``` - -### Tasks -- [ ] Add `conviction_score: float` to the `TradingAgentsGraph` output -- [ ] Prompt each analyst agent to return a numerical score alongside their text analysis -- [ ] Build `ConvictionGate` — checks all rules before passing to execution -- [ ] Add a `signal_log` database table: ticker, timestamp, conviction score, decision, action taken -- [ ] Test: force a high-conviction scenario and verify buy fires; force low and verify it doesn't -- [ ] Add a dry-run mode flag: logs what would have happened without executing - -**Prompt Addition for Each Analyst Agent** -Add to each analyst's system prompt: -``` -At the end of your analysis, always output a JSON block: -{"signal": "BUY|SELL|HOLD", "conviction": 0.85, "key_reason": "..."} -``` -Then parse this structured output in the graph state. +*Only buy when confident. Auto-exit when conditions are met.* --- -## Phase 4 — Position Monitoring & Auto-Exit -**Duration: 3–4 weeks** +## Phase 3 — Conviction Scoring and Auto-Buy Control -### Goals -This is the "profit guard" layer. Once a position is open, a separate monitoring loop checks it every N minutes and auto-exits based on your rules. +**Goal:** Not every agent decision should trigger a buy. Add conviction scoring so the platform only buys when multiple agents agree strongly. -### Exit Conditions +**Duration:** 2-3 weeks -| Condition | Rule | Notes | -|---|---|---| -| Profit target | Exit when gain ≥ 15% | Hard target | -| Trailing stop | Exit when price drops 7% from highest point reached | Locks in gains | -| Stop loss | Exit when loss ≥ 8% from entry | Hard floor | -| Reversal signal | Exit when Technical Agent says strong SELL | Agent-driven exit | -| Time-based | Exit after 30 days if none of above triggered | Prevents zombie positions | +**Prereqs:** Phase 2 complete -### Architecture -Create `tradingagents/monitoring/` -``` -monitoring/ -├── position_monitor.py ← Main loop -├── exit_rules.py ← All exit condition logic -├── price_feed.py ← Real-time price fetching -└── alert_manager.py ← Notifications -``` +| # | Task | ~Hours | Files | +| ---- | ------------------------------------------------------------------------------------ | ------ | ------------------------------------------------------------------------------------ | +| 3.1 | Add `conviction_score: float` field to `TradingAgentsGraph` output state | 2h | `tradingagents/agents/utils/agent_states.py`, `tradingagents/graph/trading_graph.py` | +| 3.2 | Update each analyst agent prompt to return structured JSON with signal + conviction | 3h | `tradingagents/agents/analysts/*.py` | +| 3.3 | Parse structured conviction output from each agent in the graph state | 2h | `tradingagents/graph/signal_processing.py` | +| 3.4 | Implement `calculate_conviction()` weighted scoring function | 2h | `portfolio/conviction_gate.py` (new) | +| 3.5 | Build `ConvictionGate` — checks threshold, min agents agree, cooldown, max positions | 3h | `portfolio/conviction_gate.py` | +| 3.6 | Add `signals` database table for logging all decisions | 2h | `database/schema.sql`, `database/db.py` | +| 3.7 | Wire ConvictionGate between graph output and execution | 2h | `execution/order_manager.py` | +| 3.8 | Test: force high-conviction scenario, verify buy fires | 1h | `tests/test_conviction.py` (new) | +| 3.9 | Test: force low-conviction scenario, verify buy is blocked | 1h | `tests/test_conviction.py` | +| 3.10 | Add dry-run mode flag — logs what would have happened without executing | 2h | `tradingagents/default_config.py`, `execution/order_manager.py` | -**The Monitor Loop** -```python -async def monitor_loop(interval_seconds=300): # Check every 5 min - while True: - positions = broker.get_positions() - for position in positions: - current_price = price_feed.get_price(position.ticker) - exit_rule = exit_rules.check(position, current_price) - if exit_rule.should_exit: - broker.place_market_sell(position.ticker, position.qty) - log_exit(position, exit_rule.reason) - await asyncio.sleep(interval_seconds) -``` -**Trailing Stop Implementation** -```python -class Position: - ticker: str - entry_price: float - qty: int - highest_price: float # Track this, update every check - entry_time: datetime +- 3.1 — Add conviction_score to graph output +- 3.2 — Update analyst prompts for structured JSON output +- 3.3 — Parse conviction output in graph state +- 3.4 — Implement weighted conviction scoring function +- 3.5 — Build ConvictionGate with all buy rules +- 3.6 — Add signals table to database +- 3.7 — Wire ConvictionGate into execution pipeline +- 3.8 — Test: high conviction triggers buy +- 3.9 — Test: low conviction blocks buy +- 3.10 — Dry-run mode -def check_trailing_stop(position, current_price, trail_pct=0.07): - # Update high-water mark - if current_price > position.highest_price: - position.highest_price = current_price - # Save updated high to DB - - # Check if we've fallen trail_pct% from the peak - trail_level = position.highest_price * (1 - trail_pct) - if current_price <= trail_level: - return ExitSignal(should_exit=True, reason="TRAILING_STOP") - return ExitSignal(should_exit=False) -``` +See [design_reference.md — Conviction Scoring Design](design_reference.md#conviction-scoring-design), [Auto-Buy Rules](design_reference.md#auto-buy-rules), and [Agent Prompt Additions](design_reference.md#agent-prompt-additions) for implementation details. -**Tasks** -- [ ] Create `positions` database table with all needed fields including `highest_price` -- [ ] Build `PriceFeed` class — uses yfinance for near-real-time quotes (15-min delay is fine for daily swing trades) -- [ ] Implement each exit rule as a separate function -- [ ] Build monitor loop as an `asyncio` task running in background -- [ ] Add reversal detection: re-run just the Technical Analyst on existing positions (not the full pipeline — too expensive) -- [ ] Wire alerts: when exit fires, log + send yourself a Telegram message (easy to set up) -- [ ] Test each exit rule in isolation with mocked prices -- [ ] Run paper trading for 2 weeks, verify exits fire correctly +### Definition of Done -**Reversal Detection (Cost-Effective Approach)** -Don't run the full 7-agent pipeline for monitoring. Instead: -```python -async def check_reversal(position): - # Only run Technical Analyst (fast, cheap, no LLM needed for basic signals) - tech_signal = technical_agent.quick_check(position.ticker) - if tech_signal.rsi > 75 and tech_signal.macd_cross == "BEARISH": - return ExitSignal(should_exit=True, reason="REVERSAL_SIGNAL") -``` +- Every `propagate()` call outputs a conviction score +- Trades only fire when conviction exceeds threshold AND 3+ agents agree +- All signals are logged to the `signals` table (bought, skipped, or rejected) +- Dry-run mode works + +--- + +## Phase 4 — Position Monitoring and Auto-Exit + +**Goal:** Once a position is open, a monitoring loop checks it on a schedule and auto-exits based on predefined rules (profit target, trailing stop, stop loss, reversal, time-based). + +**Duration:** 3-4 weeks + +**Prereqs:** Phase 3 complete + + +| # | Task | ~Hours | Files | +| ---- | ----------------------------------------------------------------------------------------- | ------- | --------------------------------------- | +| 4.1 | Ensure `positions` table has `highest_price` column for trailing stop tracking | 1h | `database/schema.sql`, `database/db.py` | +| 4.2 | Build `PriceFeed` class using yfinance for near-real-time quotes | 2h | `monitoring/price_feed.py` (new) | +| 4.3 | Implement profit target exit rule (>= 15% gain) | 1h | `monitoring/exit_rules.py` (new) | +| 4.4 | Implement trailing stop exit rule (7% drop from peak) | 2h | `monitoring/exit_rules.py` | +| 4.5 | Implement stop loss exit rule (>= 8% loss from entry) | 1h | `monitoring/exit_rules.py` | +| 4.6 | Implement time-based exit rule (30 days max hold) | 1h | `monitoring/exit_rules.py` | +| 4.7 | Implement reversal detection using only Technical Analyst (lightweight, no full pipeline) | 3h | `monitoring/exit_rules.py` | +| 4.8 | Build the async monitor loop — checks all positions every 5 min | 3h | `monitoring/position_monitor.py` (new) | +| 4.9 | Wire exit signals to `broker.place_market_sell()` and log exit reason | 2h | `monitoring/position_monitor.py` | +| 4.10 | Build alert manager — log exits + send Telegram notification | 2h | `monitoring/alert_manager.py` (new) | +| 4.11 | Test each exit rule in isolation with mocked prices | 3h | `tests/test_exit_rules.py` (new) | +| 4.12 | Run paper trading for 2 weeks, verify exits fire correctly | ongoing | manual | + + +- 4.1 — Ensure positions table has highest_price column +- 4.2 — PriceFeed class +- 4.3 — Profit target exit rule +- 4.4 — Trailing stop exit rule +- 4.5 — Stop loss exit rule +- 4.6 — Time-based exit rule +- 4.7 — Reversal detection (lightweight Technical Analyst only) +- 4.8 — Async monitor loop (5 min interval) +- 4.9 — Wire exits to broker sell + logging +- 4.10 — Alert manager (Telegram notifications) +- 4.11 — Test each exit rule in isolation +- 4.12 — 2-week paper trading validation + +See [design_reference.md — Exit Conditions and Rules](design_reference.md#exit-conditions-and-rules), [Monitor Loop](design_reference.md#monitor-loop), [Trailing Stop Implementation](design_reference.md#trailing-stop-implementation), and [Reversal Detection](design_reference.md#reversal-detection) for implementation details. + +### Definition of Done + +- Monitor loop runs continuously during market hours +- Each exit rule fires correctly when its condition is met +- All exits are logged with reason +- Telegram alerts work +- 2 weeks of paper trading with no missed exits + +--- + +# Milestone v0.3 — Portfolio Risk + Dashboard + +*Protect the whole portfolio. See what's happening in real time.* --- ## Phase 5 — Portfolio-Level Risk Controls -**Duration: 2–3 weeks** -### Goals -Protect the whole portfolio, not just individual positions. +**Goal:** Protect the portfolio as a whole, not just individual positions. Enforce hard limits on exposure, concentration, and drawdown. -### Rules to Implement +**Duration:** 2-3 weeks -**Hard Limits** -```python -PORTFOLIO_RULES = { - "max_positions": 10, # Never hold more than 10 stocks - "max_sector_exposure": 0.30, # No single sector > 30% of portfolio - "max_single_position": 0.08, # No single stock > 8% of portfolio - "max_single_exchange": 0.60, # No more than 60% in NYSE or NASDAQ alone - "daily_loss_limit": -0.03, # Stop all buys if down 3% on the day - "weekly_loss_limit": -0.07, # Stop all activity if down 7% in a week - "cash_reserve": 0.10, # Always keep 10% cash -} -``` +**Prereqs:** Phase 4 complete -**The Portfolio Guard** -```python -class PortfolioGuard: - def can_open_position(self, ticker, proposed_size) -> tuple[bool, str]: - # Check each rule before allowing a new buy - checks = [ - self._check_max_positions(), - self._check_sector_exposure(ticker), - self._check_daily_loss(), - self._check_cash_reserve(proposed_size), - ] - failures = [r for r in checks if not r.passed] - if failures: - return False, failures[0].reason - return True, "OK" -``` -**Tasks** -- [ ] Build sector classification map for your watchlist tickers -- [ ] Implement `PortfolioGuard` with each rule as a method -- [ ] Insert `PortfolioGuard.can_open_position()` check between Fund Manager approval and order execution -- [ ] Add daily P&L tracking to the database -- [ ] Test: create a scenario where 10 positions are open and verify 11th is blocked -- [ ] Test: simulate a 3% daily loss and verify no new buys are attempted -- [ ] Create a `portfolio_summary()` function for the dashboard +| # | Task | ~Hours | Files | +| ---- | --------------------------------------------------------------------------------------------- | ------ | --------------------------------------- | +| 5.1 | Create `watchlist/sector_map.json` mapping each ticker to its sector | 1h | `watchlist/sector_map.json` (new) | +| 5.2 | Implement `PortfolioGuard` class with `can_open_position()` method | 4h | `portfolio/portfolio_guard.py` (new) | +| 5.3 | Implement max positions check (10 max) | 1h | `portfolio/portfolio_guard.py` | +| 5.4 | Implement sector exposure check (no sector > 30%) | 2h | `portfolio/portfolio_guard.py` | +| 5.5 | Implement single position size check (no stock > 8%) | 1h | `portfolio/portfolio_guard.py` | +| 5.6 | Implement daily loss limit (stop buys if down 3% on the day) | 2h | `portfolio/portfolio_guard.py` | +| 5.7 | Implement cash reserve check (always keep 10%) | 1h | `portfolio/portfolio_guard.py` | +| 5.8 | Insert `PortfolioGuard.can_open_position()` between Fund Manager approval and order execution | 2h | `execution/order_manager.py` | +| 5.9 | Add `portfolio_snapshots` table for daily P&L tracking | 2h | `database/schema.sql`, `database/db.py` | +| 5.10 | Create `portfolio_summary()` function (needed for dashboard) | 2h | `portfolio/portfolio_guard.py` | +| 5.11 | Test: 10 positions open, verify 11th is blocked | 1h | `tests/test_portfolio_guard.py` (new) | +| 5.12 | Test: simulate 3% daily loss, verify no new buys | 1h | `tests/test_portfolio_guard.py` | + + +- 5.1 — Sector map JSON +- 5.2 — PortfolioGuard class skeleton +- 5.3 — Max positions check +- 5.4 — Sector exposure check +- 5.5 — Single position size check +- 5.6 — Daily loss limit +- 5.7 — Cash reserve check +- 5.8 — Wire PortfolioGuard into execution pipeline +- 5.9 — Portfolio snapshots table +- 5.10 — portfolio_summary() function +- 5.11 — Test: max positions enforcement +- 5.12 — Test: daily loss limit enforcement + +See [design_reference.md — Portfolio Guard Design](design_reference.md#portfolio-guard-design) for implementation details. + +### Definition of Done + +- All guard rules pass tests +- 11th position attempt is blocked when 10 are open +- Daily loss limit halts buying +- Portfolio snapshots are recorded daily --- -## Phase 6 — Dashboard & Observability -**Duration: 2–3 weeks** +## Phase 6 — Dashboard and Observability -### Goals -You need to see what's happening in real time. As a solo dev, a simple web dashboard is far more practical than debugging log files. +**Goal:** See what's happening in real time. A simple web dashboard is far more practical than debugging log files. -### Stack Recommendation -Use **Streamlit** — it's Python-native, fast to build, and perfect for internal tools. +**Duration:** 2-3 weeks -```bash -pip install streamlit plotly pandas -``` +**Prereqs:** Phase 5 complete -**Dashboard Pages** -*Page 1: Portfolio Overview* -- Current positions table (ticker, entry price, current price, P&L%, trailing stop level) -- Total portfolio value + daily P&L -- Cash available -- Sector exposure chart +| # | Task | ~Hours | Files | +| ---- | ---------------------------------------------------------------------------------------- | ------ | ------------------------------------ | +| 6.1 | Install Streamlit, Plotly, Pandas dependencies | 0.5h | `requirements.txt` | +| 6.2 | Set up Streamlit app shell with sidebar navigation | 2h | `dashboard/app.py` (new) | +| 6.3 | Connect app to SQLite database | 1h | `dashboard/app.py` | +| 6.4 | Build Page 1: Portfolio Overview (positions table, total value, daily P&L, sector chart) | 4h | `dashboard/app.py` | +| 6.5 | Build Page 2: Signal Feed (agent decisions log, conviction scores, pending signals) | 3h | `dashboard/app.py` | +| 6.6 | Build Page 3: Trade History (closed trades, win rate, monthly returns) | 3h | `dashboard/app.py` | +| 6.7 | Build Page 4: Agent Monitor (tickers analyzed, agent breakdown, API costs) | 3h | `dashboard/app.py` | +| 6.8 | Add auto-refresh every 60 seconds | 1h | `dashboard/app.py` | +| 6.9 | Add "pause trading" toggle that sets a flag in the DB | 2h | `dashboard/app.py`, `database/db.py` | +| 6.10 | Test: run dashboard locally alongside paper trading loop | 1h | manual | -*Page 2: Signal Feed* -- Live log of agent decisions (last 50) -- Conviction scores with color coding (green = strong buy, yellow = weak, gray = hold) -- Pending signals not yet executed -*Page 3: Trade History* -- All closed trades with entry/exit/reason/profit -- Win rate, average return, best/worst trade -- Monthly return chart +- 6.1 — Install dashboard dependencies +- 6.2 — Streamlit app shell with navigation +- 6.3 — Connect to SQLite database +- 6.4 — Page 1: Portfolio Overview +- 6.5 — Page 2: Signal Feed +- 6.6 — Page 3: Trade History +- 6.7 — Page 4: Agent Monitor +- 6.8 — Auto-refresh +- 6.9 — Pause trading toggle +- 6.10 — Test: dashboard alongside paper trading -*Page 4: Agent Monitor* -- Which tickers were analyzed today -- Agent breakdown per analysis (which agents said BUY vs SELL) -- API costs tracker (LLM call count × estimated cost) +See [design_reference.md — Dashboard Specs](design_reference.md#dashboard-specs) for page layouts. -**Tasks** -- [ ] Set up Streamlit app at `dashboard/app.py` -- [ ] Connect to your SQLite database (or Postgres if you've upgraded) -- [ ] Build each page using `st.dataframe`, `st.metric`, and Plotly charts -- [ ] Add auto-refresh every 60 seconds (`st.rerun()` with `time.sleep`) -- [ ] Deploy locally (you don't need to expose this to the internet — just run it on your machine) -- [ ] Add a simple "pause trading" toggle that sets a flag in the DB (monitor loop respects it) +### Definition of Done + +- Dashboard runs locally and shows live portfolio data +- All 4 pages render correctly +- Auto-refresh works +- Pause toggle actually stops the trading loop + +--- + +# Milestone v1.0 — Live Trading + +*Real money. Small size. Scaled carefully.* --- ## Phase 7 — Live Trading (Gradual Rollout) -**Duration: Ongoing — never rush this** -### The Graduation Criteria -Before touching real money, you must have: -- [ ] 60+ consecutive days of paper trading with no critical bugs -- [ ] All exit rules verified to have fired correctly at least 5 times each -- [ ] Portfolio guard rules verified under stress scenarios -- [ ] Trade log showing positive expectancy (avg win > avg loss) -- [ ] Manual review of every paper trade's entry/exit reasoning +**Goal:** Graduate from paper to live trading. Never rush this phase. + +**Duration:** Ongoing + +**Prereqs:** All previous phases complete + graduation criteria met + +### Graduation Criteria (all must be true before using real money) + +- 60+ consecutive days of paper trading with no critical bugs +- All exit rules have fired correctly at least 5 times each +- Portfolio guard rules verified under stress scenarios +- Trade log showing positive expectancy (avg win > avg loss) +- Manual review of every paper trade's entry/exit reasoning ### Go-Live Steps -**Week 1 with real money: $2,000 max** -- Deploy to Alpaca live account (or IBKR if you prefer) -- Max 2 positions open at once -- Position size: $200–$300 max per trade -- Monitor manually every hour during market hours -**Month 2: Scale to $10,000** -- Only if Week 1 had no execution errors -- Increase to 5 max positions -- Begin trusting the monitor loop for exits +| # | Task | ~Hours | Files | +| --- | --------------------------------------------------------------------------------------------- | ------- | -------------------------------- | +| 7.1 | Build `IBKRBroker` implementing `BrokerInterface` (optional, if using IBKR instead of Alpaca) | 4h | `execution/ibkr_broker.py` (new) | +| 7.2 | Set up Alpaca live account (or IBKR), add live credentials to `.env` | 1h | `.env` | +| 7.3 | Deploy Week 1: $2,000 max, 2 positions max, $200-300 per trade, monitor hourly | ongoing | config | +| 7.4 | After Week 1 with no execution errors: scale to $10,000, 5 max positions | ongoing | config | +| 7.5 | Month 3+: increase to target capital, weekly agent review, monthly threshold recalibration | ongoing | config | -**Month 3+: Full operation** -- Increase to your target capital -- Weekly review of agent decisions vs outcomes -- Monthly recalibration of conviction thresholds -### Broker Setup for Live NYSE/NASDAQ Trading +- 7.1 — IBKRBroker implementation (if needed) +- 7.2 — Live broker credentials +- 7.3 — Week 1: $2K deployment +- 7.4 — Month 2: scale to $10K +- 7.5 — Month 3+: full operation -**Alpaca** is the easiest path for NYSE/NASDAQ live trading. IBKR is a solid alternative with more order types: -```bash -# Alpaca live trading (same SDK as paper, just swap credentials) -pip install alpaca-py -``` +See [design_reference.md — Broker Setup Commands](design_reference.md#broker-setup-commands) and [US Regulatory Note](design_reference.md#us-regulatory-note) for broker details and PDT rules. -For IBKR: -```bash -pip install ib_insync -# Requires IBKR TWS or Gateway running locally -``` +### Definition of Done -Build `IBKRBroker` implementing the same `BrokerInterface` from Phase 2. Switching brokers is just a config change — the rest of the system doesn't care. - -### US Regulatory Note -For personal automated trading in a US brokerage account, you're operating under standard retail trading rules. If you make more than 3 day trades in a 5-day rolling window with under $25,000 in the account, you'll trigger Pattern Day Trader rules. Since TradeDog is a swing trading system (holding for days to weeks), this typically isn't an issue — but keep it in mind when sizing up. +- Live trades execute and match paper trading behavior +- No execution errors in first week +- Profitable or at least not losing beyond daily limits --- -## Database Schema - -Use **SQLite** to start (zero infrastructure). Migrate to Postgres later if needed. - -```sql --- All positions (open and closed) -CREATE TABLE positions ( - id INTEGER PRIMARY KEY, - ticker TEXT NOT NULL, - exchange TEXT NOT NULL, -- 'NYSE' or 'NASDAQ' - entry_price REAL NOT NULL, - entry_time DATETIME NOT NULL, - qty INTEGER NOT NULL, - highest_price REAL, -- For trailing stop - exit_price REAL, - exit_time DATETIME, - exit_reason TEXT, -- 'PROFIT_TARGET', 'TRAILING_STOP', etc. - status TEXT DEFAULT 'OPEN' -- 'OPEN' or 'CLOSED' -); - --- All agent signals (for analysis and debugging) -CREATE TABLE signals ( - id INTEGER PRIMARY KEY, - ticker TEXT NOT NULL, - timestamp DATETIME NOT NULL, - conviction_score REAL, - agent_decision TEXT, -- JSON of each agent's output - action_taken TEXT, -- 'BOUGHT', 'SKIPPED', 'REJECTED_BY_GUARD' - skip_reason TEXT -); - --- Daily portfolio snapshots -CREATE TABLE portfolio_snapshots ( - id INTEGER PRIMARY KEY, - snapshot_date DATE NOT NULL, - total_value REAL, - cash REAL, - num_positions INTEGER, - daily_pnl REAL, - daily_pnl_pct REAL -); - --- System events and errors -CREATE TABLE system_log ( - id INTEGER PRIMARY KEY, - timestamp DATETIME NOT NULL, - level TEXT, -- 'INFO', 'WARNING', 'ERROR' - component TEXT, - message TEXT -); -``` - ---- - -## File Structure (Target State) - -``` -TradeDog/ -├── tradingagents/ ← Upstream framework (minimal changes) -│ ├── agents/ -│ ├── dataflows/ -│ │ ├── yfinance_fallback.py ← NEW: Fallback when FinnHub fails -│ │ └── data_validator.py ← NEW: Validates data quality -│ ├── graph/trading_graph.py -│ └── default_config.py -│ -├── execution/ ← NEW: Order execution -│ ├── broker_interface.py -│ ├── paper_broker.py -│ ├── alpaca_broker.py -│ ├── ibkr_broker.py -│ └── order_manager.py -│ -├── monitoring/ ← NEW: Position monitoring -│ ├── position_monitor.py -│ ├── exit_rules.py -│ ├── price_feed.py -│ └── alert_manager.py -│ -├── portfolio/ ← NEW: Risk management -│ ├── portfolio_guard.py -│ ├── conviction_gate.py -│ └── position_sizer.py -│ -├── database/ ← NEW: Data persistence -│ ├── schema.sql -│ ├── db.py -│ └── models.py -│ -├── dashboard/ ← NEW: Streamlit UI -│ └── app.py -│ -├── watchlist/ ← NEW: Curated tickers -│ ├── watchlist.json ← NYSE/NASDAQ curated tickers -│ └── sector_map.json ← Ticker → sector classification -│ -├── scheduler/ ← NEW: Orchestrates daily run -│ └── main_loop.py -│ -├── tests/ -│ └── ... -│ -├── docs/ -│ ├── architecture.md -│ └── agent_contracts.md -│ -├── .env -├── main.py -└── requirements.txt -``` - ---- - -## Cost Estimate (Monthly) - -| Item | Cost | -|---|---| -| LLM API (Claude Sonnet for analysts, Opus for Trader/Risk) | ~$30–80/mo | -| FinnHub free tier | $0 | -| yfinance | $0 | -| Alpaca paper trading | $0 | -| IBKR live account | $0 (no monthly fee) | -| Hosting (run on your laptop) | $0 | - -Keep costs low by: analyzing each ticker once per day (not per minute), using Claude Haiku for the analyst agents, and only using a more powerful model for the final Trader and Risk Manager decisions. - ---- - -## Weekly Rhythm for a Solo Developer +## Weekly Rhythm **Every week:** + - Monday: Review last week's signal log — did the agents call it right? -- Tuesday–Thursday: Build next feature from the roadmap +- Tuesday-Thursday: Build next feature from this roadmap - Friday: Write tests, review paper trades, update docs **Every month:** + - Recalibrate conviction thresholds based on data - Review which agents are adding value vs noise - Upgrade watchlist based on what's been performing --- -## Risks & Mitigations +## Risks and Mitigations + + +| Risk | Mitigation | +| ------------------------------------ | -------------------------------------------------------- | +| LLM hallucination drives a bad trade | Conviction gate + portfolio guard as hard stops | +| API outage during market hours | Retry logic + fallback to cached data | +| Broker API failure | Always log intent before execution; reconcile on startup | +| Runaway losses | Daily loss limit halts all activity automatically | +| Overfitting to paper trading | Paper trade on different time periods before going live | +| Low liquidity stocks | Volume filter on watchlist (>1M shares/day avg) | -| Risk | Mitigation | -|---|---| -| LLM hallucination drives a bad trade | Conviction gate + portfolio guard as hard stops | -| API outage during market hours | Retry logic + fallback to cached data | -| Broker API failure | Always log intent before execution; reconcile on startup | -| Runaway losses | Daily loss limit halts all activity automatically | -| Overfitting to paper trading | Paper trade on different time periods before going live | -| Low liquidity stocks | Volume filter on watchlist (>1M shares/day avg) | --- -*This roadmap is designed to be completed one phase at a time. Finish each phase completely before starting the next. The order matters — don't skip to execution before the data layer is solid.* +*Finish each phase completely before starting the next. The order matters.* \ No newline at end of file diff --git a/docs/design_reference.md b/docs/design_reference.md new file mode 100644 index 00000000..8df0e466 --- /dev/null +++ b/docs/design_reference.md @@ -0,0 +1,498 @@ +# TradeDog — Design Reference + +*All architecture patterns, code snippets, schemas, and design decisions for TradeDog. +Referenced by [TradeDog_Roadmap.md](TradeDog_Roadmap.md) — use the roadmap for task tracking, use this doc for implementation details.* + +--- + +## Table of Contents + +- [Architecture Target](#architecture-target) +- [Data Source Strategy](#data-source-strategy) +- [Watchlist Design](#watchlist-design) +- [Market Hours and Scheduling](#market-hours-and-scheduling) +- [Execution Layer Architecture](#execution-layer-architecture) +- [Broker Interface](#broker-interface) +- [Position Sizing Formula](#position-sizing-formula) +- [Conviction Scoring Design](#conviction-scoring-design) +- [Auto-Buy Rules](#auto-buy-rules) +- [Agent Prompt Additions](#agent-prompt-additions) +- [Exit Conditions and Rules](#exit-conditions-and-rules) +- [Monitor Loop](#monitor-loop) +- [Trailing Stop Implementation](#trailing-stop-implementation) +- [Reversal Detection](#reversal-detection) +- [Portfolio Guard Design](#portfolio-guard-design) +- [Database Schema](#database-schema) +- [Dashboard Specs](#dashboard-specs) +- [Target File Structure](#target-file-structure) +- [Broker Setup Commands](#broker-setup-commands) +- [Cost Estimate](#cost-estimate) +- [US Regulatory Note](#us-regulatory-note) + +--- + +## Architecture Target + +``` +[Watchlist Scanner] + | +[Research Pipeline] <-- Fundamentals / Sentiment / News / Technical + | +[Bull vs Bear Debate] + | +[Trader -> Risk Manager -> Fund Manager] + | + Conviction Score + | +[Auto-Buy Engine] <--> [Broker API: Alpaca / IBKR] + | +[Position Monitor -- runs every N minutes] + |-- Profit target hit -> SELL + |-- Trailing stop triggered -> SELL + |-- Reversal signal detected -> SELL + |-- Time-based exit (optional) + | +[Trade Logger + Dashboard] +``` + +--- + +## Data Source Strategy + +| Source | Use Case | Cost | Notes | +|---|---|---|---| +| FinnHub | Real-time quotes, news, insider trades | Free tier | Already wired in | +| `yfinance` | OHLCV history, fundamentals fallback | Free | Add as secondary/fallback | +| Polygon.io | Higher-quality tick data if needed later | Paid | Skip for now | +| Alpha Vantage | Alternative fundamentals | Free tier | Keep as backup | + +Both FinnHub and yfinance have excellent US coverage. yfinance is the primary fallback when FinnHub returns empty or errors. + +--- + +## Watchlist Design + +Starter watchlist (~36 quality tickers across sectors): + +```json +{ + "large_cap": ["AAPL", "MSFT", "NVDA", "GOOGL", "AMZN", "META", "JPM", "UNH", "V", "MA"], + "growth": ["CRWD", "SNOW", "NET", "DDOG", "SMCI", "ARM", "PLTR"], + "value": ["BRK-B", "JNJ", "PG", "KO", "WMT", "HD", "MCD"], + "financials": ["GS", "MS", "BAC", "C", "WFC"], + "energy": ["XOM", "CVX", "COP", "SLB"] +} +``` + +**Liquidity Filter** — only analyze stocks with sufficient volume to avoid slippage: + +```python +MIN_AVG_DAILY_VOLUME = 1_000_000 # 1M shares/day minimum +MIN_MARKET_CAP = 2_000_000_000 # $2B market cap minimum +``` + +--- + +## Market Hours and Scheduling + +- NYSE/NASDAQ: 9:30 AM - 4:00 PM ET +- Pre-market analysis run: 8:00-9:15 AM ET (agents analyze, build signals) +- Market open execution window: 9:30-10:30 AM ET (buy signals fire here) +- Monitoring loop: runs every 5 min during market hours +- After-hours: position review, log summary, prep next day's watchlist + +--- + +## Execution Layer Architecture + +New module: `execution/` + +``` +execution/ +├── broker_interface.py <-- Abstract base class +├── alpaca_broker.py <-- Alpaca implementation +├── ibkr_broker.py <-- IBKR implementation (Phase 7) +├── paper_broker.py <-- Local simulation (no API needed) +└── order_manager.py <-- Order lifecycle tracking +``` + +Start with `PaperBroker` — a pure Python simulation that tracks positions in a local SQLite database. This lets you test the full loop without any API dependency. + +--- + +## Broker Interface + +Define this first. All broker implementations (Paper, Alpaca, IBKR) implement this: + +```python +class BrokerInterface: + def place_market_buy(self, ticker: str, qty: int) -> Order: ... + def place_market_sell(self, ticker: str, qty: int) -> Order: ... + def get_positions(self) -> list[Position]: ... + def get_account(self) -> AccountInfo: ... + def cancel_order(self, order_id: str) -> bool: ... +``` + +--- + +## Position Sizing Formula + +Start simple. Scale by conviction, cap at a percentage of account value: + +```python +def calculate_position_size(account_value, conviction_score, price, max_position_pct=0.05): + max_dollars = account_value * max_position_pct + dollars_to_invest = max_dollars * conviction_score + shares = int(dollars_to_invest / price) + return max(1, shares) +``` + +--- + +## Conviction Scoring Design + +The agents currently produce a BUY/SELL/HOLD decision with rationale. Extend this to produce a **conviction score (0-100)**. + +**Weighted scoring approach:** + +```python +CONVICTION_WEIGHTS = { + "technical": 0.25, # RSI, MACD, moving average signals + "fundamental": 0.25, # P/E, growth, financial health + "sentiment": 0.20, # Social/news mood + "bull_bear": 0.30, # Debate outcome (most decisive) +} + +def calculate_conviction(agent_signals: dict) -> float: + score = 0 + for agent, weight in CONVICTION_WEIGHTS.items(): + # Each agent returns -1.0 (strong sell) to +1.0 (strong buy) + score += agent_signals[agent] * weight + return score # -1.0 to +1.0 +``` + +--- + +## Auto-Buy Rules + +```python +BUY_THRESHOLD = 0.65 # Must have >65% conviction to buy +MIN_AGENTS_AGREE = 3 # At least 3 of 4 analyst agents must agree direction +MAX_POSITIONS = 10 # Never hold more than 10 stocks at once +COOLDOWN_HOURS = 24 # Don't re-analyze same stock within 24h of last trade +``` + +--- + +## Agent Prompt Additions + +Add to each analyst agent's system prompt for structured output parsing: + +``` +At the end of your analysis, always output a JSON block: +{"signal": "BUY|SELL|HOLD", "conviction": 0.85, "key_reason": "..."} +``` + +Then parse this structured output in the graph state. + +--- + +## Exit Conditions and Rules + +| Condition | Rule | Notes | +|---|---|---| +| Profit target | Exit when gain >= 15% | Hard target | +| Trailing stop | Exit when price drops 7% from highest point reached | Locks in gains | +| Stop loss | Exit when loss >= 8% from entry | Hard floor | +| Reversal signal | Exit when Technical Agent says strong SELL | Agent-driven exit | +| Time-based | Exit after 30 days if none of above triggered | Prevents zombie positions | + +--- + +## Monitor Loop + +New module: `monitoring/` + +``` +monitoring/ +├── position_monitor.py <-- Main loop +├── exit_rules.py <-- All exit condition logic +├── price_feed.py <-- Real-time price fetching +└── alert_manager.py <-- Notifications +``` + +**Core loop pseudocode:** + +```python +async def monitor_loop(interval_seconds=300): # Check every 5 min + while True: + positions = broker.get_positions() + for position in positions: + current_price = price_feed.get_price(position.ticker) + exit_rule = exit_rules.check(position, current_price) + if exit_rule.should_exit: + broker.place_market_sell(position.ticker, position.qty) + log_exit(position, exit_rule.reason) + await asyncio.sleep(interval_seconds) +``` + +--- + +## Trailing Stop Implementation + +```python +class Position: + ticker: str + entry_price: float + qty: int + highest_price: float # Track this, update every check + entry_time: datetime + +def check_trailing_stop(position, current_price, trail_pct=0.07): + if current_price > position.highest_price: + position.highest_price = current_price + # Save updated high to DB + + trail_level = position.highest_price * (1 - trail_pct) + if current_price <= trail_level: + return ExitSignal(should_exit=True, reason="TRAILING_STOP") + return ExitSignal(should_exit=False) +``` + +--- + +## Reversal Detection + +Cost-effective approach: don't run the full 7-agent pipeline for monitoring. Only run the Technical Analyst: + +```python +async def check_reversal(position): + tech_signal = technical_agent.quick_check(position.ticker) + if tech_signal.rsi > 75 and tech_signal.macd_cross == "BEARISH": + return ExitSignal(should_exit=True, reason="REVERSAL_SIGNAL") +``` + +--- + +## Portfolio Guard Design + +**Hard limits:** + +```python +PORTFOLIO_RULES = { + "max_positions": 10, # Never hold more than 10 stocks + "max_sector_exposure": 0.30, # No single sector > 30% of portfolio + "max_single_position": 0.08, # No single stock > 8% of portfolio + "max_single_exchange": 0.60, # No more than 60% in NYSE or NASDAQ alone + "daily_loss_limit": -0.03, # Stop all buys if down 3% on the day + "weekly_loss_limit": -0.07, # Stop all activity if down 7% in a week + "cash_reserve": 0.10, # Always keep 10% cash +} +``` + +**Guard class:** + +```python +class PortfolioGuard: + def can_open_position(self, ticker, proposed_size) -> tuple[bool, str]: + checks = [ + self._check_max_positions(), + self._check_sector_exposure(ticker), + self._check_daily_loss(), + self._check_cash_reserve(proposed_size), + ] + failures = [r for r in checks if not r.passed] + if failures: + return False, failures[0].reason + return True, "OK" +``` + +Insert `PortfolioGuard.can_open_position()` between Fund Manager approval and order execution. + +--- + +## Database Schema + +Use **SQLite** to start (zero infrastructure). Migrate to Postgres later if needed. + +```sql +-- All positions (open and closed) +CREATE TABLE positions ( + id INTEGER PRIMARY KEY, + ticker TEXT NOT NULL, + exchange TEXT NOT NULL, -- 'NYSE' or 'NASDAQ' + entry_price REAL NOT NULL, + entry_time DATETIME NOT NULL, + qty INTEGER NOT NULL, + highest_price REAL, -- For trailing stop + exit_price REAL, + exit_time DATETIME, + exit_reason TEXT, -- 'PROFIT_TARGET', 'TRAILING_STOP', etc. + status TEXT DEFAULT 'OPEN' -- 'OPEN' or 'CLOSED' +); + +-- All agent signals (for analysis and debugging) +CREATE TABLE signals ( + id INTEGER PRIMARY KEY, + ticker TEXT NOT NULL, + timestamp DATETIME NOT NULL, + conviction_score REAL, + agent_decision TEXT, -- JSON of each agent's output + action_taken TEXT, -- 'BOUGHT', 'SKIPPED', 'REJECTED_BY_GUARD' + skip_reason TEXT +); + +-- Daily portfolio snapshots +CREATE TABLE portfolio_snapshots ( + id INTEGER PRIMARY KEY, + snapshot_date DATE NOT NULL, + total_value REAL, + cash REAL, + num_positions INTEGER, + daily_pnl REAL, + daily_pnl_pct REAL +); + +-- System events and errors +CREATE TABLE system_log ( + id INTEGER PRIMARY KEY, + timestamp DATETIME NOT NULL, + level TEXT, -- 'INFO', 'WARNING', 'ERROR' + component TEXT, + message TEXT +); +``` + +--- + +## Dashboard Specs + +Stack: **Streamlit** + Plotly + Pandas. Run locally. + +```bash +pip install streamlit plotly pandas +``` + +**Page 1: Portfolio Overview** +- Current positions table (ticker, entry price, current price, P&L%, trailing stop level) +- Total portfolio value + daily P&L +- Cash available +- Sector exposure chart + +**Page 2: Signal Feed** +- Live log of agent decisions (last 50) +- Conviction scores with color coding (green = strong buy, yellow = weak, gray = hold) +- Pending signals not yet executed + +**Page 3: Trade History** +- All closed trades with entry/exit/reason/profit +- Win rate, average return, best/worst trade +- Monthly return chart + +**Page 4: Agent Monitor** +- Which tickers were analyzed today +- Agent breakdown per analysis (which agents said BUY vs SELL) +- API costs tracker (LLM call count x estimated cost) + +--- + +## Target File Structure + +``` +TradeDog/ +├── tradingagents/ <-- Upstream framework (minimal changes) +│ ├── agents/ +│ ├── dataflows/ +│ │ ├── yfinance_fallback.py <-- NEW: Fallback when FinnHub fails +│ │ └── data_validator.py <-- NEW: Validates data quality +│ ├── graph/trading_graph.py +│ └── default_config.py +│ +├── execution/ <-- NEW: Order execution +│ ├── broker_interface.py +│ ├── paper_broker.py +│ ├── alpaca_broker.py +│ ├── ibkr_broker.py +│ └── order_manager.py +│ +├── monitoring/ <-- NEW: Position monitoring +│ ├── position_monitor.py +│ ├── exit_rules.py +│ ├── price_feed.py +│ └── alert_manager.py +│ +├── portfolio/ <-- NEW: Risk management +│ ├── portfolio_guard.py +│ ├── conviction_gate.py +│ └── position_sizer.py +│ +├── database/ <-- NEW: Data persistence +│ ├── schema.sql +│ ├── db.py +│ └── models.py +│ +├── dashboard/ <-- NEW: Streamlit UI +│ └── app.py +│ +├── watchlist/ <-- NEW: Curated tickers +│ ├── watchlist.json <-- NYSE/NASDAQ curated tickers +│ └── sector_map.json <-- Ticker -> sector classification +│ +├── scheduler/ <-- NEW: Orchestrates daily run +│ └── main_loop.py +│ +├── tests/ +│ └── ... +│ +├── docs/ +│ ├── architecture.md +│ ├── agent_contracts.md +│ ├── design_reference.md +│ └── TradeDog_Roadmap.md +│ +├── .env +├── main.py +└── requirements.txt +``` + +--- + +## Broker Setup Commands + +**Alpaca (recommended for paper + live):** + +```bash +pip install alpaca-py +``` + +Free paper trading API, full NYSE/NASDAQ coverage, no account minimums, clean REST API + Python SDK. Same SDK for paper and live — just swap credentials. + +**IBKR (alternative for live trading):** + +```bash +pip install ib_insync +# Requires IBKR TWS or Gateway running locally +``` + +Build `IBKRBroker` implementing the same `BrokerInterface`. Switching brokers is a config change — the rest of the system doesn't care. + +--- + +## Cost Estimate + +| Item | Cost | +|---|---| +| LLM API (Claude Sonnet for analysts, Opus for Trader/Risk) | ~$30-80/mo | +| FinnHub free tier | $0 | +| yfinance | $0 | +| Alpaca paper trading | $0 | +| IBKR live account | $0 (no monthly fee) | +| Hosting (run on your laptop) | $0 | + +Keep costs low by: analyzing each ticker once per day (not per minute), using Claude Haiku for the analyst agents, and only using a more powerful model for the final Trader and Risk Manager decisions. + +--- + +## US Regulatory Note + +For personal automated trading in a US brokerage account, you operate under standard retail trading rules. If you make more than 3 day trades in a 5-day rolling window with under $25,000 in the account, you'll trigger **Pattern Day Trader** rules. Since TradeDog is a swing trading system (holding for days to weeks), this typically isn't an issue — but keep it in mind when sizing up.