From 91279fe60bb99becfb8168816ec0a179a8a0ce4c Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Tue, 7 Apr 2026 20:24:12 -0700 Subject: [PATCH 01/15] docs: add iteration system design spec Defines /iterate and /research-strategy skills + docs/iterations/ folder structure for a generic learn-improve-repeat cycle, demonstrated with the trading agent discovery pipeline. Co-Authored-By: Claude Sonnet 4.6 --- .../2026-04-07-iteration-system-design.md | 208 ++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-07-iteration-system-design.md diff --git a/docs/superpowers/specs/2026-04-07-iteration-system-design.md b/docs/superpowers/specs/2026-04-07-iteration-system-design.md new file mode 100644 index 00000000..30855b69 --- /dev/null +++ b/docs/superpowers/specs/2026-04-07-iteration-system-design.md @@ -0,0 +1,208 @@ +# Iteration System Design + +**Date:** 2026-04-07 +**Status:** Approved +**Scope:** Generic iteration system — skills + folder structure for learn-improve-repeat cycles. Demonstrated with trading agent discovery mode but applicable to any iterative system. + +--- + +## Problem + +Improving the discovery pipeline requires three distinct feedback loops running at different cadences: + +1. **Fast loop** — output quality: does the latest run produce specific, well-reasoned candidates? +2. **Slow loop** — P&L outcomes: did the picks actually work after 5–10+ trading days? +3. **Research loop** — strategy sourcing: are there techniques we haven't tried yet that could improve signal quality? + +Currently there is no structured way to capture learnings across runs, trace code changes back to evidence, or proactively search for improvements. Knowledge lives in one-off plan docs and in memory. + +--- + +## Solution + +Two Claude Code skills + a versioned knowledge base in `docs/iterations/`. + +--- + +## Folder Structure + +``` +docs/iterations/ +├── LEARNINGS.md ← master index, one-line per domain entry +├── scanners/ +│ ├── options_flow.md +│ ├── insider_buying.md +│ ├── volume_accumulation.md +│ ├── reddit_dd.md +│ ├── reddit_trending.md +│ ├── semantic_news.md +│ ├── market_movers.md +│ ├── earnings_calendar.md +│ ├── analyst_upgrades.md +│ ├── technical_breakout.md +│ ├── sector_rotation.md +│ ├── ml_signal.md +│ ├── minervini.md +│ └── ... (one file per scanner) +├── strategies/ +│ ├── analyst_upgrade.md +│ ├── momentum.md +│ ├── accumulation.md +│ └── ... (one file per strategy pattern) +├── pipeline/ +│ └── scoring.md ← LLM scoring, confidence calibration, ranking +└── research/ + └── YYYY-MM-DD-.md ← web research findings (append-only, dated) +``` + +### Domain File Schema + +Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure: + +```markdown +# + +## Current Understanding + + +## Evidence Log + + +### YYYY-MM-DD — +- What was observed +- What it implies +- Confidence: low / medium / high + +## Pending Hypotheses + +- [ ] Hypothesis description +``` + +### LEARNINGS.md Schema + +```markdown +# Learnings Index + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | 2026-04-07 | Call/put ratio <0.1 is reliable; premium filter working | +| ... | | | | +``` + +--- + +## Skill 1: `/iterate` + +**Location:** `~/.claude/plugins/cache/.../skills/iterate.md` (or project-local skills dir) + +### Trigger +Invoked manually after a discovery run or after positions are old enough for outcome data (5+ days). + +### Steps + +1. **Detect mode** + - Scans `results/discovery/` for runs not yet reflected in learning files + - Checks `data/positions/` for positions ≥5 days old with outcome data + - Sets mode: `fast` (output quality only), `pl` (P&L outcomes), or `both` + +2. **Load domain context** + - Identifies which scanners produced candidates in the target runs + - Reads the corresponding `docs/iterations/scanners/*.md` files + - Reads `docs/iterations/LEARNINGS.md` for full index awareness + +3. **Analyze** + - *Fast mode:* scores signal quality — specificity of thesis, scanner noise rate, LLM confidence calibration, duplicate/redundant candidates + - *P&L mode:* compares predicted outcome vs actual per scanner; flags scanners over/underperforming their stated confidence; computes hit rate per scanner + +4. **Write learnings** + - Appends to the evidence log in each relevant domain file + - Updates "Current Understanding" section if confidence in the new evidence is medium or higher + - Updates `LEARNINGS.md` index with new last-updated date and revised one-liner + +5. **Implement changes** + - Translates learnings into concrete code changes: scanner thresholds, priority logic, LLM prompt wording, scanner enable/disable + - Presents a diff for approval before writing + - On approval: implements and stages changes + +6. **Commit** + - Commits learning files and code changes together with message format: + `learn(iterate): — ` + +### Output +Two committed changesets traceable to the same run date: +- `docs/iterations/` — updated knowledge +- `tradingagents/` — code that encodes the knowledge + +--- + +## Skill 2: `/research-strategy` + +**Location:** same skills directory as `/iterate` + +### Two Modes + +#### Directed mode: `/research-strategy ""` +User names the topic. Skill goes deep on that specific strategy. + +#### Autonomous mode: `/research-strategy` +No topic given. Skill drives its own research agenda based on current weak spots. + +### Steps + +1. **Set agenda** + - *Directed:* topic is given + - *Autonomous:* reads `LEARNINGS.md` + domain files to identify: low-confidence scanners, pending hypotheses marked for research, gaps in pipeline coverage. Generates 3–5 research topics ranked by potential impact. + +2. **Search** + Searches across the default source list: + - **Reddit:** r/algotrading, r/quant, r/investing + - **Blogs:** QuantifiedStrategies, Hacker News (hn.algolia.com), CSS Analytics, Alpha Architect + - **GitHub:** search for quant/scanner/screener repos with recent activity + - **Academic:** SSRN quantitative finance, arXiv q-fin + - **Archives:** Quantopian community notebooks (via GitHub mirrors) + + For each source: looks for signal definition, entry/exit criteria, known edge, known failure modes, data requirements. + +3. **Cross-reference** + - Checks existing `docs/iterations/` files for overlap with already-implemented or already-discarded approaches + - Flags redundancy explicitly ("this is a variant of our existing volume_accumulation scanner") + +4. **Evaluate fit** + Scores each finding on four dimensions: + - **Data availability** — do we already have the required data source? + - **Implementation complexity** — hours estimate: trivial / moderate / large + - **Signal uniqueness** — how much does it overlap with existing scanners? + - **Evidence quality** — backtested with stats / qualitative analysis / anecdotal + +5. **Write research files** + - Saves findings to `docs/iterations/research/YYYY-MM-DD-.md` for all findings + - Adds entries to `LEARNINGS.md` + +6. **Propose and implement** + - Presents ranked findings with scores + - For the top-ranked finding (or user-selected one): drafts a scanner spec (data needed, signal logic, priority/confidence output) + - On approval: implements scanner following `@SCANNER_REGISTRY.register()` pattern, commits + +### Commit format +`research(): add scanner — ` + +--- + +## Generic Applicability + +The skill logic is domain-agnostic. To apply this system to a non-trading iterative project: + +1. Replace `results/discovery/` with the project's run output directory +2. Replace `data/positions/` with whatever measures outcome quality +3. Replace the scanner domain files with the project's equivalent components +4. Replace the research source list with domain-appropriate sources + +The folder structure and skill steps are unchanged. + +--- + +## Non-Goals + +- No automated triggering — skills are always invoked manually +- No dashboard or UI — all output is markdown + git commits +- No cross-project knowledge sharing — each project's `docs/iterations/` is independent From 05168c7a965e3150532a0ad47228ccccf9da11ca Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Tue, 7 Apr 2026 20:34:21 -0700 Subject: [PATCH 02/15] docs(iteration-system): add GitHub Actions automation layer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds daily /iterate and weekly /research-strategy cron workflows to the spec — full autonomous loop with PR-gated merges, no auto-merge to main. Co-Authored-By: Claude Sonnet 4.6 --- .../2026-04-07-iteration-system-design.md | 78 ++++++++++++++----- 1 file changed, 60 insertions(+), 18 deletions(-) diff --git a/docs/superpowers/specs/2026-04-07-iteration-system-design.md b/docs/superpowers/specs/2026-04-07-iteration-system-design.md index 30855b69..b2e28f3b 100644 --- a/docs/superpowers/specs/2026-04-07-iteration-system-design.md +++ b/docs/superpowers/specs/2026-04-07-iteration-system-design.md @@ -20,7 +20,7 @@ Currently there is no structured way to capture learnings across runs, trace cod ## Solution -Two Claude Code skills + a versioned knowledge base in `docs/iterations/`. +Two Claude Code skills + a versioned knowledge base in `docs/iterations/` + two GitHub Actions workflows that run the skills on a schedule and open PRs for review. --- @@ -93,10 +93,11 @@ Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure: ## Skill 1: `/iterate` -**Location:** `~/.claude/plugins/cache/.../skills/iterate.md` (or project-local skills dir) +**Location:** project-local `.claude/skills/iterate.md` ### Trigger -Invoked manually after a discovery run or after positions are old enough for outcome data (5+ days). +- **Automated:** GitHub Actions cron, daily at 06:00 UTC (after overnight discovery runs settle) +- **Manual:** invoke `/iterate` at any time for an on-demand iteration cycle ### Steps @@ -121,23 +122,32 @@ Invoked manually after a discovery run or after positions are old enough for out 5. **Implement changes** - Translates learnings into concrete code changes: scanner thresholds, priority logic, LLM prompt wording, scanner enable/disable - - Presents a diff for approval before writing - - On approval: implements and stages changes + - In automated mode: implements without confirmation gate + - In manual mode: presents diff for approval before writing -6. **Commit** - - Commits learning files and code changes together with message format: +6. **Commit + PR** + - Creates branch `iterate/YYYY-MM-DD` + - Commits learning files and code changes together: `learn(iterate): — ` + - Opens PR against `main` with a summary of what changed and why + - In manual mode: commits directly to current branch (no PR unless on `main`) ### Output -Two committed changesets traceable to the same run date: -- `docs/iterations/` — updated knowledge -- `tradingagents/` — code that encodes the knowledge +Per automated run: one PR containing both: +- `docs/iterations/` — updated knowledge files +- `tradingagents/` — code changes that encode the knowledge + +Human reviews PR, merges or closes. No code reaches `main` unreviewed. --- ## Skill 2: `/research-strategy` -**Location:** same skills directory as `/iterate` +**Location:** project-local `.claude/skills/research-strategy.md` + +### Trigger +- **Automated:** GitHub Actions cron, weekly on Monday at 07:00 UTC +- **Manual:** `/research-strategy` (autonomous) or `/research-strategy ""` (directed) ### Two Modes @@ -178,13 +188,20 @@ No topic given. Skill drives its own research agenda based on current weak spots - Saves findings to `docs/iterations/research/YYYY-MM-DD-.md` for all findings - Adds entries to `LEARNINGS.md` -6. **Propose and implement** - - Presents ranked findings with scores - - For the top-ranked finding (or user-selected one): drafts a scanner spec (data needed, signal logic, priority/confidence output) - - On approval: implements scanner following `@SCANNER_REGISTRY.register()` pattern, commits +6. **Implement and open PR** + - In automated mode: implements the top-ranked finding automatically (score threshold: data availability = present, evidence quality ≥ qualitative, complexity ≤ moderate) + - In manual directed mode: presents ranked findings, implements user-selected one + - Creates branch `research/YYYY-MM-DD-` + - Implements scanner following `@SCANNER_REGISTRY.register()` pattern + - Commits: `research(): add scanner — ` + - Opens PR against `main` with research findings summary and implementation notes -### Commit format -`research(): add scanner — ` +### Safety threshold for autonomous implementation +Only auto-implement if ALL of: +- Required data source already integrated (no new API keys needed) +- Complexity estimate: trivial or moderate +- Signal uniqueness score: low overlap with existing scanners +Otherwise: commits research files only, opens PR flagged `needs-review` with explanation of why auto-implementation was skipped. --- @@ -201,8 +218,33 @@ The folder structure and skill steps are unchanged. --- +## GitHub Actions Workflows + +### `.github/workflows/iterate.yml` +``` +schedule: cron '0 6 * * *' # daily at 06:00 UTC +``` +- Checks out repo +- Runs Claude Code with `/iterate` skill non-interactively +- Uses `ANTHROPIC_API_KEY` secret +- Creates branch `iterate/YYYY-MM-DD`, commits, opens PR + +### `.github/workflows/research-strategy.yml` +``` +schedule: cron '0 7 * * 1' # weekly, Monday at 07:00 UTC +``` +- Same setup as above, runs `/research-strategy` (autonomous mode) +- Creates branch `research/YYYY-MM-DD`, commits, opens PR +- If safety threshold not met: opens PR with `needs-review` label, no code changes + +### Required secrets +- `ANTHROPIC_API_KEY` — Claude API access +- `GH_TOKEN` — PAT with `repo` scope for branch creation and PR opening + +--- + ## Non-Goals -- No automated triggering — skills are always invoked manually +- No auto-merge — all PRs require human review before reaching `main` - No dashboard or UI — all output is markdown + git commits - No cross-project knowledge sharing — each project's `docs/iterations/` is independent From 8ba5b8fd7ef15908723256c405c53062cc5fd961 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 07:35:40 -0700 Subject: [PATCH 03/15] docs(iteration-system): collapse strategies/ into scanners/ MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Scanners and strategies are 1:1 in the current codebase — separate folder was artificial. Each scanner file now captures both implementation and thesis. Co-Authored-By: Claude Sonnet 4.6 --- .../specs/2026-04-07-iteration-system-design.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/superpowers/specs/2026-04-07-iteration-system-design.md b/docs/superpowers/specs/2026-04-07-iteration-system-design.md index b2e28f3b..af644df2 100644 --- a/docs/superpowers/specs/2026-04-07-iteration-system-design.md +++ b/docs/superpowers/specs/2026-04-07-iteration-system-design.md @@ -44,17 +44,14 @@ docs/iterations/ │ ├── ml_signal.md │ ├── minervini.md │ └── ... (one file per scanner) -├── strategies/ -│ ├── analyst_upgrade.md -│ ├── momentum.md -│ ├── accumulation.md -│ └── ... (one file per strategy pattern) ├── pipeline/ │ └── scoring.md ← LLM scoring, confidence calibration, ranking └── research/ └── YYYY-MM-DD-.md ← web research findings (append-only, dated) ``` +Each scanner file captures both the implementation learnings and the underlying market thesis — no separate strategies folder since scanners and strategies are currently 1:1. + ### Domain File Schema Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure: From fa3166c494b5578ce9151ce4e7662919f7a7eb38 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 07:45:03 -0700 Subject: [PATCH 04/15] docs(iteration-system): switch to rolling PR strategy At most one open PR per skill at any time. Daily runs push onto the existing branch and update the PR description. Merging resets the cycle. Prevents PR accumulation from unreviewed automated runs. Co-Authored-By: Claude Sonnet 4.6 --- .../2026-04-07-iteration-system-design.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/superpowers/specs/2026-04-07-iteration-system-design.md b/docs/superpowers/specs/2026-04-07-iteration-system-design.md index af644df2..2e1350f8 100644 --- a/docs/superpowers/specs/2026-04-07-iteration-system-design.md +++ b/docs/superpowers/specs/2026-04-07-iteration-system-design.md @@ -122,17 +122,16 @@ Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure: - In automated mode: implements without confirmation gate - In manual mode: presents diff for approval before writing -6. **Commit + PR** - - Creates branch `iterate/YYYY-MM-DD` - - Commits learning files and code changes together: - `learn(iterate): — ` - - Opens PR against `main` with a summary of what changed and why +6. **Commit + rolling PR** + - Checks for an existing open PR with branch prefix `iterate/current` + - If one exists: pushes new commits onto that branch, updates PR description with the latest findings appended + - If none exists: creates branch `iterate/current`, opens a new PR against `main` + - Commit message format: `learn(iterate): — ` - In manual mode: commits directly to current branch (no PR unless on `main`) + - On merge: next run automatically opens a fresh PR ### Output -Per automated run: one PR containing both: -- `docs/iterations/` — updated knowledge files -- `tradingagents/` — code changes that encode the knowledge +At most one open `iterate/current` PR at any time. It accumulates daily learnings until merged. Merging resets the cycle. Human reviews PR, merges or closes. No code reaches `main` unreviewed. @@ -185,13 +184,14 @@ No topic given. Skill drives its own research agenda based on current weak spots - Saves findings to `docs/iterations/research/YYYY-MM-DD-.md` for all findings - Adds entries to `LEARNINGS.md` -6. **Implement and open PR** +6. **Implement and rolling PR** - In automated mode: implements the top-ranked finding automatically (score threshold: data availability = present, evidence quality ≥ qualitative, complexity ≤ moderate) - In manual directed mode: presents ranked findings, implements user-selected one - - Creates branch `research/YYYY-MM-DD-` - - Implements scanner following `@SCANNER_REGISTRY.register()` pattern + - Checks for an existing open PR with branch prefix `research/current` + - If one exists: pushes new commits onto that branch, updates PR description with new findings appended + - If none exists: creates branch `research/current`, opens a new PR against `main` - Commits: `research(): add scanner — ` - - Opens PR against `main` with research findings summary and implementation notes + - On merge: next run automatically opens a fresh PR ### Safety threshold for autonomous implementation Only auto-implement if ALL of: From ec2b3c2a459e83bc92a818496ce7eade0870ff52 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 07:54:18 -0700 Subject: [PATCH 05/15] docs(iteration-system): add implementation plan Covers 5 tasks: knowledge base structure, /iterate command, /research-strategy command, and two GitHub Actions workflows with rolling PR logic. Co-Authored-By: Claude Sonnet 4.6 --- .../plans/2026-04-08-iteration-system.md | 1038 +++++++++++++++++ 1 file changed, 1038 insertions(+) create mode 100644 docs/superpowers/plans/2026-04-08-iteration-system.md diff --git a/docs/superpowers/plans/2026-04-08-iteration-system.md b/docs/superpowers/plans/2026-04-08-iteration-system.md new file mode 100644 index 00000000..045a96ea --- /dev/null +++ b/docs/superpowers/plans/2026-04-08-iteration-system.md @@ -0,0 +1,1038 @@ +# Iteration System Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a self-improving iteration system with two Claude Code commands (`/iterate`, `/research-strategy`) backed by a versioned knowledge base and two GitHub Actions workflows that run them daily/weekly and open rolling PRs. + +**Architecture:** A `docs/iterations/` knowledge base stores per-scanner learnings in domain files. Two `.claude/commands/` files define the skill behavior. Two GitHub Actions workflows invoke Claude non-interactively, then handle branch/PR management externally so Claude's job is analysis + file writing only. + +**Tech Stack:** Claude Code CLI (`claude -p`), GitHub Actions, `gh` CLI, Python (existing), Markdown knowledge base + +--- + +## File Map + +**Create:** +- `docs/iterations/LEARNINGS.md` — master index tracking last-analyzed run per scanner +- `docs/iterations/scanners/options_flow.md` — options flow scanner learnings +- `docs/iterations/scanners/insider_buying.md` — insider buying scanner learnings +- `docs/iterations/scanners/volume_accumulation.md` — volume accumulation scanner learnings +- `docs/iterations/scanners/reddit_dd.md` — reddit DD scanner learnings +- `docs/iterations/scanners/reddit_trending.md` — reddit trending scanner learnings +- `docs/iterations/scanners/semantic_news.md` — semantic news scanner learnings +- `docs/iterations/scanners/market_movers.md` — market movers scanner learnings +- `docs/iterations/scanners/earnings_calendar.md` — earnings calendar scanner learnings +- `docs/iterations/scanners/analyst_upgrades.md` — analyst upgrades scanner learnings +- `docs/iterations/scanners/technical_breakout.md` — technical breakout scanner learnings +- `docs/iterations/scanners/sector_rotation.md` — sector rotation scanner learnings +- `docs/iterations/scanners/ml_signal.md` — ML signal scanner learnings +- `docs/iterations/scanners/minervini.md` — Minervini scanner learnings +- `docs/iterations/pipeline/scoring.md` — LLM scoring and ranking learnings +- `.claude/commands/iterate.md` — `/iterate` Claude Code command +- `.claude/commands/research-strategy.md` — `/research-strategy` Claude Code command +- `.github/workflows/iterate.yml` — daily cron workflow +- `.github/workflows/research-strategy.yml` — weekly cron workflow + +**Modify:** none — purely additive + +--- + +## Task 1: Create Knowledge Base Folder Structure + +**Files:** +- Create: `docs/iterations/LEARNINGS.md` +- Create: `docs/iterations/scanners/*.md` (13 files) +- Create: `docs/iterations/pipeline/scoring.md` + +- [ ] **Step 1: Create directory structure** + +```bash +mkdir -p docs/iterations/scanners docs/iterations/pipeline docs/iterations/research +``` + +- [ ] **Step 2: Create LEARNINGS.md** + +```bash +cat > docs/iterations/LEARNINGS.md << 'EOF' +# Learnings Index + +**Last analyzed run:** _(none yet — will be set by first /iterate run)_ + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | — | No data yet | +| insider_buying | scanners/insider_buying.md | — | No data yet | +| volume_accumulation | scanners/volume_accumulation.md | — | No data yet | +| reddit_dd | scanners/reddit_dd.md | — | No data yet | +| reddit_trending | scanners/reddit_trending.md | — | No data yet | +| semantic_news | scanners/semantic_news.md | — | No data yet | +| market_movers | scanners/market_movers.md | — | No data yet | +| earnings_calendar | scanners/earnings_calendar.md | — | No data yet | +| analyst_upgrades | scanners/analyst_upgrades.md | — | No data yet | +| technical_breakout | scanners/technical_breakout.md | — | No data yet | +| sector_rotation | scanners/sector_rotation.md | — | No data yet | +| ml_signal | scanners/ml_signal.md | — | No data yet | +| minervini | scanners/minervini.md | — | No data yet | +| pipeline/scoring | pipeline/scoring.md | — | No data yet | +EOF +``` + +- [ ] **Step 3: Create scanner domain files** + +Create each of the 13 scanner files using this template. Fill in the "Current Understanding" section with what is already known from prior work. Run this for each scanner: + +**`docs/iterations/scanners/options_flow.md`:** +```markdown +# Options Flow Scanner + +## Current Understanding +Scans for unusual options volume relative to open interest using Tradier API. +Call/put volume ratio below 0.1 is a reliable bullish signal when combined with +premium >$25K. The premium filter is configured but must be explicitly applied. +Scanning only the nearest expiration misses institutional positioning in 30+ DTE +contracts — scanning up to 3 expirations improves signal quality. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does scanning 3 expirations vs 1 meaningfully change hit rate? +- [ ] Is moneyness (ITM vs OTM) a useful signal filter? +``` + +**`docs/iterations/scanners/insider_buying.md`:** +```markdown +# Insider Buying Scanner + +## Current Understanding +Scrapes SEC Form 4 filings. CEO/CFO purchases >$100K are the most reliable signal. +Cluster detection (2+ insiders buying within 14 days) historically a high-conviction +setup. Transaction details (name, title, value) must be preserved from scraper output +and included in candidate context — dropping them loses signal clarity. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does cluster detection (2+ insiders in 14 days) outperform single-insider signals? +- [ ] Is there a minimum transaction size below which signal quality degrades sharply? +``` + +**`docs/iterations/scanners/volume_accumulation.md`:** +```markdown +# Volume Accumulation Scanner + +## Current Understanding +Detects stocks with volume >2x average. Key weakness: cannot distinguish buying from +selling — high volume on a down day is distribution, not accumulation. Multi-day mode +(3 of last 5 days >1.5x) is more reliable than single-day spikes. Price-change filter +(<3% absolute move) isolates quiet accumulation from momentum chasing. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a price-direction filter (volume + flat/up price) improve hit rate? +- [ ] Is 3-of-5-day accumulation a stronger signal than single-day 2x volume? +``` + +**`docs/iterations/scanners/reddit_dd.md`:** +```markdown +# Reddit DD Scanner + +## Current Understanding +Scans r/investing, r/stocks, r/wallstreetbets for DD posts. LLM quality score is +computed but not used for filtering — using it (80+ = HIGH, 60-79 = MEDIUM, <60 = skip) +would reduce noise. Subreddit weighting matters: r/investing posts are more reliable +than r/pennystocks. Post title and LLM score should appear in candidate context. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does filtering by LLM quality score >60 meaningfully reduce false positives? +- [ ] Does subreddit weighting change hit rates? +``` + +**`docs/iterations/scanners/reddit_trending.md`:** +```markdown +# Reddit Trending Scanner + +## Current Understanding +Tracks mention velocity across subreddits. 50+ mentions in 6 hours = HIGH priority. +20-49 = MEDIUM. Mention count should appear in context ("47 mentions in 6hrs"). +Signal is early-indicator oriented — catches momentum before price moves. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does mention velocity (rate of increase) outperform raw mention count? +``` + +**`docs/iterations/scanners/semantic_news.md`:** +```markdown +# Semantic News Scanner + +## Current Understanding +Currently regex-based extraction, not semantic. Headline text is not included in +candidate context — the context just says "Mentioned in recent market news" which +is not informative. Catalyst classification from headline keywords (upgrade/FDA/ +acquisition/earnings) would improve LLM scoring quality significantly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Would embedding-based semantic matching outperform keyword regex? +- [ ] Does catalyst classification (FDA vs earnings vs acquisition) affect hit rate? +``` + +**`docs/iterations/scanners/market_movers.md`:** +```markdown +# Market Movers Scanner + +## Current Understanding +Finds stocks that have already moved significantly. This is a reactive scanner — +it identifies momentum after it starts rather than predicting it. Useful for +continuation plays but not for early-stage entry. Best combined with volume +confirmation to distinguish breakouts from spikes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is a volume confirmation filter (>1.5x average) useful for filtering out noise? +``` + +**`docs/iterations/scanners/earnings_calendar.md`:** +```markdown +# Earnings Calendar Scanner + +## Current Understanding +Identifies stocks with earnings announcements in the next N days. Pre-earnings +setups work best when combined with options flow (IV expansion) or insider activity. +Standalone earnings calendar signal is too broad — nearly every stock has earnings +quarterly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring options confirmation alongside earnings improve signal quality? +``` + +**`docs/iterations/scanners/analyst_upgrades.md`:** +```markdown +# Analyst Upgrades Scanner + +## Current Understanding +Detects analyst upgrades/price target increases. Most reliable when upgrade comes +from a top-tier firm (Goldman, Morgan Stanley, JPMorgan) and represents a meaningful +target increase (>15%). Short squeeze potential (high short interest) combined with +an upgrade is a historically strong setup. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does analyst tier (BB firm vs boutique) predict upgrade quality? +- [ ] Does short interest >20% combined with an upgrade produce outsized moves? +``` + +**`docs/iterations/scanners/technical_breakout.md`:** +```markdown +# Technical Breakout Scanner + +## Current Understanding +Detects price breakouts above key resistance levels on above-average volume. +Minervini-style setups (stage 2 uptrend, tight base, volume-confirmed breakout) +tend to have the highest follow-through rate. False breakouts are common without +volume confirmation (>1.5x average on breakout day). + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring volume confirmation on the breakout day reduce false positives? +``` + +**`docs/iterations/scanners/sector_rotation.md`:** +```markdown +# Sector Rotation Scanner + +## Current Understanding +Detects money flowing between sectors using relative strength analysis. Most useful +as a macro filter rather than a primary signal — knowing which sectors are in favor +improves conviction in scanner candidates from those sectors. Standalone sector +rotation signals are too broad for individual stock selection. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Can sector rotation data be used as a multiplier on other scanner scores? +``` + +**`docs/iterations/scanners/ml_signal.md`:** +```markdown +# ML Signal Scanner + +## Current Understanding +Uses a trained ML model to predict short-term price movement probability. Current +threshold of 35% win probability is worse than a coin flip — the model needs +retraining or the threshold needs raising to 55%+ to be useful. Signal quality +depends heavily on feature freshness; stale features degrade performance. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does raising the threshold to 55%+ improve precision at the cost of recall? +- [ ] Would retraining on the last 90 days of recommendations improve accuracy? +``` + +**`docs/iterations/scanners/minervini.md`:** +```markdown +# Minervini Scanner + +## Current Understanding +Implements Mark Minervini's SEPA (Specific Entry Point Analysis) criteria: stage 2 +uptrend, price above 50/150/200 SMA in the right order, 52-week high proximity, +RS line at new highs. Historically one of the highest-conviction scanner setups. +Works best in bull market conditions; underperforms in choppy/bear markets. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a market condition filter (S&P 500 above 200 SMA) improve hit rate? +``` + +- [ ] **Step 4: Create pipeline/scoring.md** + +```bash +cat > docs/iterations/pipeline/scoring.md << 'EOF' +# Pipeline Scoring & Ranking + +## Current Understanding +LLM assigns a final_score (0-100) and confidence (1-10) to each candidate. +Score and confidence are correlated but not identical — a speculative setup +can score 80 with confidence 6. The ranker uses final_score as primary sort key. +No evidence yet on whether confidence or score is a better predictor of outcomes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is confidence a better outcome predictor than final_score? +- [ ] Does score threshold (e.g. only surface candidates >70) improve hit rate? +EOF +``` + +- [ ] **Step 5: Verify structure** + +```bash +find docs/iterations -type f | sort +``` + +Expected output: 16 files (LEARNINGS.md + 13 scanner files + scoring.md + research dir exists) + +- [ ] **Step 6: Commit** + +```bash +git add docs/iterations/ +git commit -m "feat(iteration-system): add knowledge base folder structure with seeded scanner files" +``` + +--- + +## Task 2: Write the `/iterate` Command + +**Files:** +- Create: `.claude/commands/iterate.md` + +- [ ] **Step 1: Create `.claude/commands/` directory** + +```bash +mkdir -p .claude/commands +``` + +- [ ] **Step 2: Write iterate.md** + +Create `.claude/commands/iterate.md` with this exact content: + +```markdown +# /iterate + +Analyze recent discovery runs and P&L outcomes, update learning files in +`docs/iterations/`, implement concrete code improvements in `tradingagents/`, +then commit everything. In CI (`CI=true`), stop before git operations — the +workflow handles branching and PR creation. + +--- + +## Step 1: Determine What to Analyze + +Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date. + +Then scan `results/discovery/` for all run directories with dates AFTER the +last analyzed date. Each run directory contains `discovery_result.json` and +`tool_execution_logs.json`. Collect all unanalyzed runs. + +Also scan `data/recommendations/` for JSON files dated 5 or more days ago. +Load each file and extract recommendations where `status != "open"` OR where +`discovery_date` is 5+ days in the past (these have had time to play out). +For P&L analysis you need: `ticker`, `strategy_match`, `final_score`, +`confidence`, `discovery_date`, `entry_price`, `status`. + +Set your analysis mode: +- If unanalyzed runs exist → include **fast-loop** (output quality analysis) +- If mature recommendations exist → include **P&L loop** (outcome analysis) +- If neither → print "No new data to analyze since last run." and exit. + +## Step 2: Load Domain Context + +For each scanner that appears in the unanalyzed runs (check `strategy_match` +field in discovery results), read the corresponding file from +`docs/iterations/scanners/.md`. + +Also read `docs/iterations/pipeline/scoring.md`. +Also read `docs/iterations/LEARNINGS.md` for full index awareness. + +## Step 3: Analyze + +### Fast-Loop Analysis (output quality) +For each unanalyzed run's `discovery_result.json`: +- **Signal specificity**: Does each candidate have a concrete, specific thesis + or a generic one? Flag candidates with vague context. +- **Scanner noise rate**: How many candidates per scanner? Scanners producing + 10+ candidates with low scores (<65) are noisy. +- **Confidence calibration**: Is confidence (1-10) consistent with score (0-100)? + A score of 85 with confidence 5 suggests miscalibration. +- **Duplicate candidates**: Same ticker appearing from 2+ scanners — note as + confluence (positive) or redundancy (negative, if identical thesis). + +### P&L Loop Analysis (outcome analysis) +For each mature recommendation: +- Group by `strategy_match` (scanner). Compute per-scanner: hit rate (status + reflects outcome — check what non-"open" statuses mean in the data). +- Flag scanners where final_score > 80 but outcomes are poor — overconfident. +- Flag scanners where final_score < 65 but outcomes are good — undervalued. +- Note patterns: do high-confidence (9-10) picks outperform low-confidence (6-7)? + +## Step 4: Write Learnings + +For each scanner that appeared in the analysis, update its domain file in +`docs/iterations/scanners/.md`: + +1. **Append to Evidence Log**: Add a dated entry with your specific observations. + Include: what was observed, what it implies, confidence (low/medium/high). +2. **Update Current Understanding**: If your new evidence is medium or high + confidence AND contradicts or meaningfully extends the current understanding, + rewrite that section. Otherwise leave it unchanged. +3. **Update Pending Hypotheses**: Check off any hypotheses that are now answered. + Add new ones that your analysis surfaced. + +Update `docs/iterations/LEARNINGS.md`: +- Set **Last analyzed run** to today's date +- Update the one-line summary and Last Updated date for each scanner you touched + +## Step 5: Implement Code Changes + +Based on your learnings, identify concrete improvements. For each improvement: + +**Translate one learning → one code change.** Examples: +- "ML signal threshold is worse than a coin flip" → raise threshold in + `tradingagents/dataflows/discovery/scanners/ml_signal.py` +- "Options flow premium filter is configured but not applied" → add the check +- "Reddit DD LLM score computed but unused" → use it for priority assignment + +For each change: +1. Read the relevant scanner file to understand current implementation +2. Make the minimal change that encodes the learning +3. Do not refactor surrounding code — change only what the learning motivates + +Implement all changes before committing. + +## Step 6: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/ tradingagents/ +git commit -m "learn(iterate): $(date +%Y-%m-%d) — " +``` + +Then check for an existing open PR on branch `iterate/current`: +```bash +EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: push to that branch and update its description with your findings. +If none exists: create branch `iterate/current`, push, open PR against `main`. +``` +``` + +- [ ] **Step 3: Verify the command is discoverable** + +```bash +ls .claude/commands/ +``` + +Expected: `iterate.md` + +- [ ] **Step 4: Smoke-test the command manually** + +In a Claude Code session in this project, type `/iterate` and verify Claude reads +the command and starts executing Step 1 (reads LEARNINGS.md, scans results/). +You don't need to let it complete — just confirm it picks up the command file. + +- [ ] **Step 5: Commit** + +```bash +git add .claude/commands/iterate.md +git commit -m "feat(iteration-system): add /iterate Claude Code command" +``` + +--- + +## Task 3: Write the `/research-strategy` Command + +**Files:** +- Create: `.claude/commands/research-strategy.md` + +- [ ] **Step 1: Write research-strategy.md** + +Create `.claude/commands/research-strategy.md` with this exact content: + +```markdown +# /research-strategy + +Research new trading strategies or scanner improvements, evaluate fit against +the existing pipeline, write findings to `docs/iterations/research/`, and +implement the top-ranked finding as a new scanner if it qualifies. + +Usage: +- `/research-strategy` — autonomous mode: Claude picks research topics +- `/research-strategy "topic"` — directed mode: research a specific strategy + +In CI (`CI=true`), stop before git operations — the workflow handles them. + +--- + +## Step 1: Set Research Agenda + +**If a topic argument was provided** (`$ARGUMENTS` is not empty): +- Research topic = `$ARGUMENTS` +- Skip to Step 2. + +**Autonomous mode** (no argument): +- Read `docs/iterations/LEARNINGS.md` and all scanner domain files in + `docs/iterations/scanners/` +- Identify the 3-5 highest-leverage research opportunities: + - Scanners with low-confidence current understanding + - Pending hypotheses marked with `- [ ]` + - Gaps: signal types with no current scanner (e.g. dark pool flow, + short interest changes, institutional 13F filings) +- Rank by potential impact. Pick the top topic to research this run. +- Print your agenda: "Researching: — Reason: " + +## Step 2: Search + +Search the following sources for the research topic. For each source, look for: +signal definition, entry/exit criteria, known statistical edge, known failure +modes, data requirements. + +**Sources to search:** +- Reddit: r/algotrading, r/quant, r/investing (site:reddit.com) +- QuantifiedStrategies (site:quantifiedstrategies.com) +- Alpha Architect (site:alphaarchitect.com) +- CSS Analytics (site:cssanalytics.wordpress.com) +- Hacker News (hn.algolia.com query) +- GitHub: search "quant scanner " and "trading strategy " +- SSRN: search quantitative finance papers on the topic +- arXiv q-fin section + +Use WebSearch and WebFetch to retrieve actual content. Read at least 3-5 +distinct sources before forming a conclusion. + +## Step 3: Cross-Reference Existing Knowledge + +Check `docs/iterations/scanners/` and `docs/iterations/research/` for any +prior work on this topic. Flag explicitly if this overlaps with: +- An existing scanner (name it) +- A previously researched and discarded approach (cite the research file) +- A pending hypothesis in an existing scanner file + +## Step 4: Evaluate Fit + +Score the finding on four dimensions (each: ✅ pass / ⚠️ partial / ❌ fail): + +1. **Data availability**: Is the required data source already integrated in + `tradingagents/dataflows/`? Check for existing API clients. +2. **Implementation complexity**: trivial (<2 hours) / moderate (2-8 hours) / + large (>8 hours) +3. **Signal uniqueness**: Low overlap with existing scanners = good. + High overlap = flag as redundant. +4. **Evidence quality**: backtested with statistics / qualitative analysis / + anecdotal only + +**Auto-implement threshold** (all must pass for autonomous CI implementation): +- Data availability: ✅ (data source already integrated) +- Complexity: trivial or moderate +- Uniqueness: low overlap +- Evidence: qualitative or better + +## Step 5: Write Research File + +Save findings to `docs/iterations/research/$(date +%Y-%m-%d)-.md`: + +```markdown +# Research: + +**Date:** YYYY-MM-DD +**Mode:** directed | autonomous + +## Summary +<2-3 sentences on what was found> + +## Sources Reviewed +- +- +... + +## Fit Evaluation +| Dimension | Score | Notes | +|-----------|-------|-------| +| Data availability | ✅/⚠️/❌ | ... | +| Complexity | trivial/moderate/large | ... | +| Signal uniqueness | low/medium/high overlap | ... | +| Evidence quality | backtested/qualitative/anecdotal | ... | + +## Recommendation +Implement / Skip / Needs more data + +## Proposed Scanner Spec (if recommending implementation) +- **Scanner name:** `` +- **Data source:** `tradingagents/dataflows/.py` +- **Signal logic:** +- **Priority rules:** CRITICAL if X, HIGH if Y, MEDIUM otherwise +- **Context format:** "" +``` + +Add an entry to `docs/iterations/LEARNINGS.md` under a Research section. + +## Step 6: Implement (if threshold met) + +If the finding meets the auto-implement threshold: + +1. Read the scanner registry to understand the registration pattern: + `tradingagents/dataflows/discovery/scanner_registry.py` +2. Read an existing simple scanner for the code pattern, e.g.: + `tradingagents/dataflows/discovery/scanners/earnings_calendar.py` +3. Create `tradingagents/dataflows/discovery/scanners/.py` following + the same structure: class with `@SCANNER_REGISTRY.register()` decorator, + `name`, `pipeline`, `scan()` method returning list of candidate dicts with + keys: `ticker`, `source`, `context`, `priority`. +4. Register the scanner in `tradingagents/dataflows/discovery/scanners/__init__.py` + if needed (check if auto-discovery is in place). + +If threshold is NOT met: write the research file only. Add a `needs-review` note +at the top explaining why auto-implementation was skipped. + +## Step 7: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md +git commit -m "research(): " +``` + +Check for existing open PR on `research/current`: +```bash +EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty') +``` + +If exists: push to branch, update PR description. +If not: create branch `research/current`, push, open PR. +``` +``` + +- [ ] **Step 2: Smoke-test the command manually** + +In a Claude Code session, type `/research-strategy "momentum breakout"` and verify +Claude reads the command, prints its agenda, and starts searching. Let it run at +least through Step 2 to confirm web search is working. + +- [ ] **Step 3: Commit** + +```bash +git add .claude/commands/research-strategy.md +git commit -m "feat(iteration-system): add /research-strategy Claude Code command" +``` + +--- + +## Task 4: Write the `iterate.yml` GitHub Actions Workflow + +**Files:** +- Create: `.github/workflows/iterate.yml` + +- [ ] **Step 1: Write iterate.yml** + +Create `.github/workflows/iterate.yml` with this exact content: + +```yaml +name: Daily Iterate + +on: + schedule: + # 6:00 AM UTC daily — analyzes previous day's discovery run + - cron: "0 6 * * *" + workflow_dispatch: + inputs: + force: + description: "Force iterate even if no new runs detected" + required: false + default: "false" + type: choice + options: + - "false" + - "true" + +env: + PYTHON_VERSION: "3.10" + NODE_VERSION: "20" + +jobs: + iterate: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 30 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /iterate + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "/iterate" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/ tradingagents/ || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /iterate" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + git commit -m "learn(iterate): ${DATE} — automated iteration run" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="iterate/current" + DATE=$(date -u +%Y-%m-%d) + + # Check for existing open PR on this branch + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + # Push onto existing branch and update PR + git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ + git push origin HEAD:"$BRANCH" + gh pr edit "$EXISTING_PR" \ + --body "$(cat docs/iterations/LEARNINGS.md) + +--- +*Last updated: ${DATE} by automated iterate workflow*" + echo "Updated existing PR #${EXISTING_PR}" + else + # Create new branch and open PR + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" + gh pr create \ + --title "learn(iterate): automated improvements — ${DATE}" \ + --body "$(cat docs/iterations/LEARNINGS.md) + +--- +*Opened: ${DATE} by automated iterate workflow* +*Merge to apply learnings and reset the iteration cycle.*" \ + --label "automated,iteration" \ + --base main + echo "Opened new PR" + fi +``` + +- [ ] **Step 2: Validate YAML syntax** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/iterate.yml'))" && echo "YAML valid" +``` + +Expected: `YAML valid` + +- [ ] **Step 3: Add required GitHub secrets** + +Verify these secrets exist in the repo's `TradingAgent` environment (Settings → Environments → TradingAgent → Secrets): +- `ANTHROPIC_API_KEY` — for Claude Code +- `GH_TOKEN` — PAT with `repo` scope (needed for PR creation; the default `GITHUB_TOKEN` cannot create PRs that trigger other workflows) + +Check existing secrets: +```bash +gh secret list --env TradingAgent +``` + +If `GH_TOKEN` is missing, the user must add it manually via GitHub UI (PAT with `repo` scope). + +- [ ] **Step 4: Trigger manually to test** + +```bash +gh workflow run iterate.yml +sleep 10 +gh run list --workflow=iterate.yml --limit=1 +``` + +Watch the run in GitHub Actions UI. Verify: +- Claude Code installs successfully +- `/iterate` runs without crashing +- If changes produced: a PR is created or updated on `iterate/current` +- If no changes: workflow exits cleanly with "No changes produced" message + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/iterate.yml +git commit -m "feat(iteration-system): add daily iterate GitHub Actions workflow" +``` + +--- + +## Task 5: Write the `research-strategy.yml` GitHub Actions Workflow + +**Files:** +- Create: `.github/workflows/research-strategy.yml` + +- [ ] **Step 1: Write research-strategy.yml** + +Create `.github/workflows/research-strategy.yml` with this exact content: + +```yaml +name: Weekly Research Strategy + +on: + schedule: + # 7:00 AM UTC every Monday — runs after iterate (6:00 AM) + - cron: "0 7 * * 1" + workflow_dispatch: + inputs: + topic: + description: "Research topic (blank = autonomous mode)" + required: false + default: "" + +env: + NODE_VERSION: "20" + +jobs: + research: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 45 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Build research prompt + id: prompt + run: | + TOPIC="${{ github.event.inputs.topic }}" + if [ -n "$TOPIC" ]; then + echo "prompt=/research-strategy \"${TOPIC}\"" >> "$GITHUB_OUTPUT" + else + echo "prompt=/research-strategy" >> "$GITHUB_OUTPUT" + fi + + - name: Run /research-strategy + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "${{ steps.prompt.outputs.prompt }}" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /research-strategy" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + TOPIC="${{ github.event.inputs.topic }}" + if [ -n "$TOPIC" ]; then + MSG="research(${TOPIC}): ${DATE} — automated research run" + else + MSG="research(autonomous): ${DATE} — automated research run" + fi + git commit -m "$MSG" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="research/current" + DATE=$(date -u +%Y-%m-%d) + + # Summarise new research files for PR body + NEW_FILES=$(git diff HEAD~1 --name-only -- docs/iterations/research/ | head -5) + PR_BODY="## Research Findings — ${DATE} + +New research files: +${NEW_FILES} + +$(cat docs/iterations/LEARNINGS.md | head -30) + +--- +*Last updated: ${DATE} by automated research-strategy workflow* +*Merge to apply new scanner implementations and reset the research cycle.*" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ + git push origin HEAD:"$BRANCH" + gh pr edit "$EXISTING_PR" --body "$PR_BODY" + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" + gh pr create \ + --title "research: new strategy findings — ${DATE}" \ + --body "$PR_BODY" \ + --label "automated,research" \ + --base main + echo "Opened new PR" + fi +``` + +- [ ] **Step 2: Validate YAML syntax** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/research-strategy.yml'))" && echo "YAML valid" +``` + +Expected: `YAML valid` + +- [ ] **Step 3: Trigger manually to test** + +```bash +gh workflow run research-strategy.yml --field topic="momentum breakout" +sleep 10 +gh run list --workflow=research-strategy.yml --limit=1 +``` + +Watch in GitHub Actions UI. Verify: +- `/research-strategy "momentum breakout"` runs (directed mode) +- Research file written to `docs/iterations/research/` +- PR created or updated on `research/current` + +- [ ] **Step 4: Test autonomous mode** + +```bash +gh workflow run research-strategy.yml +sleep 10 +gh run list --workflow=research-strategy.yml --limit=1 +``` + +Verify Claude picks a topic from the knowledge base weak spots rather than erroring. + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/research-strategy.yml +git commit -m "feat(iteration-system): add weekly research-strategy GitHub Actions workflow" +``` + +--- + +## Completion Checklist + +- [ ] `docs/iterations/` has 16 files (LEARNINGS.md + 13 scanner files + scoring.md) +- [ ] `/iterate` command loads in Claude Code session +- [ ] `/research-strategy` command loads in Claude Code session +- [ ] `iterate.yml` workflow runs cleanly (manual trigger) +- [ ] `research-strategy.yml` workflow runs cleanly (manual trigger, directed mode) +- [ ] `research-strategy.yml` autonomous mode picks a topic from knowledge base +- [ ] Rolling PR logic verified: second workflow run updates existing PR, not creates a new one +- [ ] `GH_TOKEN` secret exists in `TradingAgent` environment From 3fb82e818072abb55523452e4f1aa4c43b27733e Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 08:09:07 -0700 Subject: [PATCH 06/15] feat(iteration-system): add knowledge base folder structure with seeded scanner files Co-Authored-By: Claude Sonnet 4.6 --- docs/iterations/LEARNINGS.md | 20 +++++++++++++++++++ docs/iterations/pipeline/scoring.md | 14 +++++++++++++ docs/iterations/scanners/analyst_upgrades.md | 14 +++++++++++++ docs/iterations/scanners/earnings_calendar.md | 13 ++++++++++++ docs/iterations/scanners/insider_buying.md | 14 +++++++++++++ docs/iterations/scanners/market_movers.md | 13 ++++++++++++ docs/iterations/scanners/minervini.md | 13 ++++++++++++ docs/iterations/scanners/ml_signal.md | 14 +++++++++++++ docs/iterations/scanners/options_flow.md | 15 ++++++++++++++ docs/iterations/scanners/reddit_dd.md | 14 +++++++++++++ docs/iterations/scanners/reddit_trending.md | 12 +++++++++++ docs/iterations/scanners/sector_rotation.md | 13 ++++++++++++ docs/iterations/scanners/semantic_news.md | 14 +++++++++++++ .../iterations/scanners/technical_breakout.md | 13 ++++++++++++ .../scanners/volume_accumulation.md | 14 +++++++++++++ 15 files changed, 210 insertions(+) create mode 100644 docs/iterations/LEARNINGS.md create mode 100644 docs/iterations/pipeline/scoring.md create mode 100644 docs/iterations/scanners/analyst_upgrades.md create mode 100644 docs/iterations/scanners/earnings_calendar.md create mode 100644 docs/iterations/scanners/insider_buying.md create mode 100644 docs/iterations/scanners/market_movers.md create mode 100644 docs/iterations/scanners/minervini.md create mode 100644 docs/iterations/scanners/ml_signal.md create mode 100644 docs/iterations/scanners/options_flow.md create mode 100644 docs/iterations/scanners/reddit_dd.md create mode 100644 docs/iterations/scanners/reddit_trending.md create mode 100644 docs/iterations/scanners/sector_rotation.md create mode 100644 docs/iterations/scanners/semantic_news.md create mode 100644 docs/iterations/scanners/technical_breakout.md create mode 100644 docs/iterations/scanners/volume_accumulation.md diff --git a/docs/iterations/LEARNINGS.md b/docs/iterations/LEARNINGS.md new file mode 100644 index 00000000..61c54c30 --- /dev/null +++ b/docs/iterations/LEARNINGS.md @@ -0,0 +1,20 @@ +# Learnings Index + +**Last analyzed run:** _(none yet — will be set by first /iterate run)_ + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | — | No data yet | +| insider_buying | scanners/insider_buying.md | — | No data yet | +| volume_accumulation | scanners/volume_accumulation.md | — | No data yet | +| reddit_dd | scanners/reddit_dd.md | — | No data yet | +| reddit_trending | scanners/reddit_trending.md | — | No data yet | +| semantic_news | scanners/semantic_news.md | — | No data yet | +| market_movers | scanners/market_movers.md | — | No data yet | +| earnings_calendar | scanners/earnings_calendar.md | — | No data yet | +| analyst_upgrades | scanners/analyst_upgrades.md | — | No data yet | +| technical_breakout | scanners/technical_breakout.md | — | No data yet | +| sector_rotation | scanners/sector_rotation.md | — | No data yet | +| ml_signal | scanners/ml_signal.md | — | No data yet | +| minervini | scanners/minervini.md | — | No data yet | +| pipeline/scoring | pipeline/scoring.md | — | No data yet | diff --git a/docs/iterations/pipeline/scoring.md b/docs/iterations/pipeline/scoring.md new file mode 100644 index 00000000..54b00039 --- /dev/null +++ b/docs/iterations/pipeline/scoring.md @@ -0,0 +1,14 @@ +# Pipeline Scoring & Ranking + +## Current Understanding +LLM assigns a final_score (0-100) and confidence (1-10) to each candidate. +Score and confidence are correlated but not identical — a speculative setup +can score 80 with confidence 6. The ranker uses final_score as primary sort key. +No evidence yet on whether confidence or score is a better predictor of outcomes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is confidence a better outcome predictor than final_score? +- [ ] Does score threshold (e.g. only surface candidates >70) improve hit rate? diff --git a/docs/iterations/scanners/analyst_upgrades.md b/docs/iterations/scanners/analyst_upgrades.md new file mode 100644 index 00000000..767dbc70 --- /dev/null +++ b/docs/iterations/scanners/analyst_upgrades.md @@ -0,0 +1,14 @@ +# Analyst Upgrades Scanner + +## Current Understanding +Detects analyst upgrades/price target increases. Most reliable when upgrade comes +from a top-tier firm (Goldman, Morgan Stanley, JPMorgan) and represents a meaningful +target increase (>15%). Short squeeze potential (high short interest) combined with +an upgrade is a historically strong setup. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does analyst tier (BB firm vs boutique) predict upgrade quality? +- [ ] Does short interest >20% combined with an upgrade produce outsized moves? diff --git a/docs/iterations/scanners/earnings_calendar.md b/docs/iterations/scanners/earnings_calendar.md new file mode 100644 index 00000000..1550247b --- /dev/null +++ b/docs/iterations/scanners/earnings_calendar.md @@ -0,0 +1,13 @@ +# Earnings Calendar Scanner + +## Current Understanding +Identifies stocks with earnings announcements in the next N days. Pre-earnings +setups work best when combined with options flow (IV expansion) or insider activity. +Standalone earnings calendar signal is too broad — nearly every stock has earnings +quarterly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring options confirmation alongside earnings improve signal quality? diff --git a/docs/iterations/scanners/insider_buying.md b/docs/iterations/scanners/insider_buying.md new file mode 100644 index 00000000..b77fc60b --- /dev/null +++ b/docs/iterations/scanners/insider_buying.md @@ -0,0 +1,14 @@ +# Insider Buying Scanner + +## Current Understanding +Scrapes SEC Form 4 filings. CEO/CFO purchases >$100K are the most reliable signal. +Cluster detection (2+ insiders buying within 14 days) historically a high-conviction +setup. Transaction details (name, title, value) must be preserved from scraper output +and included in candidate context — dropping them loses signal clarity. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does cluster detection (2+ insiders in 14 days) outperform single-insider signals? +- [ ] Is there a minimum transaction size below which signal quality degrades sharply? diff --git a/docs/iterations/scanners/market_movers.md b/docs/iterations/scanners/market_movers.md new file mode 100644 index 00000000..a4435309 --- /dev/null +++ b/docs/iterations/scanners/market_movers.md @@ -0,0 +1,13 @@ +# Market Movers Scanner + +## Current Understanding +Finds stocks that have already moved significantly. This is a reactive scanner — +it identifies momentum after it starts rather than predicting it. Useful for +continuation plays but not for early-stage entry. Best combined with volume +confirmation to distinguish breakouts from spikes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is a volume confirmation filter (>1.5x average) useful for filtering out noise? diff --git a/docs/iterations/scanners/minervini.md b/docs/iterations/scanners/minervini.md new file mode 100644 index 00000000..e2cc127c --- /dev/null +++ b/docs/iterations/scanners/minervini.md @@ -0,0 +1,13 @@ +# Minervini Scanner + +## Current Understanding +Implements Mark Minervini's SEPA (Specific Entry Point Analysis) criteria: stage 2 +uptrend, price above 50/150/200 SMA in the right order, 52-week high proximity, +RS line at new highs. Historically one of the highest-conviction scanner setups. +Works best in bull market conditions; underperforms in choppy/bear markets. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a market condition filter (S&P 500 above 200 SMA) improve hit rate? diff --git a/docs/iterations/scanners/ml_signal.md b/docs/iterations/scanners/ml_signal.md new file mode 100644 index 00000000..c824eea8 --- /dev/null +++ b/docs/iterations/scanners/ml_signal.md @@ -0,0 +1,14 @@ +# ML Signal Scanner + +## Current Understanding +Uses a trained ML model to predict short-term price movement probability. Current +threshold of 35% win probability is worse than a coin flip — the model needs +retraining or the threshold needs raising to 55%+ to be useful. Signal quality +depends heavily on feature freshness; stale features degrade performance. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does raising the threshold to 55%+ improve precision at the cost of recall? +- [ ] Would retraining on the last 90 days of recommendations improve accuracy? diff --git a/docs/iterations/scanners/options_flow.md b/docs/iterations/scanners/options_flow.md new file mode 100644 index 00000000..eb06170e --- /dev/null +++ b/docs/iterations/scanners/options_flow.md @@ -0,0 +1,15 @@ +# Options Flow Scanner + +## Current Understanding +Scans for unusual options volume relative to open interest using Tradier API. +Call/put volume ratio below 0.1 is a reliable bullish signal when combined with +premium >$25K. The premium filter is configured but must be explicitly applied. +Scanning only the nearest expiration misses institutional positioning in 30+ DTE +contracts — scanning up to 3 expirations improves signal quality. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does scanning 3 expirations vs 1 meaningfully change hit rate? +- [ ] Is moneyness (ITM vs OTM) a useful signal filter? diff --git a/docs/iterations/scanners/reddit_dd.md b/docs/iterations/scanners/reddit_dd.md new file mode 100644 index 00000000..e3164ad0 --- /dev/null +++ b/docs/iterations/scanners/reddit_dd.md @@ -0,0 +1,14 @@ +# Reddit DD Scanner + +## Current Understanding +Scans r/investing, r/stocks, r/wallstreetbets for DD posts. LLM quality score is +computed but not used for filtering — using it (80+ = HIGH, 60-79 = MEDIUM, <60 = skip) +would reduce noise. Subreddit weighting matters: r/investing posts are more reliable +than r/pennystocks. Post title and LLM score should appear in candidate context. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does filtering by LLM quality score >60 meaningfully reduce false positives? +- [ ] Does subreddit weighting change hit rates? diff --git a/docs/iterations/scanners/reddit_trending.md b/docs/iterations/scanners/reddit_trending.md new file mode 100644 index 00000000..fdb48f8d --- /dev/null +++ b/docs/iterations/scanners/reddit_trending.md @@ -0,0 +1,12 @@ +# Reddit Trending Scanner + +## Current Understanding +Tracks mention velocity across subreddits. 50+ mentions in 6 hours = HIGH priority. +20-49 = MEDIUM. Mention count should appear in context ("47 mentions in 6hrs"). +Signal is early-indicator oriented — catches momentum before price moves. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does mention velocity (rate of increase) outperform raw mention count? diff --git a/docs/iterations/scanners/sector_rotation.md b/docs/iterations/scanners/sector_rotation.md new file mode 100644 index 00000000..47608168 --- /dev/null +++ b/docs/iterations/scanners/sector_rotation.md @@ -0,0 +1,13 @@ +# Sector Rotation Scanner + +## Current Understanding +Detects money flowing between sectors using relative strength analysis. Most useful +as a macro filter rather than a primary signal — knowing which sectors are in favor +improves conviction in scanner candidates from those sectors. Standalone sector +rotation signals are too broad for individual stock selection. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Can sector rotation data be used as a multiplier on other scanner scores? diff --git a/docs/iterations/scanners/semantic_news.md b/docs/iterations/scanners/semantic_news.md new file mode 100644 index 00000000..07fdd295 --- /dev/null +++ b/docs/iterations/scanners/semantic_news.md @@ -0,0 +1,14 @@ +# Semantic News Scanner + +## Current Understanding +Currently regex-based extraction, not semantic. Headline text is not included in +candidate context — the context just says "Mentioned in recent market news" which +is not informative. Catalyst classification from headline keywords (upgrade/FDA/ +acquisition/earnings) would improve LLM scoring quality significantly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Would embedding-based semantic matching outperform keyword regex? +- [ ] Does catalyst classification (FDA vs earnings vs acquisition) affect hit rate? diff --git a/docs/iterations/scanners/technical_breakout.md b/docs/iterations/scanners/technical_breakout.md new file mode 100644 index 00000000..138678b0 --- /dev/null +++ b/docs/iterations/scanners/technical_breakout.md @@ -0,0 +1,13 @@ +# Technical Breakout Scanner + +## Current Understanding +Detects price breakouts above key resistance levels on above-average volume. +Minervini-style setups (stage 2 uptrend, tight base, volume-confirmed breakout) +tend to have the highest follow-through rate. False breakouts are common without +volume confirmation (>1.5x average on breakout day). + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring volume confirmation on the breakout day reduce false positives? diff --git a/docs/iterations/scanners/volume_accumulation.md b/docs/iterations/scanners/volume_accumulation.md new file mode 100644 index 00000000..76b99e3d --- /dev/null +++ b/docs/iterations/scanners/volume_accumulation.md @@ -0,0 +1,14 @@ +# Volume Accumulation Scanner + +## Current Understanding +Detects stocks with volume >2x average. Key weakness: cannot distinguish buying from +selling — high volume on a down day is distribution, not accumulation. Multi-day mode +(3 of last 5 days >1.5x) is more reliable than single-day spikes. Price-change filter +(<3% absolute move) isolates quiet accumulation from momentum chasing. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a price-direction filter (volume + flat/up price) improve hit rate? +- [ ] Is 3-of-5-day accumulation a stronger signal than single-day 2x volume? From 47bcbdc70b91f5e22059052e874df33d13c26981 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 08:10:26 -0700 Subject: [PATCH 07/15] fix(iteration-system): add .gitkeep to track empty research/ directory --- docs/iterations/research/.gitkeep | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 docs/iterations/research/.gitkeep diff --git a/docs/iterations/research/.gitkeep b/docs/iterations/research/.gitkeep new file mode 100644 index 00000000..e69de29b From ca24d78cf1e4fbf777c45824fff4ec48ba2fccf8 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 08:23:50 -0700 Subject: [PATCH 08/15] feat(iteration-system): add /iterate Claude Code command --- .claude/commands/iterate.md | 116 ++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 .claude/commands/iterate.md diff --git a/.claude/commands/iterate.md b/.claude/commands/iterate.md new file mode 100644 index 00000000..29620563 --- /dev/null +++ b/.claude/commands/iterate.md @@ -0,0 +1,116 @@ +# /iterate + +Analyze recent discovery runs and P&L outcomes, update learning files in +`docs/iterations/`, implement concrete code improvements in `tradingagents/`, +then commit everything. In CI (`CI=true`), stop before git operations — the +workflow handles branching and PR creation. + +--- + +## Step 1: Determine What to Analyze + +Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date. + +Then scan `results/discovery/` for all run directories with dates AFTER the +last analyzed date. Each run directory contains `discovery_result.json` and +`tool_execution_logs.json`. Collect all unanalyzed runs. + +Also scan `data/recommendations/` for JSON files dated 5 or more days ago. +Load each file and extract all recommendations. These are mature enough to +have played out — analyze them regardless of `status` field value. +For P&L analysis you need: `ticker`, `strategy_match`, `final_score`, +`confidence`, `discovery_date`, `entry_price`, `status`. + +Set your analysis mode: +- If unanalyzed runs exist → include **fast-loop** (output quality analysis) +- If mature recommendations exist → include **P&L loop** (outcome analysis) +- If neither → print "No new data to analyze since last run." and exit. + +## Step 2: Load Domain Context + +For each scanner that appears in the unanalyzed runs (check `strategy_match` +field in discovery results), read the corresponding file from +`docs/iterations/scanners/.md`. + +Also read `docs/iterations/pipeline/scoring.md`. +Also read `docs/iterations/LEARNINGS.md` for full index awareness. + +## Step 3: Analyze + +### Fast-Loop Analysis (output quality) +For each unanalyzed run's `discovery_result.json`: +- **Signal specificity**: Does each candidate have a concrete, specific thesis + or a generic one? Flag candidates with vague context. +- **Scanner noise rate**: How many candidates per scanner? Scanners producing + 10+ candidates with low scores (<65) are noisy. +- **Confidence calibration**: Is confidence (1-10) consistent with score (0-100)? + A score of 85 with confidence 5 suggests miscalibration. +- **Duplicate candidates**: Same ticker appearing from 2+ scanners — note as + confluence (positive) or redundancy (negative, if identical thesis). + +### P&L Loop Analysis (outcome analysis) +For each mature recommendation: +- Group by `strategy_match` (scanner). Compute per-scanner: how many + recommendations were made, what was the average final_score, and whether + `status` fields suggest positive or negative outcomes. +- Flag scanners where final_score > 80 but outcomes appear poor — overconfident. +- Flag scanners where final_score < 65 but outcomes appear good — undervalued. +- Note patterns: do high-confidence (9-10) picks outperform low-confidence (6-7)? + +## Step 4: Write Learnings + +For each scanner that appeared in the analysis, update its domain file in +`docs/iterations/scanners/.md`: + +1. **Append to Evidence Log**: Add a dated entry with your specific observations. + Use this format: + ``` + ### YYYY-MM-DD — + - What was observed + - What it implies + - Confidence: low / medium / high + ``` +2. **Update Current Understanding**: If your new evidence is medium or high + confidence AND contradicts or meaningfully extends the current understanding, + rewrite that section. Otherwise leave it unchanged. +3. **Update Pending Hypotheses**: Check off any hypotheses that are now answered. + Add new ones that your analysis surfaced. + +Update `docs/iterations/LEARNINGS.md`: +- Set **Last analyzed run** to today's date +- Update the one-line summary and Last Updated date for each scanner you touched + +## Step 5: Implement Code Changes + +Based on your learnings, identify concrete improvements. For each improvement: + +**Translate one learning → one code change.** Examples: +- "ML signal threshold is worse than a coin flip" → raise threshold in + `tradingagents/dataflows/discovery/scanners/ml_signal.py` +- "Options flow premium filter is configured but not applied" → add the check +- "Reddit DD LLM score computed but unused" → use it for priority assignment + +For each change: +1. Read the relevant scanner file to understand current implementation +2. Make the minimal change that encodes the learning +3. Do not refactor surrounding code — change only what the learning motivates + +Implement all changes before committing. + +## Step 6: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/ tradingagents/ +git commit -m "learn(iterate): $(date +%Y-%m-%d) — " +``` + +Then check for an existing open PR on branch `iterate/current`: +```bash +EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: push to that branch and update the PR description with your findings appended. +If none exists: create branch `iterate/current`, push, open PR against `main`. From 7398da4fc5c30e398d7588ad90e49936da09eb7f Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 09:28:18 -0700 Subject: [PATCH 09/15] fix(iterate-command): cold-start fallback, cleaner commit message, complete PR branch creation --- .claude/commands/iterate.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/.claude/commands/iterate.md b/.claude/commands/iterate.md index 29620563..f8c2ccce 100644 --- a/.claude/commands/iterate.md +++ b/.claude/commands/iterate.md @@ -10,10 +10,10 @@ workflow handles branching and PR creation. ## Step 1: Determine What to Analyze Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date. +If the file does not exist (first run), treat the Last analyzed run date as `1970-01-01` and proceed. Then scan `results/discovery/` for all run directories with dates AFTER the -last analyzed date. Each run directory contains `discovery_result.json` and -`tool_execution_logs.json`. Collect all unanalyzed runs. +last analyzed date. Each run directory contains `discovery_result.json`. Collect all unanalyzed runs. Also scan `data/recommendations/` for JSON files dated 5 or more days ago. Load each file and extract all recommendations. These are mature enough to @@ -104,13 +104,23 @@ If the environment variable `CI` is set, stop here. The workflow handles git. Otherwise: ```bash git add docs/iterations/ tradingagents/ -git commit -m "learn(iterate): $(date +%Y-%m-%d) — " ``` +Run `git commit` with a message in the format: `learn(iterate): YYYY-MM-DD — ` + Then check for an existing open PR on branch `iterate/current`: ```bash EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty') ``` If one exists: push to that branch and update the PR description with your findings appended. -If none exists: create branch `iterate/current`, push, open PR against `main`. +If none exists: create branch `iterate/current`, push, open PR against `main`: +```bash +git checkout -b iterate/current +git push -u origin iterate/current +gh pr create \ + --title "learn(iterate): automated improvements — $(date +%Y-%m-%d)" \ + --body "$(cat docs/iterations/LEARNINGS.md)" \ + --label "automated,iteration" \ + --base main +``` From ae46bd4c89609194604023dde57ea81d4258c200 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 10:08:53 -0700 Subject: [PATCH 10/15] feat(iteration-system): add /research-strategy Claude Code command --- .claude/commands/research-strategy.md | 176 ++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 .claude/commands/research-strategy.md diff --git a/.claude/commands/research-strategy.md b/.claude/commands/research-strategy.md new file mode 100644 index 00000000..19988f74 --- /dev/null +++ b/.claude/commands/research-strategy.md @@ -0,0 +1,176 @@ +# /research-strategy + +Research new trading strategies or scanner improvements, evaluate fit against +the existing pipeline, write findings to `docs/iterations/research/`, and +implement the top-ranked finding as a new scanner if it qualifies. + +Usage: +- `/research-strategy` — autonomous mode: Claude picks research topics +- `/research-strategy "topic"` — directed mode: research a specific strategy + +In CI (`CI=true`), stop before git operations — the workflow handles them. + +--- + +## Step 1: Set Research Agenda + +**If a topic argument was provided** (`$ARGUMENTS` is not empty): +- Research topic = `$ARGUMENTS` +- Skip to Step 2. + +**Autonomous mode** (no argument): +- Read `docs/iterations/LEARNINGS.md` and all scanner domain files in + `docs/iterations/scanners/` +- Identify the 3-5 highest-leverage research opportunities: + - Scanners with low-confidence current understanding + - Pending hypotheses marked with `- [ ]` + - Gaps: signal types with no current scanner (e.g. dark pool flow, + short interest changes, institutional 13F filings) +- Rank by potential impact. Pick the top topic to research this run. +- Print your agenda: "Researching: — Reason: " + +## Step 2: Search + +Search the following sources for the research topic. For each source, look for: +signal definition, entry/exit criteria, known statistical edge, known failure +modes, data requirements. + +**Sources to search:** +- Reddit: r/algotrading, r/quant, r/investing (site:reddit.com) +- QuantifiedStrategies (site:quantifiedstrategies.com) +- Alpha Architect (site:alphaarchitect.com) +- CSS Analytics (site:cssanalytics.wordpress.com) +- Hacker News: search hn.algolia.com for the topic +- GitHub: search for "quant scanner " and "trading strategy " +- SSRN: search quantitative finance papers on the topic +- arXiv q-fin section + +Use WebSearch and WebFetch to retrieve actual content. Read at least 3-5 +distinct sources before forming a conclusion. + +## Step 3: Cross-Reference Existing Knowledge + +Check `docs/iterations/scanners/` and `docs/iterations/research/` for any +prior work on this topic. Flag explicitly if this overlaps with: +- An existing scanner (name it and the file) +- A previously researched and discarded approach (cite the research file) +- A pending hypothesis in an existing scanner file (cite it) + +## Step 4: Evaluate Fit + +Score the finding on four dimensions (each: ✅ pass / ⚠️ partial / ❌ fail): + +1. **Data availability**: Is the required data source already integrated in + `tradingagents/dataflows/`? Check for existing API clients there. +2. **Implementation complexity**: trivial (<2 hours) / moderate (2-8 hours) / + large (>8 hours) +3. **Signal uniqueness**: Low overlap with existing scanners = good. + High overlap = flag as redundant. +4. **Evidence quality**: backtested with statistics / qualitative analysis / + anecdotal only + +**Auto-implement threshold** (all must pass for autonomous CI implementation): +- Data availability: ✅ (data source already integrated) +- Complexity: trivial or moderate +- Uniqueness: low overlap +- Evidence: qualitative or better + +## Step 5: Write Research File + +Save findings to `docs/iterations/research/` using filename format: +`YYYY-MM-DD-.md` where topic-slug is the topic lowercased with +spaces replaced by hyphens. + +Use this template: + +``` +# Research: + +**Date:** YYYY-MM-DD +**Mode:** directed | autonomous + +## Summary +<2-3 sentences on what was found> + +## Sources Reviewed +- : +- : +... + +## Fit Evaluation +| Dimension | Score | Notes | +|-----------|-------|-------| +| Data availability | ✅/⚠️/❌ | ... | +| Complexity | trivial/moderate/large | ... | +| Signal uniqueness | low/medium/high overlap | ... | +| Evidence quality | backtested/qualitative/anecdotal | ... | + +## Recommendation +Implement / Skip / Needs more data — + +## Proposed Scanner Spec (if recommending implementation) +- **Scanner name:** `` +- **Data source:** `tradingagents/dataflows/.py` +- **Signal logic:** +- **Priority rules:** CRITICAL if X, HIGH if Y, MEDIUM otherwise +- **Context format:** "" +``` + +Add an entry to `docs/iterations/LEARNINGS.md` under a `## Research` section +(create the section if it doesn't exist): +``` +| research/ | research/.md | YYYY-MM-DD | | +``` + +## Step 6: Implement (if threshold met) + +If the finding meets the auto-implement threshold from Step 4: + +1. Read `tradingagents/dataflows/discovery/scanner_registry.py` to understand + the `@SCANNER_REGISTRY.register()` registration pattern. +2. Read an existing simple scanner for the code pattern: + `tradingagents/dataflows/discovery/scanners/earnings_calendar.py` +3. Create `tradingagents/dataflows/discovery/scanners/.py` following + the same structure: + - Class decorated with `@SCANNER_REGISTRY.register()` + - `name` and `pipeline` class attributes + - `scan(self, state)` method returning `List[Dict]` + - Each dict must have keys: `ticker`, `source`, `context`, `priority` + - Priority values: `"CRITICAL"`, `"HIGH"`, `"MEDIUM"`, `"LOW"` +4. Check `tradingagents/dataflows/discovery/scanners/__init__.py` — if it + imports scanners explicitly, add an import for the new one. + +If threshold is NOT met: write the research file only. Add this note at the +top of the research file: +``` +> **Auto-implementation skipped:** +``` + +## Step 7: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md +``` + +Run `git commit` with a message in the format: +`research(): YYYY-MM-DD — ` + +Then check for an existing open PR on branch `research/current`: +```bash +EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: push to that branch and update PR description with new findings. +If none exists: +```bash +git checkout -b research/current +git push -u origin research/current +gh pr create \ + --title "research: new strategy findings — $(date +%Y-%m-%d)" \ + --body "$(cat docs/iterations/LEARNINGS.md | head -30)" \ + --label "automated,research" \ + --base main +``` From b554108eb7c5f39d5ef278ac64f57c2b4c08973a Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 10:11:16 -0700 Subject: [PATCH 11/15] fix(research-command): add bash commands for existing PR update path --- .claude/commands/research-strategy.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.claude/commands/research-strategy.md b/.claude/commands/research-strategy.md index 19988f74..5563f2f0 100644 --- a/.claude/commands/research-strategy.md +++ b/.claude/commands/research-strategy.md @@ -163,7 +163,13 @@ Then check for an existing open PR on branch `research/current`: EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty') ``` -If one exists: push to that branch and update PR description with new findings. +If one exists: +```bash +git push origin HEAD:research/current +gh pr edit "$EXISTING" \ + --body "$(cat docs/iterations/LEARNINGS.md | head -30)" +``` + If none exists: ```bash git checkout -b research/current From 527a56d922c9e433f03c1187718683dc5f955fa3 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 10:47:17 -0700 Subject: [PATCH 12/15] feat(iteration-system): add daily iterate GitHub Actions workflow --- .github/workflows/iterate.yml | 116 ++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 .github/workflows/iterate.yml diff --git a/.github/workflows/iterate.yml b/.github/workflows/iterate.yml new file mode 100644 index 00000000..2939132e --- /dev/null +++ b/.github/workflows/iterate.yml @@ -0,0 +1,116 @@ +name: Daily Iterate + +on: + schedule: + # 6:00 AM UTC daily — analyzes previous day's discovery run + - cron: "0 6 * * *" + workflow_dispatch: + inputs: + force: + description: "Force iterate even if no new runs detected" + required: false + default: "false" + type: choice + options: + - "false" + - "true" + +env: + NODE_VERSION: "20" + +jobs: + iterate: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 30 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /iterate + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "/iterate" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/ tradingagents/ || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /iterate" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + git commit -m "learn(iterate): ${DATE} — automated iteration run" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="iterate/current" + DATE=$(date -u +%Y-%m-%d) + SEPARATOR="---" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ + git push origin HEAD:"$BRANCH" + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Last updated: ${DATE} by automated iterate workflow*" + } > /tmp/pr_body.md + gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Opened: ${DATE} by automated iterate workflow*" + echo "*Merge to apply learnings and reset the iteration cycle.*" + } > /tmp/pr_body.md + gh pr create \ + --title "learn(iterate): automated improvements — ${DATE}" \ + --body-file /tmp/pr_body.md \ + --label "automated,iteration" \ + --base main + echo "Opened new PR" + fi From c79e4c2bcb2b6208fc67bd8f27d598fc16fe948b Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 10:49:45 -0700 Subject: [PATCH 13/15] fix(iterate-workflow): safer push logic with force-with-lease and rebase --- .github/workflows/iterate.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.github/workflows/iterate.yml b/.github/workflows/iterate.yml index 2939132e..2196de67 100644 --- a/.github/workflows/iterate.yml +++ b/.github/workflows/iterate.yml @@ -87,8 +87,9 @@ jobs: --jq '.[0].number // empty') if [ -n "$EXISTING_PR" ]; then - git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ - git push origin HEAD:"$BRANCH" + git fetch origin "$BRANCH" 2>/dev/null || true + git pull --rebase origin main + git push origin HEAD:"$BRANCH" --force-with-lease { cat docs/iterations/LEARNINGS.md echo "" @@ -99,7 +100,7 @@ jobs: echo "Updated existing PR #${EXISTING_PR}" else git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" - git push -u origin "$BRANCH" + git push -u origin "$BRANCH" --force-with-lease { cat docs/iterations/LEARNINGS.md echo "" From fa3df1a4ebbd1d8d8d393cbacabd25a2f4b28863 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 11:14:03 -0700 Subject: [PATCH 14/15] feat(iteration-system): add weekly research-strategy GitHub Actions workflow --- .github/workflows/research-strategy.yml | 125 ++++++++++++++++++++++++ 1 file changed, 125 insertions(+) create mode 100644 .github/workflows/research-strategy.yml diff --git a/.github/workflows/research-strategy.yml b/.github/workflows/research-strategy.yml new file mode 100644 index 00000000..3781bc27 --- /dev/null +++ b/.github/workflows/research-strategy.yml @@ -0,0 +1,125 @@ +name: Weekly Research Strategy + +on: + schedule: + # 7:00 AM UTC every Monday — runs autonomous research on trading strategies + - cron: "0 7 * * 1" + workflow_dispatch: + inputs: + topic: + description: "Research topic (blank = autonomous mode)" + required: false + default: "" + type: string + +env: + NODE_VERSION: "20" + +jobs: + research: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 45 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /research-strategy + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + TOPIC: ${{ inputs.topic }} + run: | + if [ -n "$TOPIC" ]; then + claude -p "/research-strategy \"$TOPIC\"" --dangerously-skip-permissions + else + claude -p "/research-strategy" --dangerously-skip-permissions + fi + + - name: Check for changes + id: changes + run: | + git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /research-strategy" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + env: + TOPIC: ${{ inputs.topic }} + run: | + DATE=$(date -u +%Y-%m-%d) + if [ -n "$TOPIC" ]; then + git commit -m "research(${TOPIC}): ${DATE} — automated research run" + else + git commit -m "research(autonomous): ${DATE} — automated research run" + fi + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="research/current" + DATE=$(date -u +%Y-%m-%d) + SEPARATOR="---" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git fetch origin "$BRANCH" 2>/dev/null || true + git pull --rebase origin main + git push origin HEAD:"$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Last updated: ${DATE} by automated research-strategy workflow*" + } > /tmp/pr_body.md + gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Opened: ${DATE} by automated research-strategy workflow*" + echo "*Merge to apply research findings and reset the research cycle.*" + } > /tmp/pr_body.md + gh pr create \ + --title "research: new strategy findings — ${DATE}" \ + --body-file /tmp/pr_body.md \ + --label "automated,research" \ + --base main + echo "Opened new PR" + fi From aaff21a770cd6314c39766372ce09e529bb42fd3 Mon Sep 17 00:00:00 2001 From: Youssef Aitousarrah Date: Wed, 8 Apr 2026 11:20:14 -0700 Subject: [PATCH 15/15] fix(research-workflow): simplify existing-PR push path, remove no-op rebase --- .github/workflows/research-strategy.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.github/workflows/research-strategy.yml b/.github/workflows/research-strategy.yml index 3781bc27..d8c02071 100644 --- a/.github/workflows/research-strategy.yml +++ b/.github/workflows/research-strategy.yml @@ -96,7 +96,6 @@ jobs: if [ -n "$EXISTING_PR" ]; then git fetch origin "$BRANCH" 2>/dev/null || true - git pull --rebase origin main git push origin HEAD:"$BRANCH" --force-with-lease { cat docs/iterations/LEARNINGS.md