diff --git a/.claude/commands/iterate.md b/.claude/commands/iterate.md new file mode 100644 index 00000000..f8c2ccce --- /dev/null +++ b/.claude/commands/iterate.md @@ -0,0 +1,126 @@ +# /iterate + +Analyze recent discovery runs and P&L outcomes, update learning files in +`docs/iterations/`, implement concrete code improvements in `tradingagents/`, +then commit everything. In CI (`CI=true`), stop before git operations — the +workflow handles branching and PR creation. + +--- + +## Step 1: Determine What to Analyze + +Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date. +If the file does not exist (first run), treat the Last analyzed run date as `1970-01-01` and proceed. + +Then scan `results/discovery/` for all run directories with dates AFTER the +last analyzed date. Each run directory contains `discovery_result.json`. Collect all unanalyzed runs. + +Also scan `data/recommendations/` for JSON files dated 5 or more days ago. +Load each file and extract all recommendations. These are mature enough to +have played out — analyze them regardless of `status` field value. +For P&L analysis you need: `ticker`, `strategy_match`, `final_score`, +`confidence`, `discovery_date`, `entry_price`, `status`. + +Set your analysis mode: +- If unanalyzed runs exist → include **fast-loop** (output quality analysis) +- If mature recommendations exist → include **P&L loop** (outcome analysis) +- If neither → print "No new data to analyze since last run." and exit. + +## Step 2: Load Domain Context + +For each scanner that appears in the unanalyzed runs (check `strategy_match` +field in discovery results), read the corresponding file from +`docs/iterations/scanners/.md`. + +Also read `docs/iterations/pipeline/scoring.md`. +Also read `docs/iterations/LEARNINGS.md` for full index awareness. + +## Step 3: Analyze + +### Fast-Loop Analysis (output quality) +For each unanalyzed run's `discovery_result.json`: +- **Signal specificity**: Does each candidate have a concrete, specific thesis + or a generic one? Flag candidates with vague context. +- **Scanner noise rate**: How many candidates per scanner? Scanners producing + 10+ candidates with low scores (<65) are noisy. +- **Confidence calibration**: Is confidence (1-10) consistent with score (0-100)? + A score of 85 with confidence 5 suggests miscalibration. +- **Duplicate candidates**: Same ticker appearing from 2+ scanners — note as + confluence (positive) or redundancy (negative, if identical thesis). + +### P&L Loop Analysis (outcome analysis) +For each mature recommendation: +- Group by `strategy_match` (scanner). Compute per-scanner: how many + recommendations were made, what was the average final_score, and whether + `status` fields suggest positive or negative outcomes. +- Flag scanners where final_score > 80 but outcomes appear poor — overconfident. +- Flag scanners where final_score < 65 but outcomes appear good — undervalued. +- Note patterns: do high-confidence (9-10) picks outperform low-confidence (6-7)? + +## Step 4: Write Learnings + +For each scanner that appeared in the analysis, update its domain file in +`docs/iterations/scanners/.md`: + +1. **Append to Evidence Log**: Add a dated entry with your specific observations. + Use this format: + ``` + ### YYYY-MM-DD — + - What was observed + - What it implies + - Confidence: low / medium / high + ``` +2. **Update Current Understanding**: If your new evidence is medium or high + confidence AND contradicts or meaningfully extends the current understanding, + rewrite that section. Otherwise leave it unchanged. +3. **Update Pending Hypotheses**: Check off any hypotheses that are now answered. + Add new ones that your analysis surfaced. + +Update `docs/iterations/LEARNINGS.md`: +- Set **Last analyzed run** to today's date +- Update the one-line summary and Last Updated date for each scanner you touched + +## Step 5: Implement Code Changes + +Based on your learnings, identify concrete improvements. For each improvement: + +**Translate one learning → one code change.** Examples: +- "ML signal threshold is worse than a coin flip" → raise threshold in + `tradingagents/dataflows/discovery/scanners/ml_signal.py` +- "Options flow premium filter is configured but not applied" → add the check +- "Reddit DD LLM score computed but unused" → use it for priority assignment + +For each change: +1. Read the relevant scanner file to understand current implementation +2. Make the minimal change that encodes the learning +3. Do not refactor surrounding code — change only what the learning motivates + +Implement all changes before committing. + +## Step 6: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/ tradingagents/ +``` + +Run `git commit` with a message in the format: `learn(iterate): YYYY-MM-DD — ` + +Then check for an existing open PR on branch `iterate/current`: +```bash +EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: push to that branch and update the PR description with your findings appended. +If none exists: create branch `iterate/current`, push, open PR against `main`: +```bash +git checkout -b iterate/current +git push -u origin iterate/current +gh pr create \ + --title "learn(iterate): automated improvements — $(date +%Y-%m-%d)" \ + --body "$(cat docs/iterations/LEARNINGS.md)" \ + --label "automated,iteration" \ + --base main +``` diff --git a/.claude/commands/research-strategy.md b/.claude/commands/research-strategy.md new file mode 100644 index 00000000..5563f2f0 --- /dev/null +++ b/.claude/commands/research-strategy.md @@ -0,0 +1,182 @@ +# /research-strategy + +Research new trading strategies or scanner improvements, evaluate fit against +the existing pipeline, write findings to `docs/iterations/research/`, and +implement the top-ranked finding as a new scanner if it qualifies. + +Usage: +- `/research-strategy` — autonomous mode: Claude picks research topics +- `/research-strategy "topic"` — directed mode: research a specific strategy + +In CI (`CI=true`), stop before git operations — the workflow handles them. + +--- + +## Step 1: Set Research Agenda + +**If a topic argument was provided** (`$ARGUMENTS` is not empty): +- Research topic = `$ARGUMENTS` +- Skip to Step 2. + +**Autonomous mode** (no argument): +- Read `docs/iterations/LEARNINGS.md` and all scanner domain files in + `docs/iterations/scanners/` +- Identify the 3-5 highest-leverage research opportunities: + - Scanners with low-confidence current understanding + - Pending hypotheses marked with `- [ ]` + - Gaps: signal types with no current scanner (e.g. dark pool flow, + short interest changes, institutional 13F filings) +- Rank by potential impact. Pick the top topic to research this run. +- Print your agenda: "Researching: — Reason: " + +## Step 2: Search + +Search the following sources for the research topic. For each source, look for: +signal definition, entry/exit criteria, known statistical edge, known failure +modes, data requirements. + +**Sources to search:** +- Reddit: r/algotrading, r/quant, r/investing (site:reddit.com) +- QuantifiedStrategies (site:quantifiedstrategies.com) +- Alpha Architect (site:alphaarchitect.com) +- CSS Analytics (site:cssanalytics.wordpress.com) +- Hacker News: search hn.algolia.com for the topic +- GitHub: search for "quant scanner " and "trading strategy " +- SSRN: search quantitative finance papers on the topic +- arXiv q-fin section + +Use WebSearch and WebFetch to retrieve actual content. Read at least 3-5 +distinct sources before forming a conclusion. + +## Step 3: Cross-Reference Existing Knowledge + +Check `docs/iterations/scanners/` and `docs/iterations/research/` for any +prior work on this topic. Flag explicitly if this overlaps with: +- An existing scanner (name it and the file) +- A previously researched and discarded approach (cite the research file) +- A pending hypothesis in an existing scanner file (cite it) + +## Step 4: Evaluate Fit + +Score the finding on four dimensions (each: ✅ pass / ⚠️ partial / ❌ fail): + +1. **Data availability**: Is the required data source already integrated in + `tradingagents/dataflows/`? Check for existing API clients there. +2. **Implementation complexity**: trivial (<2 hours) / moderate (2-8 hours) / + large (>8 hours) +3. **Signal uniqueness**: Low overlap with existing scanners = good. + High overlap = flag as redundant. +4. **Evidence quality**: backtested with statistics / qualitative analysis / + anecdotal only + +**Auto-implement threshold** (all must pass for autonomous CI implementation): +- Data availability: ✅ (data source already integrated) +- Complexity: trivial or moderate +- Uniqueness: low overlap +- Evidence: qualitative or better + +## Step 5: Write Research File + +Save findings to `docs/iterations/research/` using filename format: +`YYYY-MM-DD-.md` where topic-slug is the topic lowercased with +spaces replaced by hyphens. + +Use this template: + +``` +# Research: + +**Date:** YYYY-MM-DD +**Mode:** directed | autonomous + +## Summary +<2-3 sentences on what was found> + +## Sources Reviewed +- : +- : +... + +## Fit Evaluation +| Dimension | Score | Notes | +|-----------|-------|-------| +| Data availability | ✅/⚠️/❌ | ... | +| Complexity | trivial/moderate/large | ... | +| Signal uniqueness | low/medium/high overlap | ... | +| Evidence quality | backtested/qualitative/anecdotal | ... | + +## Recommendation +Implement / Skip / Needs more data — + +## Proposed Scanner Spec (if recommending implementation) +- **Scanner name:** `` +- **Data source:** `tradingagents/dataflows/.py` +- **Signal logic:** +- **Priority rules:** CRITICAL if X, HIGH if Y, MEDIUM otherwise +- **Context format:** "" +``` + +Add an entry to `docs/iterations/LEARNINGS.md` under a `## Research` section +(create the section if it doesn't exist): +``` +| research/ | research/.md | YYYY-MM-DD | | +``` + +## Step 6: Implement (if threshold met) + +If the finding meets the auto-implement threshold from Step 4: + +1. Read `tradingagents/dataflows/discovery/scanner_registry.py` to understand + the `@SCANNER_REGISTRY.register()` registration pattern. +2. Read an existing simple scanner for the code pattern: + `tradingagents/dataflows/discovery/scanners/earnings_calendar.py` +3. Create `tradingagents/dataflows/discovery/scanners/.py` following + the same structure: + - Class decorated with `@SCANNER_REGISTRY.register()` + - `name` and `pipeline` class attributes + - `scan(self, state)` method returning `List[Dict]` + - Each dict must have keys: `ticker`, `source`, `context`, `priority` + - Priority values: `"CRITICAL"`, `"HIGH"`, `"MEDIUM"`, `"LOW"` +4. Check `tradingagents/dataflows/discovery/scanners/__init__.py` — if it + imports scanners explicitly, add an import for the new one. + +If threshold is NOT met: write the research file only. Add this note at the +top of the research file: +``` +> **Auto-implementation skipped:** +``` + +## Step 7: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md +``` + +Run `git commit` with a message in the format: +`research(): YYYY-MM-DD — ` + +Then check for an existing open PR on branch `research/current`: +```bash +EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: +```bash +git push origin HEAD:research/current +gh pr edit "$EXISTING" \ + --body "$(cat docs/iterations/LEARNINGS.md | head -30)" +``` + +If none exists: +```bash +git checkout -b research/current +git push -u origin research/current +gh pr create \ + --title "research: new strategy findings — $(date +%Y-%m-%d)" \ + --body "$(cat docs/iterations/LEARNINGS.md | head -30)" \ + --label "automated,research" \ + --base main +``` diff --git a/.github/workflows/iterate.yml b/.github/workflows/iterate.yml new file mode 100644 index 00000000..2196de67 --- /dev/null +++ b/.github/workflows/iterate.yml @@ -0,0 +1,117 @@ +name: Daily Iterate + +on: + schedule: + # 6:00 AM UTC daily — analyzes previous day's discovery run + - cron: "0 6 * * *" + workflow_dispatch: + inputs: + force: + description: "Force iterate even if no new runs detected" + required: false + default: "false" + type: choice + options: + - "false" + - "true" + +env: + NODE_VERSION: "20" + +jobs: + iterate: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 30 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /iterate + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "/iterate" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/ tradingagents/ || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /iterate" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + git commit -m "learn(iterate): ${DATE} — automated iteration run" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="iterate/current" + DATE=$(date -u +%Y-%m-%d) + SEPARATOR="---" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git fetch origin "$BRANCH" 2>/dev/null || true + git pull --rebase origin main + git push origin HEAD:"$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Last updated: ${DATE} by automated iterate workflow*" + } > /tmp/pr_body.md + gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Opened: ${DATE} by automated iterate workflow*" + echo "*Merge to apply learnings and reset the iteration cycle.*" + } > /tmp/pr_body.md + gh pr create \ + --title "learn(iterate): automated improvements — ${DATE}" \ + --body-file /tmp/pr_body.md \ + --label "automated,iteration" \ + --base main + echo "Opened new PR" + fi diff --git a/.github/workflows/research-strategy.yml b/.github/workflows/research-strategy.yml new file mode 100644 index 00000000..d8c02071 --- /dev/null +++ b/.github/workflows/research-strategy.yml @@ -0,0 +1,124 @@ +name: Weekly Research Strategy + +on: + schedule: + # 7:00 AM UTC every Monday — runs autonomous research on trading strategies + - cron: "0 7 * * 1" + workflow_dispatch: + inputs: + topic: + description: "Research topic (blank = autonomous mode)" + required: false + default: "" + type: string + +env: + NODE_VERSION: "20" + +jobs: + research: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 45 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /research-strategy + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + TOPIC: ${{ inputs.topic }} + run: | + if [ -n "$TOPIC" ]; then + claude -p "/research-strategy \"$TOPIC\"" --dangerously-skip-permissions + else + claude -p "/research-strategy" --dangerously-skip-permissions + fi + + - name: Check for changes + id: changes + run: | + git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /research-strategy" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + env: + TOPIC: ${{ inputs.topic }} + run: | + DATE=$(date -u +%Y-%m-%d) + if [ -n "$TOPIC" ]; then + git commit -m "research(${TOPIC}): ${DATE} — automated research run" + else + git commit -m "research(autonomous): ${DATE} — automated research run" + fi + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="research/current" + DATE=$(date -u +%Y-%m-%d) + SEPARATOR="---" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git fetch origin "$BRANCH" 2>/dev/null || true + git push origin HEAD:"$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Last updated: ${DATE} by automated research-strategy workflow*" + } > /tmp/pr_body.md + gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" --force-with-lease + { + cat docs/iterations/LEARNINGS.md + echo "" + echo "$SEPARATOR" + echo "*Opened: ${DATE} by automated research-strategy workflow*" + echo "*Merge to apply research findings and reset the research cycle.*" + } > /tmp/pr_body.md + gh pr create \ + --title "research: new strategy findings — ${DATE}" \ + --body-file /tmp/pr_body.md \ + --label "automated,research" \ + --base main + echo "Opened new PR" + fi diff --git a/docs/iterations/LEARNINGS.md b/docs/iterations/LEARNINGS.md new file mode 100644 index 00000000..61c54c30 --- /dev/null +++ b/docs/iterations/LEARNINGS.md @@ -0,0 +1,20 @@ +# Learnings Index + +**Last analyzed run:** _(none yet — will be set by first /iterate run)_ + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | — | No data yet | +| insider_buying | scanners/insider_buying.md | — | No data yet | +| volume_accumulation | scanners/volume_accumulation.md | — | No data yet | +| reddit_dd | scanners/reddit_dd.md | — | No data yet | +| reddit_trending | scanners/reddit_trending.md | — | No data yet | +| semantic_news | scanners/semantic_news.md | — | No data yet | +| market_movers | scanners/market_movers.md | — | No data yet | +| earnings_calendar | scanners/earnings_calendar.md | — | No data yet | +| analyst_upgrades | scanners/analyst_upgrades.md | — | No data yet | +| technical_breakout | scanners/technical_breakout.md | — | No data yet | +| sector_rotation | scanners/sector_rotation.md | — | No data yet | +| ml_signal | scanners/ml_signal.md | — | No data yet | +| minervini | scanners/minervini.md | — | No data yet | +| pipeline/scoring | pipeline/scoring.md | — | No data yet | diff --git a/docs/iterations/pipeline/scoring.md b/docs/iterations/pipeline/scoring.md new file mode 100644 index 00000000..54b00039 --- /dev/null +++ b/docs/iterations/pipeline/scoring.md @@ -0,0 +1,14 @@ +# Pipeline Scoring & Ranking + +## Current Understanding +LLM assigns a final_score (0-100) and confidence (1-10) to each candidate. +Score and confidence are correlated but not identical — a speculative setup +can score 80 with confidence 6. The ranker uses final_score as primary sort key. +No evidence yet on whether confidence or score is a better predictor of outcomes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is confidence a better outcome predictor than final_score? +- [ ] Does score threshold (e.g. only surface candidates >70) improve hit rate? diff --git a/docs/iterations/research/.gitkeep b/docs/iterations/research/.gitkeep new file mode 100644 index 00000000..e69de29b diff --git a/docs/iterations/scanners/analyst_upgrades.md b/docs/iterations/scanners/analyst_upgrades.md new file mode 100644 index 00000000..767dbc70 --- /dev/null +++ b/docs/iterations/scanners/analyst_upgrades.md @@ -0,0 +1,14 @@ +# Analyst Upgrades Scanner + +## Current Understanding +Detects analyst upgrades/price target increases. Most reliable when upgrade comes +from a top-tier firm (Goldman, Morgan Stanley, JPMorgan) and represents a meaningful +target increase (>15%). Short squeeze potential (high short interest) combined with +an upgrade is a historically strong setup. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does analyst tier (BB firm vs boutique) predict upgrade quality? +- [ ] Does short interest >20% combined with an upgrade produce outsized moves? diff --git a/docs/iterations/scanners/earnings_calendar.md b/docs/iterations/scanners/earnings_calendar.md new file mode 100644 index 00000000..1550247b --- /dev/null +++ b/docs/iterations/scanners/earnings_calendar.md @@ -0,0 +1,13 @@ +# Earnings Calendar Scanner + +## Current Understanding +Identifies stocks with earnings announcements in the next N days. Pre-earnings +setups work best when combined with options flow (IV expansion) or insider activity. +Standalone earnings calendar signal is too broad — nearly every stock has earnings +quarterly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring options confirmation alongside earnings improve signal quality? diff --git a/docs/iterations/scanners/insider_buying.md b/docs/iterations/scanners/insider_buying.md new file mode 100644 index 00000000..b77fc60b --- /dev/null +++ b/docs/iterations/scanners/insider_buying.md @@ -0,0 +1,14 @@ +# Insider Buying Scanner + +## Current Understanding +Scrapes SEC Form 4 filings. CEO/CFO purchases >$100K are the most reliable signal. +Cluster detection (2+ insiders buying within 14 days) historically a high-conviction +setup. Transaction details (name, title, value) must be preserved from scraper output +and included in candidate context — dropping them loses signal clarity. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does cluster detection (2+ insiders in 14 days) outperform single-insider signals? +- [ ] Is there a minimum transaction size below which signal quality degrades sharply? diff --git a/docs/iterations/scanners/market_movers.md b/docs/iterations/scanners/market_movers.md new file mode 100644 index 00000000..a4435309 --- /dev/null +++ b/docs/iterations/scanners/market_movers.md @@ -0,0 +1,13 @@ +# Market Movers Scanner + +## Current Understanding +Finds stocks that have already moved significantly. This is a reactive scanner — +it identifies momentum after it starts rather than predicting it. Useful for +continuation plays but not for early-stage entry. Best combined with volume +confirmation to distinguish breakouts from spikes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is a volume confirmation filter (>1.5x average) useful for filtering out noise? diff --git a/docs/iterations/scanners/minervini.md b/docs/iterations/scanners/minervini.md new file mode 100644 index 00000000..e2cc127c --- /dev/null +++ b/docs/iterations/scanners/minervini.md @@ -0,0 +1,13 @@ +# Minervini Scanner + +## Current Understanding +Implements Mark Minervini's SEPA (Specific Entry Point Analysis) criteria: stage 2 +uptrend, price above 50/150/200 SMA in the right order, 52-week high proximity, +RS line at new highs. Historically one of the highest-conviction scanner setups. +Works best in bull market conditions; underperforms in choppy/bear markets. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a market condition filter (S&P 500 above 200 SMA) improve hit rate? diff --git a/docs/iterations/scanners/ml_signal.md b/docs/iterations/scanners/ml_signal.md new file mode 100644 index 00000000..c824eea8 --- /dev/null +++ b/docs/iterations/scanners/ml_signal.md @@ -0,0 +1,14 @@ +# ML Signal Scanner + +## Current Understanding +Uses a trained ML model to predict short-term price movement probability. Current +threshold of 35% win probability is worse than a coin flip — the model needs +retraining or the threshold needs raising to 55%+ to be useful. Signal quality +depends heavily on feature freshness; stale features degrade performance. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does raising the threshold to 55%+ improve precision at the cost of recall? +- [ ] Would retraining on the last 90 days of recommendations improve accuracy? diff --git a/docs/iterations/scanners/options_flow.md b/docs/iterations/scanners/options_flow.md new file mode 100644 index 00000000..eb06170e --- /dev/null +++ b/docs/iterations/scanners/options_flow.md @@ -0,0 +1,15 @@ +# Options Flow Scanner + +## Current Understanding +Scans for unusual options volume relative to open interest using Tradier API. +Call/put volume ratio below 0.1 is a reliable bullish signal when combined with +premium >$25K. The premium filter is configured but must be explicitly applied. +Scanning only the nearest expiration misses institutional positioning in 30+ DTE +contracts — scanning up to 3 expirations improves signal quality. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does scanning 3 expirations vs 1 meaningfully change hit rate? +- [ ] Is moneyness (ITM vs OTM) a useful signal filter? diff --git a/docs/iterations/scanners/reddit_dd.md b/docs/iterations/scanners/reddit_dd.md new file mode 100644 index 00000000..e3164ad0 --- /dev/null +++ b/docs/iterations/scanners/reddit_dd.md @@ -0,0 +1,14 @@ +# Reddit DD Scanner + +## Current Understanding +Scans r/investing, r/stocks, r/wallstreetbets for DD posts. LLM quality score is +computed but not used for filtering — using it (80+ = HIGH, 60-79 = MEDIUM, <60 = skip) +would reduce noise. Subreddit weighting matters: r/investing posts are more reliable +than r/pennystocks. Post title and LLM score should appear in candidate context. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does filtering by LLM quality score >60 meaningfully reduce false positives? +- [ ] Does subreddit weighting change hit rates? diff --git a/docs/iterations/scanners/reddit_trending.md b/docs/iterations/scanners/reddit_trending.md new file mode 100644 index 00000000..fdb48f8d --- /dev/null +++ b/docs/iterations/scanners/reddit_trending.md @@ -0,0 +1,12 @@ +# Reddit Trending Scanner + +## Current Understanding +Tracks mention velocity across subreddits. 50+ mentions in 6 hours = HIGH priority. +20-49 = MEDIUM. Mention count should appear in context ("47 mentions in 6hrs"). +Signal is early-indicator oriented — catches momentum before price moves. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does mention velocity (rate of increase) outperform raw mention count? diff --git a/docs/iterations/scanners/sector_rotation.md b/docs/iterations/scanners/sector_rotation.md new file mode 100644 index 00000000..47608168 --- /dev/null +++ b/docs/iterations/scanners/sector_rotation.md @@ -0,0 +1,13 @@ +# Sector Rotation Scanner + +## Current Understanding +Detects money flowing between sectors using relative strength analysis. Most useful +as a macro filter rather than a primary signal — knowing which sectors are in favor +improves conviction in scanner candidates from those sectors. Standalone sector +rotation signals are too broad for individual stock selection. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Can sector rotation data be used as a multiplier on other scanner scores? diff --git a/docs/iterations/scanners/semantic_news.md b/docs/iterations/scanners/semantic_news.md new file mode 100644 index 00000000..07fdd295 --- /dev/null +++ b/docs/iterations/scanners/semantic_news.md @@ -0,0 +1,14 @@ +# Semantic News Scanner + +## Current Understanding +Currently regex-based extraction, not semantic. Headline text is not included in +candidate context — the context just says "Mentioned in recent market news" which +is not informative. Catalyst classification from headline keywords (upgrade/FDA/ +acquisition/earnings) would improve LLM scoring quality significantly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Would embedding-based semantic matching outperform keyword regex? +- [ ] Does catalyst classification (FDA vs earnings vs acquisition) affect hit rate? diff --git a/docs/iterations/scanners/technical_breakout.md b/docs/iterations/scanners/technical_breakout.md new file mode 100644 index 00000000..138678b0 --- /dev/null +++ b/docs/iterations/scanners/technical_breakout.md @@ -0,0 +1,13 @@ +# Technical Breakout Scanner + +## Current Understanding +Detects price breakouts above key resistance levels on above-average volume. +Minervini-style setups (stage 2 uptrend, tight base, volume-confirmed breakout) +tend to have the highest follow-through rate. False breakouts are common without +volume confirmation (>1.5x average on breakout day). + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring volume confirmation on the breakout day reduce false positives? diff --git a/docs/iterations/scanners/volume_accumulation.md b/docs/iterations/scanners/volume_accumulation.md new file mode 100644 index 00000000..76b99e3d --- /dev/null +++ b/docs/iterations/scanners/volume_accumulation.md @@ -0,0 +1,14 @@ +# Volume Accumulation Scanner + +## Current Understanding +Detects stocks with volume >2x average. Key weakness: cannot distinguish buying from +selling — high volume on a down day is distribution, not accumulation. Multi-day mode +(3 of last 5 days >1.5x) is more reliable than single-day spikes. Price-change filter +(<3% absolute move) isolates quiet accumulation from momentum chasing. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a price-direction filter (volume + flat/up price) improve hit rate? +- [ ] Is 3-of-5-day accumulation a stronger signal than single-day 2x volume? diff --git a/docs/superpowers/plans/2026-04-08-iteration-system.md b/docs/superpowers/plans/2026-04-08-iteration-system.md new file mode 100644 index 00000000..045a96ea --- /dev/null +++ b/docs/superpowers/plans/2026-04-08-iteration-system.md @@ -0,0 +1,1038 @@ +# Iteration System Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a self-improving iteration system with two Claude Code commands (`/iterate`, `/research-strategy`) backed by a versioned knowledge base and two GitHub Actions workflows that run them daily/weekly and open rolling PRs. + +**Architecture:** A `docs/iterations/` knowledge base stores per-scanner learnings in domain files. Two `.claude/commands/` files define the skill behavior. Two GitHub Actions workflows invoke Claude non-interactively, then handle branch/PR management externally so Claude's job is analysis + file writing only. + +**Tech Stack:** Claude Code CLI (`claude -p`), GitHub Actions, `gh` CLI, Python (existing), Markdown knowledge base + +--- + +## File Map + +**Create:** +- `docs/iterations/LEARNINGS.md` — master index tracking last-analyzed run per scanner +- `docs/iterations/scanners/options_flow.md` — options flow scanner learnings +- `docs/iterations/scanners/insider_buying.md` — insider buying scanner learnings +- `docs/iterations/scanners/volume_accumulation.md` — volume accumulation scanner learnings +- `docs/iterations/scanners/reddit_dd.md` — reddit DD scanner learnings +- `docs/iterations/scanners/reddit_trending.md` — reddit trending scanner learnings +- `docs/iterations/scanners/semantic_news.md` — semantic news scanner learnings +- `docs/iterations/scanners/market_movers.md` — market movers scanner learnings +- `docs/iterations/scanners/earnings_calendar.md` — earnings calendar scanner learnings +- `docs/iterations/scanners/analyst_upgrades.md` — analyst upgrades scanner learnings +- `docs/iterations/scanners/technical_breakout.md` — technical breakout scanner learnings +- `docs/iterations/scanners/sector_rotation.md` — sector rotation scanner learnings +- `docs/iterations/scanners/ml_signal.md` — ML signal scanner learnings +- `docs/iterations/scanners/minervini.md` — Minervini scanner learnings +- `docs/iterations/pipeline/scoring.md` — LLM scoring and ranking learnings +- `.claude/commands/iterate.md` — `/iterate` Claude Code command +- `.claude/commands/research-strategy.md` — `/research-strategy` Claude Code command +- `.github/workflows/iterate.yml` — daily cron workflow +- `.github/workflows/research-strategy.yml` — weekly cron workflow + +**Modify:** none — purely additive + +--- + +## Task 1: Create Knowledge Base Folder Structure + +**Files:** +- Create: `docs/iterations/LEARNINGS.md` +- Create: `docs/iterations/scanners/*.md` (13 files) +- Create: `docs/iterations/pipeline/scoring.md` + +- [ ] **Step 1: Create directory structure** + +```bash +mkdir -p docs/iterations/scanners docs/iterations/pipeline docs/iterations/research +``` + +- [ ] **Step 2: Create LEARNINGS.md** + +```bash +cat > docs/iterations/LEARNINGS.md << 'EOF' +# Learnings Index + +**Last analyzed run:** _(none yet — will be set by first /iterate run)_ + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | — | No data yet | +| insider_buying | scanners/insider_buying.md | — | No data yet | +| volume_accumulation | scanners/volume_accumulation.md | — | No data yet | +| reddit_dd | scanners/reddit_dd.md | — | No data yet | +| reddit_trending | scanners/reddit_trending.md | — | No data yet | +| semantic_news | scanners/semantic_news.md | — | No data yet | +| market_movers | scanners/market_movers.md | — | No data yet | +| earnings_calendar | scanners/earnings_calendar.md | — | No data yet | +| analyst_upgrades | scanners/analyst_upgrades.md | — | No data yet | +| technical_breakout | scanners/technical_breakout.md | — | No data yet | +| sector_rotation | scanners/sector_rotation.md | — | No data yet | +| ml_signal | scanners/ml_signal.md | — | No data yet | +| minervini | scanners/minervini.md | — | No data yet | +| pipeline/scoring | pipeline/scoring.md | — | No data yet | +EOF +``` + +- [ ] **Step 3: Create scanner domain files** + +Create each of the 13 scanner files using this template. Fill in the "Current Understanding" section with what is already known from prior work. Run this for each scanner: + +**`docs/iterations/scanners/options_flow.md`:** +```markdown +# Options Flow Scanner + +## Current Understanding +Scans for unusual options volume relative to open interest using Tradier API. +Call/put volume ratio below 0.1 is a reliable bullish signal when combined with +premium >$25K. The premium filter is configured but must be explicitly applied. +Scanning only the nearest expiration misses institutional positioning in 30+ DTE +contracts — scanning up to 3 expirations improves signal quality. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does scanning 3 expirations vs 1 meaningfully change hit rate? +- [ ] Is moneyness (ITM vs OTM) a useful signal filter? +``` + +**`docs/iterations/scanners/insider_buying.md`:** +```markdown +# Insider Buying Scanner + +## Current Understanding +Scrapes SEC Form 4 filings. CEO/CFO purchases >$100K are the most reliable signal. +Cluster detection (2+ insiders buying within 14 days) historically a high-conviction +setup. Transaction details (name, title, value) must be preserved from scraper output +and included in candidate context — dropping them loses signal clarity. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does cluster detection (2+ insiders in 14 days) outperform single-insider signals? +- [ ] Is there a minimum transaction size below which signal quality degrades sharply? +``` + +**`docs/iterations/scanners/volume_accumulation.md`:** +```markdown +# Volume Accumulation Scanner + +## Current Understanding +Detects stocks with volume >2x average. Key weakness: cannot distinguish buying from +selling — high volume on a down day is distribution, not accumulation. Multi-day mode +(3 of last 5 days >1.5x) is more reliable than single-day spikes. Price-change filter +(<3% absolute move) isolates quiet accumulation from momentum chasing. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a price-direction filter (volume + flat/up price) improve hit rate? +- [ ] Is 3-of-5-day accumulation a stronger signal than single-day 2x volume? +``` + +**`docs/iterations/scanners/reddit_dd.md`:** +```markdown +# Reddit DD Scanner + +## Current Understanding +Scans r/investing, r/stocks, r/wallstreetbets for DD posts. LLM quality score is +computed but not used for filtering — using it (80+ = HIGH, 60-79 = MEDIUM, <60 = skip) +would reduce noise. Subreddit weighting matters: r/investing posts are more reliable +than r/pennystocks. Post title and LLM score should appear in candidate context. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does filtering by LLM quality score >60 meaningfully reduce false positives? +- [ ] Does subreddit weighting change hit rates? +``` + +**`docs/iterations/scanners/reddit_trending.md`:** +```markdown +# Reddit Trending Scanner + +## Current Understanding +Tracks mention velocity across subreddits. 50+ mentions in 6 hours = HIGH priority. +20-49 = MEDIUM. Mention count should appear in context ("47 mentions in 6hrs"). +Signal is early-indicator oriented — catches momentum before price moves. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does mention velocity (rate of increase) outperform raw mention count? +``` + +**`docs/iterations/scanners/semantic_news.md`:** +```markdown +# Semantic News Scanner + +## Current Understanding +Currently regex-based extraction, not semantic. Headline text is not included in +candidate context — the context just says "Mentioned in recent market news" which +is not informative. Catalyst classification from headline keywords (upgrade/FDA/ +acquisition/earnings) would improve LLM scoring quality significantly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Would embedding-based semantic matching outperform keyword regex? +- [ ] Does catalyst classification (FDA vs earnings vs acquisition) affect hit rate? +``` + +**`docs/iterations/scanners/market_movers.md`:** +```markdown +# Market Movers Scanner + +## Current Understanding +Finds stocks that have already moved significantly. This is a reactive scanner — +it identifies momentum after it starts rather than predicting it. Useful for +continuation plays but not for early-stage entry. Best combined with volume +confirmation to distinguish breakouts from spikes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is a volume confirmation filter (>1.5x average) useful for filtering out noise? +``` + +**`docs/iterations/scanners/earnings_calendar.md`:** +```markdown +# Earnings Calendar Scanner + +## Current Understanding +Identifies stocks with earnings announcements in the next N days. Pre-earnings +setups work best when combined with options flow (IV expansion) or insider activity. +Standalone earnings calendar signal is too broad — nearly every stock has earnings +quarterly. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring options confirmation alongside earnings improve signal quality? +``` + +**`docs/iterations/scanners/analyst_upgrades.md`:** +```markdown +# Analyst Upgrades Scanner + +## Current Understanding +Detects analyst upgrades/price target increases. Most reliable when upgrade comes +from a top-tier firm (Goldman, Morgan Stanley, JPMorgan) and represents a meaningful +target increase (>15%). Short squeeze potential (high short interest) combined with +an upgrade is a historically strong setup. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does analyst tier (BB firm vs boutique) predict upgrade quality? +- [ ] Does short interest >20% combined with an upgrade produce outsized moves? +``` + +**`docs/iterations/scanners/technical_breakout.md`:** +```markdown +# Technical Breakout Scanner + +## Current Understanding +Detects price breakouts above key resistance levels on above-average volume. +Minervini-style setups (stage 2 uptrend, tight base, volume-confirmed breakout) +tend to have the highest follow-through rate. False breakouts are common without +volume confirmation (>1.5x average on breakout day). + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does requiring volume confirmation on the breakout day reduce false positives? +``` + +**`docs/iterations/scanners/sector_rotation.md`:** +```markdown +# Sector Rotation Scanner + +## Current Understanding +Detects money flowing between sectors using relative strength analysis. Most useful +as a macro filter rather than a primary signal — knowing which sectors are in favor +improves conviction in scanner candidates from those sectors. Standalone sector +rotation signals are too broad for individual stock selection. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Can sector rotation data be used as a multiplier on other scanner scores? +``` + +**`docs/iterations/scanners/ml_signal.md`:** +```markdown +# ML Signal Scanner + +## Current Understanding +Uses a trained ML model to predict short-term price movement probability. Current +threshold of 35% win probability is worse than a coin flip — the model needs +retraining or the threshold needs raising to 55%+ to be useful. Signal quality +depends heavily on feature freshness; stale features degrade performance. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does raising the threshold to 55%+ improve precision at the cost of recall? +- [ ] Would retraining on the last 90 days of recommendations improve accuracy? +``` + +**`docs/iterations/scanners/minervini.md`:** +```markdown +# Minervini Scanner + +## Current Understanding +Implements Mark Minervini's SEPA (Specific Entry Point Analysis) criteria: stage 2 +uptrend, price above 50/150/200 SMA in the right order, 52-week high proximity, +RS line at new highs. Historically one of the highest-conviction scanner setups. +Works best in bull market conditions; underperforms in choppy/bear markets. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Does adding a market condition filter (S&P 500 above 200 SMA) improve hit rate? +``` + +- [ ] **Step 4: Create pipeline/scoring.md** + +```bash +cat > docs/iterations/pipeline/scoring.md << 'EOF' +# Pipeline Scoring & Ranking + +## Current Understanding +LLM assigns a final_score (0-100) and confidence (1-10) to each candidate. +Score and confidence are correlated but not identical — a speculative setup +can score 80 with confidence 6. The ranker uses final_score as primary sort key. +No evidence yet on whether confidence or score is a better predictor of outcomes. + +## Evidence Log +_(populated by /iterate runs)_ + +## Pending Hypotheses +- [ ] Is confidence a better outcome predictor than final_score? +- [ ] Does score threshold (e.g. only surface candidates >70) improve hit rate? +EOF +``` + +- [ ] **Step 5: Verify structure** + +```bash +find docs/iterations -type f | sort +``` + +Expected output: 16 files (LEARNINGS.md + 13 scanner files + scoring.md + research dir exists) + +- [ ] **Step 6: Commit** + +```bash +git add docs/iterations/ +git commit -m "feat(iteration-system): add knowledge base folder structure with seeded scanner files" +``` + +--- + +## Task 2: Write the `/iterate` Command + +**Files:** +- Create: `.claude/commands/iterate.md` + +- [ ] **Step 1: Create `.claude/commands/` directory** + +```bash +mkdir -p .claude/commands +``` + +- [ ] **Step 2: Write iterate.md** + +Create `.claude/commands/iterate.md` with this exact content: + +```markdown +# /iterate + +Analyze recent discovery runs and P&L outcomes, update learning files in +`docs/iterations/`, implement concrete code improvements in `tradingagents/`, +then commit everything. In CI (`CI=true`), stop before git operations — the +workflow handles branching and PR creation. + +--- + +## Step 1: Determine What to Analyze + +Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date. + +Then scan `results/discovery/` for all run directories with dates AFTER the +last analyzed date. Each run directory contains `discovery_result.json` and +`tool_execution_logs.json`. Collect all unanalyzed runs. + +Also scan `data/recommendations/` for JSON files dated 5 or more days ago. +Load each file and extract recommendations where `status != "open"` OR where +`discovery_date` is 5+ days in the past (these have had time to play out). +For P&L analysis you need: `ticker`, `strategy_match`, `final_score`, +`confidence`, `discovery_date`, `entry_price`, `status`. + +Set your analysis mode: +- If unanalyzed runs exist → include **fast-loop** (output quality analysis) +- If mature recommendations exist → include **P&L loop** (outcome analysis) +- If neither → print "No new data to analyze since last run." and exit. + +## Step 2: Load Domain Context + +For each scanner that appears in the unanalyzed runs (check `strategy_match` +field in discovery results), read the corresponding file from +`docs/iterations/scanners/.md`. + +Also read `docs/iterations/pipeline/scoring.md`. +Also read `docs/iterations/LEARNINGS.md` for full index awareness. + +## Step 3: Analyze + +### Fast-Loop Analysis (output quality) +For each unanalyzed run's `discovery_result.json`: +- **Signal specificity**: Does each candidate have a concrete, specific thesis + or a generic one? Flag candidates with vague context. +- **Scanner noise rate**: How many candidates per scanner? Scanners producing + 10+ candidates with low scores (<65) are noisy. +- **Confidence calibration**: Is confidence (1-10) consistent with score (0-100)? + A score of 85 with confidence 5 suggests miscalibration. +- **Duplicate candidates**: Same ticker appearing from 2+ scanners — note as + confluence (positive) or redundancy (negative, if identical thesis). + +### P&L Loop Analysis (outcome analysis) +For each mature recommendation: +- Group by `strategy_match` (scanner). Compute per-scanner: hit rate (status + reflects outcome — check what non-"open" statuses mean in the data). +- Flag scanners where final_score > 80 but outcomes are poor — overconfident. +- Flag scanners where final_score < 65 but outcomes are good — undervalued. +- Note patterns: do high-confidence (9-10) picks outperform low-confidence (6-7)? + +## Step 4: Write Learnings + +For each scanner that appeared in the analysis, update its domain file in +`docs/iterations/scanners/.md`: + +1. **Append to Evidence Log**: Add a dated entry with your specific observations. + Include: what was observed, what it implies, confidence (low/medium/high). +2. **Update Current Understanding**: If your new evidence is medium or high + confidence AND contradicts or meaningfully extends the current understanding, + rewrite that section. Otherwise leave it unchanged. +3. **Update Pending Hypotheses**: Check off any hypotheses that are now answered. + Add new ones that your analysis surfaced. + +Update `docs/iterations/LEARNINGS.md`: +- Set **Last analyzed run** to today's date +- Update the one-line summary and Last Updated date for each scanner you touched + +## Step 5: Implement Code Changes + +Based on your learnings, identify concrete improvements. For each improvement: + +**Translate one learning → one code change.** Examples: +- "ML signal threshold is worse than a coin flip" → raise threshold in + `tradingagents/dataflows/discovery/scanners/ml_signal.py` +- "Options flow premium filter is configured but not applied" → add the check +- "Reddit DD LLM score computed but unused" → use it for priority assignment + +For each change: +1. Read the relevant scanner file to understand current implementation +2. Make the minimal change that encodes the learning +3. Do not refactor surrounding code — change only what the learning motivates + +Implement all changes before committing. + +## Step 6: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/ tradingagents/ +git commit -m "learn(iterate): $(date +%Y-%m-%d) — " +``` + +Then check for an existing open PR on branch `iterate/current`: +```bash +EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty') +``` + +If one exists: push to that branch and update its description with your findings. +If none exists: create branch `iterate/current`, push, open PR against `main`. +``` +``` + +- [ ] **Step 3: Verify the command is discoverable** + +```bash +ls .claude/commands/ +``` + +Expected: `iterate.md` + +- [ ] **Step 4: Smoke-test the command manually** + +In a Claude Code session in this project, type `/iterate` and verify Claude reads +the command and starts executing Step 1 (reads LEARNINGS.md, scans results/). +You don't need to let it complete — just confirm it picks up the command file. + +- [ ] **Step 5: Commit** + +```bash +git add .claude/commands/iterate.md +git commit -m "feat(iteration-system): add /iterate Claude Code command" +``` + +--- + +## Task 3: Write the `/research-strategy` Command + +**Files:** +- Create: `.claude/commands/research-strategy.md` + +- [ ] **Step 1: Write research-strategy.md** + +Create `.claude/commands/research-strategy.md` with this exact content: + +```markdown +# /research-strategy + +Research new trading strategies or scanner improvements, evaluate fit against +the existing pipeline, write findings to `docs/iterations/research/`, and +implement the top-ranked finding as a new scanner if it qualifies. + +Usage: +- `/research-strategy` — autonomous mode: Claude picks research topics +- `/research-strategy "topic"` — directed mode: research a specific strategy + +In CI (`CI=true`), stop before git operations — the workflow handles them. + +--- + +## Step 1: Set Research Agenda + +**If a topic argument was provided** (`$ARGUMENTS` is not empty): +- Research topic = `$ARGUMENTS` +- Skip to Step 2. + +**Autonomous mode** (no argument): +- Read `docs/iterations/LEARNINGS.md` and all scanner domain files in + `docs/iterations/scanners/` +- Identify the 3-5 highest-leverage research opportunities: + - Scanners with low-confidence current understanding + - Pending hypotheses marked with `- [ ]` + - Gaps: signal types with no current scanner (e.g. dark pool flow, + short interest changes, institutional 13F filings) +- Rank by potential impact. Pick the top topic to research this run. +- Print your agenda: "Researching: — Reason: " + +## Step 2: Search + +Search the following sources for the research topic. For each source, look for: +signal definition, entry/exit criteria, known statistical edge, known failure +modes, data requirements. + +**Sources to search:** +- Reddit: r/algotrading, r/quant, r/investing (site:reddit.com) +- QuantifiedStrategies (site:quantifiedstrategies.com) +- Alpha Architect (site:alphaarchitect.com) +- CSS Analytics (site:cssanalytics.wordpress.com) +- Hacker News (hn.algolia.com query) +- GitHub: search "quant scanner " and "trading strategy " +- SSRN: search quantitative finance papers on the topic +- arXiv q-fin section + +Use WebSearch and WebFetch to retrieve actual content. Read at least 3-5 +distinct sources before forming a conclusion. + +## Step 3: Cross-Reference Existing Knowledge + +Check `docs/iterations/scanners/` and `docs/iterations/research/` for any +prior work on this topic. Flag explicitly if this overlaps with: +- An existing scanner (name it) +- A previously researched and discarded approach (cite the research file) +- A pending hypothesis in an existing scanner file + +## Step 4: Evaluate Fit + +Score the finding on four dimensions (each: ✅ pass / ⚠️ partial / ❌ fail): + +1. **Data availability**: Is the required data source already integrated in + `tradingagents/dataflows/`? Check for existing API clients. +2. **Implementation complexity**: trivial (<2 hours) / moderate (2-8 hours) / + large (>8 hours) +3. **Signal uniqueness**: Low overlap with existing scanners = good. + High overlap = flag as redundant. +4. **Evidence quality**: backtested with statistics / qualitative analysis / + anecdotal only + +**Auto-implement threshold** (all must pass for autonomous CI implementation): +- Data availability: ✅ (data source already integrated) +- Complexity: trivial or moderate +- Uniqueness: low overlap +- Evidence: qualitative or better + +## Step 5: Write Research File + +Save findings to `docs/iterations/research/$(date +%Y-%m-%d)-.md`: + +```markdown +# Research: + +**Date:** YYYY-MM-DD +**Mode:** directed | autonomous + +## Summary +<2-3 sentences on what was found> + +## Sources Reviewed +- +- +... + +## Fit Evaluation +| Dimension | Score | Notes | +|-----------|-------|-------| +| Data availability | ✅/⚠️/❌ | ... | +| Complexity | trivial/moderate/large | ... | +| Signal uniqueness | low/medium/high overlap | ... | +| Evidence quality | backtested/qualitative/anecdotal | ... | + +## Recommendation +Implement / Skip / Needs more data + +## Proposed Scanner Spec (if recommending implementation) +- **Scanner name:** `` +- **Data source:** `tradingagents/dataflows/.py` +- **Signal logic:** +- **Priority rules:** CRITICAL if X, HIGH if Y, MEDIUM otherwise +- **Context format:** "" +``` + +Add an entry to `docs/iterations/LEARNINGS.md` under a Research section. + +## Step 6: Implement (if threshold met) + +If the finding meets the auto-implement threshold: + +1. Read the scanner registry to understand the registration pattern: + `tradingagents/dataflows/discovery/scanner_registry.py` +2. Read an existing simple scanner for the code pattern, e.g.: + `tradingagents/dataflows/discovery/scanners/earnings_calendar.py` +3. Create `tradingagents/dataflows/discovery/scanners/.py` following + the same structure: class with `@SCANNER_REGISTRY.register()` decorator, + `name`, `pipeline`, `scan()` method returning list of candidate dicts with + keys: `ticker`, `source`, `context`, `priority`. +4. Register the scanner in `tradingagents/dataflows/discovery/scanners/__init__.py` + if needed (check if auto-discovery is in place). + +If threshold is NOT met: write the research file only. Add a `needs-review` note +at the top explaining why auto-implementation was skipped. + +## Step 7: Commit (skip if CI=true) + +If the environment variable `CI` is set, stop here. The workflow handles git. + +Otherwise: +```bash +git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md +git commit -m "research(): " +``` + +Check for existing open PR on `research/current`: +```bash +EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty') +``` + +If exists: push to branch, update PR description. +If not: create branch `research/current`, push, open PR. +``` +``` + +- [ ] **Step 2: Smoke-test the command manually** + +In a Claude Code session, type `/research-strategy "momentum breakout"` and verify +Claude reads the command, prints its agenda, and starts searching. Let it run at +least through Step 2 to confirm web search is working. + +- [ ] **Step 3: Commit** + +```bash +git add .claude/commands/research-strategy.md +git commit -m "feat(iteration-system): add /research-strategy Claude Code command" +``` + +--- + +## Task 4: Write the `iterate.yml` GitHub Actions Workflow + +**Files:** +- Create: `.github/workflows/iterate.yml` + +- [ ] **Step 1: Write iterate.yml** + +Create `.github/workflows/iterate.yml` with this exact content: + +```yaml +name: Daily Iterate + +on: + schedule: + # 6:00 AM UTC daily — analyzes previous day's discovery run + - cron: "0 6 * * *" + workflow_dispatch: + inputs: + force: + description: "Force iterate even if no new runs detected" + required: false + default: "false" + type: choice + options: + - "false" + - "true" + +env: + PYTHON_VERSION: "3.10" + NODE_VERSION: "20" + +jobs: + iterate: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 30 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Run /iterate + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "/iterate" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/ tradingagents/ || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /iterate" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + git commit -m "learn(iterate): ${DATE} — automated iteration run" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="iterate/current" + DATE=$(date -u +%Y-%m-%d) + + # Check for existing open PR on this branch + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + # Push onto existing branch and update PR + git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ + git push origin HEAD:"$BRANCH" + gh pr edit "$EXISTING_PR" \ + --body "$(cat docs/iterations/LEARNINGS.md) + +--- +*Last updated: ${DATE} by automated iterate workflow*" + echo "Updated existing PR #${EXISTING_PR}" + else + # Create new branch and open PR + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" + gh pr create \ + --title "learn(iterate): automated improvements — ${DATE}" \ + --body "$(cat docs/iterations/LEARNINGS.md) + +--- +*Opened: ${DATE} by automated iterate workflow* +*Merge to apply learnings and reset the iteration cycle.*" \ + --label "automated,iteration" \ + --base main + echo "Opened new PR" + fi +``` + +- [ ] **Step 2: Validate YAML syntax** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/iterate.yml'))" && echo "YAML valid" +``` + +Expected: `YAML valid` + +- [ ] **Step 3: Add required GitHub secrets** + +Verify these secrets exist in the repo's `TradingAgent` environment (Settings → Environments → TradingAgent → Secrets): +- `ANTHROPIC_API_KEY` — for Claude Code +- `GH_TOKEN` — PAT with `repo` scope (needed for PR creation; the default `GITHUB_TOKEN` cannot create PRs that trigger other workflows) + +Check existing secrets: +```bash +gh secret list --env TradingAgent +``` + +If `GH_TOKEN` is missing, the user must add it manually via GitHub UI (PAT with `repo` scope). + +- [ ] **Step 4: Trigger manually to test** + +```bash +gh workflow run iterate.yml +sleep 10 +gh run list --workflow=iterate.yml --limit=1 +``` + +Watch the run in GitHub Actions UI. Verify: +- Claude Code installs successfully +- `/iterate` runs without crashing +- If changes produced: a PR is created or updated on `iterate/current` +- If no changes: workflow exits cleanly with "No changes produced" message + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/iterate.yml +git commit -m "feat(iteration-system): add daily iterate GitHub Actions workflow" +``` + +--- + +## Task 5: Write the `research-strategy.yml` GitHub Actions Workflow + +**Files:** +- Create: `.github/workflows/research-strategy.yml` + +- [ ] **Step 1: Write research-strategy.yml** + +Create `.github/workflows/research-strategy.yml` with this exact content: + +```yaml +name: Weekly Research Strategy + +on: + schedule: + # 7:00 AM UTC every Monday — runs after iterate (6:00 AM) + - cron: "0 7 * * 1" + workflow_dispatch: + inputs: + topic: + description: "Research topic (blank = autonomous mode)" + required: false + default: "" + +env: + NODE_VERSION: "20" + +jobs: + research: + runs-on: ubuntu-latest + environment: TradingAgent + timeout-minutes: 45 + permissions: + contents: write + pull-requests: write + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + with: + fetch-depth: 0 + token: ${{ secrets.GH_TOKEN }} + + - name: Set up git identity + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + node-version: ${{ env.NODE_VERSION }} + + - name: Install Claude Code CLI + run: npm install -g @anthropic-ai/claude-code + + - name: Build research prompt + id: prompt + run: | + TOPIC="${{ github.event.inputs.topic }}" + if [ -n "$TOPIC" ]; then + echo "prompt=/research-strategy \"${TOPIC}\"" >> "$GITHUB_OUTPUT" + else + echo "prompt=/research-strategy" >> "$GITHUB_OUTPUT" + fi + + - name: Run /research-strategy + env: + ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} + CI: "true" + run: | + claude -p "${{ steps.prompt.outputs.prompt }}" --dangerously-skip-permissions + + - name: Check for changes + id: changes + run: | + git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md || true + if git diff --cached --quiet; then + echo "has_changes=false" >> "$GITHUB_OUTPUT" + echo "No changes produced by /research-strategy" + else + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + + - name: Commit changes + if: steps.changes.outputs.has_changes == 'true' + run: | + DATE=$(date -u +%Y-%m-%d) + TOPIC="${{ github.event.inputs.topic }}" + if [ -n "$TOPIC" ]; then + MSG="research(${TOPIC}): ${DATE} — automated research run" + else + MSG="research(autonomous): ${DATE} — automated research run" + fi + git commit -m "$MSG" + + - name: Handle rolling PR + if: steps.changes.outputs.has_changes == 'true' + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} + run: | + BRANCH="research/current" + DATE=$(date -u +%Y-%m-%d) + + # Summarise new research files for PR body + NEW_FILES=$(git diff HEAD~1 --name-only -- docs/iterations/research/ | head -5) + PR_BODY="## Research Findings — ${DATE} + +New research files: +${NEW_FILES} + +$(cat docs/iterations/LEARNINGS.md | head -30) + +--- +*Last updated: ${DATE} by automated research-strategy workflow* +*Merge to apply new scanner implementations and reset the research cycle.*" + + EXISTING_PR=$(gh pr list \ + --head "$BRANCH" \ + --state open \ + --json number \ + --jq '.[0].number // empty') + + if [ -n "$EXISTING_PR" ]; then + git push origin HEAD:"$BRANCH" --force-with-lease 2>/dev/null || \ + git push origin HEAD:"$BRANCH" + gh pr edit "$EXISTING_PR" --body "$PR_BODY" + echo "Updated existing PR #${EXISTING_PR}" + else + git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH" + git push -u origin "$BRANCH" + gh pr create \ + --title "research: new strategy findings — ${DATE}" \ + --body "$PR_BODY" \ + --label "automated,research" \ + --base main + echo "Opened new PR" + fi +``` + +- [ ] **Step 2: Validate YAML syntax** + +```bash +python3 -c "import yaml; yaml.safe_load(open('.github/workflows/research-strategy.yml'))" && echo "YAML valid" +``` + +Expected: `YAML valid` + +- [ ] **Step 3: Trigger manually to test** + +```bash +gh workflow run research-strategy.yml --field topic="momentum breakout" +sleep 10 +gh run list --workflow=research-strategy.yml --limit=1 +``` + +Watch in GitHub Actions UI. Verify: +- `/research-strategy "momentum breakout"` runs (directed mode) +- Research file written to `docs/iterations/research/` +- PR created or updated on `research/current` + +- [ ] **Step 4: Test autonomous mode** + +```bash +gh workflow run research-strategy.yml +sleep 10 +gh run list --workflow=research-strategy.yml --limit=1 +``` + +Verify Claude picks a topic from the knowledge base weak spots rather than erroring. + +- [ ] **Step 5: Commit** + +```bash +git add .github/workflows/research-strategy.yml +git commit -m "feat(iteration-system): add weekly research-strategy GitHub Actions workflow" +``` + +--- + +## Completion Checklist + +- [ ] `docs/iterations/` has 16 files (LEARNINGS.md + 13 scanner files + scoring.md) +- [ ] `/iterate` command loads in Claude Code session +- [ ] `/research-strategy` command loads in Claude Code session +- [ ] `iterate.yml` workflow runs cleanly (manual trigger) +- [ ] `research-strategy.yml` workflow runs cleanly (manual trigger, directed mode) +- [ ] `research-strategy.yml` autonomous mode picks a topic from knowledge base +- [ ] Rolling PR logic verified: second workflow run updates existing PR, not creates a new one +- [ ] `GH_TOKEN` secret exists in `TradingAgent` environment diff --git a/docs/superpowers/specs/2026-04-07-iteration-system-design.md b/docs/superpowers/specs/2026-04-07-iteration-system-design.md new file mode 100644 index 00000000..2e1350f8 --- /dev/null +++ b/docs/superpowers/specs/2026-04-07-iteration-system-design.md @@ -0,0 +1,247 @@ +# Iteration System Design + +**Date:** 2026-04-07 +**Status:** Approved +**Scope:** Generic iteration system — skills + folder structure for learn-improve-repeat cycles. Demonstrated with trading agent discovery mode but applicable to any iterative system. + +--- + +## Problem + +Improving the discovery pipeline requires three distinct feedback loops running at different cadences: + +1. **Fast loop** — output quality: does the latest run produce specific, well-reasoned candidates? +2. **Slow loop** — P&L outcomes: did the picks actually work after 5–10+ trading days? +3. **Research loop** — strategy sourcing: are there techniques we haven't tried yet that could improve signal quality? + +Currently there is no structured way to capture learnings across runs, trace code changes back to evidence, or proactively search for improvements. Knowledge lives in one-off plan docs and in memory. + +--- + +## Solution + +Two Claude Code skills + a versioned knowledge base in `docs/iterations/` + two GitHub Actions workflows that run the skills on a schedule and open PRs for review. + +--- + +## Folder Structure + +``` +docs/iterations/ +├── LEARNINGS.md ← master index, one-line per domain entry +├── scanners/ +│ ├── options_flow.md +│ ├── insider_buying.md +│ ├── volume_accumulation.md +│ ├── reddit_dd.md +│ ├── reddit_trending.md +│ ├── semantic_news.md +│ ├── market_movers.md +│ ├── earnings_calendar.md +│ ├── analyst_upgrades.md +│ ├── technical_breakout.md +│ ├── sector_rotation.md +│ ├── ml_signal.md +│ ├── minervini.md +│ └── ... (one file per scanner) +├── pipeline/ +│ └── scoring.md ← LLM scoring, confidence calibration, ranking +└── research/ + └── YYYY-MM-DD-.md ← web research findings (append-only, dated) +``` + +Each scanner file captures both the implementation learnings and the underlying market thesis — no separate strategies folder since scanners and strategies are currently 1:1. + +### Domain File Schema + +Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure: + +```markdown +# + +## Current Understanding + + +## Evidence Log + + +### YYYY-MM-DD — +- What was observed +- What it implies +- Confidence: low / medium / high + +## Pending Hypotheses + +- [ ] Hypothesis description +``` + +### LEARNINGS.md Schema + +```markdown +# Learnings Index + +| Domain | File | Last Updated | One-line Summary | +|--------|------|--------------|-----------------| +| options_flow | scanners/options_flow.md | 2026-04-07 | Call/put ratio <0.1 is reliable; premium filter working | +| ... | | | | +``` + +--- + +## Skill 1: `/iterate` + +**Location:** project-local `.claude/skills/iterate.md` + +### Trigger +- **Automated:** GitHub Actions cron, daily at 06:00 UTC (after overnight discovery runs settle) +- **Manual:** invoke `/iterate` at any time for an on-demand iteration cycle + +### Steps + +1. **Detect mode** + - Scans `results/discovery/` for runs not yet reflected in learning files + - Checks `data/positions/` for positions ≥5 days old with outcome data + - Sets mode: `fast` (output quality only), `pl` (P&L outcomes), or `both` + +2. **Load domain context** + - Identifies which scanners produced candidates in the target runs + - Reads the corresponding `docs/iterations/scanners/*.md` files + - Reads `docs/iterations/LEARNINGS.md` for full index awareness + +3. **Analyze** + - *Fast mode:* scores signal quality — specificity of thesis, scanner noise rate, LLM confidence calibration, duplicate/redundant candidates + - *P&L mode:* compares predicted outcome vs actual per scanner; flags scanners over/underperforming their stated confidence; computes hit rate per scanner + +4. **Write learnings** + - Appends to the evidence log in each relevant domain file + - Updates "Current Understanding" section if confidence in the new evidence is medium or higher + - Updates `LEARNINGS.md` index with new last-updated date and revised one-liner + +5. **Implement changes** + - Translates learnings into concrete code changes: scanner thresholds, priority logic, LLM prompt wording, scanner enable/disable + - In automated mode: implements without confirmation gate + - In manual mode: presents diff for approval before writing + +6. **Commit + rolling PR** + - Checks for an existing open PR with branch prefix `iterate/current` + - If one exists: pushes new commits onto that branch, updates PR description with the latest findings appended + - If none exists: creates branch `iterate/current`, opens a new PR against `main` + - Commit message format: `learn(iterate): ` + - In manual mode: commits directly to current branch (no PR unless on `main`) + - On merge: next run automatically opens a fresh PR + +### Output +At most one open `iterate/current` PR at any time. It accumulates daily learnings until merged. Merging resets the cycle. + +Human reviews PR, merges or closes. No code reaches `main` unreviewed. + +--- + +## Skill 2: `/research-strategy` + +**Location:** project-local `.claude/skills/research-strategy.md` + +### Trigger +- **Automated:** GitHub Actions cron, weekly on Monday at 07:00 UTC +- **Manual:** `/research-strategy` (autonomous) or `/research-strategy ""` (directed) + +### Two Modes + +#### Directed mode: `/research-strategy ""` +User names the topic. Skill goes deep on that specific strategy. + +#### Autonomous mode: `/research-strategy` +No topic given. Skill drives its own research agenda based on current weak spots. + +### Steps + +1. **Set agenda** + - *Directed:* topic is given + - *Autonomous:* reads `LEARNINGS.md` + domain files to identify: low-confidence scanners, pending hypotheses marked for research, gaps in pipeline coverage. Generates 3–5 research topics ranked by potential impact. + +2. **Search** + Searches across the default source list: + - **Reddit:** r/algotrading, r/quant, r/investing + - **Blogs:** QuantifiedStrategies, Hacker News (hn.algolia.com), CSS Analytics, Alpha Architect + - **GitHub:** search for quant/scanner/screener repos with recent activity + - **Academic:** SSRN quantitative finance, arXiv q-fin + - **Archives:** Quantopian community notebooks (via GitHub mirrors) + + For each source: looks for signal definition, entry/exit criteria, known edge, known failure modes, data requirements. + +3. **Cross-reference** + - Checks existing `docs/iterations/` files for overlap with already-implemented or already-discarded approaches + - Flags redundancy explicitly ("this is a variant of our existing volume_accumulation scanner") + +4. **Evaluate fit** + Scores each finding on four dimensions: + - **Data availability** — do we already have the required data source? + - **Implementation complexity** — hours estimate: trivial / moderate / large + - **Signal uniqueness** — how much does it overlap with existing scanners? + - **Evidence quality** — backtested with stats / qualitative analysis / anecdotal + +5. **Write research files** + - Saves findings to `docs/iterations/research/YYYY-MM-DD-.md` for all findings + - Adds entries to `LEARNINGS.md` + +6. **Implement and rolling PR** + - In automated mode: implements the top-ranked finding automatically (score threshold: data availability = present, evidence quality ≥ qualitative, complexity ≤ moderate) + - In manual directed mode: presents ranked findings, implements user-selected one + - Checks for an existing open PR with branch prefix `research/current` + - If one exists: pushes new commits onto that branch, updates PR description with new findings appended + - If none exists: creates branch `research/current`, opens a new PR against `main` + - Commits: `research(): add scanner — ` + - On merge: next run automatically opens a fresh PR + +### Safety threshold for autonomous implementation +Only auto-implement if ALL of: +- Required data source already integrated (no new API keys needed) +- Complexity estimate: trivial or moderate +- Signal uniqueness score: low overlap with existing scanners +Otherwise: commits research files only, opens PR flagged `needs-review` with explanation of why auto-implementation was skipped. + +--- + +## Generic Applicability + +The skill logic is domain-agnostic. To apply this system to a non-trading iterative project: + +1. Replace `results/discovery/` with the project's run output directory +2. Replace `data/positions/` with whatever measures outcome quality +3. Replace the scanner domain files with the project's equivalent components +4. Replace the research source list with domain-appropriate sources + +The folder structure and skill steps are unchanged. + +--- + +## GitHub Actions Workflows + +### `.github/workflows/iterate.yml` +``` +schedule: cron '0 6 * * *' # daily at 06:00 UTC +``` +- Checks out repo +- Runs Claude Code with `/iterate` skill non-interactively +- Uses `ANTHROPIC_API_KEY` secret +- Creates branch `iterate/YYYY-MM-DD`, commits, opens PR + +### `.github/workflows/research-strategy.yml` +``` +schedule: cron '0 7 * * 1' # weekly, Monday at 07:00 UTC +``` +- Same setup as above, runs `/research-strategy` (autonomous mode) +- Creates branch `research/YYYY-MM-DD`, commits, opens PR +- If safety threshold not met: opens PR with `needs-review` label, no code changes + +### Required secrets +- `ANTHROPIC_API_KEY` — Claude API access +- `GH_TOKEN` — PAT with `repo` scope for branch creation and PR opening + +--- + +## Non-Goals + +- No auto-merge — all PRs require human review before reaching `main` +- No dashboard or UI — all output is markdown + git commits +- No cross-project knowledge sharing — each project's `docs/iterations/` is independent