Merge pull request #9 from Aitous/feature/iteration-system
feat(iteration-system): self-improving iteration system
This commit is contained in:
Aitous
GitHub
commit
8c4ce43b2a
|
|
@ -0,0 +1,126 @@
|
|||
# /iterate
|
||||
|
||||
Analyze recent discovery runs and P&L outcomes, update learning files in
|
||||
`docs/iterations/`, implement concrete code improvements in `tradingagents/`,
|
||||
then commit everything. In CI (`CI=true`), stop before git operations — the
|
||||
workflow handles branching and PR creation.
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Determine What to Analyze
|
||||
|
||||
Read `docs/iterations/LEARNINGS.md` to find the **Last analyzed run** date.
|
||||
If the file does not exist (first run), treat the Last analyzed run date as `1970-01-01` and proceed.
|
||||
|
||||
Then scan `results/discovery/` for all run directories with dates AFTER the
|
||||
last analyzed date. Each run directory contains `discovery_result.json`. Collect all unanalyzed runs.
|
||||
|
||||
Also scan `data/recommendations/` for JSON files dated 5 or more days ago.
|
||||
Load each file and extract all recommendations. These are mature enough to
|
||||
have played out — analyze them regardless of `status` field value.
|
||||
For P&L analysis you need: `ticker`, `strategy_match`, `final_score`,
|
||||
`confidence`, `discovery_date`, `entry_price`, `status`.
|
||||
|
||||
Set your analysis mode:
|
||||
- If unanalyzed runs exist → include **fast-loop** (output quality analysis)
|
||||
- If mature recommendations exist → include **P&L loop** (outcome analysis)
|
||||
- If neither → print "No new data to analyze since last run." and exit.
|
||||
|
||||
## Step 2: Load Domain Context
|
||||
|
||||
For each scanner that appears in the unanalyzed runs (check `strategy_match`
|
||||
field in discovery results), read the corresponding file from
|
||||
`docs/iterations/scanners/<scanner_name>.md`.
|
||||
|
||||
Also read `docs/iterations/pipeline/scoring.md`.
|
||||
Also read `docs/iterations/LEARNINGS.md` for full index awareness.
|
||||
|
||||
## Step 3: Analyze
|
||||
|
||||
### Fast-Loop Analysis (output quality)
|
||||
For each unanalyzed run's `discovery_result.json`:
|
||||
- **Signal specificity**: Does each candidate have a concrete, specific thesis
|
||||
or a generic one? Flag candidates with vague context.
|
||||
- **Scanner noise rate**: How many candidates per scanner? Scanners producing
|
||||
10+ candidates with low scores (<65) are noisy.
|
||||
- **Confidence calibration**: Is confidence (1-10) consistent with score (0-100)?
|
||||
A score of 85 with confidence 5 suggests miscalibration.
|
||||
- **Duplicate candidates**: Same ticker appearing from 2+ scanners — note as
|
||||
confluence (positive) or redundancy (negative, if identical thesis).
|
||||
|
||||
### P&L Loop Analysis (outcome analysis)
|
||||
For each mature recommendation:
|
||||
- Group by `strategy_match` (scanner). Compute per-scanner: how many
|
||||
recommendations were made, what was the average final_score, and whether
|
||||
`status` fields suggest positive or negative outcomes.
|
||||
- Flag scanners where final_score > 80 but outcomes appear poor — overconfident.
|
||||
- Flag scanners where final_score < 65 but outcomes appear good — undervalued.
|
||||
- Note patterns: do high-confidence (9-10) picks outperform low-confidence (6-7)?
|
||||
|
||||
## Step 4: Write Learnings
|
||||
|
||||
For each scanner that appeared in the analysis, update its domain file in
|
||||
`docs/iterations/scanners/<scanner_name>.md`:
|
||||
|
||||
1. **Append to Evidence Log**: Add a dated entry with your specific observations.
|
||||
Use this format:
|
||||
```
|
||||
### YYYY-MM-DD — <run date or "P&L review">
|
||||
- What was observed
|
||||
- What it implies
|
||||
- Confidence: low / medium / high
|
||||
```
|
||||
2. **Update Current Understanding**: If your new evidence is medium or high
|
||||
confidence AND contradicts or meaningfully extends the current understanding,
|
||||
rewrite that section. Otherwise leave it unchanged.
|
||||
3. **Update Pending Hypotheses**: Check off any hypotheses that are now answered.
|
||||
Add new ones that your analysis surfaced.
|
||||
|
||||
Update `docs/iterations/LEARNINGS.md`:
|
||||
- Set **Last analyzed run** to today's date
|
||||
- Update the one-line summary and Last Updated date for each scanner you touched
|
||||
|
||||
## Step 5: Implement Code Changes
|
||||
|
||||
Based on your learnings, identify concrete improvements. For each improvement:
|
||||
|
||||
**Translate one learning → one code change.** Examples:
|
||||
- "ML signal threshold is worse than a coin flip" → raise threshold in
|
||||
`tradingagents/dataflows/discovery/scanners/ml_signal.py`
|
||||
- "Options flow premium filter is configured but not applied" → add the check
|
||||
- "Reddit DD LLM score computed but unused" → use it for priority assignment
|
||||
|
||||
For each change:
|
||||
1. Read the relevant scanner file to understand current implementation
|
||||
2. Make the minimal change that encodes the learning
|
||||
3. Do not refactor surrounding code — change only what the learning motivates
|
||||
|
||||
Implement all changes before committing.
|
||||
|
||||
## Step 6: Commit (skip if CI=true)
|
||||
|
||||
If the environment variable `CI` is set, stop here. The workflow handles git.
|
||||
|
||||
Otherwise:
|
||||
```bash
|
||||
git add docs/iterations/ tradingagents/
|
||||
```
|
||||
|
||||
Run `git commit` with a message in the format: `learn(iterate): YYYY-MM-DD — <your one-sentence summary of findings>`
|
||||
|
||||
Then check for an existing open PR on branch `iterate/current`:
|
||||
```bash
|
||||
EXISTING=$(gh pr list --head iterate/current --state open --json number --jq '.[0].number // empty')
|
||||
```
|
||||
|
||||
If one exists: push to that branch and update the PR description with your findings appended.
|
||||
If none exists: create branch `iterate/current`, push, open PR against `main`:
|
||||
```bash
|
||||
git checkout -b iterate/current
|
||||
git push -u origin iterate/current
|
||||
gh pr create \
|
||||
--title "learn(iterate): automated improvements — $(date +%Y-%m-%d)" \
|
||||
--body "$(cat docs/iterations/LEARNINGS.md)" \
|
||||
--label "automated,iteration" \
|
||||
--base main
|
||||
```
|
||||
|
|
@ -0,0 +1,182 @@
|
|||
# /research-strategy
|
||||
|
||||
Research new trading strategies or scanner improvements, evaluate fit against
|
||||
the existing pipeline, write findings to `docs/iterations/research/`, and
|
||||
implement the top-ranked finding as a new scanner if it qualifies.
|
||||
|
||||
Usage:
|
||||
- `/research-strategy` — autonomous mode: Claude picks research topics
|
||||
- `/research-strategy "topic"` — directed mode: research a specific strategy
|
||||
|
||||
In CI (`CI=true`), stop before git operations — the workflow handles them.
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Set Research Agenda
|
||||
|
||||
**If a topic argument was provided** (`$ARGUMENTS` is not empty):
|
||||
- Research topic = `$ARGUMENTS`
|
||||
- Skip to Step 2.
|
||||
|
||||
**Autonomous mode** (no argument):
|
||||
- Read `docs/iterations/LEARNINGS.md` and all scanner domain files in
|
||||
`docs/iterations/scanners/`
|
||||
- Identify the 3-5 highest-leverage research opportunities:
|
||||
- Scanners with low-confidence current understanding
|
||||
- Pending hypotheses marked with `- [ ]`
|
||||
- Gaps: signal types with no current scanner (e.g. dark pool flow,
|
||||
short interest changes, institutional 13F filings)
|
||||
- Rank by potential impact. Pick the top topic to research this run.
|
||||
- Print your agenda: "Researching: <topic> — Reason: <why this was prioritized>"
|
||||
|
||||
## Step 2: Search
|
||||
|
||||
Search the following sources for the research topic. For each source, look for:
|
||||
signal definition, entry/exit criteria, known statistical edge, known failure
|
||||
modes, data requirements.
|
||||
|
||||
**Sources to search:**
|
||||
- Reddit: r/algotrading, r/quant, r/investing (site:reddit.com)
|
||||
- QuantifiedStrategies (site:quantifiedstrategies.com)
|
||||
- Alpha Architect (site:alphaarchitect.com)
|
||||
- CSS Analytics (site:cssanalytics.wordpress.com)
|
||||
- Hacker News: search hn.algolia.com for the topic
|
||||
- GitHub: search for "quant scanner <topic>" and "trading strategy <topic>"
|
||||
- SSRN: search quantitative finance papers on the topic
|
||||
- arXiv q-fin section
|
||||
|
||||
Use WebSearch and WebFetch to retrieve actual content. Read at least 3-5
|
||||
distinct sources before forming a conclusion.
|
||||
|
||||
## Step 3: Cross-Reference Existing Knowledge
|
||||
|
||||
Check `docs/iterations/scanners/` and `docs/iterations/research/` for any
|
||||
prior work on this topic. Flag explicitly if this overlaps with:
|
||||
- An existing scanner (name it and the file)
|
||||
- A previously researched and discarded approach (cite the research file)
|
||||
- A pending hypothesis in an existing scanner file (cite it)
|
||||
|
||||
## Step 4: Evaluate Fit
|
||||
|
||||
Score the finding on four dimensions (each: ✅ pass / ⚠️ partial / ❌ fail):
|
||||
|
||||
1. **Data availability**: Is the required data source already integrated in
|
||||
`tradingagents/dataflows/`? Check for existing API clients there.
|
||||
2. **Implementation complexity**: trivial (<2 hours) / moderate (2-8 hours) /
|
||||
large (>8 hours)
|
||||
3. **Signal uniqueness**: Low overlap with existing scanners = good.
|
||||
High overlap = flag as redundant.
|
||||
4. **Evidence quality**: backtested with statistics / qualitative analysis /
|
||||
anecdotal only
|
||||
|
||||
**Auto-implement threshold** (all must pass for autonomous CI implementation):
|
||||
- Data availability: ✅ (data source already integrated)
|
||||
- Complexity: trivial or moderate
|
||||
- Uniqueness: low overlap
|
||||
- Evidence: qualitative or better
|
||||
|
||||
## Step 5: Write Research File
|
||||
|
||||
Save findings to `docs/iterations/research/` using filename format:
|
||||
`YYYY-MM-DD-<topic-slug>.md` where topic-slug is the topic lowercased with
|
||||
spaces replaced by hyphens.
|
||||
|
||||
Use this template:
|
||||
|
||||
```
|
||||
# Research: <Topic>
|
||||
|
||||
**Date:** YYYY-MM-DD
|
||||
**Mode:** directed | autonomous
|
||||
|
||||
## Summary
|
||||
<2-3 sentences on what was found>
|
||||
|
||||
## Sources Reviewed
|
||||
- <source 1>: <key finding from this source>
|
||||
- <source 2>: <key finding from this source>
|
||||
...
|
||||
|
||||
## Fit Evaluation
|
||||
| Dimension | Score | Notes |
|
||||
|-----------|-------|-------|
|
||||
| Data availability | ✅/⚠️/❌ | ... |
|
||||
| Complexity | trivial/moderate/large | ... |
|
||||
| Signal uniqueness | low/medium/high overlap | ... |
|
||||
| Evidence quality | backtested/qualitative/anecdotal | ... |
|
||||
|
||||
## Recommendation
|
||||
Implement / Skip / Needs more data — <reason>
|
||||
|
||||
## Proposed Scanner Spec (if recommending implementation)
|
||||
- **Scanner name:** `<name>`
|
||||
- **Data source:** `tradingagents/dataflows/<existing_file>.py`
|
||||
- **Signal logic:** <how to detect the signal, specific thresholds>
|
||||
- **Priority rules:** CRITICAL if X, HIGH if Y, MEDIUM otherwise
|
||||
- **Context format:** "<what to include in the candidate context string>"
|
||||
```
|
||||
|
||||
Add an entry to `docs/iterations/LEARNINGS.md` under a `## Research` section
|
||||
(create the section if it doesn't exist):
|
||||
```
|
||||
| research/<filename> | research/<filename>.md | YYYY-MM-DD | <one-line summary> |
|
||||
```
|
||||
|
||||
## Step 6: Implement (if threshold met)
|
||||
|
||||
If the finding meets the auto-implement threshold from Step 4:
|
||||
|
||||
1. Read `tradingagents/dataflows/discovery/scanner_registry.py` to understand
|
||||
the `@SCANNER_REGISTRY.register()` registration pattern.
|
||||
2. Read an existing simple scanner for the code pattern:
|
||||
`tradingagents/dataflows/discovery/scanners/earnings_calendar.py`
|
||||
3. Create `tradingagents/dataflows/discovery/scanners/<name>.py` following
|
||||
the same structure:
|
||||
- Class decorated with `@SCANNER_REGISTRY.register()`
|
||||
- `name` and `pipeline` class attributes
|
||||
- `scan(self, state)` method returning `List[Dict]`
|
||||
- Each dict must have keys: `ticker`, `source`, `context`, `priority`
|
||||
- Priority values: `"CRITICAL"`, `"HIGH"`, `"MEDIUM"`, `"LOW"`
|
||||
4. Check `tradingagents/dataflows/discovery/scanners/__init__.py` — if it
|
||||
imports scanners explicitly, add an import for the new one.
|
||||
|
||||
If threshold is NOT met: write the research file only. Add this note at the
|
||||
top of the research file:
|
||||
```
|
||||
> **Auto-implementation skipped:** <reason — which threshold failed>
|
||||
```
|
||||
|
||||
## Step 7: Commit (skip if CI=true)
|
||||
|
||||
If the environment variable `CI` is set, stop here. The workflow handles git.
|
||||
|
||||
Otherwise:
|
||||
```bash
|
||||
git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md
|
||||
```
|
||||
|
||||
Run `git commit` with a message in the format:
|
||||
`research(<topic>): YYYY-MM-DD — <one-sentence summary of finding and action>`
|
||||
|
||||
Then check for an existing open PR on branch `research/current`:
|
||||
```bash
|
||||
EXISTING=$(gh pr list --head research/current --state open --json number --jq '.[0].number // empty')
|
||||
```
|
||||
|
||||
If one exists:
|
||||
```bash
|
||||
git push origin HEAD:research/current
|
||||
gh pr edit "$EXISTING" \
|
||||
--body "$(cat docs/iterations/LEARNINGS.md | head -30)"
|
||||
```
|
||||
|
||||
If none exists:
|
||||
```bash
|
||||
git checkout -b research/current
|
||||
git push -u origin research/current
|
||||
gh pr create \
|
||||
--title "research: new strategy findings — $(date +%Y-%m-%d)" \
|
||||
--body "$(cat docs/iterations/LEARNINGS.md | head -30)" \
|
||||
--label "automated,research" \
|
||||
--base main
|
||||
```
|
||||
|
|
@ -0,0 +1,117 @@
|
|||
name: Daily Iterate
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 6:00 AM UTC daily — analyzes previous day's discovery run
|
||||
- cron: "0 6 * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
force:
|
||||
description: "Force iterate even if no new runs detected"
|
||||
required: false
|
||||
default: "false"
|
||||
type: choice
|
||||
options:
|
||||
- "false"
|
||||
- "true"
|
||||
|
||||
env:
|
||||
NODE_VERSION: "20"
|
||||
|
||||
jobs:
|
||||
iterate:
|
||||
runs-on: ubuntu-latest
|
||||
environment: TradingAgent
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GH_TOKEN }}
|
||||
|
||||
- name: Set up git identity
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Install Claude Code CLI
|
||||
run: npm install -g @anthropic-ai/claude-code
|
||||
|
||||
- name: Run /iterate
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
CI: "true"
|
||||
run: |
|
||||
claude -p "/iterate" --dangerously-skip-permissions
|
||||
|
||||
- name: Check for changes
|
||||
id: changes
|
||||
run: |
|
||||
git add docs/iterations/ tradingagents/ || true
|
||||
if git diff --cached --quiet; then
|
||||
echo "has_changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "No changes produced by /iterate"
|
||||
else
|
||||
echo "has_changes=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Commit changes
|
||||
if: steps.changes.outputs.has_changes == 'true'
|
||||
run: |
|
||||
DATE=$(date -u +%Y-%m-%d)
|
||||
git commit -m "learn(iterate): ${DATE} — automated iteration run"
|
||||
|
||||
- name: Handle rolling PR
|
||||
if: steps.changes.outputs.has_changes == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GH_TOKEN }}
|
||||
run: |
|
||||
BRANCH="iterate/current"
|
||||
DATE=$(date -u +%Y-%m-%d)
|
||||
SEPARATOR="---"
|
||||
|
||||
EXISTING_PR=$(gh pr list \
|
||||
--head "$BRANCH" \
|
||||
--state open \
|
||||
--json number \
|
||||
--jq '.[0].number // empty')
|
||||
|
||||
if [ -n "$EXISTING_PR" ]; then
|
||||
git fetch origin "$BRANCH" 2>/dev/null || true
|
||||
git pull --rebase origin main
|
||||
git push origin HEAD:"$BRANCH" --force-with-lease
|
||||
{
|
||||
cat docs/iterations/LEARNINGS.md
|
||||
echo ""
|
||||
echo "$SEPARATOR"
|
||||
echo "*Last updated: ${DATE} by automated iterate workflow*"
|
||||
} > /tmp/pr_body.md
|
||||
gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md
|
||||
echo "Updated existing PR #${EXISTING_PR}"
|
||||
else
|
||||
git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH"
|
||||
git push -u origin "$BRANCH" --force-with-lease
|
||||
{
|
||||
cat docs/iterations/LEARNINGS.md
|
||||
echo ""
|
||||
echo "$SEPARATOR"
|
||||
echo "*Opened: ${DATE} by automated iterate workflow*"
|
||||
echo "*Merge to apply learnings and reset the iteration cycle.*"
|
||||
} > /tmp/pr_body.md
|
||||
gh pr create \
|
||||
--title "learn(iterate): automated improvements — ${DATE}" \
|
||||
--body-file /tmp/pr_body.md \
|
||||
--label "automated,iteration" \
|
||||
--base main
|
||||
echo "Opened new PR"
|
||||
fi
|
||||
|
|
@ -0,0 +1,124 @@
|
|||
name: Weekly Research Strategy
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# 7:00 AM UTC every Monday — runs autonomous research on trading strategies
|
||||
- cron: "0 7 * * 1"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
topic:
|
||||
description: "Research topic (blank = autonomous mode)"
|
||||
required: false
|
||||
default: ""
|
||||
type: string
|
||||
|
||||
env:
|
||||
NODE_VERSION: "20"
|
||||
|
||||
jobs:
|
||||
research:
|
||||
runs-on: ubuntu-latest
|
||||
environment: TradingAgent
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
token: ${{ secrets.GH_TOKEN }}
|
||||
|
||||
- name: Set up git identity
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
|
||||
- name: Install Claude Code CLI
|
||||
run: npm install -g @anthropic-ai/claude-code
|
||||
|
||||
- name: Run /research-strategy
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
CI: "true"
|
||||
TOPIC: ${{ inputs.topic }}
|
||||
run: |
|
||||
if [ -n "$TOPIC" ]; then
|
||||
claude -p "/research-strategy \"$TOPIC\"" --dangerously-skip-permissions
|
||||
else
|
||||
claude -p "/research-strategy" --dangerously-skip-permissions
|
||||
fi
|
||||
|
||||
- name: Check for changes
|
||||
id: changes
|
||||
run: |
|
||||
git add docs/iterations/research/ tradingagents/ docs/iterations/LEARNINGS.md || true
|
||||
if git diff --cached --quiet; then
|
||||
echo "has_changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "No changes produced by /research-strategy"
|
||||
else
|
||||
echo "has_changes=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Commit changes
|
||||
if: steps.changes.outputs.has_changes == 'true'
|
||||
env:
|
||||
TOPIC: ${{ inputs.topic }}
|
||||
run: |
|
||||
DATE=$(date -u +%Y-%m-%d)
|
||||
if [ -n "$TOPIC" ]; then
|
||||
git commit -m "research(${TOPIC}): ${DATE} — automated research run"
|
||||
else
|
||||
git commit -m "research(autonomous): ${DATE} — automated research run"
|
||||
fi
|
||||
|
||||
- name: Handle rolling PR
|
||||
if: steps.changes.outputs.has_changes == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GH_TOKEN }}
|
||||
run: |
|
||||
BRANCH="research/current"
|
||||
DATE=$(date -u +%Y-%m-%d)
|
||||
SEPARATOR="---"
|
||||
|
||||
EXISTING_PR=$(gh pr list \
|
||||
--head "$BRANCH" \
|
||||
--state open \
|
||||
--json number \
|
||||
--jq '.[0].number // empty')
|
||||
|
||||
if [ -n "$EXISTING_PR" ]; then
|
||||
git fetch origin "$BRANCH" 2>/dev/null || true
|
||||
git push origin HEAD:"$BRANCH" --force-with-lease
|
||||
{
|
||||
cat docs/iterations/LEARNINGS.md
|
||||
echo ""
|
||||
echo "$SEPARATOR"
|
||||
echo "*Last updated: ${DATE} by automated research-strategy workflow*"
|
||||
} > /tmp/pr_body.md
|
||||
gh pr edit "$EXISTING_PR" --body-file /tmp/pr_body.md
|
||||
echo "Updated existing PR #${EXISTING_PR}"
|
||||
else
|
||||
git checkout -b "$BRANCH" 2>/dev/null || git checkout "$BRANCH"
|
||||
git push -u origin "$BRANCH" --force-with-lease
|
||||
{
|
||||
cat docs/iterations/LEARNINGS.md
|
||||
echo ""
|
||||
echo "$SEPARATOR"
|
||||
echo "*Opened: ${DATE} by automated research-strategy workflow*"
|
||||
echo "*Merge to apply research findings and reset the research cycle.*"
|
||||
} > /tmp/pr_body.md
|
||||
gh pr create \
|
||||
--title "research: new strategy findings — ${DATE}" \
|
||||
--body-file /tmp/pr_body.md \
|
||||
--label "automated,research" \
|
||||
--base main
|
||||
echo "Opened new PR"
|
||||
fi
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# Learnings Index
|
||||
|
||||
**Last analyzed run:** _(none yet — will be set by first /iterate run)_
|
||||
|
||||
| Domain | File | Last Updated | One-line Summary |
|
||||
|--------|------|--------------|-----------------|
|
||||
| options_flow | scanners/options_flow.md | — | No data yet |
|
||||
| insider_buying | scanners/insider_buying.md | — | No data yet |
|
||||
| volume_accumulation | scanners/volume_accumulation.md | — | No data yet |
|
||||
| reddit_dd | scanners/reddit_dd.md | — | No data yet |
|
||||
| reddit_trending | scanners/reddit_trending.md | — | No data yet |
|
||||
| semantic_news | scanners/semantic_news.md | — | No data yet |
|
||||
| market_movers | scanners/market_movers.md | — | No data yet |
|
||||
| earnings_calendar | scanners/earnings_calendar.md | — | No data yet |
|
||||
| analyst_upgrades | scanners/analyst_upgrades.md | — | No data yet |
|
||||
| technical_breakout | scanners/technical_breakout.md | — | No data yet |
|
||||
| sector_rotation | scanners/sector_rotation.md | — | No data yet |
|
||||
| ml_signal | scanners/ml_signal.md | — | No data yet |
|
||||
| minervini | scanners/minervini.md | — | No data yet |
|
||||
| pipeline/scoring | pipeline/scoring.md | — | No data yet |
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Pipeline Scoring & Ranking
|
||||
|
||||
## Current Understanding
|
||||
LLM assigns a final_score (0-100) and confidence (1-10) to each candidate.
|
||||
Score and confidence are correlated but not identical — a speculative setup
|
||||
can score 80 with confidence 6. The ranker uses final_score as primary sort key.
|
||||
No evidence yet on whether confidence or score is a better predictor of outcomes.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Is confidence a better outcome predictor than final_score?
|
||||
- [ ] Does score threshold (e.g. only surface candidates >70) improve hit rate?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Analyst Upgrades Scanner
|
||||
|
||||
## Current Understanding
|
||||
Detects analyst upgrades/price target increases. Most reliable when upgrade comes
|
||||
from a top-tier firm (Goldman, Morgan Stanley, JPMorgan) and represents a meaningful
|
||||
target increase (>15%). Short squeeze potential (high short interest) combined with
|
||||
an upgrade is a historically strong setup.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does analyst tier (BB firm vs boutique) predict upgrade quality?
|
||||
- [ ] Does short interest >20% combined with an upgrade produce outsized moves?
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Earnings Calendar Scanner
|
||||
|
||||
## Current Understanding
|
||||
Identifies stocks with earnings announcements in the next N days. Pre-earnings
|
||||
setups work best when combined with options flow (IV expansion) or insider activity.
|
||||
Standalone earnings calendar signal is too broad — nearly every stock has earnings
|
||||
quarterly.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does requiring options confirmation alongside earnings improve signal quality?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Insider Buying Scanner
|
||||
|
||||
## Current Understanding
|
||||
Scrapes SEC Form 4 filings. CEO/CFO purchases >$100K are the most reliable signal.
|
||||
Cluster detection (2+ insiders buying within 14 days) historically a high-conviction
|
||||
setup. Transaction details (name, title, value) must be preserved from scraper output
|
||||
and included in candidate context — dropping them loses signal clarity.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does cluster detection (2+ insiders in 14 days) outperform single-insider signals?
|
||||
- [ ] Is there a minimum transaction size below which signal quality degrades sharply?
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Market Movers Scanner
|
||||
|
||||
## Current Understanding
|
||||
Finds stocks that have already moved significantly. This is a reactive scanner —
|
||||
it identifies momentum after it starts rather than predicting it. Useful for
|
||||
continuation plays but not for early-stage entry. Best combined with volume
|
||||
confirmation to distinguish breakouts from spikes.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Is a volume confirmation filter (>1.5x average) useful for filtering out noise?
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Minervini Scanner
|
||||
|
||||
## Current Understanding
|
||||
Implements Mark Minervini's SEPA (Specific Entry Point Analysis) criteria: stage 2
|
||||
uptrend, price above 50/150/200 SMA in the right order, 52-week high proximity,
|
||||
RS line at new highs. Historically one of the highest-conviction scanner setups.
|
||||
Works best in bull market conditions; underperforms in choppy/bear markets.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does adding a market condition filter (S&P 500 above 200 SMA) improve hit rate?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# ML Signal Scanner
|
||||
|
||||
## Current Understanding
|
||||
Uses a trained ML model to predict short-term price movement probability. Current
|
||||
threshold of 35% win probability is worse than a coin flip — the model needs
|
||||
retraining or the threshold needs raising to 55%+ to be useful. Signal quality
|
||||
depends heavily on feature freshness; stale features degrade performance.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does raising the threshold to 55%+ improve precision at the cost of recall?
|
||||
- [ ] Would retraining on the last 90 days of recommendations improve accuracy?
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Options Flow Scanner
|
||||
|
||||
## Current Understanding
|
||||
Scans for unusual options volume relative to open interest using Tradier API.
|
||||
Call/put volume ratio below 0.1 is a reliable bullish signal when combined with
|
||||
premium >$25K. The premium filter is configured but must be explicitly applied.
|
||||
Scanning only the nearest expiration misses institutional positioning in 30+ DTE
|
||||
contracts — scanning up to 3 expirations improves signal quality.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does scanning 3 expirations vs 1 meaningfully change hit rate?
|
||||
- [ ] Is moneyness (ITM vs OTM) a useful signal filter?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Reddit DD Scanner
|
||||
|
||||
## Current Understanding
|
||||
Scans r/investing, r/stocks, r/wallstreetbets for DD posts. LLM quality score is
|
||||
computed but not used for filtering — using it (80+ = HIGH, 60-79 = MEDIUM, <60 = skip)
|
||||
would reduce noise. Subreddit weighting matters: r/investing posts are more reliable
|
||||
than r/pennystocks. Post title and LLM score should appear in candidate context.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does filtering by LLM quality score >60 meaningfully reduce false positives?
|
||||
- [ ] Does subreddit weighting change hit rates?
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
# Reddit Trending Scanner
|
||||
|
||||
## Current Understanding
|
||||
Tracks mention velocity across subreddits. 50+ mentions in 6 hours = HIGH priority.
|
||||
20-49 = MEDIUM. Mention count should appear in context ("47 mentions in 6hrs").
|
||||
Signal is early-indicator oriented — catches momentum before price moves.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does mention velocity (rate of increase) outperform raw mention count?
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Sector Rotation Scanner
|
||||
|
||||
## Current Understanding
|
||||
Detects money flowing between sectors using relative strength analysis. Most useful
|
||||
as a macro filter rather than a primary signal — knowing which sectors are in favor
|
||||
improves conviction in scanner candidates from those sectors. Standalone sector
|
||||
rotation signals are too broad for individual stock selection.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Can sector rotation data be used as a multiplier on other scanner scores?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Semantic News Scanner
|
||||
|
||||
## Current Understanding
|
||||
Currently regex-based extraction, not semantic. Headline text is not included in
|
||||
candidate context — the context just says "Mentioned in recent market news" which
|
||||
is not informative. Catalyst classification from headline keywords (upgrade/FDA/
|
||||
acquisition/earnings) would improve LLM scoring quality significantly.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Would embedding-based semantic matching outperform keyword regex?
|
||||
- [ ] Does catalyst classification (FDA vs earnings vs acquisition) affect hit rate?
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
# Technical Breakout Scanner
|
||||
|
||||
## Current Understanding
|
||||
Detects price breakouts above key resistance levels on above-average volume.
|
||||
Minervini-style setups (stage 2 uptrend, tight base, volume-confirmed breakout)
|
||||
tend to have the highest follow-through rate. False breakouts are common without
|
||||
volume confirmation (>1.5x average on breakout day).
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does requiring volume confirmation on the breakout day reduce false positives?
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
# Volume Accumulation Scanner
|
||||
|
||||
## Current Understanding
|
||||
Detects stocks with volume >2x average. Key weakness: cannot distinguish buying from
|
||||
selling — high volume on a down day is distribution, not accumulation. Multi-day mode
|
||||
(3 of last 5 days >1.5x) is more reliable than single-day spikes. Price-change filter
|
||||
(<3% absolute move) isolates quiet accumulation from momentum chasing.
|
||||
|
||||
## Evidence Log
|
||||
_(populated by /iterate runs)_
|
||||
|
||||
## Pending Hypotheses
|
||||
- [ ] Does adding a price-direction filter (volume + flat/up price) improve hit rate?
|
||||
- [ ] Is 3-of-5-day accumulation a stronger signal than single-day 2x volume?
|
||||
File diff suppressed because it is too large
Load Diff
|
|
@ -0,0 +1,247 @@
|
|||
# Iteration System Design
|
||||
|
||||
**Date:** 2026-04-07
|
||||
**Status:** Approved
|
||||
**Scope:** Generic iteration system — skills + folder structure for learn-improve-repeat cycles. Demonstrated with trading agent discovery mode but applicable to any iterative system.
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
Improving the discovery pipeline requires three distinct feedback loops running at different cadences:
|
||||
|
||||
1. **Fast loop** — output quality: does the latest run produce specific, well-reasoned candidates?
|
||||
2. **Slow loop** — P&L outcomes: did the picks actually work after 5–10+ trading days?
|
||||
3. **Research loop** — strategy sourcing: are there techniques we haven't tried yet that could improve signal quality?
|
||||
|
||||
Currently there is no structured way to capture learnings across runs, trace code changes back to evidence, or proactively search for improvements. Knowledge lives in one-off plan docs and in memory.
|
||||
|
||||
---
|
||||
|
||||
## Solution
|
||||
|
||||
Two Claude Code skills + a versioned knowledge base in `docs/iterations/` + two GitHub Actions workflows that run the skills on a schedule and open PRs for review.
|
||||
|
||||
---
|
||||
|
||||
## Folder Structure
|
||||
|
||||
```
|
||||
docs/iterations/
|
||||
├── LEARNINGS.md ← master index, one-line per domain entry
|
||||
├── scanners/
|
||||
│ ├── options_flow.md
|
||||
│ ├── insider_buying.md
|
||||
│ ├── volume_accumulation.md
|
||||
│ ├── reddit_dd.md
|
||||
│ ├── reddit_trending.md
|
||||
│ ├── semantic_news.md
|
||||
│ ├── market_movers.md
|
||||
│ ├── earnings_calendar.md
|
||||
│ ├── analyst_upgrades.md
|
||||
│ ├── technical_breakout.md
|
||||
│ ├── sector_rotation.md
|
||||
│ ├── ml_signal.md
|
||||
│ ├── minervini.md
|
||||
│ └── ... (one file per scanner)
|
||||
├── pipeline/
|
||||
│ └── scoring.md ← LLM scoring, confidence calibration, ranking
|
||||
└── research/
|
||||
└── YYYY-MM-DD-<topic>.md ← web research findings (append-only, dated)
|
||||
```
|
||||
|
||||
Each scanner file captures both the implementation learnings and the underlying market thesis — no separate strategies folder since scanners and strategies are currently 1:1.
|
||||
|
||||
### Domain File Schema
|
||||
|
||||
Each file in `scanners/`, `strategies/`, and `pipeline/` follows this structure:
|
||||
|
||||
```markdown
|
||||
# <Domain Name>
|
||||
|
||||
## Current Understanding
|
||||
<!-- Best-current-knowledge summary. Updated in place when evidence is strong. -->
|
||||
|
||||
## Evidence Log
|
||||
<!-- Append-only. Each entry dated. -->
|
||||
|
||||
### YYYY-MM-DD — <run or event>
|
||||
- What was observed
|
||||
- What it implies
|
||||
- Confidence: low / medium / high
|
||||
|
||||
## Pending Hypotheses
|
||||
<!-- Things to test in the next iteration. -->
|
||||
- [ ] Hypothesis description
|
||||
```
|
||||
|
||||
### LEARNINGS.md Schema
|
||||
|
||||
```markdown
|
||||
# Learnings Index
|
||||
|
||||
| Domain | File | Last Updated | One-line Summary |
|
||||
|--------|------|--------------|-----------------|
|
||||
| options_flow | scanners/options_flow.md | 2026-04-07 | Call/put ratio <0.1 is reliable; premium filter working |
|
||||
| ... | | | |
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Skill 1: `/iterate`
|
||||
|
||||
**Location:** project-local `.claude/skills/iterate.md`
|
||||
|
||||
### Trigger
|
||||
- **Automated:** GitHub Actions cron, daily at 06:00 UTC (after overnight discovery runs settle)
|
||||
- **Manual:** invoke `/iterate` at any time for an on-demand iteration cycle
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Detect mode**
|
||||
- Scans `results/discovery/` for runs not yet reflected in learning files
|
||||
- Checks `data/positions/` for positions ≥5 days old with outcome data
|
||||
- Sets mode: `fast` (output quality only), `pl` (P&L outcomes), or `both`
|
||||
|
||||
2. **Load domain context**
|
||||
- Identifies which scanners produced candidates in the target runs
|
||||
- Reads the corresponding `docs/iterations/scanners/*.md` files
|
||||
- Reads `docs/iterations/LEARNINGS.md` for full index awareness
|
||||
|
||||
3. **Analyze**
|
||||
- *Fast mode:* scores signal quality — specificity of thesis, scanner noise rate, LLM confidence calibration, duplicate/redundant candidates
|
||||
- *P&L mode:* compares predicted outcome vs actual per scanner; flags scanners over/underperforming their stated confidence; computes hit rate per scanner
|
||||
|
||||
4. **Write learnings**
|
||||
- Appends to the evidence log in each relevant domain file
|
||||
- Updates "Current Understanding" section if confidence in the new evidence is medium or higher
|
||||
- Updates `LEARNINGS.md` index with new last-updated date and revised one-liner
|
||||
|
||||
5. **Implement changes**
|
||||
- Translates learnings into concrete code changes: scanner thresholds, priority logic, LLM prompt wording, scanner enable/disable
|
||||
- In automated mode: implements without confirmation gate
|
||||
- In manual mode: presents diff for approval before writing
|
||||
|
||||
6. **Commit + rolling PR**
|
||||
- Checks for an existing open PR with branch prefix `iterate/current`
|
||||
- If one exists: pushes new commits onto that branch, updates PR description with the latest findings appended
|
||||
- If none exists: creates branch `iterate/current`, opens a new PR against `main`
|
||||
- Commit message format: `learn(iterate): <date> — <one-line summary of key finding>`
|
||||
- In manual mode: commits directly to current branch (no PR unless on `main`)
|
||||
- On merge: next run automatically opens a fresh PR
|
||||
|
||||
### Output
|
||||
At most one open `iterate/current` PR at any time. It accumulates daily learnings until merged. Merging resets the cycle.
|
||||
|
||||
Human reviews PR, merges or closes. No code reaches `main` unreviewed.
|
||||
|
||||
---
|
||||
|
||||
## Skill 2: `/research-strategy`
|
||||
|
||||
**Location:** project-local `.claude/skills/research-strategy.md`
|
||||
|
||||
### Trigger
|
||||
- **Automated:** GitHub Actions cron, weekly on Monday at 07:00 UTC
|
||||
- **Manual:** `/research-strategy` (autonomous) or `/research-strategy "<topic>"` (directed)
|
||||
|
||||
### Two Modes
|
||||
|
||||
#### Directed mode: `/research-strategy "<topic>"`
|
||||
User names the topic. Skill goes deep on that specific strategy.
|
||||
|
||||
#### Autonomous mode: `/research-strategy`
|
||||
No topic given. Skill drives its own research agenda based on current weak spots.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Set agenda**
|
||||
- *Directed:* topic is given
|
||||
- *Autonomous:* reads `LEARNINGS.md` + domain files to identify: low-confidence scanners, pending hypotheses marked for research, gaps in pipeline coverage. Generates 3–5 research topics ranked by potential impact.
|
||||
|
||||
2. **Search**
|
||||
Searches across the default source list:
|
||||
- **Reddit:** r/algotrading, r/quant, r/investing
|
||||
- **Blogs:** QuantifiedStrategies, Hacker News (hn.algolia.com), CSS Analytics, Alpha Architect
|
||||
- **GitHub:** search for quant/scanner/screener repos with recent activity
|
||||
- **Academic:** SSRN quantitative finance, arXiv q-fin
|
||||
- **Archives:** Quantopian community notebooks (via GitHub mirrors)
|
||||
|
||||
For each source: looks for signal definition, entry/exit criteria, known edge, known failure modes, data requirements.
|
||||
|
||||
3. **Cross-reference**
|
||||
- Checks existing `docs/iterations/` files for overlap with already-implemented or already-discarded approaches
|
||||
- Flags redundancy explicitly ("this is a variant of our existing volume_accumulation scanner")
|
||||
|
||||
4. **Evaluate fit**
|
||||
Scores each finding on four dimensions:
|
||||
- **Data availability** — do we already have the required data source?
|
||||
- **Implementation complexity** — hours estimate: trivial / moderate / large
|
||||
- **Signal uniqueness** — how much does it overlap with existing scanners?
|
||||
- **Evidence quality** — backtested with stats / qualitative analysis / anecdotal
|
||||
|
||||
5. **Write research files**
|
||||
- Saves findings to `docs/iterations/research/YYYY-MM-DD-<topic>.md` for all findings
|
||||
- Adds entries to `LEARNINGS.md`
|
||||
|
||||
6. **Implement and rolling PR**
|
||||
- In automated mode: implements the top-ranked finding automatically (score threshold: data availability = present, evidence quality ≥ qualitative, complexity ≤ moderate)
|
||||
- In manual directed mode: presents ranked findings, implements user-selected one
|
||||
- Checks for an existing open PR with branch prefix `research/current`
|
||||
- If one exists: pushes new commits onto that branch, updates PR description with new findings appended
|
||||
- If none exists: creates branch `research/current`, opens a new PR against `main`
|
||||
- Commits: `research(<topic>): add <scanner-name> scanner — <one-line rationale>`
|
||||
- On merge: next run automatically opens a fresh PR
|
||||
|
||||
### Safety threshold for autonomous implementation
|
||||
Only auto-implement if ALL of:
|
||||
- Required data source already integrated (no new API keys needed)
|
||||
- Complexity estimate: trivial or moderate
|
||||
- Signal uniqueness score: low overlap with existing scanners
|
||||
Otherwise: commits research files only, opens PR flagged `needs-review` with explanation of why auto-implementation was skipped.
|
||||
|
||||
---
|
||||
|
||||
## Generic Applicability
|
||||
|
||||
The skill logic is domain-agnostic. To apply this system to a non-trading iterative project:
|
||||
|
||||
1. Replace `results/discovery/` with the project's run output directory
|
||||
2. Replace `data/positions/` with whatever measures outcome quality
|
||||
3. Replace the scanner domain files with the project's equivalent components
|
||||
4. Replace the research source list with domain-appropriate sources
|
||||
|
||||
The folder structure and skill steps are unchanged.
|
||||
|
||||
---
|
||||
|
||||
## GitHub Actions Workflows
|
||||
|
||||
### `.github/workflows/iterate.yml`
|
||||
```
|
||||
schedule: cron '0 6 * * *' # daily at 06:00 UTC
|
||||
```
|
||||
- Checks out repo
|
||||
- Runs Claude Code with `/iterate` skill non-interactively
|
||||
- Uses `ANTHROPIC_API_KEY` secret
|
||||
- Creates branch `iterate/YYYY-MM-DD`, commits, opens PR
|
||||
|
||||
### `.github/workflows/research-strategy.yml`
|
||||
```
|
||||
schedule: cron '0 7 * * 1' # weekly, Monday at 07:00 UTC
|
||||
```
|
||||
- Same setup as above, runs `/research-strategy` (autonomous mode)
|
||||
- Creates branch `research/YYYY-MM-DD`, commits, opens PR
|
||||
- If safety threshold not met: opens PR with `needs-review` label, no code changes
|
||||
|
||||
### Required secrets
|
||||
- `ANTHROPIC_API_KEY` — Claude API access
|
||||
- `GH_TOKEN` — PAT with `repo` scope for branch creation and PR opening
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No auto-merge — all PRs require human review before reaching `main`
|
||||
- No dashboard or UI — all output is markdown + git commits
|
||||
- No cross-project knowledge sharing — each project's `docs/iterations/` is independent
|
||||
Loading…
Reference in New Issue