diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..a5733fdf --- /dev/null +++ b/docs/README.md @@ -0,0 +1,24 @@ +# TradingAgents Documentation + +TradingAgents is a multi-agent LLM trading research framework. This documentation set is geared toward engineers and researchers who need to understand, extend, or operate the system. It complements the project `README.md` by focusing on architecture and design rationale rather than user-facing instructions. + +## Audience and Scope +- **Core maintainers** who evolve the agent graph or add features. +- **Research engineers** integrating new data sources, LLMs, or debate strategies. +- **Ops / platform engineers** who deploy the CLI or automate experiments. + +## How to Use These Docs +1. Start with the [System Design](./system-design.md) for an end-to-end view of the architecture, flows, and constraints. +2. Drill into component guides under `components/` for implementation detail, responsibilities, and extension tips. +3. Refer back to code using the file references in each section when you need concrete implementation details. + +## Document Map +- [System Design](./system-design.md) +- Component Guides + - [Analyst Team](./components/analyst-team.md) + - [Research & Trading Pipeline](./components/research-and-trading.md) + - [Risk Management and Governance](./components/risk-management.md) + - [Data Platform & Tooling](./components/data-platform.md) + - [Runtime, CLI, and Configuration](./components/runtime-and-cli.md) + +Future additions such as runbooks or experiment playbooks should live alongside these guides to keep the `docs/` structure cohesive. diff --git a/docs/components/analyst-team.md b/docs/components/analyst-team.md new file mode 100644 index 00000000..00e71ed9 --- /dev/null +++ b/docs/components/analyst-team.md @@ -0,0 +1,49 @@ +# Analyst Team + +The analyst layer ingests raw market signals and transforms them into structured reports that seed downstream debates. Each analyst node is a LangGraph-compatible callable created in `tradingagents/agents/analysts/` and shares a common shape: + +1. Build a role-specific system prompt. +2. Bind the permitted tools from the shared `Toolkit`. +3. Invoke the LLM with the conversation history stored in `AgentState.messages`. +4. Append the generated report back into `AgentState`. + +## Roles at a Glance +| Role | Factory | Primary Data Sources | Output Field | +|------|---------|----------------------|--------------| +| Market Analyst | `create_market_analyst` | Yahoo Finance OHLCV, Stockstats indicators | `market_report` | +| Social Media Analyst | `create_social_media_analyst` | Reddit company threads, OpenAI-powered sentiment feeds | `sentiment_report` | +| News Analyst | `create_news_analyst` | Google News, Finnhub archives, Reddit global news | `news_report` | +| Fundamentals Analyst | `create_fundamentals_analyst` | Finnhub insider data, SimFin financial statements, custom OpenAI tool | `fundamentals_report` | + +All analysts conclude with a Markdown summary that downstream roles reuse verbatim. + +## Tool Selection and Control Flow +Each analyst chooses tool variants based on `Toolkit.config['online_tools']`. When set to `True`, network-bound functions collect fresh data; otherwise, they fall back to cached datasets under `tradingagents/dataflows/`. + +```mermaid +flowchart LR + subgraph Analyst Node + Prompt[LLM Prompt] + ToolCall{{Tool Decision}} + end + Toolkit[(Toolkit)] + Cache[(Offline Cache)] + APIs[(External APIs)] + + Prompt --> ToolCall + ToolCall -- online --> APIs + ToolCall -- offline --> Cache + APIs --> Toolkit + Cache --> Toolkit + Toolkit --> ToolCall + ToolCall --> Prompt + Prompt --> Report[Report -> AgentState] +``` + +Tool invocations are mediated by LangGraph `ToolNode`s created in `_create_tool_nodes()` inside `tradingagents/graph/trading_graph.py`. `ConditionalLogic.should_continue_*` inspects the last LLM message: if a tool call is present, execution loops through `ToolNode` to fetch data; otherwise, the node emits its final report and the workflow advances. + +## Extending the Analyst Layer +- Add a new tool to `Toolkit` with the `@tool` decorator so LangGraph can route invocations. +- Create a node factory mirroring the existing analysts: bind the tool subset, craft role prompts, and update the returned state dict with a new report key. +- Register the node in `GraphSetup.setup_graph()` by adding it to `analyst_nodes` and `conditional_logic`. +- Update `AgentState` (`tradingagents/agents/utils/agent_states.py`) with the new report field so downstream roles can consume it. diff --git a/docs/components/data-platform.md b/docs/components/data-platform.md new file mode 100644 index 00000000..1a4c65f9 --- /dev/null +++ b/docs/components/data-platform.md @@ -0,0 +1,47 @@ +# Data Platform & Tooling + +TradingAgents separates data acquisition from agent logic so that research nodes focus on reasoning while the `Toolkit` and `dataflows` packages handle ingress, formatting, and caching. + +## Toolkit Overview +`Toolkit` (`tradingagents/agents/utils/agent_utils.py`) exposes `@tool`-decorated functions consumable by LangGraph. Each tool encapsulates: +- Input annotations used by LangChain to validate structured tool calls. +- A thin adapter into the `dataflows.interface` module or a live API. +- Markdown-friendly string outputs to plug directly into prompts. + +The toolkit is instantiated once per `TradingAgentsGraph` and shared across agents. Configuration merges `DEFAULT_CONFIG` with user overrides via `Toolkit.update_config()`. + +## Offline vs Online Modes +`DEFAULT_CONFIG['online_tools']` determines which data plane to use. +- **Online**: Functions such as `get_YFin_data_online`, `get_global_news_openai`, or `get_fundamentals_openai` query remote APIs (OpenAI routes the heavier summarization workloads). +- **Offline**: Toolkit redirects to cached datasets stored under `tradingagents/dataflows/data_cache/` (written to on demand) or the curated archive referenced by `DEFAULT_CONFIG['data_dir']`. + +```mermaid +flowchart LR + AgentNode -->|tool call| Toolkit + Toolkit -->|online| ExternalAPIs[(YFinance, Finnhub, OpenAI Functions, Google News)] + Toolkit -->|offline| Dataflows + Dataflows --> Cache[(CSV/JSON Cache)] + ExternalAPIs --> Formatter[Markdown Formatter] + Cache --> Formatter + Formatter --> AgentNode +``` + +## Dataflows Package +Key modules under `tradingagents/dataflows/` include: +- `interface.py`: Public entry points that orchestrate date math, batching, and formatting for each provider. Functions like `get_finnhub_news` or `get_reddit_company_news` leverage helper utilities and guarantee consistent Markdown formatting. +- `yfin_utils.py`, `stockstats_utils.py`: Fetch and enrich market data (e.g., compute technical indicators before returning a report). +- `finnhub_utils.py`, `reddit_utils.py`, `googlenews_utils.py`: Read from exported datasets and support thread-safe, multi-day aggregation. +- `config.py`: Stores runtime config (`DATA_DIR`) and responds to updates from `TradingAgentsGraph` via `set_config()`. + +## Memory Storage +`FinancialSituationMemory` (`tradingagents/agents/utils/memory.py`) functions as a lightweight experience replay buffer: +- Embeddings are computed using OpenAI's `text-embedding-3-small` or Ollama's `nomic-embed-text` depending on `backend_url`. +- Recommendations are stored in an in-memory ChromaDB collection per role. +- `get_memories()` returns the best matches for prompt injection and indicates similarity scores for future weighting. + +Persisting memories across runs requires reconfiguring the Chroma client to use a file-backed storage provider. + +## Reliability Concerns +- Service limits (API quotas, tool failures) bubble up as exceptions from Toolkit calls. Consider wrapping nodes with retry logic if productionizing the framework. +- Some offline datasets referenced in `DEFAULT_CONFIG['data_dir']` are not bundled; deployments must provision the directory or override the path. +- Tool sanitization (e.g., verifying tickers, clamping lookback windows) is minimal today; additional guards can be added in `dataflows/interface.py`. diff --git a/docs/components/research-and-trading.md b/docs/components/research-and-trading.md new file mode 100644 index 00000000..b3cbbdd2 --- /dev/null +++ b/docs/components/research-and-trading.md @@ -0,0 +1,43 @@ +# Research & Trading Pipeline + +Once analyst reports are populated, the research pipeline debates the thesis, distills a plan, and hands it to the trader for execution framing. The implementation resides under `tradingagents/agents/researchers/`, `tradingagents/agents/managers/`, and `tradingagents/agents/trader/`. + +## Debate Loop +- **Bull and Bear Researchers** alternate turns (`create_bull_researcher`, `create_bear_researcher`) while `ConditionalLogic.should_continue_debate()` keeps count via `InvestDebateState['count']`. +- Each researcher pulls role-specific memories from `FinancialSituationMemory` to learn from prior scenarios and confront the opponent's argument with direct rebuttals. +- Debate transcripts are accumulated in `InvestDebateState['history']`, providing the manager with full context. + +```mermaid +sequenceDiagram + participant Bull + participant Bear + participant State as InvestDebateState + Bull->>State: Append bull argument, increment count + State-->>Bear: Provide latest bull point + memories + Bear->>State: Append bear argument, increment count + State-->>Bull: Provide latest bear point + memories +``` + +The debate stops after `2 * max_debate_rounds` turns (default `3` back-and-forths), at which point the Research Manager takes over. + +## Research Manager Responsibilities +The manager (`create_research_manager`) synthesizes debate outcomes along with analyst reports, plus lessons retrieved from memory. The manager writes two state fields: +- `investment_debate_state['judge_decision']` – narrative justification for the chosen stance. +- `investment_plan` – structured plan the trader will refine. + +Prompt design emphasizes decisive recommendations and explicit learning from past mistakes. + +## Trader Node +The trader (`create_trader`) treats the manager's plan as input, retrieves the most similar historical memories, and issues a final narrative ending with `FINAL TRANSACTION PROPOSAL: **BUY/HOLD/SELL**`. The entire response is stored as `trader_investment_plan` and later parsed by the signal processor to extract the canonical BUY/SELL/HOLD decision. + +## Reflection Hooks +Although not wired into the runtime loop yet, `tradingagents/graph/reflection.py` defines helpers for post-run analysis. Each method takes the final state, PnL feedback, and the relevant role memory to append new lessons. The design allows asynchronous reflection jobs to update the same ChromaDB collections leveraged during debates. + +## Extension Checklist +To modify or add stages in this pipeline: +1. Extend `InvestDebateState` or `AgentState` with new fields. +2. Implement the node factory with persistent memory access if needed. +3. Update `GraphSetup.setup_graph()` to insert the node and connect edges. +4. Adjust `ConditionalLogic.should_continue_debate()` if the turn-taking rules change. + +This keeps the debate logic declarative while preserving introspection via the shared state object. diff --git a/docs/components/risk-management.md b/docs/components/risk-management.md new file mode 100644 index 00000000..944c9b6a --- /dev/null +++ b/docs/components/risk-management.md @@ -0,0 +1,36 @@ +# Risk Management and Governance + +The risk layer mirrors an investment committee that challenges the trader's proposal before producing a portfolio-ready decision. Code for this stage resides in `tradingagents/agents/risk_mgmt/` and `tradingagents/agents/managers/risk_manager.py`. + +## Participants +| Node | Factory | Perspective | Key State Fields | +|------|---------|-------------|------------------| +| Risky Analyst | `create_risky_debator` | Aggressively defends upside, critiques conservative voices. | `risk_debate_state['risky_history']`, `'current_risky_response'` | +| Neutral Analyst | `create_neutral_debator` | Balances upside/downside, emphasizes sustainability. | `risk_debate_state['neutral_history']`, `'current_neutral_response'` | +| Safe Analyst | `create_safe_debator` | Protects assets, spotlights lurking threats. | `risk_debate_state['safe_history']`, `'current_safe_response'` | +| Risk Judge | `create_risk_manager` | Issues the final Buy/Sell/Hold decision and refines the trader's plan. | `risk_debate_state['judge_decision']`, `final_trade_decision` | + +All participants inspect the trader's proposal (`trader_investment_plan`) plus the four analyst reports to frame their arguments. + +## Debate Mechanics +```mermaid +flowchart TD + Start((Start)) --> Risky + Risky --> Neutral + Neutral --> Safe + Safe -->|if more rounds| Risky + Safe -->|if limit reached| Judge + Neutral -->|if limit reached| Judge + Risky -->|if limit reached| Judge + Judge --> Decision[Final Trade Decision] +``` + +`ConditionalLogic.should_continue_risk_analysis()` cycles between Risky → Safe → Neutral analysts until `risk_debate_state['count']` reaches `3 * max_risk_discuss_rounds` (default: three full turns). Each node appends its argument to the debate history and marks itself as the latest speaker. Messages include direct rebuttals to peers, encouraging adversarial dialogue rather than isolated commentary. + +## Judge Output and Signal Processing +The Risk Judge leverages role-specific memories (`risk_manager_memory`) to avoid past mistakes, rewrites the trader’s plan if necessary, and stores the narrative decision in `final_trade_decision`. The `SignalProcessor` (`tradingagents/graph/signal_processing.py`) subsequently reduces this narrative to `BUY`, `SELL`, or `HOLD` for downstream automation. + +## Governance Considerations +- **Auditability**: The full debate transcript is captured in `risk_debate_state['history']` and persisted in `TradingAgentsGraph.log_states_dict` for every run. +- **Override hooks**: Downstream systems can override or post-process `final_trade_decision` before acting, but the architecture encourages keeping governance upstream in the graph. +- **Extensibility**: Additional perspectives (e.g., compliance, portfolio construction) can plug in by augmenting `RiskDebateState` and inserting new nodes prior to the judge. diff --git a/docs/components/runtime-and-cli.md b/docs/components/runtime-and-cli.md new file mode 100644 index 00000000..eb681abf --- /dev/null +++ b/docs/components/runtime-and-cli.md @@ -0,0 +1,49 @@ +# Runtime, CLI, and Configuration + +TradingAgents can run interactively through the CLI or programmatically via the Python API, both backed by the same orchestration code. + +## Python API +Typical usage (`main.py`, `README.md`): +```python +from tradingagents.graph.trading_graph import TradingAgentsGraph +from tradingagents.default_config import DEFAULT_CONFIG + +ta = TradingAgentsGraph(debug=True, config=DEFAULT_CONFIG.copy()) +final_state, decision = ta.propagate("NVDA", "2024-05-10") +print(decision) # BUY / SELL / HOLD +``` + +- `debug=True` triggers graph streaming so intermediate messages are printed. +- `propagate()` returns both the final `AgentState` and the normalized decision from `SignalProcessor`. +- Execution directories (`dataflows/data_cache`, `results/`) are created automatically according to `config`. + +## CLI Experience +`cli/main.py` implements a Typer command that launches a Rich-based dashboard. The CLI orchestrates: +- Parameter collection (tickers, dates, model selection) via interactive prompts. +- Live progress updates using `MessageBuffer` to track agent statuses, tool calls, and partial reports. +- Final report rendering, consolidating all analyst outputs and decisions into a Markdown summary. + +To run locally: +```bash +python -m cli.main +``` +Choose the desired options and monitor agent activity in real time. + +## Configuration Surface +`DEFAULT_CONFIG` (`tradingagents/default_config.py`) exposes: +- Paths: `project_dir`, `results_dir`, `data_dir`, `data_cache_dir`. +- LLM provider settings: `llm_provider`, `deep_think_llm`, `quick_think_llm`, `backend_url`. +- Debate and recursion controls: `max_debate_rounds`, `max_risk_discuss_rounds`, `max_recur_limit`. +- Tooling: `online_tools` toggle. + +Client code can copy the dict, mutate values, and pass it into `TradingAgentsGraph`. During initialization, `set_config()` propagates the values into the `dataflows` subsystem. + +## Logging and Results +`TradingAgentsGraph` stores per-run snapshots in `log_states_dict`, keyed by trade date. Invoking `TradingAgentsGraph._log_state()` serializes analyst reports, debate histories, and final decisions into `results/` for post-run inspection. + +The CLI also caches the final Markdown report in memory; future enhancements can persist these artifacts alongside the JSON logs. + +## Deployment Notes +- The framework assumes OpenAI-compatible endpoints; alternate providers (Anthropic, Google) require the corresponding LangChain client to be configured in the config. +- For reproducible research, pin `requirements.txt` or `uv.lock` and disable online tools so all data comes from deterministic caches. +- Memory stores rely on an in-memory Chroma client. To share experiences across processes, instantiate `chromadb.PersistentClient` in `FinancialSituationMemory`. diff --git a/docs/system-design.md b/docs/system-design.md new file mode 100644 index 00000000..2295488d --- /dev/null +++ b/docs/system-design.md @@ -0,0 +1,144 @@ +# System Design + +## 1. Purpose and Capabilities +TradingAgents orchestrates a team of specialized LLM agents to perform research-driven equity trading analysis. The framework mirrors a buy-side firm structure: analysts source and synthesize signals, researchers debate positions, traders operationalize the plan, and a governance layer manages risk. The automation is built on LangGraph and LangChain primitives, letting the graph express agent interactions, tool usage, and stateful debates. + +Key capabilities: +- Multi-role analysis spanning technicals, sentiment, news, and fundamentals. +- Debate-style reasoning with configurable rounds and memory-backed reflection. +- Risk management loop that challenges trade proposals before execution. +- CLI experience for interactive runs and package APIs for programmatic use. + +## 2. System Context +```mermaid +graph TD + User[Quant Researcher / Trader] + Scheduler[Automation / Batch Jobs] + CLI[TradingAgents CLI] + SDK[TradingAgents Python API] + Graph[TradingAgentsGraph Core] + LLMProviders[OpenAI / Anthropic / Google] + DataProviders[Yahoo Finance, Finnhub, Reddit, Google News, SimFin] + Cache[Offline Data Cache] + Results[results/ logs] + + User -->|configure runs| CLI + Scheduler -->|invoke| SDK + CLI --> Graph + SDK --> Graph + Graph -->|tool calls| DataProviders + Graph -->|fallback| Cache + Graph -->|prompt invocations| LLMProviders + Graph --> Results +``` + +The `TradingAgentsGraph` (`tradingagents/graph/trading_graph.py`) is the execution nucleus shared by both the CLI (`cli/main.py`) and direct library usage (`main.py`). External dependencies are abstracted behind the `Toolkit` and `dataflows` packages. + +## 3. Architectural Layers +```mermaid +flowchart TD + subgraph Interface + CLI[[CLI: Rich + Typer]] + API[[Python API]] + end + + subgraph Orchestration + GraphSetup[[GraphSetup]] + Conditional[[ConditionalLogic]] + Propagator[[Propagator]] + SignalProcessor[[SignalProcessor]] + Reflector[[Reflector]] + end + + subgraph AgentTeams + Analysts[[Analyst Nodes]] + Researchers[[Bull/Bear Researchers + Manager]] + Trader[[Trader Node]] + Risk[[Risk Debate + Judge]] + end + + subgraph DataPlane + Toolkit[[Toolkit Tools]] + Dataflows[[Offline Dataflows]] + Memories[[FinancialSituationMemory]] + end + + Interface --> Orchestration + Orchestration --> AgentTeams + AgentTeams --> Orchestration + Orchestration --> DataPlane + DataPlane --> AgentTeams +``` + +- **Interface layer**: Either the CLI (`cli/main.py`) or library consumers call `TradingAgentsGraph.propagate()`. +- **Orchestration layer**: Builds and runs the LangGraph state machine, manages recursion limits, reflection, and signal extraction. +- **Agent teams**: Each node is a partially applied function that consumes and emits `AgentState` objects (`tradingagents/agents/utils/agent_states.py`). +- **Data plane**: Tool invocations request live data; offline caches and vector memories provide continuity between runs. + +## 4. Execution Flow +```mermaid +sequenceDiagram + participant Caller as CLI / API Client + participant Graph as TradingAgentsGraph + participant Analysts as Analyst Team + participant Researchers as Bull & Bear Researchers + participant Manager as Research Manager + participant Trader as Trader + participant Risk as Risk Debate Circle + participant Judge as Risk Manager + participant Signals as Signal Processor + + Caller->>Graph: propagate(ticker, trade_date) + Graph->>Graph: create_initial_state() + Graph->>Analysts: Market/Sentiment/News/Fundamentals nodes + Analysts-->>Graph: Enriched AgentState + reports + Graph->>Researchers: Bull/Bear alternating via ConditionalLogic + Researchers-->>Graph: Debate transcripts + Graph->>Manager: Summarize & craft investment plan + Manager-->>Trader: Updated state with plan + Graph->>Trader: Request final proposal + Trader-->>Risk: Trader plan stored in state + Graph->>Risk: Iterate risky/neutral/safe analysts + Risk-->>Judge: Debate history + recommendations + Judge-->>Graph: Final trade decision + Graph->>Signals: process_signal(final_trade_decision) + Signals-->>Caller: BUY / SELL / HOLD +``` + +Conditional edges in `ConditionalLogic` decide when analysts stop calling tools, how many debate rounds execute, and when the flow reaches the judge. + +## 5. State and Memory Management +The shared `AgentState` typed dict contains the latest reports, debate transcripts, and final decisions. Helper classes (`Propagator`, `SignalProcessor`) construct and interpret this state. + +Long-lived learning happens via `FinancialSituationMemory` (`tradingagents/agents/utils/memory.py`), which: +- Embeds summarized situations through OpenAI embeddings (or Ollama when running locally). +- Stores them in a transient ChromaDB instance per role (bull, bear, trader, judges). +- Retrieves the closest recommendations to steer future prompts. + +Reflection routines in `Reflector` append new lessons after each run, closing the learning loop. + +## 6. Data Acquisition and Tooling +`Toolkit` (`tradingagents/agents/utils/agent_utils.py`) wraps both online APIs and offline cache lookups. The data flow rules are: +- **Online mode**: Agents call OpenAI functions to fetch data (e.g., `get_YFin_data_online`, `get_global_news_openai`). +- **Offline mode**: Agents fall back to curated datasets under `tradingagents/dataflows/` (e.g., Finnhub JSON exports, SimFin CSVs, Reddit scrapes). +- Calls route through `langgraph.prebuilt.ToolNode`, so tool outputs are appended to the message stream until the analyst stops requesting data. + +The `dataflows/interface.py` module normalizes inputs (date windows, ticker symbols) and formats responses into LLM-friendly Markdown blocks. + +## 7. Configuration and Extensibility +- Default settings live in `tradingagents/default_config.py`. Consumers can copy and mutate this dict before instantiating `TradingAgentsGraph`. +- `GraphSetup.setup_graph()` accepts the analyst roster, allowing narrower runs or experiments with custom nodes. +- To add an agent, implement a node factory that accepts an LLM and toolkit/memory dependency, then register it in `GraphSetup` together with an entry in the `Toolkit` if data access is needed. +- Conditional logic thresholds (`max_debate_rounds`, `max_risk_discuss_rounds`, recursion limits) are passed into `ConditionalLogic` and `Propagator` during graph construction. + +## 8. Non-Functional Considerations +- **Latency & cost**: Runs invoke multiple LLM calls per agent and extensive tool usage; choose lighter models (`o4-mini`, `gpt-4.1-mini`) for testing. +- **Reproducibility**: With online tools enabled, outputs depend on live APIs; disable online mode for deterministic replay from cached data. +- **Statefulness**: ChromaDB collections are in-memory per process; persisting them requires swapping the `chromadb` client configuration. +- **Observability**: The CLI (`cli/main.py`) visualizes progress, but programmatic users should persist `TradingAgentsGraph.log_states_dict` if audit trails are required. + +## 9. Known Gaps and Future Enhancements +- The default `data_dir` in `DEFAULT_CONFIG` is developer-specific and should be externalized via environment configuration for portability. +- Risk and research debate limits are hard-coded when `ConditionalLogic` is instantiated inside `TradingAgentsGraph`; exposing them as config entries would simplify tuning. +- Reflection outputs are stored but not yet reinjected into prompts across runs; wiring the reflection step into the workflow is on the roadmap. +- Automated testing around agent prompts is limited; contract tests for data formatting and state transitions would improve reliability.