**Global Market News**: Implemented `get_global_market_news` in Alpha Vantage module to support generic market news (topics: economy_macro, financial_markets), fixing the lack of a primary vendor for global news.

- **Configurable Embeddings Truncation**: Added `EMBEDDING_TRUNCATION_LIMIT` env var (default 1000) to prevent `413 Payload Too Large` errors with local models.
2026-01-10 12:45:04 -06:00 · 2026-01-10 12:45:04 -06:00 · 0f46729f09
parent bfbc011a87
commit 0f46729f09
8 changed files with 391 additions and 21 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -5,6 +5,8 @@ All notable changes to the **TradingAgents** project will be documented in this
 ## [Unreleased] - 2026-01-10

 ### Added
+- **Global Market News**: Implemented `get_global_market_news` in Alpha Vantage module to support generic market news (topics: economy_macro, financial_markets), fixing the lack of a primary vendor for global news.
+- **Configurable Embeddings Truncation**: Added `EMBEDDING_TRUNCATION_LIMIT` env var (default 1000) to prevent `413 Payload Too Large` errors with local models.
 - **Local Embedding Service Support**: Added support for Anthropic to use local embedding service via URL
  - Anthropic doesn't provide embeddings API, so users can run **Hugging Face Text Embeddings Inference (TEI)** in Docker
  - Configure via `EMBEDDING_API_URL` environment variable (default: `http://localhost:11434/v1`)
@ -40,6 +42,9 @@ All notable changes to the **TradingAgents** project will be documented in this
 - **Configuration Documentation**: Enhanced `.env.example` with detailed comments and examples for all configuration options

 ### Fixed
+- **Global News Failure**: Resolved `RuntimeError: All vendor implementations failed` for `get_global_news` by correctly mapping Alpha Vantage and implementing the missing fallback logic.
+- **Error Reporting**: Improved `interface.py` to propagate detailed error messages from failed vendors to help debugging.
+- **Embedding Crash**: Fixed crashes when processing large documents with local embedding models by enforcing strict token limits via truncation.
 - **Anthropic Embedding Error**: Resolved `404 Not Found` error when using Anthropic as LLM provider by implementing automatic fallback to local embeddings (Anthropic doesn't provide an embeddings API)

 ### Technical Debt
--- a/scripts/generate_report_html.py
+++ b/scripts/generate_report_html.py
@ -0,0 +1,255 @@
+import os
+import sys
+import json
+from pathlib import Path
+
+TEMPLATE = """<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Trading Agent Report - {ticker} - {date}</title>
+    <script src="https://cdn.jsdelivr.net/npm/marked/marked.min.js"></script>
+    <style>
+        :root {
+            --bg-color: #0d1117;
+            --text-color: #c9d1d9;
+            --border-color: #30363d;
+            --accent-color: #58a6ff;
+            --sidebar-bg: #161b22;
+        }
+        
+        body {
+            font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
+            background-color: var(--bg-color);
+            color: var(--text-color);
+            margin: 0;
+            padding: 0;
+            display: flex;
+            height: 100vh;
+        }
+
+        /* Sidebar Navigation */
+        .sidebar {
+            width: 250px;
+            background-color: var(--sidebar-bg);
+            border-right: 1px solid var(--border-color);
+            padding: 20px;
+            display: flex;
+            flex-direction: column;
+        }
+
+        .logo {
+            font-size: 1.2rem;
+            font-weight: bold;
+            color: var(--accent-color);
+            margin-bottom: 30px;
+            padding-bottom: 20px;
+            border-bottom: 1px solid var(--border-color);
+        }
+
+        .nav-item {
+            padding: 10px 15px;
+            margin-bottom: 8px;
+            cursor: pointer;
+            border-radius: 6px;
+            color: var(--text-color);
+            transition: background 0.2s;
+        }
+
+        .nav-item:hover {
+            background-color: #21262d;
+        }
+
+        .nav-item.active {
+            background-color: #1f2937;
+            color: var(--accent-color);
+            border-left: 3px solid var(--accent-color);
+        }
+
+        /* Main Content */
+        .main-content {
+            flex: 1;
+            padding: 40px;
+            overflow-y: auto;
+        }
+
+        .markdown-body {
+            max-width: 900px;
+            margin: 0 auto;
+            line-height: 1.6;
+        }
+
+        /* Markdown Styles (GitHub Dark Theme approximation) */
+        h1, h2, h3 { color: #f0f6fc; border-bottom: 1px solid var(--border-color); padding-bottom: 0.3em; }
+        h1 { font-size: 2em; }
+        h2 { font-size: 1.5em; margin-top: 24px; }
+        a { color: var(--accent-color); text-decoration: none; }
+        a:hover { text-decoration: underline; }
+        
+        code {
+            background-color: rgba(110, 118, 129, 0.4);
+            padding: 0.2em 0.4em;
+            border-radius: 6px;
+            font-family: ui-monospace, SFMono-Regular, SF Mono, Menlo, Consolas, Liberation Mono, monospace;
+        }
+        
+        pre {
+            background-color: #161b22;
+            padding: 16px;
+            border-radius: 6px;
+            overflow: auto;
+        }
+        
+        pre code {
+            background-color: transparent;
+            padding: 0;
+        }
+
+        blockquote {
+            border-left: 0.25em solid #30363d;
+            color: #8b949e;
+            padding: 0 1em;
+            margin: 0;
+        }
+
+        table {
+            border-collapse: collapse;
+            width: 100%;
+            margin-top: 20px;
+        }
+        
+        table th, table td {
+            border: 1px solid var(--border-color);
+            padding: 8px 12px;
+        }
+        
+        table th {
+            background-color: #161b22;
+            font-weight: 600;
+        }
+
+        table tr:nth-child(2n) {
+            background-color: #0d1117;
+        }
+        
+        hr {
+            border: 0;
+            border-bottom: 1px solid var(--border-color);
+            margin: 24px 0;
+        }
+
+        /* Helper for replacing content */
+        .hidden { display: none; }
+    </style>
+</head>
+<body>
+
+    <div class="sidebar">
+        <div class="logo">
+            🤖 Trading Agents<br>
+            <span style="font-size: 0.8em; color: #8b949e">{ticker} | {date}</span>
+        </div>
+        <div id="nav-container">
+            <!-- Nav text will be inserted here -->
+        </div>
+    </div>
+
+    <div class="main-content">
+        <div id="content" class="markdown-body">
+            <!-- Rendered Markdown will appear here -->
+        </div>
+    </div>
+
+    <script>
+        // Start: Embedded Markdown Content
+        const reportData = {json_data};
+        // End: Embedded Markdown Content
+
+        const navContainer = document.getElementById('nav-container');
+        const contentDiv = document.getElementById('content');
+
+        function renderReport(key) {
+            // Update Active Nav
+            document.querySelectorAll('.nav-item').forEach(el => el.classList.remove('active'));
+            document.getElementById(`nav-${key}`).classList.add('active');
+            
+            // Render Content
+            contentDiv.innerHTML = marked.parse(reportData[key]);
+            window.scrollTo(0, 0);
+        }
+
+        // Initialize Navigation
+        const keys = Object.keys(reportData);
+        keys.forEach((key, index) => {
+            const navItem = document.createElement('div');
+            navItem.className = 'nav-item';
+            navItem.id = `nav-${key}`;
+            navItem.innerText = key.replace(/_/g, ' ').replace('.md', '').toUpperCase();
+            navItem.onclick = () => renderReport(key);
+            navContainer.appendChild(navItem);
+        });
+
+        // Load first report by default
+        if (keys.length > 0) {
+            renderReport(keys[0]);
+        }
+    </script>
+
+</body>
+</html>
+"""
+
+def generate_report(report_dir):
+    path = Path(report_dir)
+    if not path.exists():
+        print(f"Error: Directory {report_dir} not found.")
+        return
+
+    # Extract info from path structure: results/TICKER/DATE/reports
+    try:
+        data_parts = path.parts
+        # Assuming structure .../TICKER/DATE/reports
+        date = data_parts[-2]
+        ticker = data_parts[-3]
+    except IndexError:
+        date = "Unknown Date"
+        ticker = "Unknown Ticker"
+
+    reports = {}
+    
+    # Read all markdown files
+    for file in path.glob("*.md"):
+        try:
+            with open(file, 'r', encoding='utf-8') as f:
+                content = f.read()
+                reports[file.name] = content
+        except Exception as e:
+            print(f"Failed to read {file}: {e}")
+
+    if not reports:
+        print("No markdown files found to generate report.")
+        return
+
+    # Sort keys to ensure consistent order (e.g. Investment Plan first if possible, or alphabetical)
+    # Let's prioritize investment_plan.md
+    sorted_keys = sorted(reports.keys(), key=lambda x: (0 if "plan" in x else 1, x))
+    sorted_reports = {k: reports[k] for k in sorted_keys}
+
+    # Generate HTML
+    html_content = TEMPLATE.replace("{ticker}", ticker).replace("{date}", date)
+    html_content = html_content.replace("{json_data}", json.dumps(sorted_reports))
+
+    output_path = path / "index.html"
+    with open(output_path, 'w', encoding='utf-8') as f:
+        f.write(html_content)
+
+    print(f"✅ Generated Dashboard: {output_path}")
+    return str(output_path)
+
+if __name__ == "__main__":
+    if len(sys.argv) < 2:
+        print("Usage: python3 generate_report_html.py <report_dir>")
+        sys.exit(1)
+    
+    generate_report(sys.argv[1])
--- a/startAgent.sh
+++ b/startAgent.sh
@ -1,5 +1,15 @@
 #!/bin/bash
-/home/prem/git/antigravity-claude-proxy/startProxy.sh &
+
+# 0. Check & Start Claude Proxy
+# Check if port 10909 is open (Proxy running) using pure bash TCP check
+if ! (echo > /dev/tcp/localhost/10909) 2>/dev/null; then
+    echo "🔌 Starting Claude Proxy..."
+    /home/prem/git/antigravity-claude-proxy/startProxy.sh &
+    # Wait a moment for it to initialize
+    sleep 2
+else
+    echo "✅ Claude Proxy already running on port 10909"
+fi

 ./startEmbedding.sh

@ -19,11 +29,6 @@ if [ -f ".env" ]; then
    echo "✅ Loaded keys from .env"
 else
    echo "⚠️  No .env file found. Using default/exported keys."
-    # START: REPLACE WITH YOUR ACTUAL KEYS IF NOT USING .ENV
-    # export OPENAI_API_KEY="sk-your-key-here"
-    # export ALPHA_VANTAGE_API_KEY="your-key-here"
-    # export GOOGLE_API_KEY="your-key-here"
-    # END
 fi

 # Check if keys are set
@ -35,6 +40,44 @@ if [ -z "$GOOGLE_API_KEY" ]; then
    echo "⚠️  GOOGLE_API_KEY is missing! Set it if using Gemini."
 fi

-# 3. Start the Shadow Run (Daily Execution)
-echo "🚀 Starting Shadow Run Daily Execution..."
+# Ensure Embedding URL is set (default to local TEI port 11434)
+if [ -z "$EMBEDDING_API_URL" ]; then
+    echo "ℹ️  Setting default EMBEDDING_API_URL to http://localhost:11434/v1"
+    export EMBEDDING_API_URL="http://localhost:11434/v1"
+    export EMBEDDING_MODEL="all-MiniLM-L6-v2"
+fi
+
+if [ -z "$EMBEDDING_TRUNCATION_LIMIT" ]; then
+    export EMBEDDING_TRUNCATION_LIMIT=1000
+fi
+
+# 3. Start the Trading Agents
+echo "🚀 Starting Trading Agents..."
 python3 -m cli.main
+
+# 4. Open Reports
+echo "📊 Searching for latest generated reports..."
+# Find the latest "reports" directory by modification time (most recent last -> tail -1)
+# Works by printing timestamp (%T@) and path (%p), sorting numerically, picking last, cleaning output
+LATEST_REPORT_DIR=$(find results -type d -name "reports" -printf '%T@ %p\n' | sort -n | tail -1 | cut -f2- -d" ")
+
+if [ -n "$LATEST_REPORT_DIR" ]; then
+    echo "✅ Found reports in: $LATEST_REPORT_DIR"
+    
+    # Generate HTML Dashboard
+    echo "🎨 Generating Report Dashboard..."
+    python3 scripts/generate_report_html.py "$LATEST_REPORT_DIR"
+    
+    REPORT_HTML="$LATEST_REPORT_DIR/index.html"
+    
+    # Check if xdg-open exists (Linux)
+    if [ -f "$REPORT_HTML" ] && command -v xdg-open &> /dev/null; then
+        echo "🌐 Opening dashboard in browser..."
+        xdg-open "$REPORT_HTML" &> /dev/null &
+    else
+        echo "ℹ️  Dashboard generated at:"
+        echo "   file://$(pwd)/$REPORT_HTML"
+    fi
+else
+    echo "⚠️  No reports found to open."
+fi
--- a/test_global_news.py
+++ b/test_global_news.py
@ -0,0 +1,25 @@
+
+from tradingagents.dataflows.alpha_vantage_news import get_global_market_news
+from datetime import datetime, timedelta
+import os
+
+# Ensure env vars are loaded (assuming they are set in the shell running this)
+if not os.getenv("ALPHA_VANTAGE_API_KEY"):
+    print("WARNING: ALPHA_VANTAGE_API_KEY not set")
+
+try:
+    print("Testing get_global_market_news...")
+    curr_date = datetime.now().strftime("%Y-%m-%d")
+    print(f"Current Date: {curr_date}")
+    
+    result = get_global_market_news(curr_date=curr_date, look_back_days=7, limit=5)
+    
+    print("Success!")
+    print(f"Result type: {type(result)}")
+    if isinstance(result, str):
+        print(f"Result preview: {result[:200]}...")
+    else:
+        print(f"Result keys: {result.keys() if isinstance(result, dict) else 'Not a dict'}")
+
+except Exception as e:
+    print(f"FAILED: {e}")
--- a/tradingagents/agents/utils/memory.py
+++ b/tradingagents/agents/utils/memory.py
@ -54,12 +54,17 @@ class FinancialSituationMemory:
             masked_key = self.client.api_key[:4] + "..."
             # print(f"DEBUG: Using API Key: {masked_key}")
        
-        # Truncate text if too long (Google's limit is ~2048 tokens / 8k chars, allow buffer)
-        # OpenAI text-embedding-3 is 8191 tokens (~32k chars)
-        # Use safe limit of 9000 chars
-        if len(text) > 9000:
-             # print(f"WARNING: Truncating text for embedding. Length {len(text)} > 9000")
-             text = text[:9000]
+        # Truncate text if too long
+        # Configurable via EMBEDDING_TRUNCATION_LIMIT (default 1000 for local models)
+        # Set to -1 or 0 to disable truncation.
+        try:
+            truncation_limit = int(os.getenv("EMBEDDING_TRUNCATION_LIMIT", "1000"))
+        except ValueError:
+            truncation_limit = 1000
+
+        if truncation_limit > 0 and len(text) > truncation_limit:
+             # print(f"WARNING: Truncating text for embedding. Length {len(text)} > {truncation_limit}")
+             text = text[:truncation_limit]
        
        response = self.client.embeddings.create(
            model=self.embedding, input=text
--- a/tradingagents/dataflows/alpha_vantage.py
+++ b/tradingagents/dataflows/alpha_vantage.py
@ -2,4 +2,4 @@
 from .alpha_vantage_stock import get_stock
 from .alpha_vantage_indicator import get_indicator
 from .alpha_vantage_fundamentals import get_fundamentals, get_balance_sheet, get_cashflow, get_income_statement
-from .alpha_vantage_news import get_news, get_insider_transactions
+from .alpha_vantage_news import get_news, get_insider_transactions, get_global_market_news
--- a/tradingagents/dataflows/alpha_vantage_news.py
+++ b/tradingagents/dataflows/alpha_vantage_news.py
@ -1,3 +1,4 @@
+from datetime import datetime, timedelta
 from .alpha_vantage_common import _make_api_request, format_datetime_for_api

 def get_news(ticker, start_date, end_date) -> dict[str, str] | str:
@ -24,6 +25,35 @@ def get_news(ticker, start_date, end_date) -> dict[str, str] | str:
    
    return _make_api_request("NEWS_SENTIMENT", params)

+def get_global_market_news(curr_date, look_back_days=7, limit=50) -> dict[str, str] | str:
+    """Returns general global market news (macro economy, financial markets).
+
+    Args:
+        curr_date: Current date as string (YYYY-MM-DD) or datetime object.
+        look_back_days: Number of days to look back.
+        limit: Number of articles to return.
+
+    Returns:
+        Dictionary containing news sentiment data or JSON string.
+    """
+    # Calculate start date
+    if isinstance(curr_date, str):
+        end_dt = datetime.strptime(curr_date, "%Y-%m-%d")
+    else:
+        end_dt = curr_date
+        
+    start_dt = end_dt - timedelta(days=look_back_days)
+
+    params = {
+        "topics": "economy_macro,financial_markets",
+        "time_from": format_datetime_for_api(start_dt),
+        "time_to": format_datetime_for_api(end_dt),
+        "sort": "LATEST",
+        "limit": str(limit),
+    }
+    
+    return _make_api_request("NEWS_SENTIMENT", params)
+
 def get_insider_transactions(symbol: str) -> dict[str, str] | str:
    """Returns latest and historical insider transactions by key stakeholders.

--- a/tradingagents/dataflows/interface.py
+++ b/tradingagents/dataflows/interface.py
@ -13,7 +13,8 @@ from .alpha_vantage import (
    get_cashflow as get_alpha_vantage_cashflow,
    get_income_statement as get_alpha_vantage_income_statement,
    get_insider_transactions as get_alpha_vantage_insider_transactions,
-    get_news as get_alpha_vantage_news
+    get_news as get_alpha_vantage_news,
+    get_global_market_news as get_alpha_vantage_global_news
 )
 from .alpha_vantage_common import AlphaVantageRateLimitError
 from .alpaca import get_stock_data as get_stock_alpaca
@ -107,6 +108,7 @@ VENDOR_METHODS = {
        "local": [get_finnhub_news, get_reddit_company_news, get_google_news],
    },
    "get_global_news": {
+        "alpha_vantage": get_alpha_vantage_global_news,
        "openai": get_global_news_openai,
        "local": get_reddit_global_news
    },
@ -172,6 +174,7 @@ def route_to_vendor(method: str, *args, **kwargs):
    vendor_attempt_count = 0
    any_primary_vendor_attempted = False
    successful_vendor = None
+    errors = []

    for vendor in fallback_vendors:
        if vendor not in VENDOR_METHODS[method]:
@ -208,14 +211,17 @@ def route_to_vendor(method: str, *args, **kwargs):
                print(f"SUCCESS: {impl_func.__name__} from vendor '{vendor_name}' completed successfully")
                    
            except AlphaVantageRateLimitError as e:
+                msg = f"RATE_LIMIT: Alpha Vantage rate limit exceeded: {e}"
                if vendor == "alpha_vantage":
-                    print(f"RATE_LIMIT: Alpha Vantage rate limit exceeded, falling back to next available vendor")
-                    print(f"DEBUG: Rate limit details: {e}")
+                    print(msg)
+                    errors.append(msg)
                # Continue to next vendor for fallback
                continue
            except Exception as e:
                # Log error but continue with other implementations
-                print(f"FAILED: {impl_func.__name__} from vendor '{vendor_name}' failed: {e}")
+                msg = f"FAILED: {impl_func.__name__} from vendor '{vendor_name}' failed: {e}"
+                print(msg)
+                errors.append(msg)
                continue

        # Add this vendor's results
@ -235,8 +241,9 @@ def route_to_vendor(method: str, *args, **kwargs):

    # Final result summary
    if not results:
-        print(f"FAILURE: All {vendor_attempt_count} vendor attempts failed for method '{method}'")
-        raise RuntimeError(f"All vendor implementations failed for method '{method}'")
+        error_details = "; ".join(errors)
+        print(f"FAILURE: All {vendor_attempt_count} vendor attempts failed for method '{method}'. Errors: {error_details}")
+        raise RuntimeError(f"All vendor implementations failed for method '{method}'. Details: {error_details}")
    else:
        print(f"FINAL: Method '{method}' completed with {len(results)} result(s) from {vendor_attempt_count} vendor attempt(s)")