feat(news): enhance NewsArticle entity with sentiment fields and add Dagster orchestration specs

- Add sentiment_confidence and sentiment_label fields to NewsArticle
- Update database configuration for PostgreSQL + TimescaleDB + pgvectorscale
- Add comprehensive Dagster orchestration specifications
- Update project dependencies and tooling configuration
- Enhance test coverage for news repository functionality
- Align news domain specs with project roadmap and architecture
This commit is contained in:
Martin C. Richards 2025-11-14 22:18:33 +01:00
parent c20771bf20
commit c5ced8cd66
No known key found for this signature in database
GPG Key ID: 97EBB3B732E8C932
16 changed files with 2837 additions and 2084 deletions

1
.gitignore vendored
View File

@ -7,3 +7,4 @@ eval_results/
eval_data/ eval_data/
*.egg-info/ *.egg-info/
.env .env
.coverage

View File

@ -37,6 +37,11 @@ description = "Run tests with pytest (with database)"
depends = ["docker"] depends = ["docker"]
run = "uv run pytest" run = "uv run pytest"
[tasks.coverage]
description = "Run tests with coverage report"
depends = ["docker"]
run = "uv run pytest --cov=tradingagents --cov-report=html --cov-report=term-missing"
[tasks.lint] [tasks.lint]
description = "Run ruff linting" description = "Run ruff linting"
run = "ruff check ." run = "ruff check ."
@ -57,6 +62,11 @@ run = "ruff check --fix ."
description = "Run format, lint, and typecheck" description = "Run format, lint, and typecheck"
run = ["ruff format .", "ruff check .", "uv run pyrefly check ."] run = ["ruff format .", "ruff check .", "uv run pyrefly check ."]
[tasks.quality]
description = "Run complete quality check (format, lint, typecheck, test, coverage)"
depends = ["docker"]
run = ["ruff format .", "ruff check .", "uv run pyrefly check .", "uv run pytest --cov=tradingagents --cov-report=term-missing"]
[tasks.clean] [tasks.clean]
description = "Clean up cache and build artifacts" description = "Clean up cache and build artifacts"
run = [ run = [

View File

@ -1,6 +1,6 @@
services: services:
timescaledb: timescaledb:
build: ./db build: ./docker/db
container_name: tradingagents_timescaledb container_name: tradingagents_timescaledb
environment: environment:
POSTGRES_USER: postgres POSTGRES_USER: postgres

View File

@ -16,12 +16,27 @@ RUN echo "deb https://packagecloud.io/timescale/timescaledb/debian/ $(lsb_releas
# Install TimescaleDB for PostgreSQL 16 # Install TimescaleDB for PostgreSQL 16
RUN apt-get install -y timescaledb-2-postgresql-17 RUN apt-get install -y timescaledb-2-postgresql-17
# Install pgxman # Install pgvector manually for PostgreSQL 17
RUN apt-get update && apt-get install -y \
build-essential \
git \
postgresql-server-dev-17 \
&& rm -rf /var/lib/apt/lists/*
# Clone and build pgvector
RUN cd /tmp && \
git clone --branch v0.7.4 https://github.com/pgvector/pgvector.git && \
cd pgvector && \
make && \
make install && \
cd / && \
rm -rf /tmp/pgvector
# Install pgxman for pgvectorscale
RUN curl -sfL https://install.pgx.sh | sh - RUN curl -sfL https://install.pgx.sh | sh -
# Install pgvector and pgvectorscale using pgxman # Install pgvectorscale using pgxman
RUN pgxman install pgvector || echo "pgvector install failed" \ RUN pgxman install pgvectorscale || echo "pgvectorscale install failed"
&& pgxman install pgvectorscale || echo "pgvectorscale install failed"
# Configure PostgreSQL for TimescaleDB (instead of using timescaledb-tune) # Configure PostgreSQL for TimescaleDB (instead of using timescaledb-tune)
RUN echo "shared_preload_libraries = 'timescaledb'" >> /usr/share/postgresql/postgresql.conf.sample \ RUN echo "shared_preload_libraries = 'timescaledb'" >> /usr/share/postgresql/postgresql.conf.sample \

View File

@ -4,156 +4,183 @@
This roadmap outlines the technical development path for the personal fork of TradingAgents, focusing on building a robust data infrastructure with PostgreSQL + TimescaleDB + pgvectorscale, implementing RAG-powered agents, and establishing automated data collection pipelines with Dagster. This roadmap outlines the technical development path for the personal fork of TradingAgents, focusing on building a robust data infrastructure with PostgreSQL + TimescaleDB + pgvectorscale, implementing RAG-powered agents, and establishing automated data collection pipelines with Dagster.
## Current Status: Phase 1 - News Domain (95% Complete) **Last Updated**: 2025-11-11
The foundation has been established with core domain architecture, comprehensive testing framework, and the news domain nearly complete. ### Key Roadmap Changes
- **Pragmatic Dagster Integration**: Dagster jobs built incrementally per domain (not separate phase)
- **Accurate Timeline**: 10-14 weeks total (vs original 16-22 weeks) based on actual progress
- **Incremental Automation**: Each domain gets automated collection as it completes
- **Earlier Production Readiness**: Automated data collection starts Week 1 (not Month 4)
### Development Velocity
- **Observed Completion Rate**: News clients 85-90% complete with 600+ lines of quality tests
- **AI-Assisted Multiplier**: 3-4x faster development with spec-driven workflow
- **Target Task Velocity**: 15-20 tasks/week with AI assistance
- **Test Coverage**: Maintained 85%+ with pytest-vcr pattern
## Current Status: Phase 1 - News Domain + Dagster Integration (85% Complete)
The foundation has been established with core domain architecture, comprehensive testing framework, and the news domain clients complete.
### Completed Infrastructure ### Completed Infrastructure
- **Domain Architecture**: Clean separation of news, marketdata, and socialmedia domains - **Domain Architecture**: Clean separation of news, marketdata, and socialmedia domains
- **Testing Framework**: Pragmatic TDD with 85%+ coverage, pytest-vcr for HTTP mocking - **Testing Framework**: Pragmatic TDD with 85%+ coverage, pytest-vcr for HTTP mocking
- **Repository Pattern**: Efficient data caching and management system - **News Clients**: Google News RSS + Article Scraper with comprehensive tests (600+ lines)
- **News Domain**: Article scraping, sentiment analysis, and storage (95% complete) - **Database Stack**: PostgreSQL + TimescaleDB + pgvectorscale ready
- **Basic Agent System**: Multi-agent trading analysis framework with LangGraph - **Basic Agent System**: Multi-agent trading analysis framework with LangGraph
### Current Priorities (Next 5-7 Days)
1. **Complete News Domain Foundation** - Repository, Service, Entity layers
2. **LLM Integration** - OpenRouter sentiment analysis + vector embeddings
3. **Basic Dagster Job** - Automated daily news collection
4. **Spec Documentation** - Create status.md and tasks.md for progress tracking
## Development Phases ## Development Phases
### Phase 1: News Domain Completion (Current - 95% Complete) ### Phase 1: News Domain + Basic Dagster (Current - 85% Complete)
**Timeline**: 2-3 weeks **Timeline**: 5-7 days remaining
**Status**: 🔄 In Progress **Status**: 🔄 In Progress
#### Remaining Work #### Remaining Work (5-7 days)
- **News Processing Pipeline**: Complete article content processing and deduplication - **News Repository Layer**: PostgreSQL async operations with TimescaleDB (1-2 days)
- **Sentiment Analysis Optimization**: Fine-tune sentiment scoring algorithms - **News Service Layer**: Business logic with LLM integration (1-2 days)
- **News Repository**: Finalize PostgreSQL integration for news storage - **NewsArticle Entity**: Domain models with sentiment and embeddings (1 day)
- **Testing Coverage**: Achieve 85%+ test coverage for news domain - **OpenRouter Integration**: Sentiment analysis via LLM (1-2 days)
- **Performance Optimization**: Optimize news retrieval and search performance - **Vector Embeddings**: OpenAI embeddings via OpenRouter for semantic search (1 day)
- **Basic Dagster Job**: Daily news collection automation (1-2 days)
- **Integration Testing**: End-to-end workflow validation (1 day)
#### Key Deliverables
- News domain following Router → Service → Repository → Entity → Database pattern
- OpenRouter LLM sentiment analysis operational
- pgvectorscale vector embeddings for semantic search
- Automated Dagster job for daily news collection
- 85%+ test coverage maintained
#### Success Criteria #### Success Criteria
- ✅ All news APIs integrated and tested - ✅ Complete layered architecture implemented
- ✅ Sentiment analysis producing consistent scores - ✅ LLM sentiment scores with confidence ratings
- ✅ News data properly stored in PostgreSQL - ✅ Vector embeddings enabling semantic search
- ✅ Comprehensive test suite covering edge cases - ✅ Dagster job running daily news collection
- ✅ News domain ready for RAG integration - ✅ Query performance < 2 seconds
- ✅ News domain ready for agent integration
### Phase 2: Market Data Domain + PostgreSQL Migration (Next Priority) ### Phase 2: Market Data Domain + Dagster Integration (Next Priority)
**Timeline**: 4-6 weeks **Timeline**: 4-5 weeks
**Status**: 📋 Planned **Status**: 📋 Planned
#### Core Objectives #### Core Objectives
- **TimescaleDB Integration**: Implement hypertables for efficient time-series storage - **TimescaleDB Hypertables**: Efficient time-series storage for price/volume data
- **Market Data Collection**: Complete price, volume, and technical indicator collection - **Market Data Collection**: FinnHub/yfinance integration with retry logic
- **PostgreSQL Migration**: Move all data persistence from file-based to PostgreSQL - **PostgreSQL Migration**: Move from file-based to database storage
- **Technical Analysis**: Implement MACD, RSI, and other technical indicators - **Technical Indicators**: MACD, RSI, Bollinger Bands calculations
- **Database Schema**: Design optimized schema for market data with proper indexing - **Dagster Market Data Job**: Twice-daily price data collection automation
- **Performance Optimization**: Sub-100ms queries with proper indexing
#### Key Deliverables #### Key Deliverables
- Market data repository with TimescaleDB optimization - MarketDataRepository with TimescaleDB optimization
- Real-time and historical price data collection - MarketDataService with technical analysis calculations
- Technical analysis calculation engine - MarketData entities (Price, OHLCV, TechnicalIndicators)
- Migration scripts for moving existing data - Dagster job for automated twice-daily collection
- pytest-vcr tests for API clients
- Performance benchmarks for time-series queries - Performance benchmarks for time-series queries
#### Success Criteria #### Success Criteria
- ✅ Market data efficiently stored in TimescaleDB hypertables - ✅ TimescaleDB hypertables storing historical price data
- ✅ Sub-100ms queries for common market data retrievals - ✅ Sub-100ms queries for price lookups and indicators
- ✅ All technical indicators calculating accurately - ✅ Technical indicators calculating accurately
- ✅ Dagster job running twice daily (market open/close)
- ✅ Complete migration from file-based storage - ✅ Complete migration from file-based storage
- ✅ Market data domain ready for agent integration - ✅ Market data domain ready for agent integration
### Phase 3: Social Media Domain (Following Phase 2) ### Phase 3: Social Media Domain + Dagster Integration
**Timeline**: 3-4 weeks **Timeline**: 2-3 weeks
**Status**: 📋 Planned **Status**: 📋 Planned
#### Core Objectives #### Core Objectives
- **Reddit Integration**: Implement Reddit API for financial subreddits - **Reddit Integration**: PRAW library for financial subreddits (r/wallstreetbets, r/stocks)
- **Twitter/X Integration**: Add social sentiment from Twitter feeds - **Twitter/X Alternative**: Evaluate Reddit-only approach or alternative sources
- **Social Sentiment Analysis**: Aggregate sentiment scoring across platforms - **Social Sentiment Analysis**: OpenRouter LLM sentiment across posts
- **Cross-Domain Relations**: Link social sentiment to market data and news - **Cross-Domain Relations**: Link social sentiment to market data and news
- **pgvectorscale Preparation**: Prepare social data for vector search - **Dagster Social Media Job**: Daily social sentiment collection
- **Vector Embeddings**: Semantic search across social discussions
#### Key Deliverables #### Key Deliverables
- Reddit and Twitter data collection clients - RedditClient with pytest-vcr tests
- Social sentiment aggregation algorithms - SocialMediaRepository with PostgreSQL + pgvectorscale
- Social media data repository with PostgreSQL storage - SocialMediaService with sentiment aggregation
- Cross-domain correlation analysis tools - Dagster job for daily Reddit data collection
- Foundation for RAG implementation - Cross-domain correlation queries (social ↔ news ↔ price)
- Vector embeddings for semantic post search
#### Success Criteria #### Success Criteria
- ✅ Social media data collected from multiple sources - ✅ Reddit data collected daily from financial subreddits
- ✅ Sentiment scores integrated with market events - ✅ Sentiment scores integrated with market events
- ✅ Cross-domain relationships established in database - ✅ Cross-domain relationships queryable in database
- ✅ Social media domain ready for RAG enhancement - ✅ Dagster job running daily social collection
- ✅ Vector embeddings enabling semantic social search
- ✅ Three-domain architecture complete - ✅ Three-domain architecture complete
### Phase 4: Dagster Data Collection Orchestration #### Blockers to Resolve
**Timeline**: 3-4 weeks - **Reddit API Access**: Obtain REDDIT_CLIENT_ID and REDDIT_CLIENT_SECRET
- **Twitter/X Alternative**: Evaluate API costs or alternative data sources
### Phase 4: RAG Enhancement + Advanced Orchestration
**Timeline**: 3-4 weeks
**Status**: 📋 Planned **Status**: 📋 Planned
#### Core Objectives #### Core Objectives
- **Pipeline Architecture**: Design daily/twice-daily data collection workflows - **RAG Agent Enhancement**: All agents use vector similarity search for context
- **Data Quality Monitoring**: Implement validation and gap detection - **Historical Pattern Matching**: Semantic search for comparable market scenarios
- **Automated Backfill**: Handle missing data and API failures gracefully - **Cross-Domain RAG**: Agents query across news, price, and social data
- **Performance Monitoring**: Track pipeline health and data freshness - **Advanced Dagster Features**: Data quality monitoring, gap detection, backfill
- **Alerting System**: Notify on pipeline failures or data quality issues - **Performance Optimization**: Vector query tuning, database optimization
- **Monitoring & Alerting**: Pipeline health tracking and failure notifications
#### Key Deliverables #### Key Deliverables
- Dagster asset definitions for all data domains - RAG-enhanced agents with similarity-based context retrieval
- Automated data quality checks and validation - Cross-domain vector search (find similar market conditions)
- Gap detection and backfill capabilities - Dagster data quality checks and validation
- Automated backfill for missing historical data
- Monitoring dashboard for pipeline health - Monitoring dashboard for pipeline health
- Comprehensive logging and error handling - Performance benchmarks for vector queries (< 50ms target)
#### Success Criteria
- ✅ Fully automated data collection running daily
- ✅ Data quality monitoring with automated alerts
- ✅ Zero-downtime pipeline updates and maintenance
- ✅ Historical data gaps automatically detected and filled
- ✅ Pipeline performance metrics tracked and optimized
### Phase 5: RAG Implementation + OpenRouter Migration
**Timeline**: 4-5 weeks
**Status**: 📋 Planned
#### Core Objectives
- **pgvectorscale Integration**: Implement vector storage for historical patterns
- **RAG Agent Enhancement**: Agents use similarity search for context
- **OpenRouter Migration**: Complete migration to unified LLM provider
- **Historical Context**: Agents reference past decisions and market conditions
- **Pattern Recognition**: Semantic similarity for comparable market scenarios
#### Key Deliverables
- pgvectorscale extension configured and optimized
- Vector embeddings for all historical data
- RAG-enhanced agent decision making
- OpenRouter integration replacing all LLM providers
- Similarity search for historical pattern matching
#### Success Criteria #### Success Criteria
- ✅ All agents using RAG for contextual decisions - ✅ All agents using RAG for contextual decisions
- ✅ Vector search performing sub-50ms similarity queries - ✅ Vector similarity search < 50ms across all domains
- ✅ OpenRouter as sole LLM provider across all agents - ✅ Cross-domain queries enabling holistic analysis
- ✅ Agents demonstrating improved decision accuracy - ✅ Dagster monitoring with automated alerts
- ✅ Historical pattern matching enhancing trading analysis - ✅ Data quality metrics tracked and reported
- ✅ Historical gaps detected and auto-filled
- ✅ Production-ready data infrastructure complete
## Technical Milestones ## Technical Milestones
### Revised Timeline: 10-14 weeks (vs original 16-22 weeks)
**Phase Breakdown:**
- Phase 1 (News + Dagster): 5-7 days
- Phase 2 (Market Data + Dagster): 4-5 weeks
- Phase 3 (Social Media + Dagster): 2-3 weeks
- Phase 4 (RAG + Advanced Orchestration): 3-4 weeks
### Database Architecture ### Database Architecture
- **Month 1**: Complete PostgreSQL foundation with news domain - **Week 1**: PostgreSQL + TimescaleDB + pgvectorscale operational (News domain)
- **Month 2**: TimescaleDB hypertables optimized for market data - **Week 6**: TimescaleDB hypertables optimized for market data time-series
- **Month 3**: pgvectorscale configured for RAG implementation - **Week 9**: Three-domain database architecture complete with vector embeddings
- **Month 4**: Full database optimization and performance tuning - **Week 12**: Full RAG implementation with cross-domain similarity search
### Agent Capabilities ### Agent Capabilities
- **Month 1**: Basic multi-agent framework operational - **Week 1**: News Analysts accessing news with LLM sentiment
- **Month 2**: Agents using PostgreSQL for all data access - **Week 6**: Technical Analysts using market data with indicators
- **Month 3**: Cross-domain agent collaboration established - **Week 9**: Sentiment Analysts using social media data
- **Month 4**: RAG-powered agents with historical context - **Week 12**: All agents RAG-enhanced with historical context
### Data Pipeline Maturity ### Data Pipeline Maturity (Incremental Dagster)
- **Month 1**: Manual data collection with basic automation - **Week 1**: Daily news collection automated via Dagster
- **Month 2**: Automated collection for market data - **Week 6**: Twice-daily market data collection automated
- **Month 3**: Full three-domain automated collection - **Week 9**: Daily social media collection automated
- **Month 4**: Production-grade pipeline with monitoring and alerting - **Week 12**: Production-grade orchestration with monitoring, backfill, and alerting
## Success Metrics ## Success Metrics

File diff suppressed because it is too large Load Diff

View File

@ -10,12 +10,12 @@ Complete final 5% of news domain: add scheduled execution, LLM sentiment analysi
### 1. Scheduled Execution ### 1. Scheduled Execution
- Daily job at 6 AM UTC for all configured tickers - Daily job at 6 AM UTC for all configured tickers
- APScheduler integration (no Dagster dependency) - Dagster orchestration with partitioned schedules
- Graceful error handling with comprehensive logging - Graceful error handling with Dagster sensors and alerting
### 2. LLM Sentiment Analysis ### 2. LLM Sentiment Analysis
- OpenRouter integration using `quick_think_llm` (claude-3.5-haiku) - OpenRouter integration using `quick_think_llm` (claude-3.5-haiku)
- Structured output: `{"sentiment": "positive|negative|neutral", "confidence": 0.0-1.0}` - Structured output: `{"sentiment": "positive|negative|neutral", "confidence": 0.0-1.0, "label": "positive|negative|neutral"}`
- Best-effort processing - failures don't stop pipeline - Best-effort processing - failures don't stop pipeline
### 3. Vector Embeddings ### 3. Vector Embeddings
@ -27,54 +27,75 @@ Complete final 5% of news domain: add scheduled execution, LLM sentiment analysi
### Architecture Pattern ### Architecture Pattern
``` ```
ScheduledNewsJob → NewsService → NewsRepository → NewsArticle → PostgreSQL+pgvectorscale Dagster Job → Dagster Op → NewsService → NewsRepository → NewsArticle → PostgreSQL+pgvectorscale
``` ```
### Database Changes ### Database Changes
```sql ```sql
ALTER TABLE news_articles ALTER TABLE news_articles
ADD COLUMN sentiment_score JSONB, ADD COLUMN sentiment_confidence FLOAT,
ADD COLUMN title_embedding vector(1536), ADD COLUMN sentiment_label VARCHAR(20);
ADD COLUMN content_embedding vector(1536);
-- Vector columns already exist from 95% complete infrastructure
-- title_embedding vector(1536)
-- content_embedding vector(1536)
``` ```
### Key Integration Points ### Key Integration Points
- **Existing NewsService**: Enhance `update_news_for_symbol` method - **Existing NewsService**: Enhance `update_company_news` method
- **LLM Integration**: OpenRouter unified provider for sentiment - **LLM Integration**: OpenRouter unified provider for sentiment and embeddings
- **Vector Generation**: text-embedding-3-small model (1536 dims) - **Vector Generation**: OpenAI text-embedding-ada-002 via OpenRouter (1536 dims)
- **Job Scheduling**: APScheduler with cron trigger - **Job Scheduling**: Dagster jobs with daily partitioned schedules
## Implementation Phases ## Implementation Phases
1. **Scheduled Execution** (2-3h): APScheduler + config management 1. **Entity Layer** (2-3h): Enhance NewsArticle dataclass + migration
2. **LLM Sentiment** (3-4h): OpenRouter integration + structured prompts 2. **Repository Layer** (2-3h): RAG vector similarity search methods
3. **Vector Embeddings** (2-3h): Embedding generation + database schema 3. **LLM Integration** (4-5h): OpenRouter sentiment + embeddings clients
4. **Testing & Monitoring** (2h): Coverage + performance validation 4. **Service Enhancement** (2-3h): Integrate LLM clients into NewsService
5. **Dagster Orchestration** (3-4h): Jobs, ops, and schedules
6. **Testing & Monitoring** (2-3h): Coverage + performance validation
**Total: 9-12 hours** **Total: 15-20 hours**
## Success Criteria ## Success Criteria
- ✅ Daily automated news collection without manual intervention - ✅ Daily automated news collection via Dagster without manual intervention
- ✅ News retrieval with sentiment scores < 2 seconds response time - ✅ News retrieval with sentiment scores < 2 seconds response time
- ✅ Vector embeddings enable semantic search for News Analysts - ✅ Vector embeddings enable semantic search for News Analysts
- ✅ >95% article processing success rate despite paywall/blocking - ✅ >95% article processing success rate despite paywall/blocking
- ✅ Maintain >85% test coverage including new components - ✅ Maintain >85% test coverage including new components
- ✅ Dagster UI provides monitoring and alerting for job failures
## Dependencies ## Dependencies
- **APIs**: OpenRouter (sentiment), OpenAI (embeddings) - **APIs**: OpenRouter (sentiment + embeddings via unified provider)
- **Infrastructure**: PostgreSQL + TimescaleDB + pgvectorscale - **Infrastructure**: PostgreSQL + TimescaleDB + pgvectorscale
- **New Package**: `apscheduler` for job scheduling - **Orchestration**: Dagster for job scheduling and monitoring
- **Existing**: 95% complete news domain components - **Existing**: 95% complete news domain components (clients, repository, service)
## Configuration ## Configuration
```yaml
# Dagster workspace.yaml
schedules:
news_collection_daily:
cron_schedule: "0 6 * * *" # Daily at 6 AM UTC
execution_timezone: "UTC"
# Dagster run config
ops:
collect_news:
config:
symbols: ["AAPL", "GOOGL", "MSFT", "TSLA"]
lookback_days: 1
```
```bash ```bash
OPENROUTER_API_KEY="sk-or-..." # Environment variables
OPENAI_API_KEY="sk-..." OPENROUTER_API_KEY="sk-or-..." # Unified LLM provider
NEWS_SCHEDULE_HOUR=6 DATABASE_URL="postgresql+asyncpg://..."
NEWS_TICKERS="AAPL,GOOGL,MSFT,TSLA"
``` ```
## Risk Mitigation ## Risk Mitigation
- **API Rate Limits**: Exponential backoff + batch processing - **API Rate Limits**: Exponential backoff + batch processing
- **Paywall Blocking**: Metadata-only storage with warnings - **Paywall Blocking**: Metadata-only storage with warnings
- **Job Failures**: Monitoring + alerting for operational visibility - **Job Failures**: Dagster sensors + alerting for operational visibility
- **Performance**: Vector indexes + query optimization for <2s target - **Performance**: Vector indexes + query optimization for <2s target
- **LLM Failures**: Keyword-based fallback for sentiment, zero-vector fallback for embeddings

View File

@ -99,10 +99,10 @@ Complete the final 5% of the news domain by adding scheduled execution, LLM sent
### Architecture Alignment ### Architecture Alignment
Follows established **Router → Service → Repository → Entity → Database** pattern: Follows established **Router → Service → Repository → Entity → Database** pattern with Dagster orchestration:
``` ```
ScheduledNewsJob → NewsService → NewsRepository → NewsArticle → PostgreSQL+pgvectorscale Dagster Schedule → Dagster Job → NewsService → NewsRepository → NewsArticle → PostgreSQL+pgvectorscale
``` ```
### Database Schema Integration ### Database Schema Integration
@ -156,29 +156,70 @@ content_embedding = await embedding_client.create_embedding(
### Scheduled Execution Framework ### Scheduled Execution Framework
Use APScheduler for job orchestration (Dagster not in current dependencies): Use Dagster for job orchestration (existing dependency in project):
```python ```python
from apscheduler.schedulers.asyncio import AsyncIOScheduler from dagster import (
job,
scheduler = AsyncIOScheduler() schedule,
scheduler.add_job( ScheduleDefinition,
run_news_collection, op,
'cron', In,
hour=6, # 6 AM UTC Out,
minute=0, AssetMaterialization
timezone=timezone.utc,
id='daily_news_collection'
) )
from dagster._core.scheduler import ScheduleExecutionContext
@op
def fetch_news_for_tickers(context, tickers: list[str]) -> list[dict]:
"""Fetch news articles for configured tickers"""
pass
@op
def process_articles_with_sentiment(context, articles: list[dict]) -> list[dict]:
"""Process articles with LLM sentiment analysis and embeddings"""
pass
@op
def store_articles(context, processed_articles: list[dict]) -> None:
"""Store articles with sentiment and embeddings in database"""
pass
@job
def daily_news_collection_job():
"""Daily news collection pipeline"""
tickers = ["AAPL", "GOOGL", "MSFT", "TSLA"] # From config
articles = fetch_news_for_tickers(tickers)
processed = process_articles_with_sentiment(articles)
store_articles(processed)
@schedule(
cron_schedule="0 6 * * *", # Daily at 6 AM UTC
job=daily_news_collection_job,
execution_timezone="UTC"
)
def daily_news_collection_schedule(context: ScheduleExecutionContext):
"""Schedule for daily news collection"""
run_config = {
"ops": {
"fetch_news_for_tickers": {
"inputs": {
"tickers": ["AAPL", "GOOGL", "MSFT", "TSLA"]
}
}
}
}
return run_config
``` ```
## Implementation Approach ## Implementation Approach
### Phase 1: Scheduled Execution (2-3 hours) ### Phase 1: Dagster Scheduling Integration (2-3 hours)
1. Configure APScheduler for daily news collection 1. Create Dagster ops for news collection pipeline
2. Create job configuration management for ticker lists 2. Configure daily schedule with cron expression
3. Implement job monitoring and status tracking 3. Set up job configuration management for ticker lists
4. Add manual execution capability for testing 4. Add manual job execution capability for testing
5. Implement job monitoring and asset tracking
### Phase 2: LLM Sentiment Integration (3-4 hours) ### Phase 2: LLM Sentiment Integration (3-4 hours)
1. Integrate OpenRouter LLM for sentiment analysis 1. Integrate OpenRouter LLM for sentiment analysis
@ -218,11 +259,12 @@ scheduler.add_job(
- `NewsRepository` with async PostgreSQL operations - `NewsRepository` with async PostgreSQL operations
- `NewsArticle` domain model with validation - `NewsArticle` domain model with validation
- Comprehensive test coverage with pytest-vcr - Comprehensive test coverage with pytest-vcr
- Dagster framework for data orchestration (existing dependency)
### New Dependencies ### New Dependencies
- `apscheduler` for job scheduling
- Enhanced vector embedding capabilities - Enhanced vector embedding capabilities
- LLM client integration for sentiment analysis - LLM client integration for sentiment analysis
- Dagster scheduling integration (existing dependency)
## Configuration Management ## Configuration Management

View File

@ -1,336 +1,310 @@
# News Domain Completion - Progress Status 1→# News Domain Completion - Implementation Status
2→
## Overview 3→**Last Updated**: 2025-01-11
4→**Overall Progress**: 6.67% (1/15 tasks completed)
**Feature**: News Domain Final 5% Completion 5→**Architecture**: Dagster orchestration + OpenRouter LLM + RAG vector search
**Status**: Ready for Implementation 6→
**Total Estimated Time**: 12-16 hours with AI assistance 7→---
**Target Timeline**: 3-4 days 8→
**Current Progress**: 95% complete (infrastructure ready) 9→## Current Phase
10→
--- 11→**Phase 1: Entity Layer**
12→Status: In Progress
## Progress Summary 13→Progress: 50% (1/2 tasks completed)
14→Estimated Time Remaining: 1-2 hours
### Overall Completion: 0% (95% + 0% of final 5%) 15→
16→---
| Phase | Status | Progress | Duration | Completion | 17→
|-------|--------|----------|----------|------------| 18→## Task Status Summary
| Phase 1: Foundation | ⏳ Not Started | 0/3 tasks | 0h/4-7h | ⬜⬜⬜⬜⬜⬜⬜ | 19→
| Phase 2: Data Access | ⏳ Not Started | 0/1 tasks | 0h/2-3h | ⬜⬜⬜ | 20→### Phase 1: Entity Layer (1/2 completed)
| Phase 3: LLM Integration | ⏳ Not Started | 0/3 tasks | 0h/5-8h | ⬜⬜⬜⬜⬜⬜⬜⬜ | 21→
| Phase 4: Scheduling | ⏳ Not Started | 0/2 tasks | 0h/4-6h | ⬜⬜⬜⬜⬜⬜ | 22→| Task | Status | Priority | Time | Assigned | Completion | Completed At |
| Phase 5: Validation | ⏳ Not Started | 0/2 tasks | 0h/3-5h | ⬜⬜⬜⬜⬜ | 23→|------|--------|----------|------|----------|------------|--------------|
24→| T001: Enhance NewsArticle Dataclass | ✅ Completed | Critical | 1-2h | - | 100% | 2025-01-11 |
**Legend**: ✅ Complete | 🟡 In Progress | ⏳ Not Started | ❌ Blocked 25→| T002: Database Migration - Sentiment Fields | ⬜ Not Started | Critical | 1h | - | 0% | - |
26→
--- 27→### Phase 2: Repository Layer (0/2 completed)
28→
## Task Status Tracking 29→| Task | Status | Priority | Time | Assigned | Completion |
30→|------|--------|----------|------|----------|------------|
### Phase 1: Foundation (0% Complete) 31→| T003: NewsRepository - Vector Similarity Search | ⬜ Not Started | Critical | 2-3h | - | 0% |
32→| T004: NewsRepository - Batch Embedding Updates | ⬜ Not Started | Medium | 1h | - | 0% |
#### ⏳ T001: Database Migration - NewsJobConfig Table 33→
- **Status**: Not Started 34→### Phase 3: LLM Integration (0/3 completed)
- **Priority**: Critical 35→
- **Estimated**: 1-2 hours 36→| Task | Status | Priority | Time | Assigned | Completion |
- **Dependencies**: None 37→|------|--------|----------|------|----------|------------|
- **Progress**: 0% 38→| T005: OpenRouter Sentiment Client | ⬜ Not Started | Critical | 2-3h | - | 0% |
- **Acceptance Criteria**: 0/4 completed 39→| T006: OpenRouter Embeddings Client | ⬜ Not Started | Critical | 1-2h | - | 0% |
- [ ] `news_job_configs` table created with UUID primary key 40→| T007: Enhance NewsService - LLM Integration | ⬜ Not Started | Critical | 2-3h | - | 0% |
- [ ] JSONB fields for symbols and categories with validation 41→
- [ ] Proper indexes for enabled/frequency queries 42→### Phase 4: Dagster Orchestration (0/5 completed)
- [ ] Migration script tests with rollback capability 43→
- **Blocking Issues**: None 44→| Task | Status | Priority | Time | Assigned | Completion |
- **Next Actions**: Create Alembic migration script 45→|------|--------|----------|------|----------|------------|
46→| T008: Dagster Directory Structure | ⬜ Not Started | High | 30min | - | 0% |
#### ⏳ T002: Enhance NewsArticle Entity - Sentiment and Embeddings 47→| T009: Dagster Ops - News Collection | ⬜ Not Started | High | 2-3h | - | 0% |
- **Status**: Not Started 48→| T010: Dagster Job - Daily News Collection | ⬜ Not Started | High | 1-2h | - | 0% |
- **Priority**: Critical 49→| T011: Dagster Schedule - Daily Trigger | ⬜ Not Started | High | 1h | - | 0% |
- **Estimated**: 2-3 hours 50→| T012: Dagster Sensor - Failure Alerting | ⬜ Not Started | Medium | 1h | - | 0% |
- **Dependencies**: T001 51→
- **Progress**: 0% 52→### Phase 5: Testing & Documentation (0/3 completed)
- **Acceptance Criteria**: 0/5 completed 53→
- [ ] Add sentiment_score, sentiment_confidence, sentiment_label fields 54→| Task | Status | Priority | Time | Assigned | Completion |
- [ ] Add title_embedding and content_embedding vector fields 55→|------|--------|----------|------|----------|------------|
- [ ] Enhanced validate() method with sentiment range checks 56→| T013: Integration Tests - End-to-End Workflow | ⬜ Not Started | High | 2-3h | - | 0% |
- [ ] Updated transformations for vector handling 57→| T014: Dagster Tests | ⬜ Not Started | Medium | 1h | - | 0% |
- [ ] Embedding dimension validation (1536) 58→| T015: Documentation Updates | ⬜ Not Started | Medium | 1-2h | - | 0% |
- **Blocking Issues**: None 59→
- **Next Actions**: Extend NewsArticle dataclass 60→---
61→
#### ⏳ T003: Create NewsJobConfig Entity 62→## Dependency Graph
- **Status**: Not Started 63→
- **Priority**: Critical 64→```
- **Estimated**: 1-2 hours 65→T001 ─┬─→ T002 ──→ T003 ─────────→ T007 ──→ T009 ──→ T010 ──→ T013
- **Dependencies**: T001 66→ │ ↑ ↑ ↑ ↑
- **Progress**: 0% 67→ │ │ │ │ │
- **Acceptance Criteria**: 0/5 completed 68→ └──→ T005 ────────────────────┘ │ │ │
- [ ] NewsJobConfig dataclass with all required fields 69→ T006 ──────────────────────────────┘ │ │
- [ ] Business rule validation for job configuration 70→ T008 ──────────────────────────────────────┘ │
- [ ] Cron expression validation for frequency 71→ T011 ───────────────────────────────────────────────┘
- [ ] Symbol list validation 72→ T014 ───────────────────────────────────────────────┘
- [ ] JSON serialization for database storage 73→```
- **Blocking Issues**: None 74→
- **Next Actions**: Create new entity file 75→**Critical Path**: T001 → T002 → T003 → T007 → T009 → T010 → T013
76→
### Phase 2: Data Access (0% Complete) 77→**Parallel Opportunities**:
78→- T005 & T006 can be developed in parallel (LLM clients)
#### ⏳ T004: Enhance NewsRepository - Vector and Job Operations 79→- T009, T010, T011 can be developed in parallel after T008 (Dagster components)
- **Status**: Not Started 80→
- **Priority**: Critical 81→---
- **Estimated**: 2-3 hours 82→
- **Dependencies**: T002, T003 83→## Progress by Phase
- **Progress**: 0% 84→
- **Acceptance Criteria**: 0/5 completed 85→### Phase 1: Entity Layer
- [ ] Vector similarity search with cosine distance 86→- **Status**: In Progress
- [ ] Batch embedding update operations 87→- **Progress**: 50% (1/2 tasks)
- [ ] NewsJobConfig CRUD methods 88→- **Estimated Time**: 1-2 hours
- [ ] Optimized query performance for vector operations 89→- **Blockers**: None
- [ ] Proper async connection handling 90→- **Next Action**: Start T002 - Database Migration for Sentiment Fields
- **Blocking Issues**: Waiting for T002, T003 91→
- **Next Actions**: Extend NewsRepository class 92→### Phase 2: Repository Layer
93→- **Status**: Not Started
### Phase 3: LLM Integration (0% Complete) 94→- **Progress**: 0% (0/2 tasks)
95→- **Estimated Time**: 2-3 hours
#### ⏳ T005: OpenRouter Client - Sentiment Analysis 96→- **Blockers**: T001, T002 must complete first
- **Status**: Not Started 97→- **Next Action**: Waiting for Phase 1 completion
- **Priority**: Critical 98→
- **Estimated**: 2-3 hours 99→### Phase 3: LLM Integration
- **Dependencies**: T002 100→- **Status**: Not Started
- **Progress**: 0% 101→- **Progress**: 0% (0/3 tasks)
- **Acceptance Criteria**: 0/5 completed 102→- **Estimated Time**: 4-5 hours
- [ ] OpenRouter API integration for sentiment analysis 103→- **Blockers**: T001 must complete for client development
- [ ] Structured prompts for financial news sentiment 104→- **Next Action**: Can start T005 & T006 in parallel after T001
- [ ] Response parsing with Pydantic models 105→
- [ ] Error handling with graceful fallbacks 106→### Phase 4: Dagster Orchestration
- [ ] Retry logic with exponential backoff 107→- **Status**: Not Started
- **Blocking Issues**: Waiting for T002 108→- **Progress**: 0% (0/5 tasks)
- **Next Actions**: Create OpenRouter sentiment client 109→- **Estimated Time**: 3-4 hours
110→- **Blockers**: T007 must complete for ops/jobs, T008 has no dependencies
#### ⏳ T006: OpenRouter Client - Vector Embeddings 111→- **Next Action**: Can start T008 anytime (directory structure)
- **Status**: Not Started 112→
- **Priority**: Critical 113→### Phase 5: Testing & Documentation
- **Estimated**: 1-2 hours 114→- **Status**: Not Started
- **Dependencies**: T002 115→- **Progress**: 0% (0/3 tasks)
- **Progress**: 0% 116→- **Estimated Time**: 2-3 hours
- **Acceptance Criteria**: 0/5 completed 117→- **Blockers**: T007, T010 must complete for integration testing
- [ ] OpenRouter embeddings API integration 118→- **Next Action**: Waiting for earlier phases
- [ ] Text preprocessing for embedding generation 119→
- [ ] Batch processing for multiple articles 120→---
- [ ] 1536-dimensional vector validation 121→
- [ ] Proper error handling and retries 122→## Test Coverage Status
- **Blocking Issues**: Waiting for T002 123→
- **Next Actions**: Create OpenRouter embeddings client 124→**Current Coverage**: Baseline (from 95% complete infrastructure)
125→**Target Coverage**: ≥85%
#### ⏳ T007: Enhance NewsService - LLM Integration 126→**New Code Coverage**: 0% (no new code yet)
- **Status**: Not Started 127→
- **Priority**: Critical 128→### Coverage by Component
- **Estimated**: 2-3 hours 129→
- **Dependencies**: T005, T006 130→| Component | Coverage | Target | Status |
- **Progress**: 0% 131→|-----------|----------|--------|--------|
- **Acceptance Criteria**: 0/5 completed 132→| NewsArticle (Entity) | - | ≥85% | ⬜ Pending |
- [ ] Replace keyword sentiment with LLM analysis 133→| NewsRepository (RAG) | - | ≥85% | ⬜ Pending |
- [ ] Add embedding generation to article processing 134→| OpenRouter Sentiment Client | - | ≥85% | ⬜ Pending |
- [ ] End-to-end article processing pipeline 135→| OpenRouter Embeddings Client | - | ≥85% | ⬜ Pending |
- [ ] Proper error handling and fallback strategies 136→| NewsService (LLM Integration) | - | ≥85% | ⬜ Pending |
- [ ] Integration with existing service methods 137→| Dagster Ops | - | ≥85% | ⬜ Pending |
- **Blocking Issues**: Waiting for T005, T006 138→| Dagster Jobs | - | ≥85% | ⬜ Pending |
- **Next Actions**: Integrate LLM clients into NewsService 139→
140→---
### Phase 4: Scheduling (0% Complete) 141→
142→## Performance Benchmarks
#### ⏳ T008: APScheduler Integration - Job Scheduling 143→
- **Status**: Not Started 144→### Current Performance
- **Priority**: High 145→- **Query Time (30-day lookback)**: Not measured yet
- **Estimated**: 3-4 hours 146→- **Vector Search (top-10)**: Not measured yet
- **Dependencies**: T003, T004, T007 147→- **Batch Insert (50 articles)**: Not measured yet
- **Progress**: 0% 148→
- **Acceptance Criteria**: 0/5 completed 149→### Target Performance
- [ ] APScheduler setup with PostgreSQL job store 150→- **Query Time**: < 2 seconds for 30-day lookback
- [ ] Scheduled job execution with proper error handling 151→- **Vector Search**: < 1 second for top-10 results
- [ ] Job configuration loading and validation 152→- **Batch Insert**: < 5 seconds for 50 articles
- [ ] Status monitoring and failure recovery 153→
- [ ] CLI integration for job management 154→### Performance Test Status
- **Blocking Issues**: Waiting for T003, T004, T007 155→- [ ] Query performance baseline established
- **Next Actions**: Implement ScheduledNewsCollector 156→- [ ] Vector search performance baseline established
157→- [ ] Batch insert performance baseline established
#### ⏳ T009: CLI Integration - Job Management Commands 158→- [ ] All performance targets met
- **Status**: Not Started 159→
- **Priority**: Medium 160→---
- **Estimated**: 1-2 hours 161→
- **Dependencies**: T008 162→## Risk Assessment
- **Progress**: 0% 163→
- **Acceptance Criteria**: 0/5 completed 164→### High Risk Items
- [ ] CLI commands for job creation/management 165→1. **OpenRouter API Availability** - Mitigated with fallback strategies (keyword sentiment, zero vectors)
- [ ] Manual job execution commands 166→2. **Vector Search Performance** - Mitigated with proper pgvectorscale indexes
- [ ] Job status and monitoring commands 167→3. **Dagster Integration Complexity** - Mitigated with incremental testing approach
- [ ] Integration with existing CLI structure 168→
- [ ] Proper error handling and user feedback 169→### Medium Risk Items
- **Blocking Issues**: Waiting for T008 170→1. **LLM API Costs** - Monitor usage during development
- **Next Actions**: Extend CLI with news job commands 171→2. **Database Performance at Scale** - Test with realistic data volumes
172→3. **Test Coverage Maintenance** - Enforce ≥85% coverage requirement
### Phase 5: Validation (0% Complete) 173→
174→### Low Risk Items
#### ⏳ T010: Integration Tests - End-to-End Workflow 175→1. **Code Quality** - Enforced through TDD approach
- **Status**: Not Started 176→2. **Documentation** - Tracked as explicit task (T015)
- **Priority**: High 177→3. **Error Handling** - Comprehensive fallback strategies
- **Estimated**: 2-3 hours 178→
- **Dependencies**: T007, T008 179→---
- **Progress**: 0% 180→
- **Acceptance Criteria**: 0/5 completed 181→## Known Issues
- [ ] End-to-end workflow tests from RSS to vector storage 182→
- [ ] Agent integration tests via AgentToolkit 183→### Blocking Issues
- [ ] Performance tests for daily collection volumes 184→None currently
- [ ] Error recovery and fallback tests 185→
- [ ] Test coverage maintained above 85% 186→### Non-Blocking Issues
- **Blocking Issues**: Waiting for T007, T008 187→None currently
- **Next Actions**: Create comprehensive integration test suite 188→
189→### Technical Debt
#### ⏳ T011: Documentation and Monitoring 190→- Existing keyword-based sentiment analysis should be replaced with LLM sentiment (tracked as T005)
- **Status**: Not Started 191→- No automated vector embedding generation currently (tracked as T006)
- **Priority**: Medium 192→- No scheduled news collection (tracked as T008-T012)
- **Estimated**: 1-2 hours 193→
- **Dependencies**: T010 194→---
- **Progress**: 0% 195→
- **Acceptance Criteria**: 0/5 completed 196→## Milestone Schedule
- [ ] Updated API documentation for new methods 197→
- [ ] Job scheduling configuration examples 198→### Milestone 1: Entity & Repository Foundation
- [ ] Performance monitoring dashboard queries 199→**Target**: Day 1-2
- [ ] Troubleshooting guide for common issues 200→**Tasks**: T001, T002, T003, T004
- [ ] Agent integration documentation 201→**Status**: In Progress
- **Blocking Issues**: Waiting for T010 202→**Deliverables**:
- **Next Actions**: Update documentation and monitoring 203→- NewsArticle dataclass with sentiment fields
204→- Database migration for sentiment columns
--- 205→- RAG vector similarity search functional
206→- Batch embedding updates operational
## Success Criteria Validation 207→
208→### Milestone 2: LLM Integration
### Technical Requirements Status 209→**Target**: Day 2-3
- [ ] **OpenRouter-only LLM Integration**: Not started 210→**Tasks**: T005, T006, T007
- [ ] **Vector Embeddings with pgvectorscale**: Not started 211→**Status**: Not Started
- [ ] **APScheduler Job Execution**: Not started 212→**Deliverables**:
- [ ] **Test Coverage >85%**: Baseline established (needs monitoring) 213→- OpenRouter sentiment client operational with fallbacks
- [ ] **Query Performance <100ms**: Not tested 214→- OpenRouter embeddings client operational with fallbacks
- [ ] **Vector Search Performance <1s**: Not tested 215→- NewsService enrichment pipeline functional
- [ ] **Backward Compatibility**: Not validated 216→- find_similar_news() RAG method operational
217→
### Functional Requirements Status 218→### Milestone 3: Dagster Orchestration
- [ ] **Sentiment Analysis Pipeline**: Not implemented 219→**Target**: Day 3-4
- [ ] **Embedding Generation Pipeline**: Not implemented 220→**Tasks**: T008, T009, T010, T011, T012
- [ ] **Scheduled News Collection**: Not implemented 221→**Status**: Not Started
- [ ] **CLI Job Management**: Not implemented 222→**Deliverables**:
- [ ] **AgentToolkit Integration**: Not validated 223→- Dagster directory structure created
- [ ] **Error Handling & Fallbacks**: Not implemented 224→- News collection op functional
225→- Daily collection job operational
### Quality Requirements Status 226→- Schedule configured for 6 AM UTC
- [ ] **TDD Implementation**: Process defined, not applied 227→- Failure sensor monitoring job
- [ ] **Layered Architecture**: Pattern defined, not validated 228→
- [ ] **Async Connection Pooling**: Not implemented 229→### Milestone 4: Testing & Documentation
- [ ] **Production Monitoring**: Not implemented 230→**Target**: Day 4-5
- [ ] **Documentation Completeness**: Not updated 231→**Tasks**: T013, T014, T015
232→**Status**: Not Started
--- 233→**Deliverables**:
234→- End-to-end integration tests passing
## Current Blocking Issues 235→- Dagster component tests passing
236→- Performance benchmarks met
### Critical Blockers 237→- Documentation updated
**None currently** - All dependencies are internal to this implementation 238→
239→---
### Potential Risk Areas 240→
1. **OpenRouter API Access**: Requires valid API keys and model access 241→## Next Actions
2. **Database Migration**: Need proper PostgreSQL permissions for schema changes 242→
3. **Vector Extension**: pgvectorscale must be properly installed and configured 243→### Immediate Next Steps (Today)
4. **Performance Testing**: Need realistic data volumes for benchmark validation 244→1. **T002**: Start database migration for sentiment fields
245→2. **T008**: Create Dagster directory structure in parallel (no dependencies)
--- 246→
247→### This Week
## Weekly Progress Targets 248→1. Complete Phase 1 (Entity Layer)
249→2. Start Phase 2 (Repository Layer)
### Week 1 Target (Days 1-2) 250→3. Begin Phase 3 (LLM Integration) in parallel
- **Goal**: Complete Phase 1 & 2 (Foundation + Data Access) 251→
- **Expected Completion**: T001, T002, T003, T004 252→### Next Week
- **Target Progress**: 45% overall completion 253→1. Complete Phase 3 & 4 (LLM + Dagster)
254→2. Complete Phase 5 (Testing & Documentation)
### Week 1 Target (Days 3-4) 255→3. Deploy and monitor Dagster schedules
- **Goal**: Complete Phase 3 & 4 (LLM Integration + Scheduling) 256→
- **Expected Completion**: T005, T006, T007, T008, T009 257→---
- **Target Progress**: 90% overall completion 258→
259→## Team Notes
### Week 2 Target (Day 1) 260→
- **Goal**: Complete Phase 5 (Validation) 261→### Development Environment
- **Expected Completion**: T010, T011 262→- PostgreSQL + TimescaleDB + pgvectorscale running locally
- **Target Progress**: 100% overall completion 263→- OpenRouter API key configured
264→- Dagster installation complete
--- 265→- Python 3.13 with mise/uv
266→
## Metrics Dashboard 267→### Communication
268→- Spec documents updated to reflect Dagster architecture (spec-lite.md, design.md, tasks.md)
### Code Coverage 269→- APScheduler references removed from all specs
- **Current**: 95% (existing infrastructure) 270→- Architecture aligned with project roadmap
- **Target**: >85% (including new functionality) 271→
- **Status**: ⏳ Pending implementation 272→### Resources Needed
273→- OpenRouter API access for development/testing
### Performance Benchmarks 274→- Test database with sample news articles
- **Query Performance**: Not measured (Target: <100ms) 275→- Dagster UI for monitoring during development
- **Vector Search**: Not measured (Target: <1s) 276→
- **Batch Processing**: Not measured (Target: TBD) 277→---
- **Status**: ⏳ Pending implementation 278→
279→## Success Criteria Checklist
### Test Execution 280→
- **Unit Tests**: 0/11 tasks have tests 281→**Technical Success**:
- **Integration Tests**: 0/11 tasks have integration tests 282→- [ ] Test coverage ≥85% maintained
- **VCR Tests**: 0/3 API clients have VCR tests 283→- [ ] Query performance <2s for 30-day lookback
- **Status**: ⏳ Pending implementation 284→- [ ] Vector search <1s for top-10 results
285→- [ ] Zero breaking changes to AgentToolkit
--- 286→- [ ] Dagster jobs execute successfully
287→
## Communication & Reporting 288→**Functional Success**:
289→- [ ] OpenRouter sentiment analysis operational
### Daily Standup Format 290→- [ ] Vector embeddings enable semantic search
``` 291→- [ ] Dagster schedules running daily
Yesterday: [Tasks completed with IDs] 292→- [ ] Agent context enriched with sentiment
Today: [Tasks planned with IDs] 293→
Blockers: [Any issues requiring attention] 294→**Quality Success**:
Help Needed: [Specific areas for collaboration] 295→- [x] 1/15 tasks completed
``` 296→- [ ] All acceptance criteria met
297→- [ ] Comprehensive error handling
### Weekly Status Report Format 298→- [ ] Production-ready monitoring
``` 299→- [ ] Complete documentation
Completed: [Phase progress with task counts] 300→
In Progress: [Current focus areas] 301→---
Upcoming: [Next phase priorities] 302→
Risks: [Technical or timeline concerns] 303→**Status Key**:
Metrics: [Coverage, performance, test results] 304→- ⬜ Not Started
``` 305→- 🔄 In Progress
306→- ✅ Completed
### Milestone Checkpoints 307→- 🚫 Blocked
- **Checkpoint 1** (End of Day 2): Foundation Complete (T001-T004) 308→- ⚠️ At Risk
- **Checkpoint 2** (End of Day 4): LLM Integration Complete (T005-T009) 309→
- **Checkpoint 3** (End of Day 5): Full Implementation Complete (T001-T011) 310→**Last Status Update**: 2025-01-11 - T001 completed, updated progress tracking
---
## Notes
### Implementation Context
- Building on 95% complete news domain infrastructure
- Focus on OpenRouter-only LLM integration (no other providers)
- Maintaining backward compatibility with AgentToolkit
- Following established TDD and layered architecture patterns
### Key Success Factors
1. **Incremental Progress**: Validate each layer before proceeding
2. **Comprehensive Testing**: Maintain test coverage throughout
3. **Performance Monitoring**: Validate benchmarks at each step
4. **Error Resilience**: Implement fallbacks for all LLM dependencies
5. **Documentation**: Keep implementation and usage docs current
### Last Updated
**Date**: 2024-08-30
**By**: System
**Next Review**: Daily during implementation
---
*This status document will be updated as implementation progresses. Use this as a single source of truth for current progress and blocking issues.*

File diff suppressed because it is too large Load Diff

View File

@ -42,6 +42,8 @@ dependencies = [
"alembic>=1.13.0", "alembic>=1.13.0",
"pgvector>=0.4.1", "pgvector>=0.4.1",
"uuid-utils>=0.11.0", "uuid-utils>=0.11.0",
"dagster>=1.8.0",
"dagster-postgres>=0.24.0",
] ]
[project.optional-dependencies] [project.optional-dependencies]
@ -60,6 +62,10 @@ tradingagents = "cli.main:app"
requires = ["setuptools>=61.0", "wheel"] requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta" build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["tradingagents*", "cli*"]
exclude = ["docker*", "assets*", "alembic*", "docs*", "tests*"]
[tool.ruff] [tool.ruff]
line-length = 88 line-length = 88
target-version = "py310" target-version = "py310"

View File

@ -388,6 +388,172 @@ class TestNewsRepository:
assert result == [] assert result == []
class TestNewsArticleSentimentFields:
"""Test suite for new sentiment fields in NewsArticle."""
def test_news_article_with_sentiment_fields(self):
"""Test dataclass instantiation with new sentiment fields."""
# Arrange & Act
article = NewsArticle(
headline="Test Article",
url="https://example.com/test",
source="Test Source",
published_date=date(2024, 1, 15),
sentiment_score=0.8,
sentiment_confidence=0.95,
sentiment_label="positive",
)
# Assert
assert article.sentiment_score == 0.8
assert article.sentiment_confidence == 0.95
assert article.sentiment_label == "positive"
async def test_news_article_to_entity_includes_sentiment_fields(
self, test_db_manager
):
"""Test to_entity() maps new sentiment fields correctly."""
# Arrange
article = NewsArticle(
headline="Test Article",
url="https://example.com/test",
source="Test Source",
published_date=date(2024, 1, 15),
sentiment_score=0.75,
sentiment_confidence=0.88,
sentiment_label="positive",
)
# Act
entity = article.to_entity(symbol="TEST")
# Assert
assert entity.sentiment_score == 0.75
assert entity.sentiment_confidence == 0.88
assert entity.sentiment_label == "positive"
async def test_news_article_from_entity_includes_sentiment_fields(self, repository):
"""Test from_entity() populates new sentiment fields correctly."""
# Arrange - Create an article with sentiment fields
article = NewsArticle(
headline="Test Article",
url="https://example.com/test-from-entity",
source="Test Source",
published_date=date(2024, 1, 15),
sentiment_score=0.65,
sentiment_confidence=0.92,
sentiment_label="negative",
)
# Act - Store and retrieve
await repository.upsert(article, symbol="TEST")
retrieved_articles = await repository.list("TEST", date(2024, 1, 15))
# Assert
assert len(retrieved_articles) == 1
retrieved = retrieved_articles[0]
assert retrieved.sentiment_score == 0.65
assert retrieved.sentiment_confidence == 0.92
assert retrieved.sentiment_label == "negative"
def test_has_reliable_sentiment_with_valid_confidence(self):
"""Test has_reliable_sentiment() returns True when confidence >= 0.6."""
# Arrange
article = NewsArticle(
headline="Test Article",
url="https://example.com/test",
source="Test Source",
published_date=date(2024, 1, 15),
sentiment_score=0.8,
sentiment_confidence=0.6, # Exactly at threshold
)
# Act & Assert
assert article.has_reliable_sentiment() is True
# Test with higher confidence
article.sentiment_confidence = 0.95
assert article.has_reliable_sentiment() is True
def test_has_reliable_sentiment_with_low_confidence(self):
"""Test has_reliable_sentiment() returns False when confidence < 0.6."""
# Arrange
article = NewsArticle(
headline="Test Article",
url="https://example.com/test",
source="Test Source",
published_date=date(2024, 1, 15),
sentiment_score=0.8,
sentiment_confidence=0.59, # Just below threshold
)
# Act & Assert
assert article.has_reliable_sentiment() is False
# Test with very low confidence
article.sentiment_confidence = 0.1
assert article.has_reliable_sentiment() is False
def test_has_reliable_sentiment_with_none_values(self):
"""Test has_reliable_sentiment() returns False when fields are None."""
# Arrange - Article with no sentiment data
article = NewsArticle(
headline="Test Article",
url="https://example.com/test",
source="Test Source",
published_date=date(2024, 1, 15),
)
# Act & Assert
assert article.has_reliable_sentiment() is False
# Test with only sentiment_score
article.sentiment_score = 0.8
assert article.has_reliable_sentiment() is False
# Test with only sentiment_confidence
article.sentiment_score = None
article.sentiment_confidence = 0.9
assert article.has_reliable_sentiment() is False
async def test_news_article_roundtrip_conversion(self, repository):
"""Test to_entity() → from_entity() preserves all fields including new sentiment fields."""
# Arrange - Create article with all fields including new sentiment fields
original = NewsArticle(
headline="Roundtrip Test Article",
url="https://example.com/roundtrip-test",
source="Test Source",
published_date=date(2024, 1, 15),
summary="Test summary",
entities=["Entity1", "Entity2"],
sentiment_score=0.72,
sentiment_confidence=0.87,
sentiment_label="neutral",
author="Test Author",
category="test-category",
)
# Act - Store and retrieve (full roundtrip)
await repository.upsert(original, symbol="TEST")
retrieved_articles = await repository.list("TEST", date(2024, 1, 15))
# Assert - All fields preserved
assert len(retrieved_articles) == 1
retrieved = retrieved_articles[0]
assert retrieved.headline == original.headline
assert retrieved.url == original.url
assert retrieved.source == original.source
assert retrieved.published_date == original.published_date
assert retrieved.summary == original.summary
assert retrieved.entities == original.entities
assert retrieved.sentiment_score == original.sentiment_score
assert retrieved.sentiment_confidence == original.sentiment_confidence
assert retrieved.sentiment_label == original.sentiment_label
assert retrieved.author == original.author
assert retrieved.category == original.category
class TestDatabaseConnectionManagement: class TestDatabaseConnectionManagement:
"""Test database connection and session management.""" """Test database connection and session management."""

View File

@ -46,6 +46,13 @@ class TradingAgentsConfig:
default_lookback_days: int = 30 default_lookback_days: int = 30
default_ta_lookback_days: int = 30 default_ta_lookback_days: int = 30
# Database settings
database_url: str = field(
default_factory=lambda: os.getenv(
"DATABASE_URL", "postgresql://localhost:5432/tradingagents"
)
)
def __post_init__(self): def __post_init__(self):
"""Set computed fields after initialization.""" """Set computed fields after initialization."""
self.data_cache_dir = os.path.join(self.project_dir, "dataflows/data_cache") self.data_cache_dir = os.path.join(self.project_dir, "dataflows/data_cache")
@ -85,6 +92,9 @@ class TradingAgentsConfig:
online_tools=os.getenv("ONLINE_TOOLS", "true").lower() == "true", online_tools=os.getenv("ONLINE_TOOLS", "true").lower() == "true",
default_lookback_days=int(os.getenv("DEFAULT_LOOKBACK_DAYS", "30")), default_lookback_days=int(os.getenv("DEFAULT_LOOKBACK_DAYS", "30")),
default_ta_lookback_days=int(os.getenv("DEFAULT_TA_LOOKBACK_DAYS", "30")), default_ta_lookback_days=int(os.getenv("DEFAULT_TA_LOOKBACK_DAYS", "30")),
database_url=os.getenv(
"DATABASE_URL", "postgresql://localhost:5432/tradingagents"
),
) )
def to_dict(self) -> dict: def to_dict(self) -> dict:
@ -104,6 +114,7 @@ class TradingAgentsConfig:
"online_tools": self.online_tools, "online_tools": self.online_tools,
"default_lookback_days": self.default_lookback_days, "default_lookback_days": self.default_lookback_days,
"default_ta_lookback_days": self.default_ta_lookback_days, "default_ta_lookback_days": self.default_ta_lookback_days,
"database_url": self.database_url,
} }
def copy(self) -> "TradingAgentsConfig": def copy(self) -> "TradingAgentsConfig":
@ -122,6 +133,7 @@ class TradingAgentsConfig:
online_tools=self.online_tools, online_tools=self.online_tools,
default_lookback_days=self.default_lookback_days, default_lookback_days=self.default_lookback_days,
default_ta_lookback_days=self.default_ta_lookback_days, default_ta_lookback_days=self.default_ta_lookback_days,
database_url=self.database_url,
) )

View File

@ -46,6 +46,8 @@ class NewsArticle:
summary: str | None = None summary: str | None = None
entities: list[str] = field(default_factory=list) entities: list[str] = field(default_factory=list)
sentiment_score: float | None = None sentiment_score: float | None = None
sentiment_confidence: float | None = None # New field
sentiment_label: str | None = None # New field
author: str | None = None author: str | None = None
category: str | None = None category: str | None = None
@ -59,6 +61,8 @@ class NewsArticle:
summary=self.summary, summary=self.summary,
entities=self.entities if self.entities else None, entities=self.entities if self.entities else None,
sentiment_score=self.sentiment_score, sentiment_score=self.sentiment_score,
sentiment_confidence=self.sentiment_confidence,
sentiment_label=self.sentiment_label,
author=self.author, author=self.author,
category=self.category, category=self.category,
symbol=symbol, symbol=symbol,
@ -77,10 +81,28 @@ class NewsArticle:
summary=cast("str | None", entity.summary), summary=cast("str | None", entity.summary),
entities=cast("list[str] | None", entity.entities) or [], entities=cast("list[str] | None", entity.entities) or [],
sentiment_score=cast("float | None", entity.sentiment_score), sentiment_score=cast("float | None", entity.sentiment_score),
sentiment_confidence=cast("float | None", entity.sentiment_confidence),
sentiment_label=cast("str | None", entity.sentiment_label),
author=cast("str | None", entity.author), author=cast("str | None", entity.author),
category=cast("str | None", entity.category), category=cast("str | None", entity.category),
) )
def has_reliable_sentiment(self) -> bool:
"""
Check if the article has reliable sentiment data.
Returns True when sentiment_score is not None AND sentiment_confidence is not None
AND sentiment_confidence >= 0.6
Returns:
bool: True if sentiment data is reliable, False otherwise
"""
return bool(
self.sentiment_score is not None
and self.sentiment_confidence is not None
and self.sentiment_confidence >= 0.6
)
class NewsArticleEntity(Base): class NewsArticleEntity(Base):
"""SQLAlchemy model for news articles with vector embedding support.""" """SQLAlchemy model for news articles with vector embedding support."""
@ -113,6 +135,12 @@ class NewsArticleEntity(Base):
JSON, nullable=True JSON, nullable=True
) # Store list[str] as JSON array ) # Store list[str] as JSON array
sentiment_score: Mapped[float | None] = mapped_column(Float, nullable=True) sentiment_score: Mapped[float | None] = mapped_column(Float, nullable=True)
sentiment_confidence: Mapped[float | None] = mapped_column(
Float, nullable=True
) # New field
sentiment_label: Mapped[str | None] = mapped_column(
String(50), nullable=True
) # New field
author: Mapped[str | None] = mapped_column(String(255), nullable=True) author: Mapped[str | None] = mapped_column(String(255), nullable=True)
category: Mapped[str | None] = mapped_column(String(100), nullable=True) category: Mapped[str | None] = mapped_column(String(100), nullable=True)
@ -227,6 +255,8 @@ class NewsRepository:
"summary": article.summary, "summary": article.summary,
"entities": article.entities if article.entities else None, "entities": article.entities if article.entities else None,
"sentiment_score": article.sentiment_score, "sentiment_score": article.sentiment_score,
"sentiment_confidence": article.sentiment_confidence,
"sentiment_label": article.sentiment_label,
"author": article.author, "author": article.author,
"category": article.category, "category": article.category,
"symbol": symbol, "symbol": symbol,
@ -243,6 +273,8 @@ class NewsRepository:
"summary": stmt.excluded.summary, "summary": stmt.excluded.summary,
"entities": stmt.excluded.entities, "entities": stmt.excluded.entities,
"sentiment_score": stmt.excluded.sentiment_score, "sentiment_score": stmt.excluded.sentiment_score,
"sentiment_confidence": stmt.excluded.sentiment_confidence,
"sentiment_label": stmt.excluded.sentiment_label,
"author": stmt.excluded.author, "author": stmt.excluded.author,
"category": stmt.excluded.category, "category": stmt.excluded.category,
"symbol": stmt.excluded.symbol, "symbol": stmt.excluded.symbol,
@ -370,6 +402,8 @@ class NewsRepository:
"summary": article.summary, "summary": article.summary,
"entities": article.entities if article.entities else None, "entities": article.entities if article.entities else None,
"sentiment_score": article.sentiment_score, "sentiment_score": article.sentiment_score,
"sentiment_confidence": article.sentiment_confidence,
"sentiment_label": article.sentiment_label,
"author": article.author, "author": article.author,
"category": article.category, "category": article.category,
"symbol": symbol, "symbol": symbol,
@ -388,6 +422,8 @@ class NewsRepository:
"summary": stmt.excluded.summary, "summary": stmt.excluded.summary,
"entities": stmt.excluded.entities, "entities": stmt.excluded.entities,
"sentiment_score": stmt.excluded.sentiment_score, "sentiment_score": stmt.excluded.sentiment_score,
"sentiment_confidence": stmt.excluded.sentiment_confidence,
"sentiment_label": stmt.excluded.sentiment_label,
"author": stmt.excluded.author, "author": stmt.excluded.author,
"category": stmt.excluded.category, "category": stmt.excluded.category,
"symbol": stmt.excluded.symbol, "symbol": stmt.excluded.symbol,

View File

@ -156,8 +156,10 @@ class DatabaseManager:
def create_test_database_manager() -> DatabaseManager: def create_test_database_manager() -> DatabaseManager:
"""Create a test database manager for tests.""" """Create a test database manager for tests."""
# Use a test database URL with credentials # Use a test database URL with credentials matching docker setup
test_db_url = "postgresql://postgres:postgres@localhost:5432/tradingagents_test" test_db_url = (
"postgresql://postgres:tradingagents@localhost:5432/tradingagents_test"
)
# Create a test-specific database manager with NullPool # Create a test-specific database manager with NullPool
db_manager = DatabaseManager(test_db_url) db_manager = DatabaseManager(test_db_url)

256
uv.lock
View File

@ -111,6 +111,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/de/b9/6ffb48e82c5e97b03cecee872d134a6b6666c2767b2d32ed709f3a60a8fe/anthropic-0.54.0-py3-none-any.whl", hash = "sha256:c1062a0a905daeec17ca9c06c401e4b3f24cb0495841d29d752568a1d4018d56", size = 288774, upload-time = "2025-06-11T02:46:25.578Z" }, { url = "https://files.pythonhosted.org/packages/de/b9/6ffb48e82c5e97b03cecee872d134a6b6666c2767b2d32ed709f3a60a8fe/anthropic-0.54.0-py3-none-any.whl", hash = "sha256:c1062a0a905daeec17ca9c06c401e4b3f24cb0495841d29d752568a1d4018d56", size = 288774, upload-time = "2025-06-11T02:46:25.578Z" },
] ]
[[package]]
name = "antlr4-python3-runtime"
version = "4.13.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/33/5f/2cdf6f7aca3b20d3f316e9f505292e1f256a32089bd702034c29ebde6242/antlr4_python3_runtime-4.13.2.tar.gz", hash = "sha256:909b647e1d2fc2b70180ac586df3933e38919c85f98ccc656a96cd3f25ef3916", size = 117467, upload-time = "2024-08-03T19:00:12.757Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/89/03/a851e84fcbb85214dc637b6378121ef9a0dd61b4c65264675d8a5c9b1ae7/antlr4_python3_runtime-4.13.2-py3-none-any.whl", hash = "sha256:fe3835eb8d33daece0e799090eda89719dbccee7aa39ef94eed3818cafa5a7e8", size = 144462, upload-time = "2024-08-03T19:00:11.134Z" },
]
[[package]] [[package]]
name = "anyio" name = "anyio"
version = "4.9.0" version = "4.9.0"
@ -456,14 +465,14 @@ wheels = [
[[package]] [[package]]
name = "coloredlogs" name = "coloredlogs"
version = "15.0.1" version = "14.0"
source = { registry = "https://pypi.org/simple" } source = { registry = "https://pypi.org/simple" }
dependencies = [ dependencies = [
{ name = "humanfriendly" }, { name = "humanfriendly" },
] ]
sdist = { url = "https://files.pythonhosted.org/packages/cc/c7/eed8f27100517e8c0e6b923d5f0845d0cb99763da6fdee00478f91db7325/coloredlogs-15.0.1.tar.gz", hash = "sha256:7c991aa71a4577af2f82600d8f8f3a89f936baeaf9b50a9c197da014e5bf16b0", size = 278520, upload-time = "2021-06-11T10:22:45.202Z" } sdist = { url = "https://files.pythonhosted.org/packages/84/1b/1ecdd371fa68839cfbda15cc671d0f6c92d2c42688df995a9bf6e36f3511/coloredlogs-14.0.tar.gz", hash = "sha256:a1fab193d2053aa6c0a97608c4342d031f1f93a3d1218432c59322441d31a505", size = 275863, upload-time = "2020-02-16T20:51:12.172Z" }
wheels = [ wheels = [
{ url = "https://files.pythonhosted.org/packages/a7/06/3d6badcf13db419e25b07041d9c7b4a2c331d3f4e7134445ec5df57714cd/coloredlogs-15.0.1-py2.py3-none-any.whl", hash = "sha256:612ee75c546f53e92e70049c9dbfcc18c935a2b9a53b66085ce9ef6a6e5c0934", size = 46018, upload-time = "2021-06-11T10:22:42.561Z" }, { url = "https://files.pythonhosted.org/packages/5c/2f/12747be360d6dea432e7b5dfae3419132cb008535cfe614af73b9ce2643b/coloredlogs-14.0-py2.py3-none-any.whl", hash = "sha256:346f58aad6afd48444c2468618623638dadab76e4e70d5e10822676f2d32226a", size = 43888, upload-time = "2020-02-16T20:51:09.712Z" },
] ]
[[package]] [[package]]
@ -588,6 +597,85 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321, upload-time = "2023-10-07T05:32:16.783Z" }, { url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321, upload-time = "2023-10-07T05:32:16.783Z" },
] ]
[[package]]
name = "dagster"
version = "1.12.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "alembic" },
{ name = "antlr4-python3-runtime" },
{ name = "click" },
{ name = "coloredlogs" },
{ name = "dagster-pipes" },
{ name = "dagster-shared" },
{ name = "docstring-parser" },
{ name = "filelock" },
{ name = "grpcio" },
{ name = "grpcio-health-checking" },
{ name = "jinja2" },
{ name = "protobuf" },
{ name = "psutil", marker = "sys_platform == 'win32'" },
{ name = "python-dotenv" },
{ name = "pytz" },
{ name = "pywin32", marker = "sys_platform == 'win32'" },
{ name = "requests" },
{ name = "rich" },
{ name = "setuptools" },
{ name = "six" },
{ name = "sqlalchemy" },
{ name = "structlog" },
{ name = "tabulate" },
{ name = "tomli" },
{ name = "toposort" },
{ name = "tqdm" },
{ name = "tzdata" },
{ name = "universal-pathlib" },
{ name = "watchdog" },
]
sdist = { url = "https://files.pythonhosted.org/packages/db/59/d388dfbe5e7dfc0716a1f61f47014ec4373d37dbf175590a46bebf1a1b5a/dagster-1.12.2.tar.gz", hash = "sha256:cce66b20d5dd185b6d0415c495e1b40edc4e7f7f222e842e7706f67432d0c7ee", size = 1558635, upload-time = "2025-11-13T20:37:43.062Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/75/38/cb6461dfacfa2265a642db5adee86f3d022ca7094151154693911d356032/dagster-1.12.2-py3-none-any.whl", hash = "sha256:52b2b8873ba552d34bec0a5e31de646d8c56bef270fb020fc4a3729a0f0278a0", size = 1942153, upload-time = "2025-11-13T20:37:39.778Z" },
]
[[package]]
name = "dagster-pipes"
version = "1.12.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/1e/14/8103379523a9f3ef479683f66ee4f5ae695466fd17f506bcaddece3b4a05/dagster_pipes-1.12.2.tar.gz", hash = "sha256:7fb4c42c2fb97acc2fcc04a2b69b18091f4fe6678665800f875f2deed17a7e21", size = 21056, upload-time = "2025-11-13T20:37:51.189Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/89/ce/2a603b4a448989c111a0b62fef922709ddef617b3678c4c0a76f620ada5a/dagster_pipes-1.12.2-py3-none-any.whl", hash = "sha256:a95ec64e1a6b91023a6c2fe4c0694b49afe1c8eda856c20dd712fb3160672ae1", size = 20829, upload-time = "2025-11-13T20:37:48.506Z" },
]
[[package]]
name = "dagster-postgres"
version = "0.28.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "dagster" },
{ name = "psycopg2-binary" },
]
sdist = { url = "https://files.pythonhosted.org/packages/2e/90/bfb950930f42bb70d69879ae3da213806d95d578fae95f3fb4d9cb5cc1b3/dagster_postgres-0.28.2.tar.gz", hash = "sha256:f9b6403f836f63a47d3a3af6a5b9f2c8298d706fe738884c57d818a16a0c8f2d", size = 16409, upload-time = "2025-11-13T20:43:07.407Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ac/63/c82afecf71746a231f38bc927574d4d117c2bc88c271a8a5968cf8b45fe1/dagster_postgres-0.28.2-py3-none-any.whl", hash = "sha256:536f1c2c282634392a993f017d4854fb2f49308483c34b082d9959d7f826ba70", size = 22940, upload-time = "2025-11-13T20:43:06.258Z" },
]
[[package]]
name = "dagster-shared"
version = "1.12.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "packaging" },
{ name = "platformdirs" },
{ name = "pydantic" },
{ name = "pyyaml" },
{ name = "tomlkit" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/db/1f/280c461ecc8a6664d92533a567e9f8fad1d1dd6b7da59f80efb88ecb662b/dagster_shared-1.12.2.tar.gz", hash = "sha256:4e20354bc15df717a7546a7561f36dd6e954bd1262010ccef2dabefcec4db908", size = 77846, upload-time = "2025-11-13T20:40:49.195Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/03/88/6da6fbbc81c0e9afe835eff60964ef516219f08b390c0424f731b7a677a7/dagster_shared-1.12.2-py3-none-any.whl", hash = "sha256:b006e78adc4be46818e5c05e9401203172a984f4921770c4a22b5cdcd8d61e45", size = 91000, upload-time = "2025-11-13T20:40:47.83Z" },
]
[[package]] [[package]]
name = "dataclasses-json" name = "dataclasses-json"
version = "0.6.7" version = "0.6.7"
@ -622,6 +710,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" }, { url = "https://files.pythonhosted.org/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" },
] ]
[[package]]
name = "docstring-parser"
version = "0.17.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/b2/9d/c3b43da9515bd270df0f80548d9944e389870713cc1fe2b8fb35fe2bcefd/docstring_parser-0.17.0.tar.gz", hash = "sha256:583de4a309722b3315439bb31d64ba3eebada841f2e2cee23b99df001434c912", size = 27442, upload-time = "2025-07-21T07:35:01.868Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/55/e2/2537ebcff11c1ee1ff17d8d0b6f4db75873e3b0fb32c2d4a2ee31ecb310a/docstring_parser-0.17.0-py3-none-any.whl", hash = "sha256:cf2569abd23dce8099b300f9b4fa8191e9582dda731fd533daf54c4551658708", size = 36896, upload-time = "2025-07-21T07:35:00.684Z" },
]
[[package]] [[package]]
name = "durationpy" name = "durationpy"
version = "0.10" version = "0.10"
@ -913,6 +1010,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/d7/35/347db7d2e7674b621afd21b12022e7f48c7b0861b5577134b4e939536141/grpcio-1.73.0-cp313-cp313-win_amd64.whl", hash = "sha256:38cf518cc54cd0c47c9539cefa8888549fcc067db0b0c66a46535ca8032020c4", size = 4335872, upload-time = "2025-06-09T10:04:29.032Z" }, { url = "https://files.pythonhosted.org/packages/d7/35/347db7d2e7674b621afd21b12022e7f48c7b0861b5577134b4e939536141/grpcio-1.73.0-cp313-cp313-win_amd64.whl", hash = "sha256:38cf518cc54cd0c47c9539cefa8888549fcc067db0b0c66a46535ca8032020c4", size = 4335872, upload-time = "2025-06-09T10:04:29.032Z" },
] ]
[[package]]
name = "grpcio-health-checking"
version = "1.71.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "grpcio" },
{ name = "protobuf" },
]
sdist = { url = "https://files.pythonhosted.org/packages/53/86/20994347ef36b7626fb74539f13128100dd8b7eaac67efc063264e6cdc80/grpcio_health_checking-1.71.2.tar.gz", hash = "sha256:1c21ece88c641932f432b573ef504b20603bdf030ad4e1ec35dd7fdb4ea02637", size = 16770, upload-time = "2025-06-28T04:24:08.768Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1a/74/7bc6ab96bf1083cab2684f9c3ae434caa638de3d5c5574e8435e2c146598/grpcio_health_checking-1.71.2-py3-none-any.whl", hash = "sha256:f91db41410d6bd18a7828c5b6ac2bebd77a63483263cbe42bf3c0c9b86cece33", size = 18918, upload-time = "2025-06-28T04:23:56.923Z" },
]
[[package]] [[package]]
name = "grpcio-status" name = "grpcio-status"
version = "1.71.0" version = "1.71.0"
@ -2636,6 +2746,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/12/18/35d1d947553d24909dca37e2ff11720eecb601360d1bac8d7a9a1bc7eb08/parsel-1.10.0-py2.py3-none-any.whl", hash = "sha256:6a0c28bd81f9df34ba665884c88efa0b18b8d2c44c81f64e27f2f0cb37d46169", size = 17266, upload-time = "2025-01-17T15:38:27.83Z" }, { url = "https://files.pythonhosted.org/packages/12/18/35d1d947553d24909dca37e2ff11720eecb601360d1bac8d7a9a1bc7eb08/parsel-1.10.0-py2.py3-none-any.whl", hash = "sha256:6a0c28bd81f9df34ba665884c88efa0b18b8d2c44c81f64e27f2f0cb37d46169", size = 17266, upload-time = "2025-01-17T15:38:27.83Z" },
] ]
[[package]]
name = "pathlib-abc"
version = "0.5.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d6/cb/448649d7f25d228bf0be3a04590ab7afa77f15e056f8fa976ed05ec9a78f/pathlib_abc-0.5.2.tar.gz", hash = "sha256:fcd56f147234645e2c59c7ae22808b34c364bb231f685ddd9f96885aed78a94c", size = 33342, upload-time = "2025-10-10T18:37:20.524Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b1/29/c028a0731e202035f0e2e0bfbf1a3e46ad6c628cbb17f6f1cc9eea5d9ff1/pathlib_abc-0.5.2-py3-none-any.whl", hash = "sha256:4c9d94cf1b23af417ce7c0417b43333b06a106c01000b286c99de230d95eefbb", size = 19070, upload-time = "2025-10-10T18:37:19.437Z" },
]
[[package]] [[package]]
name = "peewee" name = "peewee"
version = "3.18.1" version = "3.18.1"
@ -2824,6 +2943,50 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/7e/cc/7e77861000a0691aeea8f4566e5d3aa716f2b1dece4a24439437e41d3d25/protobuf-5.29.5-py3-none-any.whl", hash = "sha256:6cf42630262c59b2d8de33954443d94b746c952b01434fc58a417fdbd2e84bd5", size = 172823, upload-time = "2025-05-28T23:51:58.157Z" }, { url = "https://files.pythonhosted.org/packages/7e/cc/7e77861000a0691aeea8f4566e5d3aa716f2b1dece4a24439437e41d3d25/protobuf-5.29.5-py3-none-any.whl", hash = "sha256:6cf42630262c59b2d8de33954443d94b746c952b01434fc58a417fdbd2e84bd5", size = 172823, upload-time = "2025-05-28T23:51:58.157Z" },
] ]
[[package]]
name = "psutil"
version = "7.1.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e1/88/bdd0a41e5857d5d703287598cbf08dad90aed56774ea52ae071bae9071b6/psutil-7.1.3.tar.gz", hash = "sha256:6c86281738d77335af7aec228328e944b30930899ea760ecf33a4dba66be5e74", size = 489059, upload-time = "2025-11-02T12:25:54.619Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a6/82/62d68066e13e46a5116df187d319d1724b3f437ddd0f958756fc052677f4/psutil-7.1.3-cp313-cp313t-win_amd64.whl", hash = "sha256:18349c5c24b06ac5612c0428ec2a0331c26443d259e2a0144a9b24b4395b58fa", size = 249642, upload-time = "2025-11-02T12:26:07.447Z" },
{ url = "https://files.pythonhosted.org/packages/df/ad/c1cd5fe965c14a0392112f68362cfceb5230819dbb5b1888950d18a11d9f/psutil-7.1.3-cp313-cp313t-win_arm64.whl", hash = "sha256:c525ffa774fe4496282fb0b1187725793de3e7c6b29e41562733cae9ada151ee", size = 245518, upload-time = "2025-11-02T12:26:09.719Z" },
{ url = "https://files.pythonhosted.org/packages/0f/1d/5774a91607035ee5078b8fd747686ebec28a962f178712de100d00b78a32/psutil-7.1.3-cp314-cp314t-win_amd64.whl", hash = "sha256:3792983e23b69843aea49c8f5b8f115572c5ab64c153bada5270086a2123c7e7", size = 250466, upload-time = "2025-11-02T12:26:21.183Z" },
{ url = "https://files.pythonhosted.org/packages/00/ca/e426584bacb43a5cb1ac91fae1937f478cd8fbe5e4ff96574e698a2c77cd/psutil-7.1.3-cp314-cp314t-win_arm64.whl", hash = "sha256:31d77fcedb7529f27bb3a0472bea9334349f9a04160e8e6e5020f22c59893264", size = 245756, upload-time = "2025-11-02T12:26:23.148Z" },
{ url = "https://files.pythonhosted.org/packages/55/4c/c3ed1a622b6ae2fd3c945a366e64eb35247a31e4db16cf5095e269e8eb3c/psutil-7.1.3-cp37-abi3-win_amd64.whl", hash = "sha256:f39c2c19fe824b47484b96f9692932248a54c43799a84282cfe58d05a6449efd", size = 247633, upload-time = "2025-11-02T12:26:33.887Z" },
{ url = "https://files.pythonhosted.org/packages/c9/ad/33b2ccec09bf96c2b2ef3f9a6f66baac8253d7565d8839e024a6b905d45d/psutil-7.1.3-cp37-abi3-win_arm64.whl", hash = "sha256:bd0d69cee829226a761e92f28140bec9a5ee9d5b4fb4b0cc589068dbfff559b1", size = 244608, upload-time = "2025-11-02T12:26:36.136Z" },
]
[[package]]
name = "psycopg2-binary"
version = "2.9.11"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ac/6c/8767aaa597ba424643dc87348c6f1754dd9f48e80fdc1b9f7ca5c3a7c213/psycopg2-binary-2.9.11.tar.gz", hash = "sha256:b6aed9e096bf63f9e75edf2581aa9a7e7186d97ab5c177aa6c87797cd591236c", size = 379620, upload-time = "2025-10-10T11:14:48.041Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ff/a8/a2709681b3ac11b0b1786def10006b8995125ba268c9a54bea6f5ae8bd3e/psycopg2_binary-2.9.11-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:b8fb3db325435d34235b044b199e56cdf9ff41223a4b9752e8576465170bb38c", size = 3756572, upload-time = "2025-10-10T11:12:32.873Z" },
{ url = "https://files.pythonhosted.org/packages/62/e1/c2b38d256d0dafd32713e9f31982a5b028f4a3651f446be70785f484f472/psycopg2_binary-2.9.11-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:366df99e710a2acd90efed3764bb1e28df6c675d33a7fb40df9b7281694432ee", size = 3864529, upload-time = "2025-10-10T11:12:36.791Z" },
{ url = "https://files.pythonhosted.org/packages/11/32/b2ffe8f3853c181e88f0a157c5fb4e383102238d73c52ac6d93a5c8bffe6/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8c55b385daa2f92cb64b12ec4536c66954ac53654c7f15a203578da4e78105c0", size = 4411242, upload-time = "2025-10-10T11:12:42.388Z" },
{ url = "https://files.pythonhosted.org/packages/10/04/6ca7477e6160ae258dc96f67c371157776564679aefd247b66f4661501a2/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:c0377174bf1dd416993d16edc15357f6eb17ac998244cca19bc67cdc0e2e5766", size = 4468258, upload-time = "2025-10-10T11:12:48.654Z" },
{ url = "https://files.pythonhosted.org/packages/3c/7e/6a1a38f86412df101435809f225d57c1a021307dd0689f7a5e7fe83588b1/psycopg2_binary-2.9.11-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5c6ff3335ce08c75afaed19e08699e8aacf95d4a260b495a4a8545244fe2ceb3", size = 4166295, upload-time = "2025-10-10T11:12:52.525Z" },
{ url = "https://files.pythonhosted.org/packages/f2/7d/c07374c501b45f3579a9eb761cbf2604ddef3d96ad48679112c2c5aa9c25/psycopg2_binary-2.9.11-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:84011ba3109e06ac412f95399b704d3d6950e386b7994475b231cf61eec2fc1f", size = 3983133, upload-time = "2025-10-30T02:55:24.329Z" },
{ url = "https://files.pythonhosted.org/packages/82/56/993b7104cb8345ad7d4516538ccf8f0d0ac640b1ebd8c754a7b024e76878/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ba34475ceb08cccbdd98f6b46916917ae6eeb92b5ae111df10b544c3a4621dc4", size = 3652383, upload-time = "2025-10-10T11:12:56.387Z" },
{ url = "https://files.pythonhosted.org/packages/2d/ac/eaeb6029362fd8d454a27374d84c6866c82c33bfc24587b4face5a8e43ef/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:b31e90fdd0f968c2de3b26ab014314fe814225b6c324f770952f7d38abf17e3c", size = 3298168, upload-time = "2025-10-10T11:13:00.403Z" },
{ url = "https://files.pythonhosted.org/packages/2b/39/50c3facc66bded9ada5cbc0de867499a703dc6bca6be03070b4e3b65da6c/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:d526864e0f67f74937a8fce859bd56c979f5e2ec57ca7c627f5f1071ef7fee60", size = 3044712, upload-time = "2025-10-30T02:55:27.975Z" },
{ url = "https://files.pythonhosted.org/packages/9c/8e/b7de019a1f562f72ada81081a12823d3c1590bedc48d7d2559410a2763fe/psycopg2_binary-2.9.11-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:04195548662fa544626c8ea0f06561eb6203f1984ba5b4562764fbeb4c3d14b1", size = 3347549, upload-time = "2025-10-10T11:13:03.971Z" },
{ url = "https://files.pythonhosted.org/packages/80/2d/1bb683f64737bbb1f86c82b7359db1eb2be4e2c0c13b947f80efefa7d3e5/psycopg2_binary-2.9.11-cp313-cp313-win_amd64.whl", hash = "sha256:efff12b432179443f54e230fdf60de1f6cc726b6c832db8701227d089310e8aa", size = 2714215, upload-time = "2025-10-10T11:13:07.14Z" },
{ url = "https://files.pythonhosted.org/packages/64/12/93ef0098590cf51d9732b4f139533732565704f45bdc1ffa741b7c95fb54/psycopg2_binary-2.9.11-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:92e3b669236327083a2e33ccfa0d320dd01b9803b3e14dd986a4fc54aa00f4e1", size = 3756567, upload-time = "2025-10-10T11:13:11.885Z" },
{ url = "https://files.pythonhosted.org/packages/7c/a9/9d55c614a891288f15ca4b5209b09f0f01e3124056924e17b81b9fa054cc/psycopg2_binary-2.9.11-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e0deeb03da539fa3577fcb0b3f2554a97f7e5477c246098dbb18091a4a01c16f", size = 3864755, upload-time = "2025-10-10T11:13:17.727Z" },
{ url = "https://files.pythonhosted.org/packages/13/1e/98874ce72fd29cbde93209977b196a2edae03f8490d1bd8158e7f1daf3a0/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:9b52a3f9bb540a3e4ec0f6ba6d31339727b2950c9772850d6545b7eae0b9d7c5", size = 4411646, upload-time = "2025-10-10T11:13:24.432Z" },
{ url = "https://files.pythonhosted.org/packages/5a/bd/a335ce6645334fb8d758cc358810defca14a1d19ffbc8a10bd38a2328565/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:db4fd476874ccfdbb630a54426964959e58da4c61c9feba73e6094d51303d7d8", size = 4468701, upload-time = "2025-10-10T11:13:29.266Z" },
{ url = "https://files.pythonhosted.org/packages/44/d6/c8b4f53f34e295e45709b7568bf9b9407a612ea30387d35eb9fa84f269b4/psycopg2_binary-2.9.11-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:47f212c1d3be608a12937cc131bd85502954398aaa1320cb4c14421a0ffccf4c", size = 4166293, upload-time = "2025-10-10T11:13:33.336Z" },
{ url = "https://files.pythonhosted.org/packages/4b/e0/f8cc36eadd1b716ab36bb290618a3292e009867e5c97ce4aba908cb99644/psycopg2_binary-2.9.11-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e35b7abae2b0adab776add56111df1735ccc71406e56203515e228a8dc07089f", size = 3983184, upload-time = "2025-10-30T02:55:32.483Z" },
{ url = "https://files.pythonhosted.org/packages/53/3e/2a8fe18a4e61cfb3417da67b6318e12691772c0696d79434184a511906dc/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fcf21be3ce5f5659daefd2b3b3b6e4727b028221ddc94e6c1523425579664747", size = 3652650, upload-time = "2025-10-10T11:13:38.181Z" },
{ url = "https://files.pythonhosted.org/packages/76/36/03801461b31b29fe58d228c24388f999fe814dfc302856e0d17f97d7c54d/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:9bd81e64e8de111237737b29d68039b9c813bdf520156af36d26819c9a979e5f", size = 3298663, upload-time = "2025-10-10T11:13:44.878Z" },
{ url = "https://files.pythonhosted.org/packages/97/77/21b0ea2e1a73aa5fa9222b2a6b8ba325c43c3a8d54272839c991f2345656/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:32770a4d666fbdafab017086655bcddab791d7cb260a16679cc5a7338b64343b", size = 3044737, upload-time = "2025-10-30T02:55:35.69Z" },
{ url = "https://files.pythonhosted.org/packages/67/69/f36abe5f118c1dca6d3726ceae164b9356985805480731ac6712a63f24f0/psycopg2_binary-2.9.11-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:c3cb3a676873d7506825221045bd70e0427c905b9c8ee8d6acd70cfcbd6e576d", size = 3347643, upload-time = "2025-10-10T11:13:53.499Z" },
{ url = "https://files.pythonhosted.org/packages/e1/36/9c0c326fe3a4227953dfb29f5d0c8ae3b8eb8c1cd2967aa569f50cb3c61f/psycopg2_binary-2.9.11-cp314-cp314-win_amd64.whl", hash = "sha256:4012c9c954dfaccd28f94e84ab9f94e12df76b4afb22331b1f0d3154893a6316", size = 2803913, upload-time = "2025-10-10T11:13:57.058Z" },
]
[[package]] [[package]]
name = "pyasn1" name = "pyasn1"
version = "0.6.1" version = "0.6.1"
@ -3120,6 +3283,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/81/c4/34e93fe5f5429d7570ec1fa436f1986fb1f00c3e0f43a589fe2bbcd22c3f/pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00", size = 509225, upload-time = "2025-03-25T02:24:58.468Z" }, { url = "https://files.pythonhosted.org/packages/81/c4/34e93fe5f5429d7570ec1fa436f1986fb1f00c3e0f43a589fe2bbcd22c3f/pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00", size = 509225, upload-time = "2025-03-25T02:24:58.468Z" },
] ]
[[package]]
name = "pywin32"
version = "311"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a5/be/3fd5de0979fcb3994bfee0d65ed8ca9506a8a1260651b86174f6a86f52b3/pywin32-311-cp313-cp313-win32.whl", hash = "sha256:f95ba5a847cba10dd8c4d8fefa9f2a6cf283b8b88ed6178fa8a6c1ab16054d0d", size = 8705700, upload-time = "2025-07-14T20:13:26.471Z" },
{ url = "https://files.pythonhosted.org/packages/e3/28/e0a1909523c6890208295a29e05c2adb2126364e289826c0a8bc7297bd5c/pywin32-311-cp313-cp313-win_amd64.whl", hash = "sha256:718a38f7e5b058e76aee1c56ddd06908116d35147e133427e59a3983f703a20d", size = 9494700, upload-time = "2025-07-14T20:13:28.243Z" },
{ url = "https://files.pythonhosted.org/packages/04/bf/90339ac0f55726dce7d794e6d79a18a91265bdf3aa70b6b9ca52f35e022a/pywin32-311-cp313-cp313-win_arm64.whl", hash = "sha256:7b4075d959648406202d92a2310cb990fea19b535c7f4a78d3f5e10b926eeb8a", size = 8709318, upload-time = "2025-07-14T20:13:30.348Z" },
{ url = "https://files.pythonhosted.org/packages/c9/31/097f2e132c4f16d99a22bfb777e0fd88bd8e1c634304e102f313af69ace5/pywin32-311-cp314-cp314-win32.whl", hash = "sha256:b7a2c10b93f8986666d0c803ee19b5990885872a7de910fc460f9b0c2fbf92ee", size = 8840714, upload-time = "2025-07-14T20:13:32.449Z" },
{ url = "https://files.pythonhosted.org/packages/90/4b/07c77d8ba0e01349358082713400435347df8426208171ce297da32c313d/pywin32-311-cp314-cp314-win_amd64.whl", hash = "sha256:3aca44c046bd2ed8c90de9cb8427f581c479e594e99b5c0bb19b29c10fd6cb87", size = 9656800, upload-time = "2025-07-14T20:13:34.312Z" },
{ url = "https://files.pythonhosted.org/packages/c0/d2/21af5c535501a7233e734b8af901574572da66fcc254cb35d0609c9080dd/pywin32-311-cp314-cp314-win_arm64.whl", hash = "sha256:a508e2d9025764a8270f93111a970e1d0fbfc33f4153b388bb649b7eec4f9b42", size = 8932540, upload-time = "2025-07-14T20:13:36.379Z" },
]
[[package]] [[package]]
name = "pyyaml" name = "pyyaml"
version = "6.0.2" version = "6.0.2"
@ -3493,6 +3669,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/09/15a60adddee87fb0c9d1a2ed2ba0362a80451b107a77cfc87fbe72b9aac7/stockstats-0.6.5-py2.py3-none-any.whl", hash = "sha256:89a42808a8b0f94f7fa537cee8a097ae61790b3773051a889586d51a1e8c9392", size = 31727, upload-time = "2025-05-18T08:18:51.172Z" }, { url = "https://files.pythonhosted.org/packages/bd/09/15a60adddee87fb0c9d1a2ed2ba0362a80451b107a77cfc87fbe72b9aac7/stockstats-0.6.5-py2.py3-none-any.whl", hash = "sha256:89a42808a8b0f94f7fa537cee8a097ae61790b3773051a889586d51a1e8c9392", size = 31727, upload-time = "2025-05-18T08:18:51.172Z" },
] ]
[[package]]
name = "structlog"
version = "25.5.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ef/52/9ba0f43b686e7f3ddfeaa78ac3af750292662284b3661e91ad5494f21dbc/structlog-25.5.0.tar.gz", hash = "sha256:098522a3bebed9153d4570c6d0288abf80a031dfdb2048d59a49e9dc2190fc98", size = 1460830, upload-time = "2025-10-27T08:28:23.028Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a8/45/a132b9074aa18e799b891b91ad72133c98d8042c70f6240e4c5f9dabee2f/structlog-25.5.0-py3-none-any.whl", hash = "sha256:a8453e9b9e636ec59bd9e79bbd4a72f025981b3ba0f5837aebf48f02f37a7f9f", size = 72510, upload-time = "2025-10-27T08:28:21.535Z" },
]
[[package]] [[package]]
name = "sympy" name = "sympy"
version = "1.14.0" version = "1.14.0"
@ -3521,6 +3706,15 @@ dependencies = [
] ]
sdist = { url = "https://files.pythonhosted.org/packages/ba/97/a49816dd468a18ee080cf3a04640772a9f6321790d4049cece2490c4b7ad/ta_lib-0.6.4.tar.gz", hash = "sha256:08f55bc5771a6d1ceb1a2b713aad7b05f04eb0061e980c9113571c532d32e9cb", size = 381774, upload-time = "2025-06-08T15:28:15.452Z" } sdist = { url = "https://files.pythonhosted.org/packages/ba/97/a49816dd468a18ee080cf3a04640772a9f6321790d4049cece2490c4b7ad/ta_lib-0.6.4.tar.gz", hash = "sha256:08f55bc5771a6d1ceb1a2b713aad7b05f04eb0061e980c9113571c532d32e9cb", size = 381774, upload-time = "2025-06-08T15:28:15.452Z" }
[[package]]
name = "tabulate"
version = "0.9.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/ec/fe/802052aecb21e3797b8f7902564ab6ea0d60ff8ca23952079064155d1ae1/tabulate-0.9.0.tar.gz", hash = "sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c", size = 81090, upload-time = "2022-10-06T17:21:48.54Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/40/44/4a5f08c96eb108af5cb50b41f76142f0afa346dfa99d5296fe7202a11854/tabulate-0.9.0-py3-none-any.whl", hash = "sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f", size = 35252, upload-time = "2022-10-06T17:21:44.262Z" },
]
[[package]] [[package]]
name = "tenacity" name = "tenacity"
version = "9.1.2" version = "9.1.2"
@ -3607,6 +3801,24 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/6e/c2/61d3e0f47e2b74ef40a68b9e6ad5984f6241a942f7cd3bbfbdbd03861ea9/tomli-2.2.1-py3-none-any.whl", hash = "sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc", size = 14257, upload-time = "2024-11-27T22:38:35.385Z" }, { url = "https://files.pythonhosted.org/packages/6e/c2/61d3e0f47e2b74ef40a68b9e6ad5984f6241a942f7cd3bbfbdbd03861ea9/tomli-2.2.1-py3-none-any.whl", hash = "sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc", size = 14257, upload-time = "2024-11-27T22:38:35.385Z" },
] ]
[[package]]
name = "tomlkit"
version = "0.13.3"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/cc/18/0bbf3884e9eaa38819ebe46a7bd25dcd56b67434402b66a58c4b8e552575/tomlkit-0.13.3.tar.gz", hash = "sha256:430cf247ee57df2b94ee3fbe588e71d362a941ebb545dec29b53961d61add2a1", size = 185207, upload-time = "2025-06-05T07:13:44.947Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/75/8539d011f6be8e29f339c42e633aae3cb73bffa95dd0f9adec09b9c58e85/tomlkit-0.13.3-py3-none-any.whl", hash = "sha256:c89c649d79ee40629a9fda55f8ace8c6a1b42deb912b2a8fd8d942ddadb606b0", size = 38901, upload-time = "2025-06-05T07:13:43.546Z" },
]
[[package]]
name = "toposort"
version = "1.10"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/69/19/8e955d90985ecbd3b9adb2a759753a6840da2dff3c569d412b2c9217678b/toposort-1.10.tar.gz", hash = "sha256:bfbb479c53d0a696ea7402601f4e693c97b0367837c8898bc6471adfca37a6bd", size = 11132, upload-time = "2023-02-27T13:59:51.834Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/f6/17/57b444fd314d5e1593350b9a31d000e7411ba8e17ce12dc7ad54ca76b810/toposort-1.10-py3-none-any.whl", hash = "sha256:cbdbc0d0bee4d2695ab2ceec97fe0679e9c10eab4b2a87a9372b929e70563a87", size = 8500, upload-time = "2023-02-25T20:07:06.538Z" },
]
[[package]] [[package]]
name = "tqdm" name = "tqdm"
version = "4.67.1" version = "4.67.1"
@ -3684,6 +3896,8 @@ dependencies = [
{ name = "backtrader" }, { name = "backtrader" },
{ name = "chainlit" }, { name = "chainlit" },
{ name = "chromadb" }, { name = "chromadb" },
{ name = "dagster" },
{ name = "dagster-postgres" },
{ name = "eodhd" }, { name = "eodhd" },
{ name = "feedparser" }, { name = "feedparser" },
{ name = "finnhub-python" }, { name = "finnhub-python" },
@ -3744,6 +3958,8 @@ requires-dist = [
{ name = "backtrader", specifier = ">=1.9.78.123" }, { name = "backtrader", specifier = ">=1.9.78.123" },
{ name = "chainlit", specifier = ">=2.5.5" }, { name = "chainlit", specifier = ">=2.5.5" },
{ name = "chromadb", specifier = ">=1.0.12" }, { name = "chromadb", specifier = ">=1.0.12" },
{ name = "dagster", specifier = ">=1.8.0" },
{ name = "dagster-postgres", specifier = ">=0.24.0" },
{ name = "eodhd", specifier = ">=1.0.32" }, { name = "eodhd", specifier = ">=1.0.32" },
{ name = "feedparser", specifier = ">=6.0.11" }, { name = "feedparser", specifier = ">=6.0.11" },
{ name = "finnhub-python", specifier = ">=2.4.23" }, { name = "finnhub-python", specifier = ">=2.4.23" },
@ -3870,6 +4086,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/5c/23/c7abc0ca0a1526a0774eca151daeb8de62ec457e77262b66b359c3c7679e/tzdata-2025.2-py2.py3-none-any.whl", hash = "sha256:1a403fada01ff9221ca8044d701868fa132215d84beb92242d9acd2147f667a8", size = 347839, upload-time = "2025-03-23T13:54:41.845Z" }, { url = "https://files.pythonhosted.org/packages/5c/23/c7abc0ca0a1526a0774eca151daeb8de62ec457e77262b66b359c3c7679e/tzdata-2025.2-py2.py3-none-any.whl", hash = "sha256:1a403fada01ff9221ca8044d701868fa132215d84beb92242d9acd2147f667a8", size = 347839, upload-time = "2025-03-23T13:54:41.845Z" },
] ]
[[package]]
name = "universal-pathlib"
version = "0.3.6"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "fsspec" },
{ name = "pathlib-abc" },
]
sdist = { url = "https://files.pythonhosted.org/packages/76/db/6874223d251a2e146dae57a27ca8cb1f71e7e135aa51ad394173ffe18fc0/universal_pathlib-0.3.6.tar.gz", hash = "sha256:d8640454ff08305fc639f7980e8bad4a7d38e82f6389ff993fb0e7b2a4969de9", size = 249113, upload-time = "2025-11-13T17:05:29.882Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/47/5d/fc1f5478eb486a59549e0dbea5827633bbba01139b549968d4936154b756/universal_pathlib-0.3.6-py3-none-any.whl", hash = "sha256:ff10a86e5340ad986b6f04847bb64ba397dff7467450234ffa2ab5ff135641d8", size = 78715, upload-time = "2025-11-13T17:05:28.101Z" },
]
[[package]] [[package]]
name = "update-checker" name = "update-checker"
version = "0.18.0" version = "0.18.0"
@ -4009,6 +4238,27 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/58/dd/56f0d8af71e475ed194d702f8b4cf9cea812c95e82ad823d239023c6558c/w3lib-2.3.1-py3-none-any.whl", hash = "sha256:9ccd2ae10c8c41c7279cd8ad4fe65f834be894fe7bfdd7304b991fd69325847b", size = 21751, upload-time = "2025-01-27T14:22:09.421Z" }, { url = "https://files.pythonhosted.org/packages/58/dd/56f0d8af71e475ed194d702f8b4cf9cea812c95e82ad823d239023c6558c/w3lib-2.3.1-py3-none-any.whl", hash = "sha256:9ccd2ae10c8c41c7279cd8ad4fe65f834be894fe7bfdd7304b991fd69325847b", size = 21751, upload-time = "2025-01-27T14:22:09.421Z" },
] ]
[[package]]
name = "watchdog"
version = "6.0.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/db/7d/7f3d619e951c88ed75c6037b246ddcf2d322812ee8ea189be89511721d54/watchdog-6.0.0.tar.gz", hash = "sha256:9ddf7c82fda3ae8e24decda1338ede66e1c99883db93711d8fb941eaa2d8c282", size = 131220, upload-time = "2024-11-01T14:07:13.037Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/68/98/b0345cabdce2041a01293ba483333582891a3bd5769b08eceb0d406056ef/watchdog-6.0.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:490ab2ef84f11129844c23fb14ecf30ef3d8a6abafd3754a6f75ca1e6654136c", size = 96480, upload-time = "2024-11-01T14:06:42.952Z" },
{ url = "https://files.pythonhosted.org/packages/85/83/cdf13902c626b28eedef7ec4f10745c52aad8a8fe7eb04ed7b1f111ca20e/watchdog-6.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:76aae96b00ae814b181bb25b1b98076d5fc84e8a53cd8885a318b42b6d3a5134", size = 88451, upload-time = "2024-11-01T14:06:45.084Z" },
{ url = "https://files.pythonhosted.org/packages/fe/c4/225c87bae08c8b9ec99030cd48ae9c4eca050a59bf5c2255853e18c87b50/watchdog-6.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a175f755fc2279e0b7312c0035d52e27211a5bc39719dd529625b1930917345b", size = 89057, upload-time = "2024-11-01T14:06:47.324Z" },
{ url = "https://files.pythonhosted.org/packages/a9/c7/ca4bf3e518cb57a686b2feb4f55a1892fd9a3dd13f470fca14e00f80ea36/watchdog-6.0.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:7607498efa04a3542ae3e05e64da8202e58159aa1fa4acddf7678d34a35d4f13", size = 79079, upload-time = "2024-11-01T14:06:59.472Z" },
{ url = "https://files.pythonhosted.org/packages/5c/51/d46dc9332f9a647593c947b4b88e2381c8dfc0942d15b8edc0310fa4abb1/watchdog-6.0.0-py3-none-manylinux2014_armv7l.whl", hash = "sha256:9041567ee8953024c83343288ccc458fd0a2d811d6a0fd68c4c22609e3490379", size = 79078, upload-time = "2024-11-01T14:07:01.431Z" },
{ url = "https://files.pythonhosted.org/packages/d4/57/04edbf5e169cd318d5f07b4766fee38e825d64b6913ca157ca32d1a42267/watchdog-6.0.0-py3-none-manylinux2014_i686.whl", hash = "sha256:82dc3e3143c7e38ec49d61af98d6558288c415eac98486a5c581726e0737c00e", size = 79076, upload-time = "2024-11-01T14:07:02.568Z" },
{ url = "https://files.pythonhosted.org/packages/ab/cc/da8422b300e13cb187d2203f20b9253e91058aaf7db65b74142013478e66/watchdog-6.0.0-py3-none-manylinux2014_ppc64.whl", hash = "sha256:212ac9b8bf1161dc91bd09c048048a95ca3a4c4f5e5d4a7d1b1a7d5752a7f96f", size = 79077, upload-time = "2024-11-01T14:07:03.893Z" },
{ url = "https://files.pythonhosted.org/packages/2c/3b/b8964e04ae1a025c44ba8e4291f86e97fac443bca31de8bd98d3263d2fcf/watchdog-6.0.0-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:e3df4cbb9a450c6d49318f6d14f4bbc80d763fa587ba46ec86f99f9e6876bb26", size = 79078, upload-time = "2024-11-01T14:07:05.189Z" },
{ url = "https://files.pythonhosted.org/packages/62/ae/a696eb424bedff7407801c257d4b1afda455fe40821a2be430e173660e81/watchdog-6.0.0-py3-none-manylinux2014_s390x.whl", hash = "sha256:2cce7cfc2008eb51feb6aab51251fd79b85d9894e98ba847408f662b3395ca3c", size = 79077, upload-time = "2024-11-01T14:07:06.376Z" },
{ url = "https://files.pythonhosted.org/packages/b5/e8/dbf020b4d98251a9860752a094d09a65e1b436ad181faf929983f697048f/watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:20ffe5b202af80ab4266dcd3e91aae72bf2da48c0d33bdb15c66658e685e94e2", size = 79078, upload-time = "2024-11-01T14:07:07.547Z" },
{ url = "https://files.pythonhosted.org/packages/07/f6/d0e5b343768e8bcb4cda79f0f2f55051bf26177ecd5651f84c07567461cf/watchdog-6.0.0-py3-none-win32.whl", hash = "sha256:07df1fdd701c5d4c8e55ef6cf55b8f0120fe1aef7ef39a1c6fc6bc2e606d517a", size = 79065, upload-time = "2024-11-01T14:07:09.525Z" },
{ url = "https://files.pythonhosted.org/packages/db/d9/c495884c6e548fce18a8f40568ff120bc3a4b7b99813081c8ac0c936fa64/watchdog-6.0.0-py3-none-win_amd64.whl", hash = "sha256:cbafb470cf848d93b5d013e2ecb245d4aa1c8fd0504e863ccefa32445359d680", size = 79070, upload-time = "2024-11-01T14:07:10.686Z" },
{ url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" },
]
[[package]] [[package]]
name = "watchfiles" name = "watchfiles"
version = "0.20.0" version = "0.20.0"