TradingAgents/docs/migration/rollback-notes.md

# TradingAgents backend migration and rollback notes draft

Status: draft
Audience: backend/application maintainers
Scope: migrate toward application-service boundary and result-contract-v1alpha1 with rollback safety

## Current progress snapshot (2026-04)

Mainline has moved beyond pure planning, but it has not finished the full boundary migration:

- `Phase 0` is effectively done: contract and architecture drafts exist.
- `Phase 1-4` are **partially landed**:
  - backend services now project `v1alpha1`-style public payloads;
  - result contracts are persisted via `result_store.py`;
  - `/ws/analysis/{task_id}` and `/ws/orchestrator` already wrap payloads with `contract_version`;
  - recommendation and task-status reads already depend on application-layer shaping more than route-local reconstruction.
- `Phase 5` is **partially landed** via the task lifecycle boundary slice:
  - `status/list/cancel` now route through backend task services instead of route-local orchestration;
  - `web_dashboard/backend/main.py` is still too large outside that slice;
  - reports/export and other residual route-local orchestration are still pending;
  - compatibility fields still coexist with the newer contract-first path.

Also note that research provenance / node guard / profiling work is now landed on the orchestrator side. That effort complements the backend migration but should not be confused with “application boundary fully complete.”

**Recent improvements (2026-04-16)**:
- Orchestrator error classification now includes comprehensive provider × base_url matrix validation
- Timeout configuration validation warns when analyst/research timeouts may be insufficient for multi-analyst profiles
- All provider mismatches (anthropic, openai, google, xai, ollama, openrouter) are now detected before graph initialization

## 1. Migration objective

Move backend delivery code from route-local orchestration to an application-service layer without changing the quant+LLM merge kernel behavior.

Target outcomes:

- stable result contract (`v1alpha1`)
- thin FastAPI transport
- application-owned task lifecycle and mapping
- rollback-safe migration using dual-read/dual-write where useful

## 2. Current coupling hotspots

Primary hotspot: `web_dashboard/backend/main.py`

It currently combines:

- route handlers
- task persistence
- subprocess creation and monitoring
- progress/stage state mutation
- result projection into API fields
- report export concerns

This file is the first migration target.

## 3. Recommended migration sequence

## Phase 0: contract freeze draft

Deliverables:

- agree on `docs/contracts/result-contract-v1alpha1.md`
- agree on application boundary in `docs/architecture/application-boundary.md`

Rollback:

- none needed; documentation only

## Phase 1: introduce application service behind existing routes

Actions:

- add backend application modules for analysis status, live signals, and report reads
- keep existing route URLs unchanged
- move mapping logic out of route functions into service/mappers

Compatibility tactic:

- routes still return current payload shape if frontend depends on it
- internal service also emits `v1alpha1` DTOs for verification comparison

Rollback:

- route handlers can call old inline functions directly via feature flag or import switch

Current status:

- partially complete on mainline via `analysis_service.py`, `job_service.py`, and `result_store.py`
- task lifecycle (`status/list/cancel`) is now service-routed
- not complete enough yet to claim `main.py` is only a thin adapter

## Phase 2: dual-read for task status

Why:

Task status currently lives in memory plus `data/task_status/*.json`. During migration, new service storage and old persisted shape may diverge.

Recommended strategy:

- read preference: new application store first
- fallback read: legacy JSON task status
- compare key fields during shadow period: `status`, `progress`, `current_stage`, `decision`, `error`

Rollback:

- switch read preference back to legacy JSON only
- leave new store populated for debugging, but non-authoritative

## Phase 3: dual-write for task results

Why:

To avoid breaking status pages and historical tooling during rollout.

Recommended strategy:

- authoritative write: new application store
- compatibility write: legacy `app.state.task_results` + `data/task_status/*.json`
- emit diff logs when new-vs-legacy projections disagree

Guardrails:

- dual-write only for application-layer payloads
- do not dual-write alternate domain semantics into `orchestrator/`

Rollback:

- disable new-store writes
- continue legacy writes only

## Phase 4: websocket and live signal migration

Actions:

- make `/ws/analysis/{task_id}` and `/ws/orchestrator` render application contracts
- keep websocket wrapper fields stable while migrating internal body shape

Suggested compatibility step:

- send legacy event envelope with embedded `contract_version`
- update frontend consumers before removing legacy-only fields

Rollback:

- restore websocket serializer to legacy shape
- keep application service intact behind adapter

Current status:

- partially complete on mainline
- `/ws/orchestrator` already emits `contract_version`, `data_quality`, `degradation`, and `research`
- `/ws/analysis/{task_id}` already reads application-shaped task state

## Phase 5: remove route-local orchestration

Actions:

- delete dead inline task mutation helpers from `main.py`
- keep routes as thin adapter layer
- preserve report retrieval behavior

Rollback:

- only safe after shadow metrics show parity
- otherwise revert to Phase 3 dual-write mode, not direct deletion

## 4. Suggested feature flags

Environment-variable style examples:

- `TA_APP_SERVICE_ENABLED=1`
- `TA_RESULT_CONTRACT_VERSION=v1alpha1`
- `TA_TASKSTORE_DUAL_READ=1`
- `TA_TASKSTORE_DUAL_WRITE=1`
- `TA_WS_V1ALPHA1_ENABLED=0`

These names are placeholders; exact naming can be chosen during implementation.

## 5. Verification checkpoints per phase

For each migration phase, verify:

- same task ids are returned for the same route behavior
- stage transitions remain monotonic
- completed tasks persist `decision`, `confidence`, and degraded-path outcomes
- failure path still preserves actionable error text
- live websocket payloads preserve ticker/date ordering expectations

## 6. Rollback triggers

Rollback immediately if any of these happen:

- task status disappears after backend restart
- WebSocket clients stop receiving progress updates
- completed analysis loses `decision` or confidence fields
- degraded single-lane signals are reclassified incorrectly
- report export or historical report retrieval cannot find prior artifacts

## 7. Explicit non-goals during migration

- do not rewrite `orchestrator/signals.py` merge math as part of boundary migration
- do not rework provider/model selection semantics in the same change set
- do not force frontend redesign before contract shadowing proves parity
- do not implement a new strategy layer inside the application service

## 8. Minimal rollback playbook

If production or local verification fails after migration cutover:

1. disable application-service read path
2. disable dual-write to new store if it corrupts parity checks
3. restore legacy route-local serializers
4. keep generated comparison logs/artifacts for diff analysis
5. re-run backend tests and one end-to-end manual analysis flow

## 9. Review checklist

A migration plan is acceptable only if it:

- preserves orchestrator ownership of quant+LLM merge semantics
- introduces feature-flagged cutover points
- supports dual-read/dual-write only at application/persistence boundary
- provides a one-step rollback path at each release phase

## 10. Maintainer note

When updating migration status, keep these three documents aligned:

- `docs/architecture/application-boundary.md`
- `docs/contracts/result-contract-v1alpha1.md`
- `docs/architecture/research-provenance.md`

The first two describe backend/application convergence; the third describes orchestrator-side research degradation and profiling semantics that now feed those contracts.