charieswei
/
TradingAgents
mirror of https://github.com/TauricResearch/TradingAgents.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
TradingAgents/.claude/skills/project-alignment-validation/examples/misalignment-examples.md

15 KiB
Raw Blame History

Misalignment Examples - What Not To Do

Real-world examples of alignment failures and how to avoid them.

Example 1: Keyword Stuffing Without Intent

The Wrong Approach ❌

Feature Description: "Add user authentication security login JWT token session management authorization role-based access control RBAC permissions security audit logging compliance GDPR HIPAA encryption SSL TLS security headers CORS CSRF protection security best practices OWASP Top 10 security vulnerability scanning penetration testing security review."

What Happened:

  • Developer stuffed description with security keywords
  • Validation script passes (all keywords present)
  • But actual feature is just basic login form
  • No JWT, no RBAC, no audit logging, no compliance features
  • Result: False alignment, wasted effort on wrong feature

The Right Approach ✓

Feature Description: "Add basic user authentication with username/password, session management, and password hashing using bcrypt."

Intent Analysis:

  • Clear scope: Basic authentication only
  • Specific technology: bcrypt for password hashing
  • Honest about what's included (sessions) and what's not (JWT, RBAC)
  • Result: True alignment, realistic expectations

Lesson

Focus on intent, not keyword density. Honest, specific descriptions lead to better alignment validation.

Example 2: Ignoring Explicit Out-of-Scope

The Wrong Approach ❌

PROJECT.md:

## SCOPE
Out of Scope: Payment processing, billing, subscriptions

Feature Request: "Add user account upgrades with credit card processing"

Developer Reasoning: "Users need to upgrade accounts, so this must be in scope. Payment processing is just a detail."

What Happened:

  • Ignored explicit out-of-scope item
  • Built Stripe integration
  • Product owner: "We're not doing payments in MVP!"
  • Result: Wasted 2 weeks, feature rejected

The Right Approach ✓

Alternative Feature: "Add account tier system (Free/Pro/Enterprise) with manual upgrade workflow. Admin can upgrade users via dashboard. Payment integration deferred to v2.0."

Reasoning:

  • Achieves goal: Users can have different account levels
  • Respects scope: No payment processing
  • Provides value: Validates tiered pricing model
  • Result: Feature approved, ships in MVP

Lesson

Out-of-scope means OUT. Find alternative approaches that respect boundaries.

Example 3: Relaxing Constraints Without Approval

The Wrong Approach ❌

PROJECT.md CONSTRAINTS:

- API response time < 200ms (P95)
- 80% test coverage minimum
- Python 3.9+ compatibility

Developer Action:

  • Implements complex ML feature
  • Response time: 3 seconds
  • Test coverage: 45%
  • Uses Python 3.11-only features
  • Reasoning: "The feature is too complex for these constraints"

What Happened:

  • Demo fails: "This is way too slow"
  • Tests fail in CI: "Python 3.9 compatibility broken"
  • Code review rejected: "Where are the tests?"
  • Result: Feature blocked, sprint wasted

The Right Approach ✓

Before Implementation:

  1. Recognize constraints will be violated
  2. Document trade-offs and options
  3. Escalate to stakeholders: 
    "ML feature requires either:
A) Relax response time to 3s (async processing)
B) Simplify algorithm to meet 200ms
C) Defer feature until infrastructure improved

Recommendation: Option A with async processing"
  4. Get explicit approval to relax constraints
  5. Update PROJECT.md with new constraints

Result: Feature approved with realistic expectations, constraints updated

Lesson

Constraints exist for a reason. Get approval before violating them.

Example 4: Architecture Deviation Without Documentation

The Wrong Approach ❌

PROJECT.md ARCHITECTURE:

- RESTful API with JSON responses
- Stateless services
- PostgreSQL database

Developer Implementation:

  • Adds GraphQL endpoint (not REST)
  • Uses in-memory session storage (not stateless)
  • Adds MongoDB for caching (not PostgreSQL only)

Developer Reasoning: "GraphQL is better than REST. We need sessions for UX. MongoDB is faster for caching."

What Happened:

  • New developer joins: "Architecture doc says REST, but I see GraphQL?"
  • Deployment fails: "Where's MongoDB configured?"
  • Scaling issues: "In-memory sessions don't work across multiple servers"
  • Result: Confusion, bugs, scaling problems

The Right Approach ✓

Before Implementation:

  1. Recognize architecture deviations
  2. Document rationale for changes
  3. Update PROJECT.md: 
    ## ARCHITECTURE

### API Layer
- Primary: RESTful API with JSON
- GraphQL endpoint for complex queries (added 2024-03)
  - Rationale: Better handling of nested data structures
  - Scope: Read-only queries, write operations still use REST

### Session Management
- Stateless JWT tokens (primary)
- Redis-backed sessions for admin panel (added 2024-03)
  - Rationale: Admin features need complex session state
  - Scope: Admin routes only, user-facing remains stateless

### Data Store
- PostgreSQL (primary database)
- Redis (session storage, caching)
- Migration from in-memory to Redis: docs/redis-migration.md

Result: Architecture documented, team aligned, scaling works

Lesson

Document architectural decisions. Update PROJECT.md when you deviate.

Example 5: Goal Misalignment Through Feature Creep

The Wrong Approach ❌

PROJECT.md GOALS:

1. Launch MVP in 8 weeks
2. Validate product-market fit
3. Keep costs under $5k/month

Developer Additions:

  • Comprehensive analytics dashboard (2 weeks)
  • A/B testing framework (2 weeks)
  • Advanced user segmentation (1 week)
  • Email campaign automation (2 weeks)
  • Social media integrations (1 week)

Developer Reasoning: "These features help validate product-market fit, so they align with Goal #2."

What Happened:

  • MVP delayed by 8 weeks (16 weeks total)
  • Budget exceeded: $12k/month for all services
  • Goal #1 (launch in 8 weeks) completely missed
  • Goal #3 (costs < $5k) violated by 240%
  • Result: Competitor launched first, missed market opportunity

The Right Approach ✓

MVP Features Only:

  • Basic event tracking (3 days) - validates core metrics
  • Simple user feedback form (1 day) - validates PMF
  • Google Analytics integration (1 day) - basic analytics

Defer to v2.0:

  • Advanced analytics → After MVP validation
  • A/B testing → When traffic justifies it
  • Segmentation → When user base grows
  • Automation → When manual process proven valuable
  • Social media → When core product validated

Result: MVP shipped in 8 weeks, budget under $5k, validated market fit

Lesson

Every feature has a cost. Align with primary goals, defer secondary features.

Example 6: Literal Scope Interpretation

The Wrong Approach ❌

PROJECT.md SCOPE:

In Scope: User authentication
Out of Scope: Third-party integrations

Developer Question: "Should we support OAuth (Google, GitHub)?"

Wrong Interpretation: "OAuth involves third-party services (Google, GitHub), so it's out of scope per the 'no third-party integrations' rule. Feature rejected."

What Happened:

  • Implemented only username/password auth
  • Users complain: "No social login?"
  • Competitor has Google sign-in
  • Result: Worse UX, competitive disadvantage

The Right Approach ✓

Semantic Interpretation:

Analysis:
- "Third-party integrations" in out-of-scope likely means:
  - Business integrations (Salesforce, HubSpot)
  - Analytics platforms (Segment, Mixpanel)
  - Payment processors (Stripe, PayPal)

- OAuth is:
  - Authentication mechanism (serves "User authentication" goal)
  - Industry standard (improves UX)
  - Not a "business integration"

Recommendation: OAuth is IN SCOPE

Clarify PROJECT.md:

## SCOPE

In Scope:
- User authentication (local credentials, OAuth providers)

Out of Scope:
- Third-party business integrations (CRM, analytics, payments)
- Note: Authentication providers (OAuth) are IN SCOPE

Result: Better UX, competitive feature parity, clear documentation

Lesson

Understand intent, not just words. Use semantic validation, not literal pattern matching.

Example 7: Skipping Constraint Validation

The Wrong Approach ❌

PROJECT.md CONSTRAINTS:

- GDPR compliant (EU data residency)
- SOC 2 Type II certification required
- All data encrypted at rest and in transit

Developer Implementation:

  • Uses AWS US-East (not EU region)
  • Stores user data in plain text
  • No encryption for backups
  • No data processing agreements

Developer Reasoning: "We'll handle compliance later, let's ship features first."

What Happened:

  • Legal review: "We can't launch in EU with this"
  • Customer security audit: "Failed - no encryption"
  • Compliance officer: "This will take 6 months to fix"
  • Result: Launch blocked, massive refactoring needed

The Right Approach ✓

Before Implementation:

  1. Read ALL constraints carefully
  2. Design for compliance from day 1:
    • AWS EU-West region
    • Database encryption enabled
    • Backup encryption enabled
    • DPA templates prepared
  3. Get security review early
  4. Document compliance measures

Result: Launch approved, no last-minute delays

Lesson

Constraints are not optional. Validate compliance early, not later.

Example 8: Assuming Implicit Scope

The Wrong Approach ❌

PROJECT.md SCOPE:

In Scope: User dashboard with profile management

Developer Assumptions: "Dashboard means charts, so I'll add:

  • Real-time analytics graphs
  • Data export to CSV/Excel
  • Custom report builder
  • Data visualization library
  • Historical trend analysis"

What Happened:

  • 4 weeks spent on charts
  • Product owner: "We just needed a profile page"
  • Actual MVP need: Name, email, password change form
  • Result: 3.5 weeks wasted on wrong features

The Right Approach ✓

Clarify Before Implementing:

Question to Product Owner:
"PROJECT.md mentions 'user dashboard'. Can you clarify scope?
- Profile management (name, email, password)?
- Usage statistics/analytics?
- Account settings?
- Other features?
"

Product Owner Response:
"Just basic profile for MVP:
- View/edit name and email
- Change password
- Delete account

Analytics for v2.0 after we have data."

Update PROJECT.md:

## SCOPE

In Scope (MVP):
- User profile management (view/edit name, email, password, account deletion)

Future Scope (v2.0):
- User dashboard with analytics
- Usage statistics
- Custom reports

Result: Correct feature built in 0.5 weeks, clear expectations

Lesson

Don't assume implicit scope. Clarify ambiguous requirements before implementing.

Example 9: Documentation Drift Leading to Confusion

The Wrong Approach ❌

PROJECT.md (written 6 months ago):

## TECH STACK
- Python 3.8
- SQLite database
- Flask framework
- Bootstrap 4 UI

Actual Current Stack:

  • Python 3.11 (upgraded 3 months ago)
  • PostgreSQL (migrated 4 months ago)
  • FastAPI (replaced Flask 2 months ago)
  • React UI (replaced Bootstrap 1 month ago)

What Happened:

  • New developer joins, reads PROJECT.md
  • Sets up Python 3.8 environment
  • Uses Flask examples from PROJECT.md
  • Code doesn't run: "No Flask installed?"
  • Result: 2 days wasted, developer frustrated

The Right Approach ✓

Update PROJECT.md with Each Change:

3 months ago (Python upgrade):

## TECH STACK
- Python 3.11 (upgraded from 3.8 on 2024-06-01 for performance)

2 months ago (Flask → FastAPI):

## TECH STACK
- FastAPI (migrated from Flask on 2024-07-01 for async support)

Result: Documentation always current, new developers productive immediately

Lesson

Update documentation with code changes. PROJECT.md is a living document.

Example 10: Ignoring Trade-off Communication

The Wrong Approach ❌

Feature: "Real-time notifications"

Implementation Choices (not communicated):

  • Uses WebSocket connections (battery drain on mobile)
  • Keeps connections open (server cost $2k/month more)
  • No graceful degradation (breaks if WebSocket unavailable)

What Happened:

  • Mobile users: "App drains battery!"
  • Finance: "Server costs doubled!"
  • Network team: "Fails on corporate firewalls!"
  • Result: Feature rollback, reputation damage

The Right Approach ✓

Document Trade-offs:

## Real-Time Notifications - Trade-off Analysis

### Approach: WebSockets
Pros:
- True real-time updates
- Modern UX

Cons:
- Battery drain on mobile (~10% per hour)
- Server cost increase ($2k/month)
- Firewall issues in some networks

### Mitigation:
- Mobile: Allow users to disable real-time (reduce to polling)
- Cost: Limit connections to active users only
- Network: Fallback to long-polling if WebSocket fails

### Alternative Considered:
- Server-Sent Events (SSE): Server-only push
- Polling: Simpler, less efficient
- Push notifications: iOS/Android only

### Decision: WebSockets with fallback
- Approved by: Product Owner, CTO
- Date: 2024-03-15
- Review: Monitor battery impact, server costs monthly

Result: Informed decision, stakeholders aligned, mitigation in place

Lesson

Communicate trade-offs explicitly. Let stakeholders make informed decisions.

Common Anti-Patterns Summary

1. Keyword Stuffing

❌ Stuffing keywords to pass validation ✓ Honest, specific feature descriptions

2. Scope Wishful Thinking

❌ "Users want it, so it must be in scope" ✓ "Does PROJECT.md explicitly include this?"

3. Constraint Denial

❌ "Constraints don't apply to my feature" ✓ "How do I meet constraints or get approval?"

4. Architecture Cowboy Coding

❌ "I'll add whatever tech I want" ✓ "Does this fit architecture or should I propose change?"

5. Feature Creep Justification

❌ "This loosely relates to a goal, ship it" ✓ "Does this directly serve primary goals?"

6. Literal Scope Reading

❌ "The exact word isn't there, so no" ✓ "What's the intent behind scope section?"

7. Compliance Later

❌ "We'll handle security/compliance later" ✓ "Design for compliance from day 1"

8. Assumption Over Clarification

❌ "Dashboard probably means charts" ✓ "Let me clarify what dashboard includes"

9. Documentation Neglect

❌ "We'll update docs later" ✓ "Update PROJECT.md with each decision"

10. Trade-off Silence

❌ Hide trade-offs, surprise stakeholders later ✓ Document and communicate trade-offs upfront

How to Avoid Misalignment

1. Read PROJECT.md Thoroughly

  • Don't skim
  • Understand GOALS, SCOPE, CONSTRAINTS, ARCHITECTURE
  • Ask questions if unclear

2. Validate Early

  • Check alignment before implementing
  • Use alignment checklist
  • Get stakeholder confirmation

3. Document Decisions

  • Record rationale for choices
  • Update PROJECT.md with changes
  • Communicate trade-offs

4. Communicate Proactively

  • Don't hide constraint violations
  • Escalate conflicts early
  • Keep stakeholders informed

5. Update Continuously

  • PROJECT.md evolves with project
  • Document changes as you make them
  • Review alignment regularly

See Also:

  • alignment-scenarios.md - Correct alignment examples
  • ../docs/alignment-checklist.md - Systematic validation
  • ../docs/semantic-validation-approach.md - Understanding intent
  • project-md-structure-example.md - Well-structured PROJECT.md