Closed the loop from data collection to model improvement. The ML filter now retrains weekly on all historical data, tracks what would have happened on blocked trades, and reports results via Telegram. Pair confluence features from combo analysis add 5 new predictive signals.

1. ML Threshold Calibration

• Block threshold calibrated to base rate — Original thresholds (block <40%, boost >70%) would have blocked every trade since the base win rate is 30% (system profits via 3.25:1 payoff ratio, not win rate). Recalibrated: block <15% (half base rate), boost >40%.
• Validation — Blocked trades had 4.3% actual WR, boosted trades had 72.2% WR. Strong discrimination.

2. Self-Improving ML Pipeline

• Weekly retrain with model comparison — Sunday 23:00 UTC retrain compares accuracy and AUC delta against previous model. Feature importance tracking identifies which signals matter most.
• Live prediction audit — Joins ml_predictions with closed positions to verify ML discrimination is maintained in production (blocked vs boosted actual win rates).
• Telegram retrain report — Comprehensive report sent after each retrain: training stats, model comparison, threshold calibration, top 5 features, live audit, shadow P&L summary.

3. Shadow P&L for Blocked Trades

• Counterfactual tracking — Every blocked trade (ML filter, event guard, anti-stack, regime gate, etc.) logs entry price to shadow_pnl table. Price checked 4h later to determine if the trade would have won.
• Guard effectiveness validation — Shadow P&L by block reason shows whether each guard is filtering good or bad trades. Reported weekly in ML retrain Telegram report.

4. Pair Confluence Features

• Combo analysis on 652 historical trades — Atlas+Pulse pair: 62.2% WR (+£1,017, 45 trades). Oracle+Pulse: 8.3% WR. Cipher+Oracle: 7.7% WR. Strong predictive signal in which agents agree.
• 5 binary pair features in ML model — has_atlas_pulse, has_cipher_oracle, has_oracle_pulse, has_atlas_cipher, has_atlas_sentinel. Top feature after retrain: has_atlas_sentinel at 33.2% importance.

5. Weekly Anomaly Flags

• 7 automated flags in Telegram digest — Trade drought (5+ days), win rate drop (>15% vs prior week), agent accuracy below 30%, concentration risk (single instrument >60% of P&L), high ML block rate (>50%), shadow P&L showing over-aggressive guards, large weekly drawdown (>£50).

Comprehensive upgrade across six areas: mathematical edge, position sizing, agent intelligence, execution quality, and ML-based trade filtering. Built on the +£1,102 all-time paper performance (671 trades, 70% win rate last week).

1. Market Hours & Execution Fixes

• Market hours guard (Layer 0) — Skips instruments where the market is closed. Parses all IG config formats (24/7, 24/5, 08-21, 09:30-18:30). Prevents wasteful API calls and IG MARKET_CLOSED rejections.
• Extended rejection memory — MARKET_CLOSED and MARKET_ROLLED rejections now tracked with longer cooldown. Instruments aren't retried until is_market_open() returns True.

2. Kelly Criterion Edge & Position Sizing

• Edge formula replaced — Old: expected_value / avg_loss × conviction_ratio. New: (p × b - q) / b × conviction_ratio × regime_mult. Proper Kelly criterion with regime multipliers (trending=1.0, volatile=0.7, ranging=0.8).
• Position sizing from edge — kelly_raw = edge instead of avg_confidence × 0.5. Size now scales with actual mathematical edge, not just confidence.
• R:R enforcement (Layer 8) — Rejects trades where limit_distance / stop_distance < 1.5. Every trade must have asymmetric payoff.
• Pre-trade margin check — Skips if estimated margin exceeds 90% of available balance.

3. Non-Linear Agent Weights

• Weight formula — Old: accuracy × 2 (range 0.3–2.5). New: accuracy^1.5 × 3 (range 0.2–3.0). Cipher (83.7% acc) now gets 2.30 weight vs Sentinel (36.6%) at 0.66 — a 3.5x gap.
• EMA decay alpha — 0.10 → 0.15 for faster adaptation to recent performance.

4. Structure Agent (6th Voting Agent)

• Wyckoff phase detection — Identifies accumulation, distribution, markup, and markdown phases from 20-bar price/volume windows.
• Volume Profile (POC/Value Area) — Point of Control and Value Area computed from candle data. Signal: price above/below POC.
• Market structure (swing analysis) — Detects HH+HL (uptrend) and LH+LL (downtrend) patterns from swing highs and lows.
• Support/Resistance levels — Key levels from price clustering. Signal from proximity and reaction.
• Conviction sizing updated — {3: 1.0, 4: 1.5, 5: 2.5, 6: 3.5} to accommodate the 6th agent.

5. Congressional Trading Data

• House Stock Watcher collector — Fetches congressional stock transactions from public S3 data. Maps tickers to instruments (NVDA→NAS100, XOM→OIL_BRENT, GLD→XAU/USD). Daily at 10:00 UTC.
• Oracle integration — Congressional purchase/sale signals feed into Oracle's fundamental convergence logic for SP500, NAS100, FTSE100, OIL_BRENT, XAU/USD, coffee.

6. XGBoost ML Trade Filter (Layer 9)

• 15-feature binary classifier — Features: conviction, avg_confidence, num_dissenting, RSI, BB %B, ATR ratio, composite signal, Fear & Greed, ADX, BB width pctl, regime one-hot (3), hour of day, day of week.
• Win probability gate — Block if win_prob < 15% (well below 30% base rate). Boost conviction +1 if win_prob > 40%. Thresholds calibrated to base rate since system profits via R:R, not win rate. Graceful degradation: returns neutral 0.5 until 50+ training samples.
• Weekly retrain — Walk-forward split (80/20 chronological). Sunday 23:00 UTC. Model saved to data/xgb_trade_filter.pkl.
• Audit trail — Every ML prediction logged to ml_predictions table with features, win probability, and decision.

Account consolidation. Disabled the spreadbet account entirely — Mercurius now operates CFD-only. All UI simplified: removed the account selector dropdown, hardcoded to the CFD account. One account, one mode, less surface area for confusion.

Virtual bankroll. Set the virtual bankroll to £100,000 for all position sizing calculations. The IG demo account maintains ~£10K actual balance, but all sizing, risk parameters, and P&L tracking use the £100K virtual figure. This decouples position sizing from the demo balance constraint and lets the system trade at the scale it was designed for.

Risk parameters scaled to match:

Strategy presets scaled. Per-trade sizing across all presets updated to reflect the new bankroll:

P&L tracking decoupled from IG balance. The dashboard now uses DB-based P&L (sum of realized + unrealized from the positions table) instead of comparing the IG balance against starting capital. The header shows virtual portfolio value (bankroll + DB P&L), making performance reporting independent of IG account fluctuations.

Trade history archived. All pre-v2 data (638 positions, 1,868 trades) moved to archive tables (positions_archive, trades_archive). V2_CUTOFF set to May 20, 2025 — all stats, benchmarks, and performance tracking start fresh from this date.

Dashboard visual uplift. Refined CSS across all pages — improved card layouts, table styling, navigation consistency, typography spacing, and custom scrollbars. The system looks as serious as its ambitions.

After 21 days of live demo trading (651 closed positions), a comprehensive analysis revealed that the system was massively over-trading with inverted conviction signals. This overhaul restructures the entire trading pipeline.