Direction accuracy = the AI's predicted direction (long or short) moved at least 2% in that direction within a 28-day window, regardless of whether the strict target/stop was hit. This is the right metric for an AI macro-signal system because it measures whether the underlying thesis played out — exit rule tuning is a separate problem.

Confidence score is the model's own pre-trade self-assessment (1–5). The calibration above shows that the score is informative: higher-confidence calls have meaningfully higher hit rates.

Setup grade is the model's pre-trade self-rating of the whole opportunity (A+ / A / B+ / B / C), weighing catalyst clarity, regime alignment, indicator confluence, and sector setup. Tracking accuracy by grade tells us whether the model's own confidence ranking is calibrated — if A+ resolves at 90% directional and B+ at 50%, that's a clean signal to size differently on A+ setups vs B+ setups.

Methodology notes (full transparency):

• Resolved trades = predictions whose 28-day window has closed (target hit, stop hit, or expiry passed).
• Pre-April 2026 trades were reconstructed from the scanner's own log files after a database event; all assumed long direction (all in-DB rows are long). These rows have no setup_grade — they only contribute to the by-confidence accuracy view, not by-grade.
• Backfilled peak prices use daily High/Low from yfinance — slightly more aggressive than the production resolver's snapshot-at-15:30 tracking.
• Grade-accuracy samples start small (only post-April alerts have grades) and grow as new scans run. A bucket with n < 5 is flagged as "low sample" and should not be acted on yet.