Class AlphaValidation

java.lang.Object

com.quantfinlib.alpha.AlphaValidation

public final class AlphaValidation
extends Object

Validation for alpha factors — the overfitting defense, run before any capital-weighted conclusion is drawn:

• Walk-forward — pick the best factor variant on a training window by in-sample IC, measure it on the following unseen window, roll forward. The IS→OOS gap is the overfitting, measured.
• K-fold (blocked) cross-validation — the IC recomputed on k contiguous time blocks. Time-series data forbids shuffled folds (they leak adjacent bars across the train/test line), so blocks it is; a factor that only works in one block is a regime story, not a signal.
• Monte Carlo robustness — a permutation test: re-pair score dates with return dates at random to build the null distribution of mean IC, and report where the observed value falls. This asks the right question ("could this IC arise from no relationship?") without any normality assumption.
• Parameter sensitivity — mean IC across a parameter sweep, plus the worst drop between adjacent parameters. A real effect degrades smoothly as parameters move; a spike at exactly one value is the signature of a lucky backtest.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	AlphaValidation.CrossValidationResult	Per-block ICs with their dispersion — consistency across regimes.
static final record	AlphaValidation.Fold	One walk-forward fold: what was chosen, and how it did out of sample.
static final record	AlphaValidation.RobustnessResult	Observed mean IC against its permutation null distribution.
static final record	AlphaValidation.SensitivityResult	IC across the sweep plus the worst adjacent-parameter drop.
static final record	AlphaValidation.WalkForwardResult	All folds plus the aggregate in-sample vs out-of-sample comparison.

Method Summary

Modifier and Type	Method	Description
static AlphaValidation.CrossValidationResult	crossValidate(AlphaContext ctx, AlphaFactor factor, int horizon, int startIndex, int k)	Splits the evaluation range into k contiguous blocks and recomputes the mean IC inside each.
static AlphaValidation.RobustnessResult	monteCarloRobustness(AlphaContext ctx, AlphaFactor factor, int horizon, int startIndex, int trials, long seed)	Permutation test on the score/return pairing: per trial, scores from date tᵢ are paired with forward returns from a shuffled date tⱼ, destroying any true predictive link while preserving both marginal distributions.
static AlphaValidation.SensitivityResult	parameterSensitivity(AlphaContext ctx, List<AlphaFactor> sweep, int horizon, int startIndex)	Evaluates each candidate (an ORDERED parameter sweep — neighbors in the list must be neighbors in parameter space) and reports the worst IC drop between adjacent candidates.
static AlphaValidation.WalkForwardResult	walkForward(AlphaContext ctx, List<AlphaFactor> candidates, int horizon, int startIndex, int trainBars, int testBars)	Rolls a train/test split across the sample: each fold picks the candidate with the best training-window mean IC and scores it on the next testBars unseen bars.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

walkForward

public static AlphaValidation.WalkForwardResult walkForward(AlphaContext ctx,
 List<AlphaFactor> candidates,
 int horizon,
 int startIndex,
 int trainBars,
 int testBars)

Rolls a train/test split across the sample: each fold picks the candidate with the best training-window mean IC and scores it on the next testBars unseen bars.

Evaluation dates lie on ONE global grid (startIndex, stepping by the horizon) shared by every fold: consecutive folds' training windows overlap by trainBars − testBars, so scoring per fold would recompute the same (candidate, date) work up to trainBars/testBars times — instead the whole IC matrix is computed once (forward returns shared across candidates, too) and folds average slices of it. Window containment still holds: a date contributes to a window only when its ENTIRE forward window fits inside it.

Parameters:

candidates - the factor variants competing (e.g. one factor across a lookback grid)

crossValidate

public static AlphaValidation.CrossValidationResult crossValidate(AlphaContext ctx,
 AlphaFactor factor,
 int horizon,
 int startIndex,
 int k)

Splits the evaluation range into k contiguous blocks and recomputes the mean IC inside each. (Stateless factors have nothing to fit, so this is a pure consistency check — the honest reading of "cross-validation" for unfitted signals.)

monteCarloRobustness

public static AlphaValidation.RobustnessResult monteCarloRobustness(AlphaContext ctx,
 AlphaFactor factor,
 int horizon,
 int startIndex,
 int trials,
 long seed)

Permutation test on the score/return pairing: per trial, scores from date tᵢ are paired with forward returns from a shuffled date tⱼ, destroying any true predictive link while preserving both marginal distributions. The p-value is the fraction of trials whose |mean IC| reaches the observed |mean IC| (two-sided, add-one smoothed so p is never exactly 0).

Deliberate conservatism: a signal whose scores never change over time (a static ranking) is invariant under date permutation, so it earns p ≈ 1 regardless of its in-sample IC — correctly so, because a time-invariant cross-section against persistent drifts is one effective observation, however many dates it is sampled on. Only signals whose time variation aligns with return variation can earn a small p here.

parameterSensitivity

public static AlphaValidation.SensitivityResult parameterSensitivity(AlphaContext ctx,
 List<AlphaFactor> sweep,
 int horizon,
 int startIndex)

Evaluates each candidate (an ORDERED parameter sweep — neighbors in the list must be neighbors in parameter space) and reports the worst IC drop between adjacent candidates. Small drop = plateau = robust; large drop = the chosen parameter is a lucky spike.