Package com.quantfinlib.backtest.validation
@NullMarked
package com.quantfinlib.backtest.validation
Validation tools that test a backtest for overfitting. The main
classes:
ParameterGrid +
GridSearchOptimizer enumerate
and rank parameter combinations;
WalkForwardAnalyzer optimizes
on rolling train windows and evaluates on unseen test windows, stitching
out-of-sample equity (capital carries across folds) and reporting the
walk-forward efficiency ratio;
SharpeValidation applies the
Bailey/López de Prado probabilistic and deflated Sharpe (the
multiple-testing haircut for grid-picked winners) plus the minimum
track record length (how many periods before the record means
something, in closed form);
BlockBootstrap hands the
backtest Sharpe its sampling distribution (stationary Politis-Romano
blocks; an iid resample destroys the autocorrelation and understates
the uncertainty, the classic route to false confidence): the
question is whether the 5th percentile is still positive, not whether
1.2 is a good number.
Two more layers of defense:
PurgedKFold generates
cross-validation splits that purge training samples whose label windows
overlap the test fold and embargo the serially-correlated echo after it
— ordinary K-fold on financial data leaks the test answers into
training through forward-looking labels; and
OverfitProbability asks the
question that comes before any single track record: is the selection
process itself noise-mining? (CSCV: how often does the in-sample winner
rank below the out-of-sample median across every symmetric
train/test split.)
-
ClassDescriptionStationary block bootstrap (Politis-Romano) — the confidence interval your backtest's Sharpe ratio deserves and almost never gets.Exhaustive strategy parameter search: backtests every grid combination and ranks by an objective (e.g.One grid point: its parameters, metrics, objective and equity curve.Monte Carlo trade reshuffling: the answer to "was my equity curve's shape luck?".Drawdown distribution across trade-order shuffles.Probability of backtest overfitting via combinatorially symmetric cross-validation — CSCV (Bailey, Borwein, Lopez de Prado & Zhu 2015, "The probability of backtest overfitting").A named parameter grid for strategy optimization;
ParameterGrid.combinations()enumerates the cartesian product in deterministic order.Purged K-fold cross-validation splits with an embargo — the fix for the quiet leak that ordinary K-fold has on financial data (Lopez de Prado, Advances in Financial Machine Learning, ch. 7).One fold: test on[testFrom, testTo), train ontrainIndices(ascending, purged and embargoed).Sharpe ratio significance tests (Bailey & López de Prado): Probabilistic Sharpe Ratio — the probability the true Sharpe exceeds a benchmark, adjusting for track length and non-normal returns (skew, kurtosis). Deflated Sharpe Ratio — PSR against the Sharpe you'd expect from the best of N random trials: the multiple-testing haircut for a strategy picked from a parameter grid.Builds a strategy instance from one parameter combination.Walk-forward analysis.