Bias-Corrected feature selection for financial time series forecasting: a large-scale evaluation across 40 global assets

In financial forecasting, model selection has become a crucial problem because large-dimensional spaces of technical indicators need to be analyzed using meaningful signals in return series that are noisy and weakly predictive. Algorithms such as the feature selection algorithm (FSA), which apply simulated annealing to find sparse predictive subsets, have demonstrated good performance but have a methodological weakness: they habitually predict positive price changes by approximately 54% rather than a neutral 50/50 benchmark. This directional bias can distort long–short strategies, inflate hidden risk exposures, and undermine the reliability of model-driven trading systems. Although previous studies have shown the advantages of FSA in terms of sparsity and accuracy, its distributional balance in predictions has not yet been addressed, posing an unmet need in financial machine learning. To address this drawback, this paper presents the bias-corrected feature selection algorithm (BFSA), which extends the FSA loss to impose a direct penalty for deviations from a neutral benchmark of 0.5. The robustness of the penalty (λ_bias) was optimized using asset-specific five-fold chronological cross-validation, which enables BFSA to control its bias correction across varying volatility regimes and structural properties. BFSA was tested on daily Open, High, Low, Close, Volume (OHLCV) data for 40 assets tracking developed and emerging equity indices, major US-based equities, sector ETFs, commodities, and crypto markets, spanning over a decade of market history. BFSA achieved an average rank of 4.20, representing the strongest performance among all non-trivial predictive models. Only the Null model (1.48) and LASSO (1.85) obtained lower ranks, reflecting their simplicity rather than genuine predictive structure. BFSA outperformed the original FSA (5.55), all tree-based ensembles (ranks 9–12), and all deep learning hybrids (ranks 15–25), demonstrating that explicit bias control materially improves generalization in noisy financial settings. Cross-validated λ tuning improved performance in approximately 48% of assets and slightly degraded it in 52%, resulting in a near-zero net change in MSE. This indicates that the default λ_bias = 30 already has a robust general setting, while cross-validation primarily safeguards against severe hyperparameter misspecification in high-volatility assets. The wide dispersion in optimal λ values (4–135) highlights the structural heterogeneity of directional bias across asset classes. BFSA materially reduced the upward directional skew of FSA (mean prediction 0.537) and produced nearly balanced outputs (0.503), substantially improving calibration for long–short and market-neutral strategies.