Category: statistical arbitrage

I just returned from a two-day conference in New York, FutureAlpha (formerly QuantStrats). This year, the theme focused largely on data, machine learning, and AI. While some speakers were very enthusiastic about the potential of AI to generate alpha, our panel was more conservative. The consensus among the panelists was to use ML and AI to enhance and improve risk management. Along this theme, in this post, I discuss the use of generative AI in trading.

Integrating Structured and Unstructured Data with LLMs and RAG

Traditional quantitative methods often rely on structured data, such as time series. With the emergence of Large Language Models (LLMs), it is now possible to process unstructured data. A new line of research focuses on integrating unstructured data analysis into traditional frameworks.

Along this line, Reference [1] proposed the use of LLMs together with retrieval-augmented generation (RAG) to process both structured and unstructured data concurrently. Specifically, the authors developed a system that first applies LLMs to detect regime shifts using time-series techniques, then employs RAG to integrate external knowledge into the model’s decision-making process. By retrieving relevant information from a vector database and combining it with the model’s capabilities, RAG improves both the interpretability and effectiveness of trading strategies.

Findings

-The paper studies methods for fine-tuning open-source Large Language Models to enhance quantitative trading strategies.

-It integrates numerical data, such as prices and technical indicators, with textual data, including news and sentiment.

-The approach uses Retrieval-Augmented Generation with a vector database to process and contextualize textual information.

-The study focuses on fully fine-tuning smaller models to achieve cost efficiency and scalability.

-It proposes a hybrid framework that combines LLM capabilities with traditional quantitative methods.

-The framework incorporates real-time data pipelines and adaptive model tuning.

-The results show improvements in predictive accuracy and risk-adjusted returns.

-The integration of multimodal data helps address challenges in combining structured and unstructured information.

-Fine-tuned smaller models improve regime detection and trading decision accuracy while maintaining efficiency.

-Additional techniques enhance model performance and robustness, supporting practical applications in quantitative finance.

In short, incorporating RAG into the framework enhances the model’s ability to understand complex macroeconomic environments and adapt trading strategies as conditions evolve. Experimental results show significant gains in predictive accuracy and risk-adjusted returns, demonstrating the practical value of these fine-tuning methods in finance.

Reference

[1] Li, C., Chan, C.H.R., Huang, S.H., Choi, P.M.S. (2025). Integrating LLM-Based Time Series and Regime Detection with RAG for Adaptive Trading Strategies and Portfolio Management. In: Choi, P.M.S., Huang, S.H. (eds) Finance and Large Language Models. Blockchain Technologies. Springer, Singapore.

Can AI Trade? Modeling Investors with Large Language Models

The previous paper focuses on improving trading performance by integrating LLMs with quantitative models and data, while another line of research explores how LLMs behave as autonomous agents within market environments.

Reference [2] utilized LLMs to construct trading agents in the financial markets. Specifically, the author used LLMs to emulate various types of investors: value investors, momentum traders, market makers, retail traders, etc.

Findings

-The paper develops a simulated stock market in which large language models act as heterogeneous trading agents.

-The framework includes realistic market features such as an order book, market and limit orders, partial fills, dividends, and equilibrium clearing.

-Agents operate with different strategies, information sets, and endowments, and communicate decisions using structured outputs while explaining reasoning in natural language.

-The results show that LLMs can consistently follow instructions and implement strategies such as value investing, momentum trading, and market making.

-LLM agents process market information and respond meaningfully to prices, dividends, and historical data.

-The simulated market exhibits realistic dynamics, including price discovery, bubbles, underreaction, and liquidity provision.

-The framework enables controlled analysis of agent behavior under different market conditions, similar to interpretability methods in machine learning.

-It provides a cost-effective way to test financial theories that lack closed-form solutions.

-The study highlights that LLM behavior is highly sensitive to prompts, which can lead to correlated actions across agents.

-This correlation may amplify volatility and introduce systemic risks, emphasizing the need for careful testing before real-world deployment.

In short, the article concluded that trading strategies generated by large language models are effective, but could introduce new systemic risks to financial markets because these agents would act in a correlated manner.

Reference

[2] Alejandro Lopez-Lira, Can Large Language Models Trade? Testing Financial Theories with LLM Agents in Market Simulations, arXiv:2504.10789

Closing Thoughts

In this issue, the discussion highlights two complementary directions in applying LLMs to finance. On one hand, integrating LLMs with quantitative models and multimodal data can improve predictive accuracy and risk-adjusted returns. On the other hand, treating LLMs as autonomous trading agents reveals how their behavior can shape market dynamics, including liquidity, price discovery, and potential instability. Taken together, the results suggest that while LLMs offer meaningful opportunities in trading and risk management, their impact depends critically on implementation, prompting, and control of system-wide behavior.

Applications of machine learning in finance continue to evolve rapidly. In previous posts, we discussed both the uses and the challenges of applying machine learning in financial markets. In this installment, we continue that discussion by highlighting new research on machine learning approaches for pricing complex derivatives and identifying signals that may precede major market downturns.

Speeding Up Derivatives Pricing Using Machine Learning

A financial derivative is a contract whose value depends on the price of an underlying asset such as a stock, bond, commodity, or index. Accurate valuation of financial derivatives and their associated sensitivity factors is important for both investment and hedging purposes. However, many complex derivatives exhibit path-dependency and early-exercise features, which means that closed-form solutions rarely exist, and numerical methods must be used.

The issue with numerical methods is that they are often slow. As a result, efforts are being made to improve the efficiency of numerical techniques for valuing financial derivatives. Reference [1] proposed a fast valuation method based on machine learning. It developed a hybrid two-stage valuation framework that applies a machine learning algorithm to highly accurate derivative valuations incorporating full volatility surfaces. The volatility surface is parameterized, and a Gaussian Process Regressor (GPR) is trained to learn the nonlinear mapping from the complete set of pricing inputs directly to the valuation outputs. Once trained, the GPR delivers near-instantaneous valuation results.

Findings

-The study develops a machine learning framework for pricing derivative products whose valuation depends on volatility surfaces.

-Volatility surfaces are parameterized using the five-parameter SVI model with a one-factor term structure adjustment to generate realistic synthetic market scenarios.

-High-accuracy valuations for variance swaps and American put options are computed using conventional numerical methods and used to create training and testing datasets.

-A Gaussian Process Regressor is trained to learn the nonlinear relationship between input risk factors, such as volatility surface parameters, strike, and interest rate, and valuation outputs including prices and Greeks.

-The trained model achieves high accuracy, with approximately 0.5% relative error for variance swap fair strikes and 1.7–3.5% relative error for American put prices and first-order Greeks.

-The model is less accurate for the Gamma Greek due to discontinuities in the strike dimension.

-After training, the machine learning model produces valuations almost instantly, achieving a speed improvement of three to four orders of magnitude compared with traditional numerical methods.

-The results demonstrate that machine learning can enable real-time risk analytics, dynamic hedging, and large-scale scenario analysis for derivatives.

-The framework is general and can be extended to other path-dependent derivatives with early exercise features.

In summary, the authors developed an efficient method to price complex financial derivatives using a machine learning technique. However, it is noted that GPR’s performance in valuing higher-order greeks is noticeably less accurate. Additionally, the study was conducted using synthetic data, so it would be useful to see the method applied to real-world scenarios.

Reference

[1] Lijie Ding, Egang Lu, Kin Cheung,  Fast Derivative Valuation from Volatility Surfaces using Machine Learning, arXiv:2505.22957

Forecasting Market Crashes with Machine Learning Techniques

Reference [2] examines how machine learning can be used to predict market crashes within the Adaptive Market Hypothesis framework.

The study considers three categories of factors:

1. Internal factors, such as technical indicators designed to capture endogenous market dynamics, including momentum, trend strength, and money flow arising from investor behavior and adaptive learning;
2. External factors, including macroeconomic and commodity variables that proxy for systematic, exogenous risks affecting fundamental valuations; and
3. Volatility features that quantify market fear and uncertainty.

The author evaluates the performance of three predictive models—logistic regression, random forest, and a long short-term memory (LSTM) network.

Findings

-While the Efficient Market Hypothesis suggests crashes cannot be predicted, the Adaptive Market Hypothesis allows for temporary periods of predictability as market conditions evolve.

-The analysis compares a traditional econometric model, Logistic Regression, with machine learning approaches, including Random Forest and LSTM.

-The models use a feature set combining technical, macroeconomic, and volatility-based indicators.

-Model performance is evaluated using metrics designed for imbalanced classification problems, where crash events are rare but economically significant.

-Empirical results show that the LSTM provides the best balance between precision and recall, although Logistic Regression remains competitive.

-The findings highlight that simpler models can still perform effectively, supporting the value of model parsimony in turbulent market environments.

-The results also support the Adaptive Market Hypothesis by showing that market predictability evolves over time and depends on changing conditions.

-Logistic Regression performs well as an early-warning system due to its high recall, although it generates many false positives.

-The LSTM model improves precision while maintaining strong recall, suggesting that capturing temporal patterns in financial data enhances predictive performance.

-Overall, the study concludes that market crashes are not entirely random, but their prediction depends on the appropriate balance between model complexity and practical application.

In short, the study concludes that market crashes are difficult to forecast but not entirely random, and different models capture different aspects of predictability. Logistic regression functions well as a high-recall early warning tool, while LSTM models provide more balanced signals.

Reference

[2] Michele Della Mura, Predicting Stock Market Crashes, A Comparative Analysis of Econometric and Machine Learning Models, Politecnico di Torino, 2025

Closing Thoughts

Taken together, these studies illustrate the expanding role of machine learning in modern quantitative finance. One line of research demonstrates how machine learning models can dramatically accelerate the pricing of complex derivatives while maintaining high accuracy, enabling real-time risk management and large-scale scenario analysis. Another line of work explores the ability of both traditional econometric methods and advanced machine learning models to identify signals that may precede market crashes. Collectively, these findings show that machine learning is reshaping financial modeling, though simpler approaches can still play a meaningful role.

Herding behavior has been extensively studied and is well understood in equity markets, but far less so in other asset classes such as commodities and cryptocurrencies. In this post, we explore key aspects of herding behavior in crypto and commodity markets.

Investor Behavior in Crypto During Geopolitical Shocks

Herd behavior refers to the tendency of investors to follow the actions of a larger group, often ignoring their own analysis or information. This collective movement can lead to asset bubbles during bull markets and sharp sell-offs during downturns. Understanding herd behavior is essential for identifying potential mispricings and avoiding emotionally driven decisions.

Herding behavior has been well studied in the equity markets, but less so in the cryptocurrency market. One might expect stronger herding in crypto due to the prevalence of young, inexperienced traders and the fact that crypto markets are under-regulated, less transparent, and highly volatile. However, existing studies have produced inconclusive results.

Reference [1] extends the research on herding in the crypto space by examining behavior during major geopolitical events, such as the COVID-19 pandemic and the Russia–Ukraine war.

Findings

-The study finds strong evidence of market-wide herding behavior in cryptocurrency markets by analyzing the relationship between return dispersion and market returns.

-Geopolitical risk (GPR) significantly amplifies herding, with severe herding detected across nearly all model specifications.

-The GPR Threat index has a stronger impact on herding than the GPR Act index, indicating that perceived geopolitical threats matter more than realized events.

-Herding behavior is asymmetric, occurring more intensely during bearish market conditions than bullish ones.

-Imitative trading is particularly pronounced during periods of market stress, confirming the presence of asymmetric herding.

-The strongest herding effects are observed during extreme geopolitical and global events, notably the COVID-19 pandemic and the Russia–Ukraine war.

-The findings suggest that herding in cryptocurrency markets is largely intentional, reflecting low information symmetry, weak disclosure, and limited information quality.

-Actual geopolitical events (GPR Act) tend to lose explanatory power because market participants rapidly process and price in the information once it is released.

-When realized geopolitical shocks exceed investor expectations, uncertainty rises sharply and herding intensifies.

In short, the authors found that herding intensifies during such events and is clearly present throughout these periods.

Reference

[1] Phasin Wanidwaranan, Jutamas Wongkantarakorn, Chaiyuth Padungsaksawasdi, Geopolitical risk, herd behavior, and cryptocurrency market, The North American Journal of Economics and Finance Volume 80, September 2025, 102487

Does Herding Behavior Exist in the Commodity Markets?

Herding behavior has been shown to exist in equity markets. Reference [2] examines the herding behavior in the commodity markets.

Findings

-The study investigates herding behavior in commodity ETFs using high-frequency microstructure data and a GARCH model that incorporates cross-sectional and market volatility at 15-, 30-, 45-, and 60-minute intervals.

-During periods of market instability and the COVID-19 pandemic, agricultural and metal-based ETFs generally exhibit weaker herding behavior, while energy-based ETFs tend to herd more.

-Under normal market conditions, herding typically emerges at frequencies longer than 30 minutes.

-Broad basket commodity ETFs and energy-based ETFs display herding behavior across multiple frequencies rather than at a single time scale.

-A notable exception is agricultural ETFs during the COVID-19 pandemic, where herding is observed across all frequencies, representing a key and unusual finding.

-Correlation analysis shows that commodity ETFs become less correlated with each other as time progresses.

-Lower observation frequencies are associated with weaker correlations across ETFs, except in the energy sector.

-The results suggest that herding behavior varies significantly by commodity type, market regime, and observation frequency.

The findings provide insights for investors, economists, and policymakers, particularly for designing diversification, hedging strategies and mitigating risks such as asset price bubbles and financial instability.

Reference

[2] Ah Mand, Abdollah and Sifat, Imtiaz and Ang, Wei Kee and Choo, Jian Jing, Herding Behavior in Commodity Markets. SSRN 4502804

Closing Thoughts

Taken together, these two studies show that herding behavior extends well beyond equity markets and plays a meaningful role in both cryptocurrencies and commodity ETFs, particularly under stress. In crypto markets, herding is strongly amplified by geopolitical risk, bearish conditions, and extreme events. In commodity ETFs, herding is more nuanced and highly dependent on asset class, market regime, and trading frequency, with energy and broad commodity baskets exhibiting persistent herding, while agricultural and metal ETFs remain relatively resilient except during extreme volatility.

Overall, the evidence suggests that herding is regime-dependent, frequency-specific, and asset-class-specific, with important implications for risk management, diversification, and the design of trading and hedging strategies during periods of market stress.

Pairs trading, or statistical arbitrage (stat arb), is a classic, well-established quantitative trading strategy, and it is still in use today. I discussed its profitability in a previous post, and in this installment, we continue that discussion.

Pairs Selection Methods

Reference [1] provides a thorough review of the pairs trading literature between 2016 and 2023.

Pair selection is a critical step in pairs trading, and the paper offers a comprehensive review of the various pair selection methods used in practice. They are:

1-Distance Methods

Use SSE/SAE of normalized price differences to identify co-moving assets. Simple, intuitive, and historically profitable across markets, even after costs.

2-Cointegration Methods

Exploit long-run equilibrium relationships. Strong empirical support across equities and bonds, with advances in regime switching and external-factor integration.

3-Stochastic Control Methods

Model pairs trading as a continuous-time optimization problem. Incorporate jumps, regime changes, and stochastic volatility, showing strong performance but facing practical frictions.

4-Time Series Methods

Use GARCH, OU, and fractional OU to model short-term dynamics and volatility clustering. Adaptive thresholds improve returns; hybrid models are an emerging area.

5-Other Methods

Copulas capture tail dependence; Hurst exponent methods capture long memory; entropic approaches address model uncertainty. These improve robustness under nonlinear dynamics.

Overall, the review helps practitioners adapt stat-arb techniques to new markets and regimes. While simple methods once worked well, today’s competitive environment often requires more sophisticated approaches, though success still depends on model design, data quality, and market regime.

Profitability of Pairs Trading

There is an ongoing debate in the literature—some argue that “pairs trading is dead,” while others maintain that it remains profitable. From this review paper [1], we learn the following.

1- Pairs trading remains profitable, but returns are weaker and more conditional

The survey explicitly notes that profitability persists, but is not uniform and depends on market conditions, costs, and implementation details:

Empirical evidence consistently shows that distance-based pairs trading can be profitable across different markets, asset classes, and time horizons.

However, this is immediately tempered elsewhere by declining performance stability:

Performance is not uniform over time: profitability tends to vary with market volatility, and Sharpe ratios decline in certain subperiods.

2. Transaction costs and competition materially erode profits

Modern profitability survives only after careful cost control, unlike the early 2000s results:

Even after accounting for realistic transaction costs, the strategy remains profitable in several markets.

3. Advanced methods outperform naïve approaches

The paper makes clear that simple Gatev-style [2] implementations are no longer sufficient:

The apparent simplicity of GGR’s strategy becomes less evident as more sophisticated models and techniques have been introduced.

And later:

Regime-switching structures … demonstrate superior performance, particularly under frequent or pronounced regime shifts.

In short, the paper does not argue that pairs trading has stopped working, but it makes clear that the simple, mechanical versions that worked in the 1990s and early 2000s no longer deliver robust returns. Profitability today is weaker, highly dependent on market regimes, and much more sensitive to transaction costs and execution. What survives is not the original Gatev–Goetzmann–Rouwenhorst method, but more adaptive, model-driven implementations that account for changing volatility, correlations, and liquidity.

Reference

[1] Sun, Y. (2025). A survey of statistical arbitrage pairs trading strategies with non-machine learning methods, 2016-2023. WNE Working Papers, 19/2025 (482). Faculty of Economic Sciences, University of Warsaw

[2] Gatev, E., Goetzmann, W., & Rouwenhorst, K. G. (2006). Journal of Financial Economics, 81(1), 105–141.

Closing Thoughts

The paper provides a thorough review of all existing pair selection methods, which are critical to pairs trading. It also concludes that current profitability is weaker, highly dependent on market regimes, and significantly more sensitive to transaction costs and execution.

Most research in portfolio management focuses on alpha generation; however, another critical component of portfolio construction is position sizing. In this post, we examine key considerations in position sizing, including the Kelly criterion and the martingale betting system.

Does Kelly Portfolio Outperform the Market?

A method for capital allocation and position sizing is to employ the Kelly criterion. The Kelly criterion aims to optimize the expected growth rate of capital, maximizing the anticipated value of the logarithm of wealth. This strategy is rooted in John Kelly’s paper, “A New Interpretation of Information Rate.” According to Kelly, in repeated bets, a bettor should act to maximize the expected growth rate of capital, thus maximizing expected wealth at the end.

Reference [1] applies Thorp’s approach, as outlined in “The Kelly Criterion in Blackjack, Sports Betting and the Stock Market,” [2]  to construct a portfolio in the Norwegian stock market. The formula computes the optimal investment fraction in a set of assets, considering the expected excess returns of the assets and the inverse of the variance-covariance matrix.

Findings

-The study evaluates the performance of a growth-optimal Kelly portfolio in the Norwegian stock market over the period February 2003 to December 2022.

-It assesses abnormal performance using the CAPM, Fama–French three-factor model, and Carhart four-factor model.

-The Kelly portfolio achieves a higher compound annual growth rate (14.1%) and higher ending wealth than the benchmark index, which grows at 12%.

-It also outperforms a Markowitz portfolio, which delivers lower growth and final wealth.

-The Kelly portfolio and the benchmark exhibit similar Sharpe ratios (0.58), while the Kelly portfolio attains a higher Sortino ratio (0.95).

-Factor regressions indicate an annualized alpha of 16.8% for the Kelly portfolio, statistically significant at the 1% level before transaction costs.

-However, the factor models display very low explanatory power, suggesting that the estimated alpha may be overstated.

-Once transaction costs are incorporated, the Kelly portfolio no longer outperforms the benchmark in terms of final wealth.

-After costs, the alpha remains only marginally significant at the 10% level, implying limited real-world risk-adjusted excess returns.

This paper presents several interesting findings,

-First, the correlation of the Kelly portfolio with the market is nearly zero.

-Second, the performance is sensitive to transaction costs. We believe that with lower transaction costs, the Kelly portfolio has the potential to outperform the market and display zero correlation with it.

-Third, the Kelly portfolio surpasses the Markowitz mean-variance portfolio in performance.

We also concur with the author that the utilization of options can further enhance the risk-adjusted return.

Reference

[1] Jon Endresen and Erik Grødem, The Kelly criterion, an empricial study of the growth optimal Kelly portfolio, backtested on the Oslo Stock Exchange, 2023, Norwegian School of Economics.

[2] Thorp, E. O., The Kelly Criterion in Blackjack Sports Betting and the Stock Market, in: Zenios, S.A. & Ziemba, W.T., Handbook of Asset and Liability Management, Volume 1, 387–428, 2006

Enhanced Martingale Betting System with Stop Policy

The martingale betting system is a popular gambling strategy that involves doubling one’s wager after each loss in the pursuit of recovering previous losses and securing a profit equal to the original bet. The underlying idea is that, statistically, a win will eventually occur, allowing the player to recoup losses and gain a net profit equal to the initial stake. While simple in concept, the martingale system carries inherent risks, as it assumes unlimited funds for doubling bets and disregards the fact that losing streaks can persist longer than expected. Thus, this system will eventually result in bankruptcy.

Reference [3] however argues that different perspectives exist regarding whether stock price movements adhere strictly to a random walk, often modeled as a geometric Brownian motion. This suggests a potential for enhancement in the martingale betting system. The author has subsequently introduced an enhanced martingale betting system that includes a stop policy.

Findings

-The paper proposes an Improved Martingale Betting System (IMBS) by modifying the traditional martingale strategy with a stop policy and adapting it from casino gambling to intraday trading.

-The IMBS is empirically tested using TAIEX (TX) futures across three intraday trading strategies.

-Results show that the IMBS delivers strong performance and is applicable to TX intraday trading and related markets.

-The study finds that returns increase with leverage up to a certain threshold, beyond which traditional martingale strategies face a high probability of bankruptcy.

-By controlling key parameters—specifically leverage scaling (a), the number of steps (n), and total leverage—the IMBS significantly outperforms both the Equal-Weight Betting System (EWBS) and the traditional Martingale Betting System (MBS).

-The inclusion of a stop-loss mechanism further improves performance and risk control.

-Empirical tests indicate that IMBS performs particularly well when combined with price breakout strategies, which are identified as the most profitable approach for TX intraday trading.

In short, after testing on real data, the article concludes that

-The conventional martingale betting system inevitably leads to bankruptcy,

-With the integration of a stop policy, the new and improved martingale betting system demonstrates enhanced efficacy.

Reference

[3] Ting-Yuan Chen, and Szu-Lang Liao, Improved Martingale Betting System for Intraday Trading in Index Futures—Evidence of TAIEX Futures, Asian Journal of Economics and Business, Year:2023, Vol.4 (2), PP.339-366

Closing Thoughts

Taken together, the two studies highlight the trade-off between growth maximization and risk control in position sizing. The Kelly-based approach demonstrates strong theoretical and empirical growth performance, but its apparent alpha weakens once transaction costs and model limitations are accounted for, raising questions about real-world applicability. By contrast, the Improved Martingale Betting System shows that disciplined leverage control and stop policies can materially improve intraday trading outcomes relative to naive martingale schemes, especially when combined with breakout strategies. Overall, both strands of research suggest that position sizing is as critical as signal generation, and that practical constraints, parameter calibration, and market frictions ultimately determine whether theoretically attractive sizing rules translate into sustainable performance.

Fractal Market Hypothesis is an alternative framework that models financial markets through long-memory and multi-scale dynamics. There is a growing trend in the industry to incorporate it—first in analyzing the behavior of underlying assets, and more recently in the pricing of financial derivatives such as futures. In this post, we will examine these developments.

Fractal Market Hypothesis: Quantification and Usage

The Fractal Market Hypothesis (FMH) is a theory that suggests that financial markets behave in the same way as natural phenomena and are subject to the same physical laws as found in nature. It suggests that financial markets are composed of similar patterns which repeat over and over again at different scales. These patterns can be used to identify market trends and can help investors make more informed decisions.

The Fractal Market Hypothesis is one of the alternatives to the Efficient Market Hypothesis (EMH) which states that all available information is already factored into the price of a security. The other alternative is the Adaptive Market Hypothesis (AMH).

Reference [1] examined how the fractal nature of the financial market can be quantified and used in investment analysis.

Fractal Market Hypothesis (FMH):

-Suggests financial markets mimic natural phenomena, governed by the same physical laws.

-Identifies repeating patterns at different scales in financial markets.

-Offers a quantitative description of how financial time series change.

Comparison with Efficient Market Hypothesis (EMH):

-FMH contrasts with EMH, which claims all available information is already reflected in security prices.

-FMH, along with Adaptive Market Hypothesis (AMH), presents alternatives to EMH.

Quantification and Usage of FMH:

-The paper quantifies the fractal nature of developed and developing market indices.

-FMH posits self-similarity in financial time series due to investor interactions and liquidity constraints.

-Market stability is influenced by liquidity and investment horizon heterogeneity.

Market Dynamics and Stability:

-FMH suggests that during normal conditions, diverse investor objectives maintain liquidity and orderly price movements.

-Under stressed conditions, herding behavior reduces liquidity, leading to market destabilization through panic selling.

Reference

[1] A. Karp and Gary Van Vuuren, Investment implications of the fractal market hypothesis, 2019 Annals of Financial Economics 14(01):1950001

Fractional Geometric Brownian Motion and Its Application to Futures Arbitrage

While the previous paper discusses the FMH from an investment perspective, Reference [2] reflects a recent trend in quantitative research—namely, incorporating the FMH into the pricing of financial derivatives.

The paper proposed an extension based on fractional Brownian motion (FBM), which incorporates trend fractal dimensions (FTD)—distinguishing between upward (D⁺) and downward (D⁻) dimensions—combined with momentum lifecycle theory.

The authors developed a pricing framework for futures under this setup. Because FBM is not a semi-martingale in the classical sense, they adjusted the drift of the log-price process to reconcile fractal dynamics with approximate arbitrage-free pricing.

Afterward, they constructed a futures pricing model and designed an arbitrage strategy based on the futures–cash basis. The strategy operates as follows:

-Rule 1: Execute a positive arbitrage (sell futures, buy spot/ETF) when the basis series enters the low reversal phase, as identified by the conditions on D⁺ and D⁻.

-Rule 2: Close the positive arbitrage position (buy futures, sell spot/ETF) when the basis series enters the high reversal phase, or, depending on market rules and strategy design, open a negative arbitrage position.

Findings

-The study challenges traditional futures pricing models based on the efficient market hypothesis, noting their limitations in capturing complex market behavior and their tendency to produce significant pricing errors.

-It introduces the fractal market hypothesis (FMH) as a more effective framework that accounts for long memory and multi-scale market dynamics.

-A fractal futures pricing model is developed by incorporating the Hurst exponent and a cash-futures arbitrage strategy that uses trend fractal dimensions (D⁺ and D⁻) and momentum lifecycle logic to generate dynamic trading signals.

-Empirical testing using CSI 300 data shows that the fractal model substantially reduces pricing errors relative to the traditional cost-of-carry model.

-The proposed fractal-based arbitrage strategy achieves higher returns, stronger risk-adjusted performance, and lower drawdowns compared to conventional static-threshold approaches.

-Backtesting results indicate a total return of 12.71% versus 7.06% for the traditional strategy, with a positive Sharpe ratio of 0.32 compared to a negative −0.61.

-The strategy demonstrates exceptional resilience during market stress, such as the 2015 crash, limiting losses to −0.83% while traditional approaches lost −5.82%.

-This robustness under extreme conditions highlights the model’s effectiveness for both profitability and capital preservation.

Overall, the findings validate the practical value of the fractal market hypothesis for developing adaptive, accurate, and profitable pricing and arbitrage tools.

Reference

[2] Xu Wu and Yi Xiong, A fractal market perspective on improving futures pricing and optimizing cash-and-carry arbitrage strategies, Quantitative Finance and Economics, Volume 9, Issue 4, 713–744.

Closing Thoughts

In summary, both articles underscore the growing relevance of the Fractal Market Hypothesis as an alternative framework for understanding modern financial markets. The first article outlines FMH’s theoretical foundation, emphasizing its focus on multi-scale behavior, liquidity, and investor horizon heterogeneity. The second article extends this perspective into practical applications, demonstrating how fractal-based pricing models and arbitrage strategies can outperform traditional approaches and remain resilient under stress. Together, they show that FMH is evolving from a descriptive theory into a useful quantitative tool for pricing, risk management, and strategy design.

Gold prices have risen sharply in recent months, prompting renewed debate over whether the market has reached its peak. In this post, we examine quantitative models used to forecast gold prices and evaluate their effectiveness in capturing volatility and market dynamics. However, gold is not only a speculative vehicle, it also functions as an effective hedging instrument. We explore both aspects to provide a comprehensive view of gold’s role in modern portfolio management.

Comparative Analysis of Gold Forecasting Models: Statistical vs. Machine Learning Approaches

Gold is an important asset class, serving as both a store of value and a hedge against inflation and market uncertainty. Therefore, performing predictive analysis of gold prices is essential. Reference [1] evaluated several predictive methods for gold prices. It examined not only classical, statistical approaches but also newer machine learning techniques. The study used data from 2021 to 2025, with 80% as in-sample data and 20% as validation data.

Findings

-The study analyzes gold’s forecasting dynamics, comparing traditional statistical models (ARIMA, ETS, Linear Regression) with machine learning methods (KNN and SVM).

-Daily gold price data from 2021 to 2025 were used for model training, followed by forecasts for 2026.

-Descriptive analysis showed moderate volatility (σ = 501.12) and strong cumulative growth of 85%, confirming gold’s ongoing role as a strategic safe-haven asset.

-Empirical results indicate that Linear Regression (R² = 0.986, RMSE = 35.7) and ETS models achieved superior forecasting accuracy compared to ARIMA, KNN, and SVM.

-Machine learning models (KNN and SVM) underperformed, often misrepresenting volatility and producing higher forecast errors.

-The results challenge the assumption that complex algorithms necessarily outperform traditional methods in financial forecasting.

-Forecasts for 2026 project an average gold price of $4,659, corresponding to a 58.6% potential return.

-The study cautions that these forecasts remain sensitive to macroeconomic shocks and market uncertainties.

-The findings emphasize that simpler, transparent, and interpretable models can outperform more complex machine learning approaches in volatile market conditions.

In short, the paper shows that,

-Linear Regression and ETS outperformed ARIMA, KNN, and SVM, delivering the lowest error and highest explanatory power,

-Machine learning models (KNN, SVM) did not outperform traditional statistical methods, emphasizing the value of interpretability and stability in volatile markets.

Another notable aspect of the study is its autocorrelation analysis, which reveals that, unlike equities, gold does not exhibit clear autocorrelation patterns—its price behavior appears almost random. The paper also suggested improving the forecasting model by incorporating macroeconomic variables.

Reference

[1] Muhammad Ahmad, Shehzad Khan, Rana Waseem Ahmad, Ahmed Abdul Rehman, Roidar Khan, Comparative analysis of statistical and machine learning models for gold price prediction, Journal of Media Horizons, Volume 6, Issue 4, 2025

Using Gold Futures to Hedge Equity Portfolios

Hedging is a risk management strategy used to offset potential losses in one investment by taking an opposing position in a related asset. By using financial instruments such as options, futures, or derivatives, investors can protect their portfolios from adverse price movements. The primary goal of hedging is not to maximize profits but to minimize potential losses and provide stability.

Reference [2] explores hedging basic materials portfolios using gold futures.

Findings

-The study examines commodities as alternative investments, hedging instruments, and diversification tools.

-Metals, in particular, tend to be less sensitive to inflation and exhibit low correlation with traditional financial assets.

-Investors can gain exposure to metals through shares of companies in the basic materials sector, which focus on exploration, development, and processing of raw materials.

-Since not all companies in this sector are directly linked to precious metals, the study suggests including gold futures to enhance portfolio diversification.

-The research compares a portfolio composed of basic materials sector stocks with a similar portfolio hedged using gold futures.

-Findings show that hedging with gold reduces both profits and losses, providing a stabilizing effect suitable for risk-averse investors.

-The analysis used historical data from March 1, 2018, to March 1, 2022, and tested several portfolio construction methods, including equal-weight, Monte Carlo, and mean-variance approaches.

-Between March 2022 and November 2023, most portfolios without gold futures experienced losses, while portfolios with short gold futures positions showed reduced drawdowns and more stable performance.

-The basis trading strategy using gold futures did not change the direction of returns but significantly mitigated volatility and portfolio swings.

In short, the study concludes that hedging base metal equity portfolios with gold futures can effectively reduce PnL volatility and enhance portfolio stability, offering a practical approach for conservative investors and professional asset managers.

Reference

[2] Stasytytė, V., Maknickienė, N., & Martinkutė-Kaulienė, R. (2024), Hedging basic materials equity portfolios using gold futures, Journal of International Studies, 17(2), 132-145.

Closing Thoughts

In summary, gold can serve as an investment, a speculative vehicle, and a hedging instrument. In the first article, simpler models such as Linear Regression and ETS outperformed complex algorithms in forecasting gold prices, emphasizing the importance of interpretability in volatile markets. In the second, incorporating gold futures into base metal portfolios reduced profit and loss volatility, offering stability for risk-averse investors. Together, the studies highlight gold’s dual function as both a return-generating asset and a tool for risk management.

Identifying market regimes is essential for understanding how risk, return, and volatility evolve across financial assets. In this post, we examine two quantitative approaches to regime detection.

Hedge Effectiveness Under a Four-State Regime Switching Model

Identifying market regimes is important for understanding shifts in risk, return, and volatility across financial assets. With the advancement of machine learning, many regime-switching and machine learning methods have been proposed. However, these methods, while promising, often face challenges of interpretability, overfitting, and a lack of robustness in real-world deployment.

Reference [1] proposed a more “classical” regime identification technique. The authors developed a four-state regime switching (PRS) model for FX hedging. Instead of using a simple constant hedge ratio, they classified the market into regimes and optimized hedge ratios accordingly.

Findings

-The study develops a four-state regime-switching model for optimal foreign exchange (FX) hedging using forward contracts.

-Each state corresponds to distinct market conditions based on the direction and magnitude of deviations of the FX spot rate from its long-term trend.

-The model’s performance is evaluated across five currencies against the British pound over multiple investment horizons.

-Empirical results show that the model achieves the highest risk reduction for the US dollar, euro, Japanese yen, and Turkish lira, and the second-best performance for the Indian rupee.

-The model demonstrates particularly strong performance for the Turkish lira, suggesting greater effectiveness in hedging highly volatile currencies.

-The model’s superior results are attributed to its ability to adjust the estimation horizon for the optimal hedge ratio according to current market conditions.

-This flexibility enables the model to capture asymmetry and fat-tail characteristics commonly present in FX return distributions.

-Findings indicate that FX investors use short-term memory during low market conditions and long-term memory during high market conditions relative to the trend.

-The model’s dynamic structure aligns with prior research emphasizing the benefits of updating models with recent data over time.

-Results contribute to understanding investor behavior across market regimes and offer practical implications for mitigating behavioral biases, such as panic during volatile conditions.

In short, the authors built a more efficient hedging model by splitting markets into four conditions instead of two, adjusting hedge ratios and memory length depending on the volatility regime. This significantly improves hedge effectiveness, especially in volatile currencies.

We believe this is an efficient method that can also be applied to other asset classes, such as equities and cryptocurrencies.

Reference

[1] Taehyun Lee, Ioannis C. Moutzouris, Nikos C. Papapostolou, Mahmoud Fatouh, Foreign exchange hedging using regime-switching models: The case of pound sterling, Int J Fin Econ. 2024;29:4813–4835

Using the Gaussian Mixture Models to Identify Market Regimes

Reference [2] proposed an approach that uses the Gaussian Mixture Models to identify market regimes by dividing it into clusters. It divided the market into 4 clusters or regimes,

Cluster 0: a disbelief momentum before the breakout zone,

Cluster 1: a high unpredictability zone or frenzy zone,

Cluster 2: a breakout zone,

Cluster 3: the low instability or the sideways zone.

Findings

-Statistical analysis indicated that the S&P 500 OHLC data followed a Gaussian (Normal) distribution, which motivated the use of Gaussian Mixture Models (GMMs) instead of k-means clustering, since GMMs account for the distributional properties of the data.

-Traditional trading strategies based on the Triple Simple Moving Average (TSMA) and Triple Exponential Moving Average (TEMA) were shown to be ineffective across all market regimes.

-The study identified the most suitable regimes for each strategy to improve portfolio returns, highlighting the importance of regime-based application rather than uniform use.

-This combined approach of clustering with GMM and regime-based trading strategies demonstrated potential for improving profitability and managing risks in the S&P 500 futures market.

In short, the triple moving average trading systems did not perform well. However, the authors managed to pinpoint the market regimes where the trading systems performed better, relatively speaking.

Reference

[2] F. Walugembe, T. Stoica, Evaluating Triple Moving Average Strategy Profitability Under Different Market Regimes, 2021, DOI:10.13140/RG.2.2.36616.96009

Closing Thoughts

Both studies underscore the importance of regime identification and adaptive modeling in financial decision-making. The four-state regime-switching hedging model demonstrates how incorporating changing market conditions enhances risk reduction in foreign exchange markets, while the Gaussian Mixture Model approach illustrates how clustering can effectively capture distinct market phases in equity trading. Together, they highlight the value of data-driven, regime-aware frameworks in improving both risk management and trading performance.

Much emphasis has been placed on developing accurate and robust financial models, whether for pricing, trading, or risk management. However, a crucial yet often overlooked component of any quantitative system is the reliability of the underlying data. In this post, we explore some issues with financial data and how to address them.

How to Deal with Missing Financial Data?

In the financial industry, data plays a critical role in enabling managers to make informed decisions and manage risk effectively. Despite the critical importance of financial data, it is often missing or incomplete. Financial data can be difficult to obtain due to a lack of standardization and regulatory requirements. Incomplete or inaccurate data can lead to flawed analysis, incorrect decision-making, and increased risk.

Reference [1] studied the missing data in firms’ fundamentals and proposed methods for imputing the missing data.

Findings

-Missing financial data affects more than 70% of firms, representing approximately half of total market capitalization.

-The authors find that missing firm fundamentals exhibit complex, systematic patterns rather than occurring randomly, making traditional ad-hoc imputation methods unreliable.

-They propose a novel imputation method that utilizes both time-series and cross-sectional dependencies in the data to estimate missing values.

-The method accommodates general systematic patterns of missingness and generates a fully observed panel of firm fundamentals.

-The paper demonstrates that addressing missing data properly has significant implications for estimating risk premia, identifying cross-sectional anomalies, and improving portfolio construction.

-The issue of missing data extends beyond firm fundamentals to other financial domains such as analyst forecasts (I/B/E/S), ESG ratings, and other large financial datasets.

-The problem is expected to be even more pronounced in international data and with the rapid expansion of Big Data in finance.

-The authors emphasize that as data sources grow in volume and complexity, developing robust imputation methods will become increasingly critical.

In summary, the paper provides foundational principles and general guidelines for handling missing data, offering a framework that can be applied to a wide range of financial research and practical applications.

We think that the proposed data imputation methods can be applied not only to fundamental data but also to financial derivatives data, such as options.

Reference

[1] Bryzgalova, Svetlana and Lerner, Sven and Lettau, Martin and Pelger, Markus, Missing Financial Data SSRN 4106794

Predicting Realized Volatility Using High-Frequency Data: Is More Data Always Better?

A common belief in strategy design is that ‘more data is better.’ But is this always true? Reference [2] examined the impact of the quantity of data in predicting realized volatility. Specifically, it focused on the accuracy of volatility forecasts as a function of data sampling frequency. The study was conducted on crude oil, and it used GARCH as the volatility forecast method.

Findings

-The research explores whether increased data availability through higher-frequency sampling leads to improved forecast precision.

-The study employs several GARCH models using Brent crude oil futures data to assess how sampling frequency influences forecasting performance.

-In-sample results show that higher sampling frequencies improve model fit, indicated by lower AIC/BIC values and higher log-likelihood scores.

-Out-of-sample analysis reveals a more complex picture—higher sampling frequencies do not consistently reduce forecast errors.

-Regression analysis demonstrates that variations in forecast errors are only marginally explained by sampling frequency changes.

-Both linear and polynomial regressions yield similar results, with low adjusted R² values and weak correlations between frequency and error metrics.

-The findings challenge the prevailing assumption that higher-frequency data necessarily enhance forecast precision.

-The study concludes that lower-frequency sampling may sometimes yield better forecasts, depending on model structure and data quality.

-The paper emphasizes the need to balance the benefits and drawbacks of high-frequency data collection in volatility prediction.

-It calls for further research across different assets, markets, and modeling approaches to identify optimal sampling frequencies.

In short, increasing the data sampling frequency improves in-sample prediction accuracy. However, higher sampling frequency actually decreases out-of-sample prediction accuracy.

This result is surprising, and the author provided some explanation for this counterintuitive outcome. In my opinion, financial time series are usually noisy, so using more data isn’t necessarily better because it can amplify the noise.

Another important insight from the article is the importance of performing out-of-sample testing, as the results can differ, sometimes even contradict the in-sample outcomes.

Reference

[2] Hervé N. Mugemana, Evaluating the impact of sampling frequency on volatility forecast accuracy, 2024, Inland Norway University of Applied Sciences

Closing Thoughts

Both studies underscore the central role of high-quality data in financial modeling, trading, and risk management. Whether it is the frequency at which data are sampled or the completeness of firm-level fundamentals, the integrity of input data directly determines the reliability of forecasts, model calibration, and investment decisions. As financial markets become increasingly data-driven, the ability to collect, process, and validate information with precision will remain a defining edge for both researchers and practitioners.

A key challenge in system development is that trading performance often deteriorates after going live. In this post, we look at why this happens by examining the post-publication decay of stock anomalies, and we address a practical question faced by every trader: when a system is losing money, is it simply in a drawdown or has it stopped working altogether?

Why and How Systematic Trading Strategies Decay After Going Live

Testing and validating a trading strategy is an important step in trading system development. It’s a commonly known fact that a well-optimized trading strategy’s performance often deteriorates after it goes live. Thus, developing a robust strategy that performs well out-of-sample is quite a challenge.

Reference [1] attempts to answer the question: why a strategy’s performance decays after going live.

Findings

-The paper investigates which ex-ante characteristics can predict the out-of-sample decline in risk-adjusted performance of published stock anomalies.

-The analysis covers a broad cross-section of anomalies documented in finance and academic journals, with the post-publication period defined as out-of-sample.

-Predictors of performance decay are based on two hypotheses: (1) arbitrage capital flowing into newly published strategies, and (2) in-sample overfitting due to multiple hypothesis testing.

-Publication year alone accounts for 30% of the variance in Sharpe ratio decay, with Sharpe decay increasing by 5 percentage points annually for newly published factors.

-Three overfitting-related variables—signal complexity (measured by the number of operations required) and two measures of in-sample sensitivity to outliers—add another 15% of explanatory power.

-Arbitrage-related variables are statistically significant but contribute little additional predictive power.

-The study tests both hypotheses using explanatory variables and univariate regressions, finding significant coefficients from both sets.

In short, the results indicate that performance decay is driven jointly by overfitting and arbitrage effects.

Reference

[1] Falck, Antoine Rej, Adam and Thesmar, David, Why and How Systematic Strategies Decay, SSRN 3845928

When to Stop Trading a Strategy?

When a trading system is losing money, an important question one should ask is: Are we in a drawdown, or has the system stopped working? The distinction is crucial because the two situations require different solutions. If we are in a drawdown, it means that our system is still working and we just have to ride out the losing streak. On the other hand, if our system has stopped working, we need to take action and find a new system.

Reference [2] attempted to answer this question.

Findings

-The paper examines how to distinguish between normal unlucky streaks and genuine degradation in trading strategies.

-It argues that excessively long or deep drawdowns should trigger a downward revision of the strategy’s assumed Sharpe ratio.

-A quantitative framework is developed using exact probability distributions for the length and depth of the last drawdown in upward-drifting Brownian motions.

-The analysis shows that both managers and investors systematically underestimate the expected length and depth of drawdowns implied by a given Sharpe ratio.

I found that the authors have some good points. But I don’t think that the assumption that the log P&L of a strategy follows a drifted Brownian process is realistic.

Note that a trading strategy’s P&L can often exhibit serial correlation. This is in contradiction with the assumption above.

Reference

[2] Adam Rej, Philip Seager, Jean-Philippe Bouchaud, You are in a drawdown. When should you start worrying? arxiv.org/abs/1707.01457v2

Closing Thoughts

Both papers address the critical issue of strategy persistence and performance decay, though from different perspectives. The first highlights how published anomalies tend to lose risk-adjusted returns over time, with evidence pointing to both overfitting in backtests and arbitrage capital crowding as drivers of performance decay. The second provides a quantitative framework for assessing when drawdowns signal genuine deterioration rather than normal variance, showing that investors often underestimate the length and depth of drawdowns implied by a given Sharpe ratio. Taken together, these studies underscore the need for investors to treat historical performance with caution, monitor strategies rigorously, and account for both statistical fragility and realistic drawdown expectations in portfolio management.