Trend Following Mean Oscillating Strategy

Introduction

Welcome to this detailed exploration of the 'Trend Following Mean Oscillating' (TFMO) trading strategy. As part of this analysis, we will dissect an algorithmic trading script that backtests and optimizes this strategy on historical foreign exchange (FX) data for ten major currency pairs. 

The TFMO strategy is a variant of trend following trading strategies that identifies potential trading opportunities based on the oscillation of prices around certain mean levels. It incorporates elements of both trend following and mean reversion, making it a hybrid approach that aims to capture both trending and range-bound market conditions. 

The TFMO strategy in our script operates on a dual timeframe: it uses daily high and low prices to compute moving averages (for capturing the overall trend) and resamples these averages to a 5-minute frequency (for generating trading signals). The script's objective is to identify the optimal parameters for the strategy that maximize its performance, as measured by the Sharpe ratio. 

This blog post will guide you through the different components of the script, explain the concepts behind the TFMO strategy, describe how the strategy is backtested and optimized, and discuss how the results are analyzed. We hope this deep dive will provide insights into the development of algorithmic trading strategies and the use of backtesting and optimization in strategy development. 

Join us as we unpack the layers of this fascinating strategy, step by step. Let's get started.

Data Preparation

To effectively run the TFMO strategy, the script requires historical foreign exchange (FX) data. The data used in the script is stored in a Hierarchical Data Format (HDF5) file named 'fx10_1m.h5'. This file contains one-minute bar data for ten major currency pairs. Bar data includes the high, low, close, and volume for each time period (in this case, one minute). 

In algorithmic trading, data preparation is a critical step. It involves cleaning and formatting the data to fit the requirements of the strategy. For the TFMO strategy, the script performs several key steps to prepare the data for backtesting:

1. Loading the Data**: The script loads the data using the `vbt.HDFData.fetch` function from the vectorbtpro library. This function reads data from HDF5 files, which are ideal for storing large volumes of numerical data, like financial market data.

2. Resampling**: After loading the data, the script resamples the one-minute bars to two other timeframes: 5-minute bars and daily bars. The resampling is done using the `resample` method, which aggregates the high-frequency data into lower frequency data. The 5-minute data is used to generate trading signals, while the daily data is used to calculate the moving averages that guide the strategy.

3. Filtering**: The script filters out the weekend data from the resampled data sets. Since the FX markets are typically closed over the weekend, this step is necessary to avoid any distortions in the strategy caused by the non-trading period.

4. Splitting**: Finally, the script separates the high, low, and close prices from the resampled data. These price series are used in various stages of the strategy, such as indicator calculation and signal generation.

Through these steps, the script transforms the raw, high-frequency data into a format suitable for backtesting the TFMO strategy. The aim is to ensure the data aligns with the strategy's requirements, thereby providing accurate and reliable backtest results.

import vectorbtpro as vbt
import numpy as np
import pandas as pd
import gc
import time

fx10_1m = vbt.HDFData.fetch('fx10_1m.h5')

fx10_5m = fx10_1m.resample('5T')
fx10_d = fx10_1m.resample('d')

fx10_5m = fx10_5m[fx10_5m.index.dayofweek < 5]
fx10_d = fx10_d[fx10_d.index.dayofweek < 5]

high_d = fx10_d.get('High')
low_d = fx10_d.get('Low')
close_5m = fx10_5m.get('Close')

del fx10_1m, fx10_5m, fx10_d

TFMO Strategy Definition

The Trend Following Mean Oscillating (TFMO) strategy is a unique combination of trend-following and mean-reverting techniques. It aims to identify potential trading opportunities based on the oscillation of prices around certain mean levels. Here's a more detailed explanation of the TFMO strategy and how it calculates the indicators and trading signals:

Indicators

The TFMO strategy uses Simple Moving Averages (SMAs) as its primary indicators. It calculates two types of SMAs on the high and low prices of the daily data: a short-term SMA and a long-term SMA. The period for these SMAs is defined by the `short_wndw` and `long_wndw` parameters, respectively. The short-term SMA is expected to react faster to price changes, while the long-term SMA provides a slower and more stable signal that tracks the underlying trend.

The SMAs are calculated using the `vbt.talib("SMA")` function from the vectorbtpro library. The `.run` method is used to apply the SMA function to the high and low data, and the `.real` property is accessed to obtain the SMA values. The resulting SMA series are then shifted one period into the future using the `.shift()` method to avoid lookahead bias.

The short and long SMAs are then resampled to the 5-minute frequency using a custom resampling operation. This operation is necessary because the SMAs are initially calculated on the daily data, but the trading signals are generated based on the 5-minute close prices. The resampling is performed using the `vbt.Resampler` class and the `.resample_closing` method.

Trading Signals

The trading signals are derived from the SMAs and the 5-minute close prices. The script calculates entry and exit levels for both long and short trades as a function of the short-term SMA range (the difference between the SMA of high and low). 

For long trades, a buy signal is generated when two conditions are met:

1. The 5-minute close is above the upper long-term SMA.
2. The 5-minute close crosses below the buy level, which is calculated as the lower short-term SMA plus a certain multiple (`entry_mult`) of the short-term SMA range.

The long trade is exited when any of the following conditions are met:

1. The 5-minute close crosses above the long target level, which is calculated as the lower short-term SMA plus a certain multiple (`target_mult`) of the short-term SMA range.
2. The 5-minute close crosses below the long stop level, which is calculated as the upper short-term SMA minus a certain multiple (`stop_mult`) of the short-term SMA range.
3. The 5-minute close crosses below the lower long-term SMA.

The conditions for short trades are similar but in the opposite direction.

Through this combination of moving averages and price levels, the TFMO strategy aims to capture both trends and reversions to the mean in the FX market.

def TFMO_func(high_d, low_d, id_close, short_wndw, long_wndw, entry_mult, target_mult, stop_mult) :    

    sma_upr = vbt.talib("SMA").run(high_d, short_wndw, skipna=True).real.shift()

    sma_lwr = vbt.talib("SMA").run(low_d, short_wndw, skipna=True).real.shift()

    lma_upr = vbt.talib("SMA").run(high_d, long_wndw, skipna=True).real.shift()

    lma_lwr = vbt.talib("SMA").run(low_d, long_wndw, skipna=True).real.shift()

    resampler = vbt.Resampler(source_index=high_d.index, 

                              target_index= id_close.index, 

                              source_freq='d', 

                              target_freq='5T') 

    sma_upr_5m = sma_upr.vbt.resample_closing(resampler)

    sma_lwr_5m = sma_lwr.vbt.resample_closing(resampler)

    sma_range_5m = sma_upr_5m - sma_lwr_5m

    lma_upr_5m = lma_upr.vbt.resample_closing(resampler)

    lma_lwr_5m = lma_lwr.vbt.resample_closing(resampler)

    buy_lvl = sma_lwr_5m + (sma_range_5m * entry_mult)

    sell_lvl = sma_upr_5m - (sma_range_5m * entry_mult)

    long_tgt_lvl = sma_lwr_5m + (sma_range_5m * target_mult)

    short_tgt_lvl = sma_upr_5m - (sma_range_5m * target_mult)

    long_stp_lvl = sma_upr_5m - (sma_range_5m * stop_mult)

    short_stp_lvl = sma_lwr_5m + (sma_range_5m * stop_mult)

    buy_ent_cond1 = id_close.values > lma_upr_5m.values

    buy_ent_cond2 = buy_lvl.vbt.crossed_below(id_close)

    sell_ent_cond1 = id_close.values < lma_upr_5m.values

    sell_ent_cond2 = sell_lvl.vbt.crossed_above(id_close)

    long_ext_cond1 = long_tgt_lvl.vbt.crossed_below(id_close)

    long_ext_cond2 = long_stp_lvl.vbt.crossed_above(id_close)

    long_ext_cond3 = id_close.vbt.crossed_below(lma_lwr_5m)

    short_ext_cond1= short_tgt_lvl.vbt.crossed_above(id_close)

    short_ext_cond2 = short_stp_lvl.vbt.crossed_below(id_close)

    short_ext_cond3 = id_close.vbt.crossed_above(lma_upr_5m)   

    long_entry_sigs = buy_ent_cond1 & buy_ent_cond2

    short_entry_sigs = sell_ent_cond1 & sell_ent_cond2

    long_exit_sigs = long_ext_cond1.values | long_ext_cond2.values | long_ext_cond3.values

    short_ext_sigs = short_ext_cond1.values | short_ext_cond2.values | short_ext_cond3.values

    return long_entry_sigs, short_entry_sigs, long_exit_sigs, short_ext_sigs

TFMO_strat = vbt.IF(class_name= 'strat_TFMO',

                    input_names= ['high_d', 'low_d', 'id_close'], 

                    param_names= ['short_wndw', 'long_wndw', 'entry_mult', 'target_mult', 'stop_mult'],

                    output_names= ['long_entry_sigs', 'short_entry_sigs', 'long_exit_sigs', 'short_ext_sigs']

                   ).with_apply_func(TFMO_func, keep_pd= True)

Backtesting and Optimization

Backtesting and optimization are fundamental processes in the development of trading strategies. Backtesting involves testing a trading strategy on historical data to assess its performance, while optimization seeks to find the optimal parameters for the strategy that maximize its performance. Let's take a closer look at how these processes are implemented in our TFMO strategy script.

Backtesting with vectorbtpro

The script uses the vectorbtpro library to conduct backtesting. This library provides powerful tools for backtesting and analyzing trading strategies in Python. Two main components are used in our script: the IndicatorFactory (IF) and the Portfolio class.

The IndicatorFactory (IF) is used to define a custom indicator function for the TFMO strategy. The indicator function takes the high, low, and close prices, along with the strategy parameters (window sizes for SMA and LMA, and multipliers for entry, target, and stop levels) as inputs, and outputs trading signals (entry and exit signals for long and short trades). The `.run` method is used to execute this function with the given inputs and parameters.

The Portfolio class is used to simulate trading based on the signals generated by the indicator function. The `.from_signals` method is used to create a portfolio from the entry and exit signals, and several arguments are specified, including the close prices, trading frequency, trade size, initial cash, and slippage. The method returns a Portfolio object, which is used to calculate various performance metrics of the strategy.

Parameter Optimization

To find the optimal parameters for the TFMO strategy, the script performs a random search optimization. It enters a loop where it randomly generates 1000 sets of strategy parameters and backtests the strategy on the FX data for each parameter set. The random parameter values are generated using the `np.random.choice` function from the numpy library, which randomly selects values from a given array.

The performance of each backtest is evaluated using several metrics, including total trades, total return, win rate, and Sharpe ratio. These metrics are calculated using the `.stats` method of the Portfolio object, which returns a pandas Series of various performance statistics.

Storing the Results

The results of each backtest, including the parameter values and performance metrics, are stored in a dictionary. Each key in the dictionary corresponds to a parameter or performance metric, and its value is a list that stores the corresponding values from each backtest.

Finally, the results dictionary is converted to a pandas DataFrame using the `pd.DataFrame` function. This DataFrame provides a convenient format for analyzing and visualizing the results of the backtests. It is saved as a CSV file using the `.to_csv` method, and the top 10 parameter sets sorted by the Sharpe ratio are printed to the console.

Through backtesting and optimization, the script aims to find the best parameters for the TFMO strategy that maximize its performance, as measured by the Sharpe ratio. This process helps to ensure the strategy is robust and likely to perform well on unseen data.

results = {'SMA' : [], 'LMA' : [], 'Ent_Mul' : [], 'Tgt_Mul' : [], 'Stp_Mul' : [], 

           'Tot_Trades' : [], 'Tot_Ret' : [], 'Win_Rate' : [], 'Sharpe' : []}

for i in range(1000) : 

    gc.collect()

    try :      

        sma_period = np.random.choice(np.arange(2, 6, 1, dtype= int))

        lma_period = np.random.choice(np.arange(6, 31, 1, dtype= int))

        entry_mult = np.random.choice(np.arange(0.5, 2.1, 0.1, dtype= float))

        tgt_mult = np.random.choice(np.arange(2.1, 5.1, 0.1, dtype= float))

        stp_mult = np.random.choice(np.arange(0.5, 2.1, 0.1, dtype= float))  

        entries_exits = TFMO_strat.run(high_d= high_d, low_d= low_d, id_close= close_5m,

                                       short_wndw= sma_period, long_wndw= lma_period, 

                                       entry_mult= entry_mult, target_mult= tgt_mult, 

                                       stop_mult= stp_mult)    

        pf = vbt.Portfolio.from_signals(close= close_5m, freq= '5T',

                                        entries= entries_exits.long_entry_sigs, 

                                        exits= entries_exits.long_exit_sigs, 

                                        short_entries= entries_exits.short_entry_sigs, 

                                        short_exits= entries_exits.short_ext_sigs, 

                                        size=1000, size_type='amount', init_cash='auto', 

                                        slippage= 0.001)    

        stts = pf.stats(['total_trades','total_return', 'win_rate', 'sharpe_ratio'])    

        results['SMA'].append(sma_period)

        results['LMA'].append(lma_period) 

        results['Ent_Mul'].append(entry_mult) 

        results['Tgt_Mul'].append(tgt_mult) 

        results['Stp_Mul'].append(stp_mult) 

        results['Tot_Trades'].append(stts.loc['Total Trades'])

        results['Tot_Ret'].append(stts.loc['Total Return [%]'])

        results['Win_Rate'].append(stts.loc['Win Rate [%]']) 

        results['Sharpe'].append(stts.loc['Sharpe Ratio'])

        print('This is iteration: {}'.format(i))

        time.sleep(1)

    except :

        break

res_df = pd.DataFrame(results)

res_df.to_csv('simulation_results_2.4.csv')

print(res_df.sort_values(by= 'Sharpe', ascending= False).head(10))

Analysis of Results

Once the backtesting and optimization process is completed, the next step is to analyze the results to identify the best performing set of parameters for the strategy. In our TFMO strategy script, the results of each backtest are stored in a pandas DataFrame. Each row in the DataFrame represents a single backtest with a particular set of parameters, and the columns store the parameter values and performance metrics.

Interpreting the Performance Metrics

The performance of each backtest is evaluated using four key metrics: total trades, total return, win rate, and Sharpe ratio. Here's a brief explanation of each metric:

- Total Trades**: This is the total number of trades (both long and short) executed by the strategy during the backtest. This metric gives an indication of the trading frequency of the strategy.
- Total Return**: This is the total return of the strategy over the backtest period, expressed as a percentage. It measures the overall profitability of the strategy.

- Win Rate**: This is the percentage of trades that were profitable. A higher win rate indicates a higher probability of making profitable trades, but it doesn't necessarily mean a higher total return, as it doesn't take into account the size of the wins and losses.

- Sharpe Ratio**: This is a measure of the risk-adjusted return of the strategy. It is calculated as the average return per unit of risk, with risk measured as the standard deviation of returns. A higher Sharpe ratio indicates a better risk-adjusted performance.

Analyzing the Results with pandas

The pandas DataFrame provides a convenient and powerful tool for analyzing the results. The `.head` method can be used to quickly check the top rows of the DataFrame, and the `.sort_values` method can be used to sort the DataFrame by a particular column.

In our script, the DataFrame is sorted by the Sharpe ratio in descending order, and the top 10 rows are printed to the console. This gives us the 10 sets of parameters that yielded the highest Sharpe ratio in the backtests.

It's also possible to perform more complex analyses with pandas, such as grouping the results by one or more parameters and calculating the average performance metrics for each group, or plotting the distribution of the performance metrics to visualize the results.

Through this analysis, we can identify the optimal parameters for the TFMO strategy that not only yield the highest total return, but also balance risk and reward effectively, as measured by the Sharpe ratio.

     SMA  LMA  Ent_Mul  Tgt_Mul  ...  Tot_Trades    Tot_Ret   Win_Rate    Sharpe

199    2   10      2.0      4.6  ...       279.9 -35.540980  30.332704 -0.560936

807    4    7      1.8      4.2  ...       367.1 -39.003175  32.322065 -0.573465

469    4    7      1.8      4.8  ...       362.2 -39.296354  31.735201 -0.577743

233    4   28      2.0      4.5  ...       247.4 -39.011270  24.163028 -0.588620

118    2   29      1.7      4.9  ...       333.8 -38.254087  26.312191 -0.589926

567    4   30      2.0      3.7  ...       252.1 -37.711092  25.879305 -0.591009

938    4   30      1.8      4.9  ...       290.5 -39.924817  23.717481 -0.591328

973    5   20      1.9      4.9  ...       283.8 -41.072182  25.411719 -0.599928

658    4    6      1.8      4.3  ...       383.8 -38.866538  31.842439 -0.603399

617    5   12      2.0      4.0  ...       283.6 -40.513000  31.729561 -0.603474

Conclusion

Over the course of this blog post, we've delved into the mechanics of the Trend Following Mean Oscillating (TFMO) strategy, from its data requirements and strategy definition to the processes of backtesting and optimization, and finally, to the analysis of results. This journey has allowed us to gain a deeper understanding of how this hybrid trading strategy operates and how it can be fine-tuned for improved performance.

Backtesting and optimization are indispensable in the process of algorithmic trading strategy development. They allow us to assess a strategy's performance on historical data and find the optimal parameters that maximize its expected performance. However, it's important to bear in mind that backtesting comes with certain limitations. Notably, a strategy's performance in the past may not necessarily predict its future performance due to changing market conditions. Furthermore, over-optimization or "curve fitting" can lead to the creation of a model that performs exceptionally well on the backtest data but fails to generalize to new, unseen data.

While our exploration of the TFMO strategy has been comprehensive, there are several avenues for further investigation and improvement. For instance, we could incorporate additional or different indicators into the strategy, experiment with other optimization techniques, or test the strategy on different types of financial instruments or timeframes. 

In conclusion, the TFMO strategy, backed by rigorous backtesting and optimization, showcases the potential of algorithmic trading strategies. It represents a starting point from which traders and developers can build and refine their unique trading systems. As the world of algorithmic trading continues to evolve, so too will the techniques and strategies employed in the quest for profitable trading opportunities. 

Thank you for joining us on this exploration of the TFMO strategy. We hope that it has provided you with valuable insights and inspiration for your own algorithmic trading journey.