In this post we demonstrate ideas to test for parameter sensitivity.
Here we have a strategy with 5x parameters. 3x being look back periods for a specific indiactor. The other 2x being an entry level threshold and an exit level threshold.
I decided to change the original parameters by up to 50% in either direction of the original. It might look something like this:
# Original param1 = 116 param2 = 10 param3 = 69 entry_level = 6 exit_level= 85
So if you multiply any of the params by 0.5… you obtain a value… and in this example lets multiply the first parameter, param1: 116 * 0.5 = 58. This provides the new upper and lower limit. The range now becomes:
58 to 174.
This has created a range of parameters up to a maximum of 50% change in either direction of the original. Do this for all parameters. Note if your indicator only goes to 100 for example then that would be your maximum versus the full 50% range. Same in the opposite direction. Parameter lower limit obviously 0.
Next run a monte carlo running randomly through the parameter ranges. I tend to do 5000 iterations then look at the distribution. An example output of this process is below:
Here we show some key metrics. Average trade profit. This is an important value because on average we want to know what we make per trade and it has to be high enough to withstand slippage and commissions.
Next we have the win % where depending on which money management system you are using, often this is an input variable in money management calculations. An example would be half Kelly formula.
And of course we have the net ending balance of the strategy itself less commissions (I didnt add slippage in this example).
The plots above show my original parameter setting results by marking a straight vertical red line on the plot.
This red line was some optimal combination again found through a random search. As we can see its not THE optimal as to the right side of the red line we have parameter sets which beat the original.
Besides this what can we learn – we see that by running 5000 iterations and randomly selecting a parameter from a range which includes up to a 50% change in either direction from the original. That we land in positive expectancy across our selected trading metrics. We may observe the mean values below:
Needless to say. The random parameter selection might not choose the most extreme 50% parameter change at any given iteration. Thus across the 5 parameter sets we would be combining a range of changes in either direction up to a 50% maximum. One iteration might might look like:
Param1 = +/- 10% change from original
Param2 = No change from original
Param 3 = +/- 40% change from original
Param 4 = +/- 12 % change from original
Param 5 = +/- 5% change from original
And run another:
Param1 = +/- 45% change from original
Param2 = +/- 8% change from original
Param 3 = +/- 2% change from original
Param 4 = +/- 22 % change from original
Param 5 = +/- 34% change from original
So on and so forth for 5000 iterations.
How do you obtain confidence that you can protect your initial strategy investment when going live?
This type of study essentially says we may place a strategy live and have a degree in confidence in changing parameters.
What would I do in real life?
I would place 3 parameters bands in live trading. Whichever parameter set triggered first is the trade that is taken. Thus I am covering a parameter band and I am covered if the market changes away from my original parameters…..
Food for thought!
If you have an interest in this type of research feel free to drop me a line.