Parameter Sensitivity Analysis

In this post we demonstrate ideas to test for parameter sensitivity.

Here we have a strategy with 5x parameters. 3x being look back periods for a specific indiactor. The other 2x being an entry level threshold and an exit level threshold.

I decided to change the original parameters by up to 50% in either direction of the original. It might look something like this:

# Original 
param1 = 116
param2 = 10
param3 = 69
entry_level = 6
exit_level= 85

So if you multiply any of the params by 0.5… you obtain a value… and in this example lets multiply the first parameter, param1: 116 * 0.5 = 58. This provides the new upper and lower limit. The range now becomes:

58 to 174.

This has created a range of parameters up to a maximum of 50% change in either direction of the original. Do this for all parameters. Note if your indicator only goes to 100 for example then that would be your maximum versus the full 50% range. Same in the opposite direction. Parameter lower limit obviously 0.

Next run a monte carlo running randomly through the parameter ranges. I tend to do 5000 iterations then look at the distribution. An example output of this process is below:

1 (1)1 (2)1 (3)

Here we show some key metrics. Average trade profit. This is an important value because on average we want to know what we make per trade and it has to be high enough to withstand slippage and commissions.

Next we have the win % where depending on which money management system you are using, often this is an input variable in money management calculations. An example would be half Kelly formula.

And of course we have the net ending balance of the strategy itself less commissions (I didnt add slippage in this example).

The plots above show my original parameter setting results by marking a straight vertical red line on the plot.

This red line was some optimal combination again found through a random search. As we can see its not THE optimal as to the right side of the red line we have parameter sets which beat the original.

Besides this what can we learn – we see that by running 5000 iterations and randomly selecting a parameter from a range which includes up to a 50% change in either direction from the original. That we land in positive expectancy across our selected trading metrics. We may observe the mean values below:

params

Needless to say. The random parameter selection might not choose the most extreme 50% parameter change at any given iteration. Thus across the 5 parameter sets we would be combining a range of changes in either direction up to a 50% maximum. One iteration might might look like:

Param1 = +/- 10% change from original

Param2 = No change from original

Param 3 = +/- 40% change from original

Param 4 = +/- 12 % change from original

Param 5 = +/- 5% change from original

And run another:

Param1 = +/- 45% change from original

Param2 = +/- 8% change from original

Param 3 = +/- 2% change from original

Param 4 = +/- 22 % change from original

Param 5 = +/- 34% change from original

So on and so forth for 5000 iterations.

How do you obtain confidence that you can protect your initial strategy investment when going live?

This type of study essentially says we may place a strategy live and have a degree in confidence in changing parameters.

What would I do in real life?

I would place 3 parameters bands in live trading. Whichever parameter set triggered first is the trade that is taken. Thus I am covering a parameter band and I am covered if the market changes away from my original parameters…..

Food for thought!

If you have an interest in this type of research feel free to drop me a line.

Follow me: @flare9x / Github

Author: Andrew Bannerman

Integrity Inspector. Quantitative Analysis is a favorite past time.

One thought on “Parameter Sensitivity Analysis”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s