There are many letters in the article, so for the most impatient at the end I made important conclusions, having read which, I hope, there will be a desire to read everything else.
I thought a lot over the weekend, about Trading и судьбы мира, and as a result I realized one very important thing, which will radically change my approach to the development of trading systems. On the one hand, this is a very simple thing in its essence., but very difficult for an unprepared person to understand. If you still have questions after reading — re-read or ask questions in the comments.
Anyway — я на 99% sure, that you haven't read about it anywhere before, and i'm sure, that it will change your idea of the development of trading systems.
so, let's try to figure out how the development and testing of trading systems usually occurs. It doesn't matter what idea is at the heart of the system, это может быть и technical analysis, and any other approach.
At the first stage, clear formal rules are developed., determining how the system gives a signal to open a position, open position volume management, stop-loss and take-profit, etc.. More advanced traders at the same stage are actively exploring opportunities for money management, results from reinvestment, time and value of maximum drawdown.
Next, the resulting system is tested on historical data..
Based on the results obtained, the system is being modernized, goes to the basket, optimized.
Further, if all is well, the stage of trading with real money begins.
Friends (in this place I run the risk of making myself mortal enemies from among the vendors of software for the development and testing of trading systems)!
All your backtesting is completely useless! Any results obtained from a run of the system do not mean anything., they are just a consequence of chance. Below I will explain in detail why this is so..
The most important reason, because of which everything happens — . Read this word to yourself, and then repeat three more times: dispersion, dispersion, dispersion.
It is the dispersion that kills any point in testing trading systems.. Dispersion multiplies by zero all efforts to optimize trading systems in the classical sense. Variance brings your results out of sight: «Based on historical data, in real use, the system gives in a year 10.000 пп. arrived», in view: «Based on historical data, in real use, the system gives in a year 10.000 пп. plus / minus 15.000 пп. arrived».
Просто вдумайтесь. You are testing the system on historical data. You get the result +10.000 пп. arrived. But in reality it means, that launching it next year, you can get, how 25.000 пп. arrived, so 5.000 пп. losses!
Now imagine, what did you test the system. Got the result in 5.000 пп. убытока, and sent it to the basket. But it really is, could be the same system, which gave on other data 10.000 пп. arrived!
Let's try to understand why this is happening., how great is the influence of this very variance and how to do to eliminate this influence.
Как я уже написал выше — не имеет значение, what exactly is your trading strategy based on. In general, we can assume, that by opening a position you are likely to make money. And with some you will lose. Let's imagine for simplicity of calculations, that you have a trading system, which in 60% cases we bring to you 500 пп. arrived, and in 40% cases of loss in 500 пп. (it is very important to understand here, that these %, obtained NOT based on historical data, it is received from nowhere, absolutely reliable knowledge).
Let's model now, how events can develop, when trading this system for 100 deals.
I used a tester based on a random number generator, which with probability 60% плюсует 500 пп. to our result, and with probability 40% subtracts the same 500 пп.
Our subconscious mind immediately tells us this picture.:
You would take such a trading system to work?
It seems, we have a good expectation, the system should give a good profit. And indeed, often enough (more often than other options) so it happens.
But now take a look at other options for the development of events.:
All these pictures show the results of the work of exactly the same trading system.! In all cases, we had 60% probability of earning 500 pp., And 40% the likelihood of losing.
You would take such a trading system to work, если ваш тестер показал бы вам подобный schedule Equity?
Of course, in most cases, our virtual assistant, which I use to draw these graphs shows some kind of average picture:
It seems that there is growth, and there is a drawdown.
But (!). You never know for sure (if a, certainly, you are not dealing with this issue on purpose), which of these pictures will your testing show based on historical data.
Quite possible, what a crazy profit, which you see on the graph — just a banal accident. It so happened.
And maybe, what a trading system, which went to the trash last week — just a failure. It just happened so, what's in history, where did you test it (maybe even two or three, easily!) she was corny unlucky.
Hope, if you've read this far, I managed to convey to you a simple idea — the effect of variance on the result of testing the system on historical data is great. Very large. It is much more than you ever thought.
The same understanding, can now be transferred to the process of optimizing the trading system. In most cases, the historical equity of the trading system is improving, not because of a real improvement in the expectation of your system (60/40 which we took as a base), but simply due to the influence of variance: «It just happened so».
Я сейчас вспоминаю, that in one of the books I read (like Taleb had it in «Черном лебеде», just such a thought about, that we tend to grossly underestimate the role of chance in our lives, who has not read ). Unfortunately, Until now, I have not seen anywhere such a simple and accessible explanation of this idea in relation to the process of testing and optimizing trading systems.
well, OK. I blew your brain, and your idea of basic approaches. Let's try to understand now, what to do with it and what conclusions follow from this:
- Testing the system on historical data means nothing in 99% cases. You cannot take the results of historical testing on faith, unless you factor in the effect of variance on them.
- For that, to understand how strong the effect of variance on the result is, you need to know the true value of expectation for your trading system. Where to get it from? I dont know. But I know for sure, what to use for this testing data on historical data — mistake. Perhaps experts in mathematical statisticians will help, but for me it is still a mystery. What to do now!? Basically, value of expectation, obtained on the basis of historical data will gradually approach the true value with an increase in the number of transactions on the test. But you need to understand, that this is a very decent growth. For most trading systems in our market, this approach is not possible in principle., due to short market history, and constantly changing market conditions. So, for example, for our example (60/40) distance in 1000 transactions can still give results that differ significantly (!):
- And who to believe in this case? You can reach a more or less stable result if you increase the distance to 10.000 deals:
- But such a volume of data for testing on the Russian market on a sane timeframe simply does not exist.. Conventional system, clock-built, for example, in the entire history will make a maximum of several hundred transactions, which means it will take you to an area of very high uncertainty. Now remove the inadequate from history 2008 year, years earlier 2006 (due to a fundamentally different conjuncture) and you will understand that even a system at 15 minutes cannot be tested normally.
This is why I say: backtesting trading systems is absolutely pointless.
What to do with it? I can't imagine.
