Many, when testing systems for American stocks, do not attach importance to this, that there are no dropouts in their tests for various reasons, stock. And completely in vain, as the results vary greatly. In the picture on the left, a test of one of the systems for 9 years excluding retired shares, and on the right, taking into account. It can be seen that the annual profit differs in 3 Times, and the Sharpe ratio is twice. Well, the rest of the indicators and coefficients also vary greatly..
What, in my opinion, is the main point of testing systems on historical data? Ответ простой — weed out knowingly unprofitable or knowingly non-earning systems. For example, a novice trader read in one of the books, article, seminar, etc.. o super-systems, which just can't help but win, because she is correct, you have to believe in it and trade disciplinedly. He has two paths — trade it disciplinedly, gradually draining money over 2-3 years or test in any specially created program for this, to make sure that in the past she did not bring money, which means, there is a possibility that it will not bring in the future. The second path will be much shorter 2-3 years, and depending on the skills of coding trading signals(no programming skills) it may take from 2 to 15 minutes.
Когда-то давно, when I just started playing on stock exchanges, мне говорили — “how can you trade, you don't have a system, trade is not yours, without a system, you will still merge… develop your system…”. I honestly did not know what a system is and how it can be developed, but did not admit it and said that I have a system, and I believed in it myself — thought, what if I trade, this is already the system. Then I read that the system turns out to be very simple — price is higher than EMA — Buy, below the EMA — Sell. I looked at the graph — and exactly, looks extremely logical, because when the price is above the EMA, she grows, and when under EMA — falls. I realized that I had found the grail, now it will be very easy to make money. And I started trading EMA with a period 21 on the EURUSD currency pair. The first trade was closed by stop loss, but i didn't worry, I knew I had the grail. I didn't know what a saw was yet, but gradually I began to feel it, when several deals in a row closed in negative territory. I doubted….. and decided to check the grail on historical data manually. Tested for several days and made sure that the grail is draining…. By the way, I was very lucky that at that time seminars were not yet so widespread, manuals and forum discussions with zombies about the benefits of developing iron discipline and blind faith in the correctness of the method, where the main postulates are risk / reward not less than 1:3 and stop 5 Cents — Sort of, if you stick, sooner or later you will get rich. I represent, what would have happened if the brainwashing struck the still immature consciousness of a novice speculator.
Fortunately, I came across a program, on which it was possible to check what results the coded system showed in the past on the selected instruments. I already had different ideas at that time, и очень много, about 1000 :). Imagine, how many years would have to be spent in real trading, or, as a last resort on demo, to make sure they don't work in an almost efficient market. And if you blindly adhere to discipline, how much money would have to be drained. And with the help of the program, after acquiring skills in coding trading signals, this process will take, maximum, 1 day. And all. And you can, with a clear conscience, discard these ideas and not trade.
Someone will say — it is not always necessary to blindly adhere to the system, since the system will not be able to foresee what the human brain can. For example, if the price suddenly went against the position — close without thinking, no matter what the system says there — because everything is simple, you just have to watch the price. This, By the way, very common opinion. Me too, By the way, I once believed in this, until I began to constantly notice that if I had not been engaged in amateur performances, but strictly followed the system, then the profit would be much higher. And this disease lasted for a relatively long time, it was very difficult to recover from her. Even now, sometimes it washes away “улучшить” system at some point. well, and that, that with a system, you need to constantly monitor the price, simply speaks of the absence of clear rules for entering and exiting a position.
In general, what I wanted to say — what testing, it's not the same, как многие считают: “stop testing, you need to trade”, “mathematical self-sufficiency”, “art for art's sake”, “all life will pass for tests”, “Rezvyakov does not test”, “theorists” and so on. This, first of all, huge time savings. Why 98% traders drain money? If they tested their fix ideas on past data and were convinced of their inconsistency, they just wouldn't bet money on it and, so there would be a chance of them(The money) save.
З.Ы. All of the above does not apply to very experienced players. They are experienced for that because they went through all the hardships and they themselves are like a program for testing — testing takes place in their heads instantly at a speed, incomparable with the speed of a computer and a program. Honor and praise them :)
Yuri, when you had a press conference, You said you can ask questions right here.
Yes, certainly.
“I will have a question about testing. Describe, if not difficult, testing and optimization process in a few sentences. In the literature, it is recommended to divide the entire testing period into several intervals and test the system separately at each of them..
An idea comes to mind, the program encodes inputs, exits, test starts, analyzes the results. The literature recommends correctly. For example, parameters for the system are selected on the interval 2000-2007 of the year, and then if the results are good, the site is tested on the interval 2007-2010 of the year. If negative results are obtained in this interval, then the parameters were just adjusted and the system does not inspire confidence. If on this interval the results are just as good, that is, the hope that the system will work in the future. The testing interval should include various market conditions — bearish, bull markets, saw, high, low volatility. Then you need to check transactions manually, at least deals 50. Often, after such a check, it turns out that the system looks quite cunningly into the future due to the peculiarities of a particular testing program..
See also. How do you test the inputs, feet, exits. All separately and then only combine into the system. Or else first “come up with” system and then test it entirely.
Бывает по разному, depending on the idea. For example, if we catch an instant price change, then the exit should be immediately when the movement is exhausted, since the idea of the system — моментум. If the idea — catch big movements completely, then trailing exits. BUT–a-a understood, You probably mean to test the outputs for random inputs and inputs for random outputs.? Not, I very rarely do this and only for the purpose of specialized research. And in general, I immediately test the system completely, where the inputs and outputs correspond to the idea of the system.
How do you evaluate the test results and how do you compare the systems with each other based on the results of such testing.
First of all, I look at the equity of the tested system and you can see everything from it.. It should be smooth, without large drawdowns. From the coefficients, I watch everything that is possible, but if you arrange them in order of importance, then it will be like this:
Recovery factor Коэффициент Шарпа (for portfolios) Average profit per trade Profit factor
I don't like it when the ratio of winning / losing trades is less 40%. AND, respectively, when the size of the average losing trade significantly exceeds the value of the average winning trade.
There is such a book – R. Colby “Encyclopedia of technical market indicators”. Paid attention to her? There the author briefly describes the testing procedure., and then gives the results of testing various systems and indicators in the MetaStock program.
Yes, but long ago, so I don't really remember. If I'm not mistaken, it just tests different indicators one by one? I also tested all available indicators in due time.. But it's better to combine them — alone as filters, others as a trigger. But every day the markets are becoming more efficient and traditional indicator systems are becoming, vice versa, неэффективными :)
He also describes the testing procedure and indicators in his book. “Computer analysis of futures markets” C. Lebeau.
Yes, remember, I listened to this book on headphones while jogging in the park about four years ago. I liked her — the author writes truthfully, does not promise mountains of gold.
Already studied everything that is possible. And from which side to approach testing, I did not fully understand.
Well i guess, if from the very beginning, then you just need to take ready-made public systems and study them a little at different markets, and there your ideas will appear.
The other day I had a creative conversation with a stock trader from Moscow under the nickname osmar92 (), which inspired me to test one idea, prompted by the distinguished osmar92.
—————————————————————————————————— osmar92: I prefer to re-login many times, liquidating a position, if I think, that I am right in the assessment.
JC: And it does not happen in this case so, that the loss from frequent prepayments will exceed, finally held, successful trade, in which you were right? :)
osmar92: TOof course this can be, but this is insurance for a possible loss. It's better to do less because of the cost, than to be in a decent minus. The task is not to do much, and often a little. That, what makes an experienced trader cannot be copied by an inexperienced one as experience suggests, what to do in a specific situation. I just gave this approach as an illustration., that you don't need to cling or as they say marry a trade, opportunities are enough. Get off where it didn't work and look elsewhere. After all, beginners cling and stand in a losing position until the cock crows. To your question, I can ask a counter question. What if I jumped off correctly and found an even better trade?. which worked in 2 times better than the previous one, instead of , to sit in it.
JC:So lucky. Maybe, lucky not by accident, but based on your experience and acquired intuition for such situations. But, Admit it, that an initially unprofitable situation can shoot in the right direction and predict in advance, which position out of several will be the most profitable, That's impossible. We're just playing out our statistical advantage in the long run.. One strictly according to the system, others based on acquired intuition. Second, unfortunately not available to me — probably killed the development of intuition with a strict systemic approach. Therefore, I try to check by testing on history — Will small losses from prepayments exceed large profits from one trade?. It happens in different ways, sometimes exceed, sometimes not. That is, I wanted to say that this is all ambiguous. :)
osmar92:Basically true. Although my vision, which can even help you in systemic( in the full sense of the word, since I also have a system), if you enter a trade, then it should immediately produce the result. Based on my experience and statistics, usually the most successful trades are, who immediately start working in the right direction, otherwise the entry point will be incorrect and therefore the system needs to be configured. We are all different, different people have different hemispheres. But the future belongs to intuitive systems, based on mathematics and intuitive modeling. THEN, what they try to call neural networks, still far from that self-learning and flexibility , which is inherent in man
JC:It may well be like this. It will be necessary to test this idea., for example, if after entering for some time the price has not passed some minimum distance in the desired direction, then take appropriate action.
osmar92: I am more , than sure, that statistics will be on the side of that, what did I say.
here you go, we start testing the idea of, what if the price after opening a position for a certain time has not passed the minimum distance in the desired direction, then we close the position as potentially unpromising. For the inputs, let's take the widely accepted standard Aberration system.. Everyone knows her — standard breakout system, built on Bollinger channels. When the upper channel is broken, we enter long by buy-stop, upon breakdown of the lower, respectively, short. Exit on the stop of the middle line of Bollinger Bands, or , which is the same, by SMA with a period like the channels. The system itself is . We modify this system — if through N(1,2,3,4,5) days the price will not pass M(1,2,3,4,5)%, then we close the position the next day at the opening as unpromising. We will optimize the parameters and find the best solution. If the condition for the rate of price growth is satisfied, then we close by condition, given by the system — on the stop of the middle line of the Bollinger Channel. We test on a portfolio of all liquid American futures for the last 10 years.
As a result, the most optimal steel parameters — if for 5 days the price did not pass 1%, then close the position. These are extreme parameters — highest number of days and lowest percentage. And the further into the forest, the more firewood and the most unprofitable options were options with 1 And 2 days since 5,4,3 Percent. I.e, the more requests we have for fast profit growth in a short time after opening a position, the more we get a loss, and what is more indulgent to growth, the more tolerant of price stagnation after opening a position, the more we earn in the future. It turns out the opposite is true — the result is not what you were going to get, and the reverse. Here is the increase in equity for 10 years, provided that if for 5 days after entry the price does not pass 1%, then close the position. Remind, this is the best result of those, who asked.
But the best result — this is the perfect result, therefore, you need to choose the average to get closer to reality. And the average result will be obtained under the condition — if the price is for 3 days after entering did not add at least 2%, then close the position.
And finally,, let's take a look at the result, when we simply do not take into account any minimum price movements for a certain time after opening a position. Just closes, as expected according to the crawling foot system (Bollinger Channel median line). And let the price behave in the intervals as it wishes — do not pay attention to her twitching. And we get the best option:
What conclusion? My so: foresee, whether the position is against us or for us — That's impossible. Now goes against us, in a second it can soar in our favor that it will not seem a little. Or vice versa. We can say that the first option happens much more often., it can be unfounded to assert the opposite, but it's better to check it out anyway, the benefit of programs for this business is now different — just choose. Otherwise, intuition can fail.…..
In this case, speculation about whether the price has stopped and therefore it is necessary to leave, whether the price has gone against us and therefore intervention is required, whether the price has risen so much that it is time to take profit and other subjective fantasies — only harm the system. That's why — opened a position, put a stop and turn off the computer so that your hands do not itch improve the system. We turn on the computer tomorrow, rearrange the crawling stop and turn it off again. And so every day :)
Testing on history (bye by hand) systems, based on sharp spikes in volume. Works against the trend, you are on the turns. Stop: 6пп, profits from 4 up to 40pp.
Here's the Friday deal, which is also in the statistics:
According to the results, it almost coincides with trend systems, but the drawdown is several times less, you can't get rid of them at all.
