Tuesday
Jan062009
Startup Robustness
Tuesday, January 6, 2009 at 12:37PM Some systems are inherently weak on
startup. i.e. they may be slow acquiring
positions or tend to buy all stocks under the same
market conditions. So it is a good idea to
test random entry dates and look for poor
performance on a one year backtest. There is
nothing worse than suffering a massive drawdown
right after you start investing.
I
have noticed that many systems I have previously
developed suffer from poor startup. For example,
let's say there is a buy rule Rank > 95. When the
system is up and running for a period of time most ly
top-ranked stocks will be bought i.e. with rank 99.8 or
something like that. But when starting up a new
system all stocks above the buy rule Rank > 95 will
be bought. So there is an inherent dilution
on startup. The alternative is to buy over several
weeks or months and not have your capital fully utilized
during that time period (also not desirable).
Another issue is that the system may buy all of the stocks in the middle or start of a bear market. So I want to know what happens in this situation.
The third issue is that a system may take a long time to acquire positions if the buy rules are too tight. This will show up with poor profit during the first year.
For these reasons I like to perform a series of one-year backtests with random start dates. To achieve this I simply select a one year test and each run I subtract 1 week, 4 weeks or 3 months. I try to mix it up a little so that I am not systematically testing a pattern of dates.
The results for the Industry Breakout system are as follows:
All in all I am quite satisfied with these results. By the way, I started a port based on this trading system a little while ago. The links are as follows:
Industry Breakout Portfolio: http://www.portfolio123.com/port_summary.jsp?portid=403557
Ranking System: http://www.portfolio123.com/app/ranking-system/72662
Custom Universe: http://www.portfolio123.com/screen_summary.jsp?mt=7&screenid=20173
Steve
(NOTE: Portfolio123 was used for all tests unless stated otherwise.)
Another issue is that the system may buy all of the stocks in the middle or start of a bear market. So I want to know what happens in this situation.
The third issue is that a system may take a long time to acquire positions if the buy rules are too tight. This will show up with poor profit during the first year.
For these reasons I like to perform a series of one-year backtests with random start dates. To achieve this I simply select a one year test and each run I subtract 1 week, 4 weeks or 3 months. I try to mix it up a little so that I am not systematically testing a pattern of dates.
The results for the Industry Breakout system are as follows:
| Start Date | Annual Profit | Max D.D. |
| 21/05/01 | 19.61% | 7.33% |
| 18/06/01 | 25.54% | 3.25% |
| 25/06/01 | 25.54% | 3.25% |
| 25/09/01 | 25.85% | 2.74% |
| 02/10/01 | 25.85% | 2.74% |
| 30/10/01 | 26.06% | 2.74% |
| 30/01/02 | 4.87% | 7.72% |
| 27/02/02 | 5.95% | 7.65% |
| 27/05/02 | 8.24% | 8.46% |
| 27/08/02 | 44.15% | 8.47% |
| 24/09/02 | 62.14% | 8.47% |
| 01/10/02 | 55.80% | 8.47% |
| 31/12/02 | 111.16% | 6.52% |
| 31/03/03 | 104.50% | 7.68% |
| 28/04/03 | 120.66% | 6.58% |
| 28/07/03 | 55.26% | 10.95% |
| 28/10/03 | 30.97% | 9.94% |
| 04/11/03 | 33.15% | 9.33% |
| 04/02/04 | 46.69% | 12.11% |
| 03/03/04 | 47.73% | 10.92% |
| 03/06/04 | 55.33% | 10.54% |
| 03/09/04 | 53.36% | 11.73% |
| 01/10/04 | 49.54% | 10.32% |
| 08/10/04 | 36.44% | 13.23% |
| 08/01/05 | 27.72% | 10.76% |
| 08/04/05 | 65.62% | 7.13% |
| 06/05/05 | 74.18% | 7.13% |
| 06/08/05 | 26.53% | 14.88% |
| 03/09/05 | 25.69% | 15.31% |
| 03/12/05 | 32.28% | 13.19% |
| 03/03/06 | 17.37% | 13.00% |
| 03/06/06 | 32.94% | 10.95% |
| 01/07/06 | 40.35% | 10.95% |
| 29/07/06 | 45.73% | 10.95% |
| 05/08/06 | 43.17% | 10.24% |
| 02/09/06 | 39.40% | 11.35% |
| 02/12/06 | 34.03% | 9.79% |
| 30/12/06 | 32.24% | 9.12% |
| 30/03/07 | 32.93% | 8.12% |
| 06/04/07 | 25.66% | 8.98% |
| 04/05/07 | 28.96% | 9.38% |
| 01/06/07 | 34.57% | 8.53% |
| 08/06/07 | 14.52% | 9.63% |
| 08/09/07 | 20.35% | 9.79% |
| 08/12/07 | 13.24% | 8.15% |
| 01/05/08 | 20.50% | 3.88% |
All in all I am quite satisfied with these results. By the way, I started a port based on this trading system a little while ago. The links are as follows:
Industry Breakout Portfolio: http://www.portfolio123.com/port_summary.jsp?portid=403557
Ranking System: http://www.portfolio123.com/app/ranking-system/72662
Custom Universe: http://www.portfolio123.com/screen_summary.jsp?mt=7&screenid=20173
Steve
(NOTE: Portfolio123 was used for all tests unless stated otherwise.)
Reader Comments (2)
Steve:
Good articles on testing robustness. I learned a few things.
I looked at the P123 port using the link and noticed that the industry breakout buy rule is turned off. Is this right? I thought you decided to use it in your "Checking Under the Hood" article.
Glenn
Good catch Glenn. I started the Port a little while ago and it looks like I had some different thoughts at the time. I will start the correct version this coming weekend.
Steve