The New Screener Backtest

In my last post I mentioned that the Portfolio123 website functionality had changed.  In this post I am providing a summary of tests I performed with the new screener backtest functionality and also provide a comparison of my results versus the old screener backtest. 

As I mentioned in the previous post there have been some screener backtest enhancements.  The following is a direct quote from Portfolio123 staff summarizing the enhancements.

• Price used can now be Open, Hi/Lo avg, and Close

• Daily rebalance. During a daily screen backtest the ranks are computed weekly, but the rules daily. Therefore this feature would not work too well if your ranking system has a lot of TA functions

• Slippage parameter. You can specify a slippage for a screen backtest. It's an approximation but it should generate more realistic backtests.

• During a backtests of Weekly or greater frequency, the rebalancing will normalize to Mondays, regardless of the starting date. This avoids situations where a starting date happens to be on, for ex, a Thursday. In this case the first screen will run on Thursday (using the prev week ranks) and all the rest will rebalance on Monday. This is how simulations work as well.

I am primarily concerned with the first and last enhancements on the above list.  At the moment I don't particularly care about daily rebalancing or slippage.  I only consider slippage once I am much further along in my system development process.

As I mentioned in my previous post the normalization to Mondays presents a problem since I have already done a lot of repetitive data gathering with a non-Monday start date.  So my first task is to compare new backtest results with the old.  I ran one test case (X > 1.30) and compared results with what I got from the old backtester.

The peak value is less with the new screener backtest algorithm but I don't view it as significant at this point in time.  The results are similar enough that I don't feel it necessary to re-run all of backtests I did previously.

Next I decided to take a look at "Next Open" rebalancing versus "Next Close".  The screener backtest previously only offered Next Close.  Comparing the two entry times is of interest to me because I am not a fan of using market orders at the open.  This period of time tends to be chaotic and it is easy to get burned.  So here are the results again uX > 1.30.

From this chart it can be seen that the results for the two rebalance times (market open and market close) are fairly similar i.e. there is no significant penalty going with the market close.  It might be worthwhile using a limit order at the market open.  If not filled then convert to a market order near the market close.

Finally I decided to look at each sector individually to see if I can find the sweet spot at n = 160 mentioned in my last post.  As I was running the tests on individual sectors I noticed that cyclicals, noncyclicals and utilities rarely exceeded the threshold I was using for this test set to X > 1.20.

As can be seen the results are all over the place.  No sweet spot to be found here.  The one conclusion I can draw from this test is that each sector has it's own characteristics. I need a way to tailor the threshold X > ... based on the characteristics of the particular industry or sector.  I am also wondering if there is a way to adapt the lookback period for each industry.  This second thought may be pretty challenging using Portfolio123.