Monday
Jan122009
Ranking System Robustness
Monday, January 12, 2009 at 12:27PM
Evaluation
of a stock ranking system requires the ability to
objectively measure performance, something that is
lacking with the tools offered at Portfolio123.
My intention with this post is not to be critical of Portfolio123. The stock investment tools offered by this website are excellent. But one area lacking is in the area of ranking system evaluation. There is very little insight provided by this tool other than annualized performance on a bucket-by-bucket basis. One cannot examine how many stocks are in each bucket or what the stock symbols are. You can't tell if an anomaly caused certain performance...
And there has been much discussion within the Portfolio123 community about what specific characteristics are important for a ranking system. Some individuals are only interested in the return of the top bucket. Some (like myself) want to see continual improvement in annualized return from left bucket to right.
In order to move forward with ranking system robustness testing / evaluation I found it necessary to invent my own evaluation tool. The tool is based on Sharpe and Sortino Ratios - industry standard performance evaluation statistics. The only twist I used was to replace the Risk-Free Return (RFR) by a Minimum Acceptable Return (MAR). The RFR is usually the interest payed on T-Bill notes or some other interest-bearing security. The RFR is fine when the objective is to compare widely varying investment strategies against one another. But my objective is to compare ranking systems drawing from stocks within a custom stock universe. So I use my custom universe of stocks as the MAR.
The tool I created is in the form of an EXCEL spreadsheet. It requires the user to run screener 5 year backtests, download the results and cut and paste into the spreadsheet. It is set up for backtest results for: the entire stock universe as well as 5, 10, 20, 40, 80, 160 top ranked stocks.
My intention with this post is not to be critical of Portfolio123. The stock investment tools offered by this website are excellent. But one area lacking is in the area of ranking system evaluation. There is very little insight provided by this tool other than annualized performance on a bucket-by-bucket basis. One cannot examine how many stocks are in each bucket or what the stock symbols are. You can't tell if an anomaly caused certain performance...
And there has been much discussion within the Portfolio123 community about what specific characteristics are important for a ranking system. Some individuals are only interested in the return of the top bucket. Some (like myself) want to see continual improvement in annualized return from left bucket to right.
In order to move forward with ranking system robustness testing / evaluation I found it necessary to invent my own evaluation tool. The tool is based on Sharpe and Sortino Ratios - industry standard performance evaluation statistics. The only twist I used was to replace the Risk-Free Return (RFR) by a Minimum Acceptable Return (MAR). The RFR is usually the interest payed on T-Bill notes or some other interest-bearing security. The RFR is fine when the objective is to compare widely varying investment strategies against one another. But my objective is to compare ranking systems drawing from stocks within a custom stock universe. So I use my custom universe of stocks as the MAR.
The tool I created is in the form of an EXCEL spreadsheet. It requires the user to run screener 5 year backtests, download the results and cut and paste into the spreadsheet. It is set up for backtest results for: the entire stock universe as well as 5, 10, 20, 40, 80, 160 top ranked stocks.
My
next set of posts will be on Ranking System robustness
and will use this tool.
Steve
Steve
Reader Comments (3)
Interesting. I have been using only 5 or 10 buckets lately in looking at P123 after working through some the "Quantitative Strategies for Creating Alpha" book. You may want to see what the pros use, which I think is www.clarifi.com.
I checked out www.clarifi.com. You know the old saying "if you have to ask how much it costs you can't afford it". The book you mentioned is on my wish list along with about a zillion other things. I've never been a proponent of large number of buckets :)
Steve
i think that those stocks were taken up for the express purpose of taking them down for great profit. That`s the only explanation for making loans like that and then removing the uptick rule at that particular time. I don`t know exactly how who pulled this off, but I`ll bet the letters CDS have something to do with it. Find out who had the uptick rule rescinded, and you`ve got your thief.
Zinc Mini tips