I spend a lot of time talking about our industry (and all the warts and snake-oil that is sold) with the boys at BlackStar Funds. Below is a summary of a back and forth where we discuss survivor bias, as well as just how rare a Sharpe Ratio of 1 is over long periods of time.
In this case, we’re talking about Commodity Trading Advisors (CTAs). The 25 largest CTA’s with at least 10 years of performance history:
Avg CAGR: 13.2%
Avg Volatility: 14.9%
Avg Max DD: -18.1%
Avg Sharpe: 0.66
These number are AFTER the fact, and do not include the results of funds that have disappeared (and this effect is HUGE). They are the largest CTA’s because they’ve had the best returns (or the best marketing).
We have a done a ton of posts in the past that show how survivor bias misstates returns by 4% or more per year.
Out of 140 CTA’s with 10+ years of track record (regardless of assets), only 3 have Sharpe ratios greater than 1.
For some great background reading on professionals and Sharpe Ratios check out this old post. Also here is a nice old post of mine on hedge funds and survivor bias. This great white paper shows that CTA’s do indeed add value, but they capture most of it in their fees.
The biggest of these is AAA Capital Management, a discretionary Energy trader with $640 million AUM. The second is Newton Capital Partners, a global macro fund that trades stock indexes, currencies, and government bonds with $178 million AUM. The third is Capricorn Advisory Management, a dedicated currency manager with $62 million AUM.
Even with survivorship bias almost nobody has been able to sustain a Sharpe ratio above 1.
Below is the cumulativeve distribution of the Sharpe ratios for the 140 CTA’s with the longest track records. Look familiar??? Remember, this is with all the funds that disappeared because they blew up.
Meb, do you know of a place to get historical index data on the managed futures area? The Rydex fund is an awful proxy, and the best-fit index I can find is the Credit Suisse/Tremont Managed Futures index. Unfortunately I can't find historical daily/monthly pricing to run tactical trials on. The Altegris 40 has similar issues.
I don't find this surprising, and BlackStar mentioned the same issue to me. My question is whether the Sharpe ratio is the best method for analyzing and evaluating the risk vs. return characteristics of CTA's?
I did not mean to suggest that the Sharpe ratio is the most appropriate method of evaluating CTA's. I'm only pointing out what the real Sharpe ratios are…in response to people constantly telling me that a “good” CTA should have a Sharpe ratio higher than 2.
As a portfolio manager with lots of tricks in his bag, I take any *independent* risk that has a forward-looking Sharpe ratio, after transactions costs and/or fees, bigger than 0.01. Simple Markowitz theory sez to take risks in proportion to their Sharpe ratio and you have a killer aggregate overall Sharpe ratio.
I.e., give 1% of your CTA allocation to each of 100 truly unique managers with “merely” 0.5 Sharpe Ratios and you get an aggregate of 5.0, WAAY off the top of this chart. There's no limit, as long as you're not inadvertently taking the same bet under different labels. That's one of the rubs in this era of “All Correlations Go To 1.”
The other, of course, as hinted at by your survivorship bias notes, is that we are seeing the prettiest information that can be legally presented, AND ONLY THAT. These types of studies are utterly clueless about HOW the CTA achieved his results. *Most* due-diligence confirmatory studies by even very careful investors merely confirm that the accounting is correct; they are incapable of determining (a) that trading skill dominated luck in the historical data, and even more clueless in saying (b) that the skill will apply to whatever markets we have in 2010. (Hence, “Past performance…”)
I have a very careful way of making those assessments that, alas, works for stock styles but is irrelevant to CTA skill determination. Oh, well.
[...] a follow up to our earlier post “Let’s Talk About Professional Funds“, here is a great graphic from the guys at BlackStar showing the unsustainability of a Sharpe [...]
Eric,
Likewise I didn't mean to suggest that you did think the Sharpe ratio is the best measure for CTA's. Instead, I am really curious what you and others think the best way to judge CTA's is. I have my own way of doing that, but I am not sure I would call it a consistent or rigorous method. It has worked for me relative to my objectives, but I think that people doing the kind of research you do would be instructive.
We used to manage a fund of funds that allocated specifically to CTA's. So I've studied and talked to lots of them. I don't think it's responsible to just look at the returns, drawdown, volatility, assets under management, margin to equity, etc. Random chance alone will ensure that some total lunatics will look good, even for many years. I found that putting forth my own questions in the due diligence process is a quick way to identify CTA's that have the three R's (responsible, reasonable, realistic). The first thing I ask about is the number of markets they are involved with. Are they motivated by convenience (just trading the easy to trade markets) or are they doing the things necessary to diversify their risks and opportunity set? Second, I ask about their risk controls. Do they know how much risk they are taking at any given time? Can they quantify it? How accurate has this estimate been over time? Third, do they have a theory as to why they should be able to extract profits in a negative sum game? Do they know what their edge is and can they articulate it? I also try to get a feel for why they are in the business. Are they ego-driven, emotional gamblers? Or are they rational, calm, patient speculators?
Actually that sounds a lot more like my way of looking at it. People, process, performance (and the last is not meant to mean just the returns, but the characteristics of the returns, the patterns and how they match up with the what I find or hear in looking at the first two p's.)
I'd be interested in learning more about how you evaluate equities managers. Full disclosure, I manage a long/short equity fund.
Yes, performance is necessary but it really should be last on the list. Not very intuitive, but I've logged enough hours with random number simulators to know that great performance sometimes accompanies baseless strategies…in which case it is ALWAYS heavily marketed.
What you are not taking into account are those funds which do not report because they are not interested in raising funds. Many of the top performing CTA's are not in the databases either because they are private or they have evolved into more traditional hedge funds and other strategies in order to put more $ to work. Many of the top macro funds today are such cases. Also, most funds shut down due to operational issues, not blow-ups. So dont presume that all funds that exit an index are because of terrible performance.
Secondly, you have to understand the limitations of the industry. Most strategies are not scalable, really only trend following really is. Because trend following is a long volatility strategy, a category which is not known for high sharpe ratios (many small losses, than large gains… which doesnt reflect well since sharpe penalizes upside vol) what you are left with are a collection of low sharpe trend following funds that are the largest and have the longest track records. For example, there arent many funds that have a more impressive track record (by any other measure) than Trendstrand over the past 14 years yet they're sharpe is just .95. And many other trend followers, i.e. Eckhardt, Saxon, etc., have evolved their strategies over the years. The same program that swung for the fences in the 80’s or 90’s and produced volatile performance, now has significantly increased risk controls, yet the prior performance is going to wipe the manager from your 10 year sharpe filter, despite the fact that their evolved systems produce a much higher sharpe.
And Eric, curious as to who told you that a “good” CTA should have a Sharpe ratio higher than 2”?
Thirdly, it is difficult to use a 10 year scale since many funds which yield higher sharpes are quant based and require electronic markets. Most commodity markets have evolved from pit to electronic just in the last 10 years. So a funds like QIM for example, who has a 9 year track with a 1.33 sharpe is missing from the list but it would have been impossible for them to have existed 15 years ago.
- And if somebody is selecting a portfolio of CTA’s to out-perform a CTA index it is one thing, but if you are simply looking to provide the largest benefit to other aspects of your portfolio by providing inverse correlation, you shouldn’t want a high sharpe ratio!
4thly, the whole notion that you can index managed futures is only true as far as trend following is concerned. And I would argue that the top funds, which mix multiple uncorrelated systems to increase risk adjusted returns, will always out-perform any attempt at an indexed product. You might read my piece here: http://www.pionline.com/assets/docs/CO6646984.PDF
Eric mentioned one of his questions asks “ how do you generate returns from a zero sum game”. This is one area where CTA’s have a direct advantage since a large number of investment dollars have flowed into the markets which only play the long side. Many CTA’s have a significant advantage as they tend to be more agnostic about the direction of the market and stand to “collect” returns from the long only investment dollars when markets fall. This is a huge source of returns.
Eric, one more question. You say “The first thing I ask about is the number of markets they are involved with. Are they motivated by convenience (just trading the easy to trade markets)”. What do you mean by this? What markets are easy to trade? I have yet to find one, and if a CTA did have a market they found easy to trade, please let me know so I can invest!
- In terms of questions to ask, as mentioned, ½ (or more) of all funds that shut down are due to operational reasons, not blow ups as many might think. So once operationally things check out, the majority of the remaining questions should be risk management. If all max positions were on, what the largest loss a program can lose in any one day if all positions met their max risk or stop level? Does the manager scale back on positions as a draw-down becomes greater? Do they use hedges using instruments whose relationships may change in the future, exposing unseen risks? Etc. (Notice that of these risks tend to lower the sharpe….)
Lastly, managers lose their edge, lose top talent, retire, etc., all of which cause CTA’s to close and from producing 10+ year track records. I sophisticated CTA investor should be able to spot many of these risks and should view their CTA portfolio as highly dynamic.
Best,
Ryan
my opinion is one should use the modified Sharpe ratio… since it punishes downside volatility, one should take only the semi-standard deviation of the returns, meaning only the ones that are negative or under the Rf rate…
As you say, Transtred only has a sharpe of 0.9 or so, and yet they have a CAGR of 18% with a max drawdown of 7% which is ridiculous low for a trend following strategy
my opinion is one should use the modified Sharpe ratio… since it punishes downside volatility, one should take only the semi-standard deviation of the returns, meaning only the ones that are negative or under the Rf rate…
As you say, Transtred only has a sharpe of 0.9 or so, and yet they have a CAGR of 18% with a max drawdown of 7% which is ridiculous low for a trend following strategy