I was cleaning out some files and came across the fantastic James Montier PDF “The Seven Sins of Fund Management“. (If you want the Cliff’s note version check out his recent book The Little Book of Behavioral Investing.)
There was a great quote from Lao Tzu, a 6th century BC poet, “Those who have knowledge don’t predict. Those who predict don’t have knowledge.”
This reminded me of a conversation I had with the Chief Economist of a major I-bank when I just graduated college. We were out skateboarding (technically on a FlowBoard) down the paved streets of San Francisco. The FlowBoard allows the user to basically snowboard down paved streets. We would take a lift (car) back up then skate back down Rivera or Sloate streets. Pretty amazing and one of those experiences you only get in the Bay Area. Imagine going skateboarding with the head of Goldman in NYC…
Anyways, one of the simple questions I had was – “If people have such a rotten record of forecasting, specifically, why do we have a Federal Reserve Chairman/Committee adjusting interest rates at all? Why not simply peg them to some basket of commodities, goods, or other inflation measure.” He went on an hour long diatrtibe, and that debate is not the point of this post. Rather, I wanted to point out a few of the great charts from the PDF highlighting the terrible record of forecasting in general. From Montier:
The two most common biases are over-optimism and overconfidence. Overconfidence refers to a situation whereby people are surprised more often than they expect to be. Effectively people are generally much too sure about their ability to predict. This tendency is particularly pronounced amongst experts. That is to say, experts are more overconfident than lay people. This is consistent with the illusion of knowledge driving overconfidence.
Dunning and colleagues have documented that the worst performers are generally the most overconfident. They argue that such individuals suffer a double curse of being unskilled and unaware of it. Dunning et al argue that the skills needed to produce correct responses are virtually identical to those needed to self-evaluate the potential accuracy of responses. Hence the problem.
Why do we persist in using forecasts in the investment process? The answer probably lies in behaviour known as anchoring. That is in the face of uncertainty we will cling to any irrelevant number as support. So it is little wonder that investors cling to forecasts, despite their uselessness.
The first chart shows economists attempts to forecast the rate of inflation as measured by the GDP deflator. Sadly it reveals a pattern that will become all too common in the next few charts. Economists are really very good at telling you what has just happened! They constantly seem to lag reality. Inflation forecasts appear to be largely a function of past inflation rates.
The economists are bad, but what about bond forecasters?
Not only are bond forecasters bad at guessing the level of the yield, they can’t get the direction of yield changes right either. The table below (in PDF) shows that when yields were forecast to rise, they actually fell 55% of the time!
The same thing occurs for both equity strategists and analysts. So, the question is, why forecast at all?
Meb,
I agree with the anchoring explanation in the face of uncertainty. It seems to me that the bigger problem is not with the forecaster but with the consumer of the forecast. I have seen it working with my clients. They crave some type of prediction or forecast. It doesn't even seem to matter if whatever prediction you make is wrong or right. I have given clients predictions that have ultimately been wrong, however, they remember them as being correct! Spend a few minutes reading the comments in any of the more forecast oriented blogs and you will see the dysfunction in the client/consumer.
Meb – Really good points. At the bottom of the last post you reference a pdf file that shows how bond forecasters get it wrong – I don't see a pdf with a table – Am I missing it?
Mark
I remember reading an SF Chronicle article about the Dunning study about the unskilled and unaware right about the same time I graduated college and being floored by how it completely explained a part of the world that so confounded me.
http://www.sfgate.com/cgi-bin/article.cgi?f=/c/...
I think the answer to your last question is that some people, even if skilled, lack the self-awareness necessary to believe they just can't accurately predict something. It's an issue of perception. It's the same reason that someone in a pick-up basketball game will vehemently argue that the ball is not out on them (while I was sitting on the sideline and clearly saw it go off them) to the point of having an aneurysm. I don't believe that person is intentionally lying about the call (otherwise they wouldn't be so upset). I think they actually really believe that it couldn't possibly have gone off them (how could it go off someone that good?) and are so sure of themselves, they feel like the other person is cheating them.
I think certain people are just wired that way and it's hard for someone that isn't wired like that to understand the mindset of someone that has that much belief in themselves. That Dunning study also mentioned that (consistent with your Lao Tzu quote) those that were objectively found to be better at something systematically underpredicted their own strengths as if they thought there were plenty of other people who were better than they were in the same endeavors.
That's why I tend to search out predictions and forecasts from people who don't usually make them and am suspicious of anyone who is always sure about everything.
I love Montier… Thanks for sharing this and the link to the book. Did not know he had written one.
Flowboarding, surfing, skiing, you're a real cool Californian hipster aren't you?
Cool!
(even more cool to do it with a big I-Bank dude…)
It's in Montier's PDF at the top.
Interesting…
[...] “Genius” label. Anyone willing to call themselves a genius in our field needs to go read my last blog post on the Folly of Forecasting and the must read Montier behavioral PDF the Seven Sins of Fund [...]
Meb for Fed chairman!!
I don't think forecasting should be dismissed altogether. There are two completely different approaches to forecasting: one is a gut-feel prognostication type forecasting and then there is forecasting based on quantitative models. I just started reading “Super Crunchers” which is all about drawing a distinction between “expert opinion” and data backed models. One key difference is that the quantitative modelers know the error bars on their estimates, but “experts” never seem to review history to see if their guesses are worth anything!
One reason to forecast is to make money. But, an interesting “feature” of investing is that you can easily go broke with a sound forecasting model by over-betting as a result of over-confidence. If you bet on a system with a 0.7 win rate with the assumption that it has a 0.8 win rate you go broke. But if you only count on winning at 60% of the time then you make money.
The overconfidence thing reminds me of the Bertrand Russell quote: “The whole problem with the world is that fools and fanatics are always so certain of themselves, but wiser people so full of doubts.”
Forecasting by its design implies someone will find it actionable but there are a myriad of biases to be accounted for, some with unintended consequence.
Think of a CFO with a motivational bias with an incentive to reach an outcome.
Punishment: “This forecast is too low, they will cut our budget”….or
Reward: “Lower forecasts means lower targets that we can meet”.
Below is a relative link:
http://www.bbsconsultants.com/documents/06_2009...
Author Malcolm Gladwell demonstrates the hazards of statistical reasoning and shows this kind of thing in his writings. It's kinda like what Dirty Harry said, “A man's gotta know his limitations”.
One additional is “illusory superiority”…recall Svenson's safety driving study wherein 93% of Americans and 63% of Swedes put themselves in the top 50 ( above median) percent.
Clearly we can't all be better than average, but we have put a lot at stake in Mr. Bernanke being an excellent driver.
take a run at this test. it demonstrates very clearly how experts fool themselves into confidence. am going to make any new money manager sit this in future
http://www.projectionpoint.com/
another board that i really like is tierney. check it out tierneyrides.com
now living in austin, it's the closest thing to snowboarding i've got!
another board that i really like is tierney. check it out tierneyrides.com
now living in austin, it's the closest thing to snowboarding i've got!