Performance Prediction and Factor Models (Chapter 22, Part 2)
In Part 1, we saw that statistical tests need 20+ years of data to reliably separate skilled managers from lucky ones. But the problems run even deeper. This section of Chapter 22 covers the traps that make performance evaluation even less reliable than the basic statistics suggest, and what actually works for predicting who will trade well.
Previous: Performance Evaluation
When the Numbers Lie: Distributional Problems
The standard t-test assumes that returns follow a normal distribution, that classic bell curve. But actual portfolio returns have fatter tails than the normal distribution. Extreme outcomes happen more often than the math assumes. This means t-tests are actually less powerful than the calculations in Part 1 suggested. And those calculations were already discouraging.
But the really dangerous problem is asymmetric distributions. Harris introduces the “peso problem,” and it’s one of the most important concepts in the whole chapter.
Here’s the setup: some trading strategies produce small positive returns almost all the time, but very occasionally produce massive losses. Until the big loss hits, the manager looks like a genius. Standard statistical tests will confirm the manager is skilled. But the manager is not skilled. They’re running a strategy that’s essentially picking up pennies in front of a steamroller.
Harris gives a vivid example: a manager can hold an index portfolio and sell out-of-the-money options. Most of the time, the options expire worthless and the manager collects premiums, consistently beating the market by a small amount. But when the market gets volatile enough to trigger those options, the losses are enormous. No skill required. Just a willingness to take hidden risk.
The name comes from the Mexican peso in the 1960s and 70s. Mexico fixed its exchange rate but had higher inflation than the U.S. So you could earn higher interest rates by investing in Mexican debt. It worked great, until Mexico suddenly devalued the peso and you lost everything overnight. A Chicago professor reportedly planned to avoid this by having his former students (now Mexican economists) call him before any devaluation. They did call. He was traveling and couldn’t be reached.
Pyramid Schemes and Fraudulent Returns
Harris points out that statistical tests are useless when the data itself is fake. Pyramid schemes produce remarkably good returns right up until they collapse. If you analyze those returns with a t-test, you’ll conclude the manager is incredibly skilled.
Charles Ponzi ran his scheme in 1920 by claiming to trade postal reply coupons for 400% returns. He paid early investors with money from new investors. It took in $9.5 million from over 10,000 investors before collapsing.
Return smoothing is a subtler version of the same problem. Managers of portfolios with illiquid assets can be slow to adjust valuations, making returns look artificially smooth and low-risk. When reality catches up, the crash is sudden and severe.
The Sample Selection Bias
This is probably the most important concept in the chapter. Harris calls it responsible for “more trading losses than any other cause.”
The sample selection bias works like this: you only see the winners. You hear about Warren Buffett because he won. You don’t hear about the thousands of managers who lost. When you evaluate Buffett’s record, you need to ask not whether an average random manager could have achieved those returns, but whether the best manager out of thousands of competitors could have achieved them by chance.
Harris does the math on this. In a group of 10,000 unskilled managers with well-diversified portfolios, the luckiest one will beat the market by almost 27% in a typical year. Over ten years, the luckiest will beat the market by an annual average of more than 8%.
That’s stunning. These are unskilled managers. Pure luck.
Mutual fund companies exploit this systematically. They start many new funds, keep the winners, and kill the losers by merging them into other funds. The surviving funds look impressive. But you never see the ones that died. This survivorship bias makes the average surviving fund look much better than the average fund that ever existed.
Harris describes his “favorite fraud” to illustrate the point. Send 20,480 people a market prediction letter. To half, predict the market will rise. To the other half, predict it will fall. Next month, write only to the 10,240 who got the correct prediction. Split them again. After ten rounds, 20 people have seen you correctly predict the market ten times in a row. They’ll be absolutely convinced you’re a genius. You just need a mailing list and stamps.
Regression to the Mean
When sample selection pushes past averages up, subsequent performance invariably drops. This is regression to the mean.
Harris cites a striking example: between 1979 and 1985, commodity trading advisors who launched public funds had an average monthly return of 4.1% in the 36 months before going public. In their first year after going public, their average monthly return was 0.23%. The advisors looked incredible before they went public. Once actual tracking began, they were mediocre.
Even Warren Buffett shows this pattern. In his first 26 years, Berkshire Hathaway beat the S&P 500 by 13.2% annually. In the next 10 years, the margin dropped to 6.84%. Still impressive, but the regression is clear.
Our Brains Are Working Against Us
Harris makes a fascinating evolutionary argument. Our ancestors survived by learning that past events predict future events. Don’t go near the place where the tiger was. Eat the berries that didn’t kill you last time. Natural selection hardwired us to believe that performance is persistent.
But survival is a game against nature, which doesn’t actively adapt against you. Trading is a game against other smart people who are constantly trying to beat you. Being good isn’t enough. You have to be better than your opponents. Our evolutionary history trained us to appreciate absolute advantage but not comparative advantage.
We also tend to attribute success to skill and failure to luck, especially our own success. We remember our winners and forget our losers. “This may be the most dangerous selection bias that we face,” Harris writes.
What Actually Works: Comparative Advantage
If statistical methods can’t reliably predict who will trade well, what can?
Harris argues that economic theory provides a better answer. Trading is a zero-sum game. Long-term winners are those who have a comparative advantage over their opponents. Not just an absolute advantage (being good) but a comparative advantage (being better than the competition).
This is like winning the Olympic marathon. A 2:20 time is incredibly fast and will win most marathons. But at the Olympics, it would barely crack the top 40. You don’t win by running fast. You win by running faster than everyone else.
The factors that predict manager performance include intelligence, experience, education, training, creativity, memory, discipline, drive, and access to data. But most professional managers have all of these. They’re competing against each other, not against average people.
Harris provides tables of factors correlated with performance for both individual managers and firms. But he emphasizes that the single most important predictor is whether a manager understands the concept of comparative advantage itself. Traders who don’t think about why others will lose to them have no reason to expect they’ll win.
“Managers who are not constantly thinking about their comparative advantages cannot know when they should trade.”
Key Takeaways
The core message is both simple and hard to accept:
Past performance does not predict future returns. It’s not just a legal disclaimer on mutual fund ads. It’s a mathematical and statistical reality. Over human time frames, luck dominates skill as a determinant of good performance.
The only reliable way to assess a manager is to evaluate their comparative advantages directly. Can they articulate why they’ll win and why others will lose to them? Do they understand that trading is a zero-sum game? Do they have resources and skills that their competitors lack?
If you can’t identify your own comparative advantage as a trader, you probably don’t have one. And if you don’t have one, you should invest in index funds.
It’s not the most exciting conclusion. But it might be the most important one in the entire book.
Next: Index and Portfolio Markets
This post is part of a series retelling “Trading and Exchanges: Market Microstructure for Practitioners” by Larry Harris (Oxford University Press, 2003). The goal is to make these concepts accessible to everyone, not just finance professionals.