Fri. Dec 1st, 2023

The Art and Science of Performance Benchmarking

For the past two decades, I’ve measured portfolio performance against benchmarks for individuals and institutions. None of us can control markets, but it’s critical that our clients know how they have performed relative to appropriate benchmarks.

Though that sounds very simple, it isn’t. I’ll explain why and then offer some suggestions on how to do this for your clients.

The benchmarking game

Let’s start with an example other than investments. Like 88% of drivers, I think I’m above average. You may have heard of the “Lake Wobegon” effect – the human tendency to overestimate ourselves. One reason we can fool ourselves that we’re all above average is by using different standards. I think I’m an excellent driver because my benchmark is saving time, getting from point A to point B quickly without endangering others or getting a ticket. My wife’s standard is getting from point A to point B while following all laws, including the speed limit. My benchmark may be wrong but, being only human, I’m unlikely to ever admit it.

The same goes for benchmarking investment performance. Here’s an example. A recent Barron’s article highlighted a study from Bank of America on the failure of the $3.27 trillion invested in target-date funds. The article stated:

2040 target-date funds’ performance has been lackluster, while offering minimal protection against volatility and declines than other options. Over the past 28 years, the funds – meant for investors who plan to retire in 2040 – returned a total of 750%, underperforming the 1,494% logged by the S&P 500 and even the 866% logged by a balanced 60% stock/40% bond allocation.

Not only have 2040 target-date funds lagged the index by a compounded annual average 2.4 percentage points, but they have done so without less volatility – 13.6% compared with 14.9% for the S&P 500 since March 1994, according to BofA.

This benchmarking indicates a huge shortfall and says we and our clients should avoid these types of funds. But is that a valid conclusion? I agree with Morningstar’s John Rekenthaler that it’s not. Has the Bank of America consistently avoided international stocks and only recommended large-cap U.S. stocks (S&P 500) over this timeframe? Does it manage portfolios for no fees? As Rekenthaler put it, the bank is using “perfect hindsight” in selecting indexes to benchmark against to show a point it wants to make. He does his own analysis by benchmarking against the actual asset allocation of these funds, throwing in fees, and concluded “the funds look fine when compared with the existing alternatives.”

Selecting the right benchmarks

While I happen to agree with Rekenthaler’ s benchmarking much more than Bank of America’s, his isn’t perfect either. Admittedly, I’ve never done a perfect benchmarking. That’s because, when it comes to benchmarking, there is judgment involved. There are over three million indexes and those numbers are increasing constantly.

There is clearly crooked benchmarking. The most common I see is comparing a portfolio’s total return to the S&P 500 index. Any index includes only the part of the return that comes from price appreciation. It strips out the dividend portion. Comparing a U.S. stock portfolio to the raw S&P 500 index is comparing the total return of the total U.S. stock market to part of the return of part of the U.S. stock market. I once spoke with behavioral economist Dan Ariely on this subject. He noted that it’s human to pick a benchmark that will make us look good. But using the raw price of only the S&P 500 index goes past being human and into the realm of being evil. Still, I see it often.

Building a custom benchmark using some of these indexes involves a ton of judgement. How granular should the benchmarks be? Should we benchmark a small-cap-value portfolio against a small-cap-value index? Should we compare an alternative market-neutral fund against an index of all market-neutral funds? If we benchmark at the most granular level, it becomes useless. Using literally an “apple to apple” example, we would benchmark Apple stock against Apple stock.

Beyond that, when a portfolio shifts asset allocations over time, should we shift the weightings of the benchmarks? Doing so might reward the manager for performance chasing.

How I benchmark

I’m a believer that broader is better. The more granular the benchmark, the more likely the portfolio will perform exactly like the benchmark, which renders the performance analysis close to worthless. Fees should be included in the benchmark but at the lowest level. I use the following five benchmarks:

1. U.S. stocks – Vanguard Total Stock Index (VTI)
2. International stocks – Vanguard Total International Stock Index (VXUS)
3. U.S. Bonds – Vanguard Total bond Market Index (BND)
4. Real estate – Vanguard REIT Index (VNQ)
5. Cash – Vanguard Federal Money Market Fund (VMFXX)

I use Vanguard because these funds have been around the longest (giving more data to benchmark) and are among those with the lowest fees. These are returns that could have been achieved after considering costs. However, these benchmarks are far from perfect. For example, a small-cap-value fund has significantly more risk than the broad market. Investment-grade corporate bond funds have more risk than funds that are mostly backed by the U.S. government. I would point out to the client that I haven’t adjusted for these risk differentials.

There is far more judgment involved in private investments (which I typically recommend against holding) such as hedge funds or private equity funds. Here I look at the underlying investments (or stated objective) but then add an illiquidity premium, which can be as high as 4%, depending upon the terms of the investment. Even if a fund says it provides quarterly liquidity, if it has the right to gate redemptions, I will pump up the benchmark. Liquid alternative assets are even harder. For example, a market-neutral fund should be benchmarked to the risk-free rate; I’d use cash but note the fund has far more risk.

The benchmarks I’m suggesting are extremely subjective. My main goal is to help you think about the appropriate benchmarks. The more difficult it is to determine the appropriate benchmark, the less likely the holding belongs in a portfolio. If you can’t explain the fund to your client, you should rethink using it.

Using any hindsight in weighting the benchmarks creates a bias. I typically use the average portfolio weighting over the longest period my client has data. For example, if my client has four years of data, I take the average weighting of those four years. If weightings changed significantly over those years, the impact of that market timing will be reflected in the analysis.

You can’t eliminate bias

Bias is part of human nature, though we can work hard to minimize it in the subjective art and science of picking benchmarks.

We would have an incentive to use higher performing benchmarks if we were analyzing someone else’s portfolio, especially if we had something to gain by showing underperformance. But if it’s a portfolio we recommended, just the opposite is true.

To minimize bias, I suggest the following:

1. Select benchmarks independent of selecting of money managers.
2. Select benchmarks ahead of time.
3. Benchmarks must be consistent over time.
4. Broader is better (total stock market index or total international stock market index).
5. Match characteristics rather than using a narrow focus (market neutral, managed futures).
6. Weighting should be the average over the time period to separate alpha caused by:

  • Manager
  • Timing

Summing it up

Because our clients deserve to know how their portfolio has performed relative to reasonable benchmarks, there must be a high dose of science thrown in along with the art. The average return of the U.S. stock market over any period of time is a cap-weighted total-stock-market fund less the lowest fee. Some point to a Wall Street Journal article noting it’s not quite technically true because of IPOs, stock repurchases, and the fact that indexes are reconstituted. I disagree; the first two are minor while total-stock index funds have very little reconstitution.

Even portfolios I’ve advised on typically underperform the benchmark. Though I usually recommend buying the benchmarks, there are often holdings I don’t sell due to tax consequences. It’s hard to beat appropriate benchmarks.


Leave a Reply

Your email address will not be published. Required fields are marked *