Can you beat the market by randomly picking stocks?

Just because an investment idea initially sounds smart doesn’t mean it is sound. As Investors’ Chronicle’s de facto Ideas editor, I can say this with a degree of authority. Often, what first appears to be a lightbulb shatters under further scrutiny or real-world conditions.

A cursory scan of my own notes and unfinished to-do lists reveals lots of clever duds. One such idea I am repeatedly drawn to, for example, is randomness. As I will unpack below, taking a blindfolded dart-throwing approach to the stock market hasn’t worked very well of late, especially for UK shares. But there are a few reasons why randomised stockpicking has a certain appeal.

For one, it feels appropriate, given the seemingly random dispersion of stock returns. The one-year total return of the average FTSE All-Share constituent, for example, differs from the unweighted market average by 25 percentage points. A stock selected at random a year ago has, with hindsight, had just a one-in-25 chance of a flat performance (defined as a 1 per cent total return up or down).

Randomisation is also a proven way to remove human biases from investing. This isn’t because humans cannot hope to beat the market through intelligence. But they are up against it. Index fund provider S&P’s latest SPIVA report is a handy reminder of the tendency of active managers to fail to generate alpha. The longer they are invested, the data shows, the more likely they are to fail.

Our weekly stock screens try to adjust for this by applying rules. As the long-term performance of many of our screens shows, this is a worthwhile activity. Then again, factor-based investing does not and will never guarantee outperformance. Nor is it ever clear which balance of growth, momentum, valuation and fundamental factors will work in the future. On some level, screening is a codified expression of human bias, based on what has worked before.

This argument also chimes with the so-called random walk theory, which states that movements in asset prices are essentially directionless. If this is an essential property of pricing (a view many contest) it follows that attempts to apply fundamental analysis are either worthless or misleading.

All very interesting in theory. The reality, however, is a little less straightforward. And while randomly picking stocks will sometimes beat the market, we need to acknowledge the possibility that it might just add up to a very inefficient or even doomed bid to match passive returns.

To demonstrate this point, the table below shows 30 attempts to outperform the FTSE All-Share’s 2.3 per cent negative total return over the past year, using different-sized portfolios of randomly selected stocks from the index. If we sidestep the imperfections in the current constituent list, which does not include companies that have de-listed or been taken over, we see that the odds have been stacked heavily against the randomised stockpicker.

While the average total return improved with larger portfolio sizes (as modern portfolio theory suggests) the randomised method only managed to outperform the benchmark three times.

My guess is that for random stockpicking to have a chance of working, at least half of an index’s stocks need to be in positive territory (over the past year, just a third meet that threshold). What this ill-conceived idea does reinforce, however, is the fact that positive recent returns in the UK market have strongly correlated to company size. On an equal-weighted basis, the average company up at least 10 per cent in a year is worth £11.5bn, compared with the £4.4bn market average.

This isn’t simply a case of a few medium-sized companies getting much bigger. Of the 29 companies that were worth at least £17bn a year ago, only one is worth less than £17bn now, and the average has posted a total return of 5.8 per cent. That’s 8.1 percentage points ahead of the market-weighted benchmark and 13.8 per cent up on the median All-Share constituent.

It’s also a stunning 15.4 percentage points ahead of the average return of firms whose market value were less than £17bn but more than £1bn a year ago. Whatever we can say about the strong recent showing from the UK’s largest listed companies, there’s nothing random about it.

Get all the latest Ideas Farm data

Shares hitting new highs and lows

Companies smashing broker forecasts

Big director buys and sells

London's most shorted shares

Fund managers' favourite international shares