Introduction

It’s a common feeling: financial markets seem impossibly complex, a domain of arcane mathematics and high-speed computers. Or, they’re brutally simple, driven by the raw emotions of greed and fear. For over a century, some of the most brilliant minds in economics and mathematics have tried to discover the hidden “rules” that govern this global game of risk and reward. What they found is often far more surprising and counter-intuitive than we might assume.

These thinkers weren’t just trying to get rich; they were on a scientific quest to understand the machinery of the market. This intellectual journey saw a lone French student accidentally lay the groundwork for modern finance, a Chicago heir systematically prove the experts were guessing, and, in a final, stunning twist, the very father of market efficiency take a sledgehammer to his own elegant creation. Their discoveries reveal a market that is part science, part psychology, and endlessly fascinating.

This article distills five of the most impactful and mind-bending takeaways from this history. They reveal a world where randomness was discovered in 1900, where experts lose to coin flips, and where the greatest champions of a theory are sometimes the ones to dismantle it.

1. The Big Idea: Market Randomness Is Over 100 Years Old (And It Came With a Warning)

The idea that stock prices are random isn’t a modern “quant” invention—it’s from the year 1900.

Long before computers and complex algorithms, a French mathematics student named Louis Bachelier studied the price fluctuations on the Paris Bourse for his 1900 doctoral thesis. His core insight was revolutionary: “the mathematical expectation of the speculator is zero.” In plain English, this means that for every investor betting a stock will go up, there is another betting it will go down. When you average all these bets together, the expected gain is zero. The average investor, therefore, cannot beat the average—they are the average. Price movements must be unpredictable, wandering randomly.

This groundbreaking work, which presaged Einstein’s theories on Brownian motion, was largely ignored for over 50 years. But what’s even more fascinating is that it came with an immediate warning from Bachelier’s own thesis grader, the celebrated mathematician Henri Poincaré. He saw a major limitation in applying pure mathematics to human behavior, noting that individuals in a crowd don’t act independently.

“When men are brought together, they no longer decide by chance and independently of each other, but react upon one another. Many causes come into action, they trouble the men and draw them this way and that, but there is one thing they cannot destroy, the habits they have of Panurge’s sheep.”

Panurge, a character from Rabelais’s satirical novels, gets a flock of sheep to jump off a ship by throwing the lead ram overboard. More than a century later, this fundamental tension—between mathematical randomness and the psychological reality of human herd behavior—remains the central debate in finance. This debate, however, remained purely theoretical for decades, until one researcher decided to put it to the test not with mathematics, but with cold, hard data on expert failure.

2. The Bombshell: The Experts Can’t Beat a Coin Flip

The most powerful evidence against stock-picking came from systematically proving the experts were terrible at it.

In the early 1930s, Alfred Cowles, a Chicago heir, decided to rigorously test a simple question: “Can Stock Market Forecasters Forecast?” He wasn’t interested in anecdotes; he wanted data. Aided by a state-of-the-art Hollerith (IBM) punch card calculating machine, he and his team examined thousands of stock picks made over several years by professional forecasting services, financial publications, and major insurance companies.

His conclusion, published in a 1932 paper, was a bombshell: the answer was no. He found that the performance of the experts was “little, if any, better than what might be expected to result from pure chance.” To drive the point home, Cowles and his helpers created their own random forecasts by shuffling cards. On the whole, the randomly shuffled cards beat the professionals. The finding was so shocking it made headlines.

“Rates Luck Above Wall St. Experts: Alfred Cowles 3d Asserts That Turn of Card Is Preferable to Following Forecasters.”

This research had a profound impact. If the so-called experts couldn’t reliably beat a random draw, perhaps the goal of investing shouldn’t be to outsmart the market at all. Perhaps the goal should be to simply be the market. This startlingly simple idea helped inspire the creation of the first index funds, pioneered by John Bogle, which have since transformed how millions of people invest. This stark empirical finding laid the groundwork for passive investing, but at the same time, a new wave of theorists was building an even more radical case for market rationality—one that argued the market wasn’t just hard to beat, but was, in theory, perfectly wise.

3. The Absurdity: A Company’s Finances Don’t Matter (In Theory)

One of the most influential theories in finance argues that a company is like a pizza: how you slice it doesn’t change its size.

Until the late 1950s, finance was a field of “common sense, judgment, and tradition,” ruled by empirical research and rules of thumb. That changed dramatically with the “M&M propositions,” a pair of papers by economists Franco Modigliani and Merton Miller. They used deductive logic to arrive at a conclusion that seemed, on its face, absurd. In a “rational and perfect economic environment,” they argued, two key things are irrelevant to a company’s total value:

  • How the company raises money (by issuing stock vs. taking on debt).
  • How the company distributes profits (by paying cash dividends vs. retaining the earnings).

Their point was that a company’s value comes from the earning power of its assets, not from how that earning power is “packaged” or financed. This flew in the face of conventional Wall Street wisdom, which obsessed over dividend policies and capital structure. To explain this counter-intuitive concept, Miller used a famous analogy.

“The pizza delivery man comes to Yogi Berra after the game and says, Yogi, how do you want this pizza cut, into quarters or eighths? Yogi says, cut it into eight pieces. I’m feeling hungry tonight.”

While seemingly abstract, the M&M propositions were immensely important. They replaced traditions and rules of thumb with deductive logic, helping transform finance into a field ruled by economic theory. But this new theoretical perfection, which argued that markets would rationally see through corporate packaging, would soon face its greatest challenge from an economist who noticed that the market’s behavior didn’t look rational at all.

4. The Takedown: The Market Is Far Too Jumpy to Be “Right”

The biggest blow to the “rational market” idea was a simple observation: prices are way more volatile than the profits they’re based on.

By the 1980s, the Efficient Market Hypothesis—the idea that stock prices reflect all available information and are therefore “correct”—was the dominant theory in academic finance. But economist Robert Shiller decided to test this with a devastatingly simple question: If a stock’s price is the market’s best guess of its future dividend payments, shouldn’t the price chart and the subsequent dividend chart look roughly similar? One should be the rational shadow of the other.

Shiller compared the historical volatility of the S&P 500 index to the historical volatility of the actual dividends paid out by the companies in that index. His finding was stark: they looked nothing alike. Stock prices jumped around dramatically more than the changes in real-world profits could ever justify. Defenders of market efficiency, like Robert Merton, countered that corporate managers artificially smooth dividend payments, so of course they are less volatile than prices. But Shiller argued that this “excess volatility” suggested something other than rational calculation was driving prices—something like mass psychology, fads, or what he called “social dynamics.” He argued that the academic community had made a massive logical error.

The leap from observing that it is hard to predict stock price movements to concluding that those prices must therefore be right was, he declared… “one of the most remarkable errors in the history of economic thought.”

Shiller’s data-driven critique opened the floodgates for the field of “behavioral finance,” which re-introduced human psychology into the heart of market theory. Yet the most astonishing challenge to the elegant models of market rationality wouldn’t come from an outside critic, but from the high priest of the theory itself.

5. The Twist: The Father of Market Efficiency Dismantled His Own Theory

The ultimate challenge to rational market theory came from its most famous champion.

Eugene Fama, a professor at the University of Chicago, was the intellectual father of the Efficient Market Hypothesis. For decades, the primary tool for testing his theory was another elegant model called the Capital Asset Pricing Model (CAPM). CAPM proposed that a stock’s risk—and therefore its expected return—could be explained by a single factor: its sensitivity to overall market movements, a measure known as “beta.” It was a clean, simple, and powerful idea.

Then, in the early 1990s, the unthinkable happened. Fama, alongside his colleague Kenneth French, published research demonstrating that CAPM simply did not work. They analyzed decades of data and found that a stock’s beta did a poor job of explaining its returns. Instead, other factors—namely, the company’s size (smaller companies did better) and its price-to-book ratio (so-called “value” stocks outperformed)—were much stronger predictors.

Fama wasn’t abandoning market efficiency, but to save the theory, he had to redefine risk. The old, elegant definition (beta) was dead, so he proposed a new, messier definition based on size and value factors. He was publicly destroying the elegant, single-factor model of risk that the entire academic edifice was built upon. The reaction from market critics was one of stunned admiration, as articulated by efficient market critic Robert Haugen of the University of California at Irvine:

“The Pope said God was dead. At least the God of CAPM.”

It was a remarkable moment of intellectual honesty. The data became so compelling that the theory’s most famous creator was forced to concede that his original, beautiful model was wrong, paving the way for a more complex and multi-faceted understanding of market risk.

Conclusion

The story of how we think about the stock market is a journey away from elegant simplicity and toward a messier, more realistic truth. The early theories of perfectly rational, all-knowing markets have been chipped away by evidence of herd behavior, expert failure, excess volatility, and the quiet confessions of their own creators. We’ve moved from a world of pure mathematics to one that must account for human psychology.

But acknowledging that the “rational market” is a myth doesn’t magically make us as individuals perfectly rational. We are still prone to the same biases, manias, and panics that send markets careening. This leaves us with a profound challenge. If we know the market isn’t perfectly wise, but we also know that we as individuals are full of biases and prone to herd behavior, how should we approach the challenge of investing our money for the future?