bad statistics

There was something bothering me a few weeks ago about an argument I remembered from James Surowiecki‘s book, the Wisdom of the Crowds, which is surprising as I had read it about two years ago.  It regards the passage below, from page 8:
“

What this means is that the stock market had, almost immediately, labeled Morton Thiokol as the company that was responsible for the Challenger disaster… the steep decline in Thiokol’s stock price – especially compared with the slight declines in the stock prices of its competitors – was an unmistakable sign that the investors believed Thiokol was responsible… on the day of the disaster there were no public comments singling out Thiokol as the guilty party… regardless, the market was right… six months after the explosion… Thiokol was held liable for the accident.

”
First off, the word “competitors” is not accurate – because the correct comparison to be made is to other companies who manufactured parts for the space shuttle, not companies who were competing with Thiokol but had nothing to do with the spacecraft; but no worries, the paper he cites is a-OK in this regard.

The claim being made is that because the stock price of Thiokol went down further than the other manufacturers, the market must have “known” they were guilty.  Given the nature of insider trading, this is a possibility.  But can you make such a claim from the evidence?

Let’s look at the data, then: a few of the large companies that built components for the space shuttle included Lockheed, Martin Marietta, and Rockwell International.  Here are graphs of the stock prices, and traded volumes, of their ticker symbols (LK, ML, and ROK, respectively, along with Thiokol’s MTI) the month of the disaster (thanks to my friend Salman for digging up some of the historical data for me – the horizontal axis is in days the stock market was open, so day -18 is January 2, 1986):

while all 4 stocks have a jump in volume on the 28th (the day of the disaster), Thiokol’s certainly does dip lower than the rest.  According to the paper written by professors Mulherin and Maloney, Thiokol dipped 12% compared to 3% for the others.  They mention that the “data show no evidence of trading by insiders on January 28, 1986,” yet go on to make the claim that there was “no ambiguity that the stock market quickly isolated Morton Thiokol as the cause of the accident.”  Moreover, “the fact that market liquidity was available to maintain a market in Lockheed, Martin Marietta, and Rockwell while the market for Morton Thiokol dried up suggests that the stock market discerned the guilty party within minutes of the announcement of the crash” (an older draft of the paper used “is evidence that” in place of “suggests that”).

Even if it was statistically significant, I’d say the evidence is still inconclusive, simply because you can’t rely on correlation to be anything more robust than an indicator.  Perhaps I should proffer an alternative explanation for the larger fall in Thiokol’s stock price, then?  How about, Morton Thiokol was a much smaller, less-diversified company than any of the other three, and had the most to lose from the space shuttle disaster?  If I could get my hands on some of the annual reports to see what percentage of their revenues come from NASA contracts…

Nevertheless, I don’t think the argument holds water.  But I do love it when people anthropomorphize the markets

the Stock Market Reaction to the Challenger Crash