Were astrologers the original p-hackers?

By Brian Clegg

Science writers rarely mention astrology, other than to moan when someone accidentally uses the word instead of astronomy. There is, of course, no scientific basis for astrology, but when we are considering history of science it is impossible to ignore astrology as many of the early astronomers earned a fair amount of their living doing a spot of astrology on the side. This didn’t mean that they necessarily believed in it (though Roger Bacon, for example, makes an argument for it as an environmental influence, as opposed to a predictor of the future), but it brought in the cash and often the support of the nobility.

The reality with astrology and other fortune telling approaches is that, even though it has no basis for working, inevitably some of the predictions will come true. If every single prediction didn’t happen, it would actually be a very significant outcome – astrologers would be successfully predicting what wasn’t going to happen.

I was struck the other day when writing about the dubious scientific result-generating mechanism of p-hacking that astrologers, in effect, do the same thing. The social sciences generally accept a ‘p-value’ of 0.05 as meaning that a finding is statistically significant. Getting this value tells us that if there were no genuine cause (the so-called ‘null hypothesis’ were true), the measured effect should occur about 1 time in 20 (hence 1/20 = 0.05). Before the practice was identified as being extremely dodgy, p-hackers would take a set of data from a study and slice it up in many different ways. If they got, say 40 different outcomes, then with a 1 in 20 chance of an apparent effect when the null hypothesis was true, the chances would be high that at least one of the outcomes would appear significant without any basis.

In a similar way, if a horoscope makes enough predictions, especially if they are vague, it is very likely that at least one of them will prove to be true. I admit Monty Python is showing its age, but the show’s horoscope sketch still holds up well for demonstrating this exact principle when we get to the spectacles:

See all Brian's online articles or subscribe to a weekly email free here

Now Appearing is the blog of science writer Brian Clegg (www.brianclegg.net), author of Inflight Science, Before the Big Bang and The God Effect.