As Deirdre McCloskey has been arguing for some time (see here an intro to her book on the topic), in the absence of background theory (a mechanism, etc) statistical significance is largely meaningless concept. It is just a way to generate 'results' (and publications/grants) in social science. (It gives free license to data-minding.) Paradoxically, as I noticed a while ago, it makes 'results' too easy to come by. At best it might motivate further research. There is now a lovely reductio ad absurdum of its use from one of the great experimental disciplines, social psychology.
Full disclosure: I have not read the underlying paper that I am about to discuss critically (just the blog summary). But the news is that a "paper providing evidence for [the] existence of ["precognition," that is, "knowledge of unpredictable future events"] has been accepted for publication by the leading social psychology journal." First, the clever part: "the experimental set-up well-studied psychological phenomena and simply reversed the sequence, so that the event generally interpreted as the cause happened after the tested behaviour rather than before it." Second the striking result: "The effects he recorded were small but statistically significant...[description of one such test in which] volunteers guessed correctly 53.1 per cent of the time." Third, the methodological overreach: "That may sound unimpressive – truly random guesses would have been right 50 per cent of the time, after all. But well-established phenomena such as the ability of low-dose aspirin to prevent heart attacks are based on similarly small effects, notes Melissa Burkley of Oklahoma State University in Stillwater." http://www.newscientist.com/article/dn19712-evidence-that-we-can-see-the-future-to-be-published.html
At the casino folk have had better (and worse!) runs of luck.
Recent Comments