Homo Scientificus (or whatever it should be in latin)
A bit of discipline hopping to start with.
Homo Ecomomicus is the theoretical construct that economists use to build their models of how economies work, and how interventions (such as a change in tax rate) will affect the economy. Homo Economicus is the idea that homo sapiens are perfectly rational and narrowly self-interested beings. Now, it kind of goes without saying that Homo Economicus is basically completely wrong, and a mad starting assumption to base most of economics on. But it is simple, and it allows elegant, sophisticated models of behaviour, that have founding principles, and so lots of people like it and use it (even if it is (a) obviously wrong and (b) was demonstrated to be wrong by Tversky and Kahnemann and others, decades ago).
Now to the pre-registration debate (which for some reason, makes me disproportionately incandescent with rage), and Homo Scientificus. Most people accept that there are some bad practices being performed in science, including not publishing results, selectively publishing results, or p-hacking (not a new idea, Ronald Coase said in the 1960s: “If you torture the data enough, nature will always confess”). Agreeing that there are some bad practices is one thing, what to do about it is another. So this is where I want to introduce the idea of Homo Scientificus, which will make it much easier to formulate a response.
Homo Scientificus (HS) is a strange beast. HS’s goal is to further his/her career at all costs through acquiring by-lines in scientific publications (rather than find out real things about the world). What’s strange is that HS has a very specific moral compass; HS will not commit outright fraud by making data up; that would be plain wrong! That said, HS will do a range of dodgy practices that he/she knows are a bit wrong, such as p-hacking etc but draws the line at outright fraud.
Now, armed with the construct of HS and the assumption that all scientists are like HS, we can develop a robust response to all the dodgy practices. Pre-registration. Pre-registering experiments will mean that HS can no longer do p-hacking, selective publishing of data etc and the truthfulness of all scientific output will improve, followed shortly by cold fusion, downloadable brains, and the AI singularity.
But maybe, just maybe (like Homo Economicus), HS is not a good construct. Maybe it only applies to a very small subset of scientists out there. Maybe lots of real scientists would be perfectly happy to commit fraud, but given the way things are at the moment, a bit of p-hacking is a safer and easier approach to take. (I mean fraud is difficult: what random distribution to use to generate the data, how many made-up subjects; how large an effect size will be plausible). Maybe, making your results up would be even more attractive in a pre-registered world, since, presumably, there’ll be fewer implausible, unexpected studies being published to great acclaim in Current Biology, so a fraudulent study with a nice unexpected result might stand out even more.
At the other end of the moral spectrum, many scientists may be much more moral than HS, not wanting to publish dodgy results (lying awake at night worrying about whether what they write is true). Their p-hacking/selective publishing could be a question of education/ignorance, and requires some form of awareness campaign (like the one currently ongoing). Pre-registration for these scientists is using a sledge-hammer to crack a nut (or FMRI to ask a behavioural psychology question).
Many perfectly sensible, thoughtful, informed scientists think that pre-registration will come with costs (whether pre-registration becomes obligatory, or whether it just becomes the cultural norm and leads to down-weighting of non-pre-registered studies). Costs in time, costs in making much of science more conservative by rewarding conservative studies, and down-weighting novel approaches to asking and answering questions. And for these costs not to outweigh the benefits, pre-registration requires the existence of a cryptozoological being such as Homo Scientificus.