Could conference submission be preregistration?

If we care about the answer to a particular question, preregistration – registering hypotheses and analyses ahead of time so that they are not data-dependent – is an important strategy for improving the strength of the evidence from studies bearing on that question. Of course, preregistration has some pros and cons. In my mind, the most notable these is that prereg is more appropriate for large, expensive, confirmatory studies than small, cheap, exploratory studies that are easily replicated (see my post about this topic).

In brief: My worry about pre-registered journal papers is that they can be very expensive in terms of research effort. If no one really cares about a hypothesis, then it's not a big deal not to publish on it. But if you preregister your crazy, speculative claim, then you may be stuck writing a paper telling everyone something they already expected: that your crazy idea, which would have been cool if true, is actually false. And writing papers is hard work: it takes a long time, and has severe opportunity costs. You could be doing new research during the time you are writing a careful, clear, and comprehensive paper on a thing that no one cares about because it wasn't likely to be true and indeed isn't.*

Nevertheless, there's no denying that it's good to be able to see an unbiased sample of experimental hypotheses. So here's a thought.

Something I always tell students NOT to do is to submit to conferences before they are done collecting data. This practice means that you have to impose your own biases on your preliminary data, and it can put you in an awkward position if you write a strongly hypothesis-driven abstract about data that don't end up supporting your spin on them.

But what about if we exploited this issue? We could create a track at conferences where you would submit an abstract on what you were going to do but hadn't yet done – essentially a prereg track. Then we'd have a particular poster session for seeing the results. All we'd need to do is to make sure that the conference abstracts themselves were indexed appropriately, and perhaps require an updated, post data-collection addendum. The upsides would be A) a chance for folks (especially undergrad and early grad students) to get an opportunity to present their work, and B) a low-cost preregistration mechanism.

The Cognitive Science Society already has a track for lightly-reviewed "member abstracts" – essentially posters on work that isn't done enough to merit a six-page paper. Why not "pre data-collection abstracts" too?

---
* Let me emphasize here that I'm not talking about hypotheses where a null result is important and informative, e.g. as in intervention work, or tests of theoretically-central claims. I'm talking about the kind of exploratory work – trying to play around with novel theoretical ideas – that characterizes a lot of research in cognitive science.