Misperception of incentives for publication

There's been a lot of conversation lately about negative incentives in academic science. A good example of this is Xenia Schmalz's nice recent post. The basic argument is, professional success comes from publishing a lot and publishing quickly, but scientific values are best served by doing slower, more careful work. There's perhaps some truth to this argument, but it overstates the misalignment in incentives between scientific and professional success. I suspect that people think that quantity matters more than quality, even if the facts are the opposite.

Let's start with the (hopefully uncontroversial) observation that number of publications will be correlated at some magnitude with scientific progress. That's because for the most part, if you haven't done any research you're not likely to be able to publish, and if you have made a true advance it should be relatively easier to publish.* So there will be some correlation between publication record and theoretical advances.

Now consider professional success. When we talk about success, we're mostly talking about hiring decisions. Though there's something to be said about promotion, grants, and awards as well, I'll focus here on hiring.** Getting a postdoc requires the decision of a single PI, while faculty hiring generally depend on committee decisions. It seems to me that many people believe these hiring decisions comes down to the weight of the CV. That doesn't square with either my personal experience or the incentive structure of the situation. My experiences suggest that the quality and importance of the research is paramount, not the quantity of publications. And more substantively, the incentives surrounding hiring also often favor good work.***

At the level of hiring a postdoc, what I personally consider is the person's ideas, research potential, and skills. I will have to work with someone closely for the next several years, and the last person I want to hire is someone sloppy and concerned only with career success. Nearly all postdoc advisors that I know feel the same way, and that's because our incentive is to bring someone in who is a strong scientist. When a PI interviews for a postdoc, they talk to the person about ideas, listen to them present their own research, and read their papers. They may be impressed by the quantity of work the candidate has accomplished, but only in cases where that work is well-done and on an exciting topic. If you believe that PIs are motivated at all by scientific goals – and perhaps that's a question for some people at this cynical juncture, but it's certainly not one for me – then I think you have to believe that they will hire with those goals in mind.

Was Piaget a Bayesian?

tl;dr: Analogies between Piaget's theory of development and formal elements in the Bayesian framework.

Intro

I'm co-teaching a course with Alison Gopnik at Berkeley this quarter. It's called "What Changes?" and the goal is to revisit some basic ideas about what drives developmental changes. Here's the syllabus, if you're interested. As part of the course, we read the first couple of chapters of Flavell's brilliant book, "The Developmental Psychology of Jean Piaget." I had come into contact with Piagetian theory before of course, but I've never spent that much time engaging with the core ideas. In fact, I don't actually teach Piaget in my intro to developmental psychology course. Although he's clearly part of the historical foundations of the discipline, to a first approximation, a lot of what he said turned out to be wrong.

In my own training and work, I've been inspired by probabilistic models of cognition and cognitive development. These models use the probability calculus to represent degrees of belief in different hypotheses, and have been influential in a wide range of domains from perception and decision-making to communication and social cognition.¹ But as I have gotten more interested in the measurement of developmental change (e.g., in Wordbank or MetaLab, two new projects I've been involved in recently), I've become a bit more frustrated with these probabilistic tools, since there hasn't been as much progress in using them to understand children's developmental change (in contrast to progress characterizing the nature of particular representations). Hence my desire to teach this course and understand what other theoretical frameworks had to contribute.

Despite the seeming distance between the modern Bayesian framework and Piaget, reading Flavell's synthesis I was surprised to see that many of the key Piagetian concepts actually had nice parallels in Bayesian theory. So this blogpost is my attempt to translate some of these key concepts in theory into a Bayesian vocabulary.² It owes a lot to our class discussion, which was really exciting. For me, the translation highlights significant areas of overlap between Piagetian and Bayesian thinking, as well as some nice places where the Bayesian theory could grow.