"Psychological plausibility" considered harmful

"goto" statement (programiz.com)

goto is considered harmful in programming languages.

The fundamental enterprise of cognitive science is to create a theory of what minds are. A major component of this enterprise is the creation of models – explicit theoretical descriptions that capture aspects of mind, usually by reference to the correspondence between their properties and some set of data. These models can be posed in a wide variety of frameworks or formalisms, from symbolic architectures to neural networks and probabilistic models.

Superficially, there are many arguments one can make against a particular model of mind. You can say that it doesn't fit the data, that it's overfit, that there are many possible alternative models, that it predicts absurd consequences, that it has a hack to capture some phenomenon, that it has too many free parameters, and so forth. But nearly all of these superficially different arguments boil down to well-posed statistical criticisms.

Consider a theory to be a compression of a large mass of data into a more parsimonious form, e.g. the "minimum description length" framework. For a given set of data, the total description length is the length of the theory (including its parameters), the predicted data from the theory, and some metric over the deviation of the data from those predictions. Under this kind of setup, the critiques above boil down to the following two critiques:

1. There's a theory that compresses the data more, either (a) by having fewer free parameters, or (b) by being overall more parsimonious. 
2.  If we add more data, your theory won't compress the new data well at all, where the new data are either (a) other pre-existing experiments that weren't considered in the initial analysis, or (b) genuinely new data ("model predictions"). Concerns about overfitting and generalization fall squarely into this bucket. 

Of course, we don't have a single description language for all theories, and so it's often hard to compare across different frameworks or formalisms. But within a formalism, it's typically pretty easy to say "this change to a model increased or decreased fit and/or parsimony." In linear regression, AIC and BIC are metrics for doing this sort of model comparison and selection. In the general Bayesian or statistical framework, the tradeoff between parsimony and fit to data is a natural consequence of the paradigm and has been formalized to good effect.

In this context, I want to call out one kind of critique as distinct from this set: the critique that a model is not "psychologically plausible." In my view, any way you read this kind of critique, it's harmful and should be replaced with other language. Here are some possible interpretations:

1. "Model X is psychologically implausible" means "model X is inconsistent with other data." This is perhaps the most common argument from plausibility. For example, "your model assumes that people can remember everything they hear."  Often this is an instance of argument (2a) above, only with an appeal to uncited, often non-quantitative data, so it is impossible to argue against. If there is an argument on the basis of memory/computation limits, citing the data makes it possible to have a discussion about possible modifications to model architecture (and the rationale for doing so). And often it becomes clearer what is at stake, as in the case of e.g. asking a model of word segmentation to account for data about explicit memory (discussion here and here) when the phenomenon itself may rely on implicit mechanisms.

2. "Model X is psychologically implausible" means "model X doesn't fit with framework Y." Different computational frameworks have radically different limitations, e.g. parallel frameworks make some computations easy while symbolic architectures make others easy. Consider Marr & Poggio's 1976 paper on stereo disparity, which shows that a computation that could be intractable using one model of "plausible" resources actually turns out to be very doable with a localist neural net.* We don't know  what the brain can compute. Limiting your interpretation of one model by reference to some other model (which is in turn necessarily wrong) creates circularity. Perhaps these arguments are best thought of as "poverty of the imagination" arguments.

3. "Model X is psychologically implausible" means "model X is posed at a higher/lower level of abstraction than other models I have in mind." To me, this is a standard question about the level of abstraction at which a model is posed – is it at the level of what neurons are doing, what psychological processes are involved, or the structure of the computation necessary in a particular environment. (This is the standard set of distinctions proposed by Marr, but there may even be other useful ones to make). As I've recently argued, from a scientific perspective I think it's pretty clear we want descriptions of mind at every level of abstraction. Perhaps some of these arguments are in fact requests to be clearer about levels of description (or even rejections of the levels of description framework).

In other words, arguments from psychological plausibility are harmful. Some possible interpretations of such arguments are reasonable – that a model should account for more data or be integrated with other frameworks. In these cases, the argument should be stated directly in a way that allows a response via modification of the model or the data that are considered. But other interpretations of plausibility arguments are circular claims or confusions about level of analysis. Either way, such arguments lump together a number of different possibilities without providing the clarity necessary to move the discussion forward.

---
Thanks to Ed Vul, Steve Piantadosi, Tim Brady, and Noah Goodman for helpful discussion, and * = HT Vikash Mansinghka. (Small edits 2/12/14.)

Early communicative vocalizations and the raspberry

(Fresh raspberries.)

M has begun blowing raspberries. It started one day just before she turned five months. I picked her up at day care and a teacher asked me whether she had been doing it at home – but this was the first I had heard of it. When I got her home and went to change her diaper, she rewarded me with a big wet "thbbbbt!" Since then, she deploys them unpredictably, with the raspberry even being her dominant vocalization some days.

What do the raspberries mean? I don't know – but they have an interesting feature: they are not necessarily either happy or sad. Sometimes M is struggling to get something out of reach, grunting with frustration, and then she turns aside and blows what seems to me (and other unbiased observers, like her mom) to be a sad raspberry. Other times, she looks at her grandmother or grandfather, smiles, and blows a happy one.

A gorgeous new paper by Oller and colleagues argues that this affective flexibility is an important characteristic of infants' early vocalizations. The study is very simple, coding the affective expression (e.g. smile, grimace, etc.) that goes along with a range of baby noises, including laughter, crying, squealing, growling, and "vocants" (coos and their siblings). Unlike laughter and cries, the other three vocalizations are all remarkably flexible. Like M's raspberries, they go along with – perhaps are even used to express – sadness, happiness, or something else altogether. Even infants 3 - 4 months old showed this same flexibility.

This early flexibility account seems like it goes along with a number of other proto-communicative responses that I've seen in M. Each of these is individually-consistent with conditioning of a specific response to a social stimulus. But they are also not inconsistent with the early beginnings of communication: 

• When M is hungry at night, she cries harder when she sees or hears me come into the room. Presumably this is because she knows she has an audience?
• M looks at me when I say her name (also when I say other things, but her name and various nicknames seem especially attention-grabbing)
• M has started putting her arms up in a way that could be interpreted as "pick me up!"
• Once (around 4 months), M cooed loudly, and I turned around in surprise. Her face lit up with a bigger smile than usual, as if to say "I didn't know that would happen!"

Again, none of these give strong evidence in favor of a communicative account over pure stimulus-response learning. But all seem like great candidates for further investigation – either to test whether they are communicative  or else to understand better how communication emerges out of simple social associations. 

(A post I drafted last month, hastily wrapped up.)

Oller DK, Buder EH, Ramsdell HL, Warlaumont AS, Chorna L, & Bakeman R (2013). Functional flexibility of infant vocalization and the emergence of language. Proceedings of the National Academy of Sciences of the United States of America, 110 (16), 6318-23 PMID: 23550164

Emergence of empathy and separation anxiety

M is six months this week (and sleeping in this morning; hence, time for blogging).

Over the weekend I noticed the emergence of two social behaviors – separation anxiety and empathic crying – that I hadn't previously thought were related. But this developmental co-incidence is making me reconsider.

Re "separation anxiety": M is an extremely social baby and up until now has loved being held by other people, spending time with different teachers at daycare, etc. So I was quite surprised when – for the first time – she burst into tears when a friend was holding her and I left the room. The same exact thing happened the next day with a different friend and M's mom, and then again at daycare yesterday.

From BabyCenter (that reliable source for all things developmental):

At around 7 months your baby will realize that he's independent of you. While this is an exciting cognitive milestone, this new understanding of separateness can make him anxious. He knows that you can leave him, but he doesn't know that you'll always come back, so he's likely to burst into tears when you leave, even for a minute.

I'm somewhat skeptical about the Freudian gloss about individuation that's given – and the connections to Piagetian object permanence also freak me out a bit (given that the different markers of this visual ability have moved around so much developmentally and depend on so many other aspects of cognition, e.g. here and here). Nevertheless, the phenomenon is striking.

And re empathic crying: a much-cited study suggests that even newborns cry more when they hear other babies crying. And indeed, I had noticed that M was a little bit more likely to cry when someone was crying nearby. But my primary observation was that she mostly didn't look visibly distressed by their distress. I was surprised, then, that during this last weekend she cared so much about our friends' sad toddlers and babies. To over-interpret a bit, it seemed like she got a pained look every time they screamed – as though she didn't want to continue carefree play while someone else was in so much discomfort.

On the Freudian/Piagetian gloss of separation anxiety, these two behaviors should likely have very little to do with one another. But if you posit that there is some underlying social understanding that is developing, it's not completely implausible that they are coincident. Perhaps the same understanding of their own vulnerability without the parent is also helping the child feel sympathy for a distressed other...

(edits 1/14)

A belated git migration

It's coming up on conference paper season, specifically for the Cognitive Science Conference. I love how deadlines like CogSci push research forward, giving us an intermediate goal to shoot for. But when lots of folks in the lab are writing papers separately, keeping track of all the drafts can get unwieldy very fast. My resolution this year is that no one will send me any more zip files of a directory called "CogSciPaperFinal." File naming practices like this one have been caricatured before,* but they get even worse when I'm constantly trying to track something like 6 - 8 different papers going forward at the same time.

Towards that end, our last lab meeting of the quarter was on version control software. In a nutshell, version control packages allow individuals and collaborative groups to work together on a project (usually a software codebase) and provide tools for keeping track of and merging changes to the project. It's painfully clear that we're late to the party: virtually no one in industry works on a large project without version control, but, as is frequently noted, scientists are not good software engineers.

We are starting a lab-wide push to keeping track of all of our writing and code using git and github. This transition will mean a bit of discomfort – hopefully not pain – but it's a far better method for storing our work and sharing it with collaborators. If you haven't played with git, I recommend looking at this nice tutorial by NYU's John McDonnell. I also found it very useful to do the TryGit tutorial. The lab's (currently very empty) github page is here. Hopefully in a couple of months it'll be substantially fuller...

---
* HT: Michael Waskom.

Computation under interference

(ENIAC, the first electronic general purpose computer; courtesy Wikipedia).

What if you have a very powerful computer, but it only works some of the time? Maybe it's made from vacuum tubes, and when they overheat or when some dust or a fly ends up in the works, one of those tubes burns out. Then the computer is down for weeks on end. But even if it's nonfunctional most of the time, when it's working, it's turing complete.

I'm coming to think that babies are this sort of computer. Perhaps the biggest puzzle in cognitive development is the amazing things that babies can sometimes do in very controlled settings (think moral reasoning) and yet the tremendous amount they can't do the rest of the time (think anything except eat, sleep, poop, suck, swipe and occasionally give you a charming smile...). I wonder if one way to reconcile these two different conceptions of infants is by thinking about the challenges of regulating their arousal – in much the same way you need to regulate the temperature of the vacuum tubes to get optimal computing performance.

Sometimes M gives me an incredibly intelligent look and does something unexpected. In the past weeks, she's been trying to pick up her chair to see the bottom, cooing systematically in response to me, or pulling out and reinserting her pacifier in her mouth. But other times she is glassy-eyed because she's concentrating on eating, or wiggling because she has indigestion. Tiredness is the biggest cause of cognitive failures. When I'm tired, I get grouchy, and my reactions are slower. When M is tired, everything goes to pieces. For a while she would even forget how to swallow: Milk would come pouring out of her mouth because she was sucking it in but forgetting to put it away down her throat...

When I was starting to think about M's cognitive development, right after she was born, I described her crying as a feedback look, where arousal leads to more and more arousal unless there is some internal regulation or external noise in the system. Having observed her for a few more months, I'm increasingly convinced that crying is only one small part of this process.

In fact, most of M's psychological world – perhaps ours as well, though it's well-hidden – seems like it's about regulation of attention (think temperature in the vacuum tube room). Part of this is learning to attend to what is interesting in the world (say, her father's face rather than the blinds). Another partis learning to suppress attention to all kinds of stimuli, including both visual stimuli and internal sensations (like gurgling in the stomach or wetness in the diaper). When she gets tired this all stops happening. Internal sensations get amplified, external ones don't get attended to. The vacuum tubes start burning out, and only a long, relaxing nap will help.

Confounds in developmental time

(Looking for developmental dissociations between processes can be a profitable research strategy, but such dissociations may be affected by external events like the transition to formal schooling.) 

As a developmental psychologist, I'm primarily interested in answering "how" questions: How do children figure out how objects work, learn the meanings of words, or recognize the beliefs or goals of others? Yet along the way, I can't help interacting with the (less interesting) more descriptive set of "when" questions: When do children show evidence of object permanence, learn their first word, or pass false belief tasks? And in studying any individual phenomenon, answers to "how" questions can be informed by estimates of when a particular behavior is first observed.

But here's an issue that has been bothering me for a while. Our "when" estimates – derived as they are from the behavior of middle-class kids in the US and Europe – are not independent from one another. They are instead highly correlated, because of external milestones in the lives of the children we are studying. Transitions to preschool or to kindergarten are major drivers of new behaviors. Worse, because teachers are active readers of developmental psychology, new school experiences likely involve explicit practice of exactly the kinds of skills we're interested in studying.

One possible example of this issue comes from a lovely talk I heard by Yuko Munakata at the Cognitive Development Society meetintg. Munakata has a deep body of recent work on the development of children's executive function (roughly, the ability to shift flexibly between different sets of behaviors according to context or task; review here). She documents transitions in children's executive function, including the transition from reacting to a stimulus to proactive preparation – choosing the proper behavior for a particular situation ahead of time. To be clear, nothing in Munakata's work depends on the precise timing of these transitions. Yet suspiciously, many of the transitions she studies happen in the same age range (4 - 6 years) when children are transitioning to school, an environment where their executive functions are being challenged and perhaps even trained.

A second example (very far from my area of expertise) comes from a comment made by Kate McLean in a recent brownbag talk she gave at Stanford. McLean studies identity development in adolescents, and she noted a big uptick in the quality of narratives in later high school. When she probed more deeply, however, she uncovered an external driver: late high schoolers were all engaged in the same social ritual: college application essays.

The research in these examples is not necessarily compromised by the presence of external events. But nevertheless, these kinds of events are big factors that might affect study outcomes in ways we wouldn't otherwise predict. From my perspective, I wonder how much the cognitive constructs I am interested in – pragmatics, language learning, theory of mind reasoning – are affected by individual children's transition to preschool, since the period around age 3 - 4 is a time of tremendous development for all of these abilities.

Studies that dissociate age and school shouldn't be too complicated to do, for either executive control or for other constructs. And these sorts of studies might give us some insights into the ways that (pre-) school experiences support the development and refinement of cognition. I recently heard the term "academic redshirting": holding children back from starting school so that they are older and do better than their peers when they finally start. This is a fairly intense (and controversial) strategy for getting kids ahead, but it might create an interesting natural opportunity for studying cognitive development...

What can a four-month-old do?

If you read papers on babies, you get the sense that mostly they just stare at stuff. The vast majority of research on babies uses visual attention – usually time looking at a screen or puppet show – as its dependent measure. Some experiments use more exotic dependent variables, like operant conditioning of kicks, pacifier sucking rate, or even smiles. But since Fantz's work in the 1960s, habituation and related looking time paradigms have dominated the field. Although we're reminded occasionally that babies cry, fuss, poop, and sleep, developmentalists appear far and away most interested in looking (very nice review and critique of this idea by Dick Aslin here).

As a reader of that literature, it's been consistently amazing to me to see what M can do, even as a little baby. She is about to turn four months old next week, and the the range of her behaviors is astonishing. Even more interesting is that some of this behavioral repertoire gives clear signals to underlying cognitive processes. Here's a quick list of some things I've noticed:

Eating. M takes a lot of her meals from a bottle. Early on, she showed no recognition of the bottle itself until it touched her cheek or lip, activating the rooting reflex. But around a month or six weeks ago ago she started showing signs of recognizing the bottle as an object, and responding to it before it reached her mouth. At first, the evidence seemed inconclusive to me – she was reaching (at that point mostly unsuccessfully) for many objects, so a reach for the bottle didn't seem diagnostic. But now she shows clear signs of recognition: When she is hungry and sees the bottle, she vocalizes and opens her mouth. Although I haven't tested this systematically, her recognition seems fairly viewpoint-invariant: she can recognize the bottle in many different orientations. This provides converging evidence for object categorization in 3 - 4 month-olds. It also seems like it could be a neat method for studying vision – think specially engineered bottles with different shapes and colors...

Smiling. Ever since around six weeks, M has been a very smiley baby. She greets people with a big smile, sometimes even smiling when she is otherwise quite fussy. It's kind of fun (in a slightly sad way) to watch smiles war with crying. If she is starting to fuss you can smile at her and see a reciprocal smile fight its way through her pouty face. But so far I have seen no evidence that her smiles reflect recognition of me or her mom: she gives them quite indiscriminately right now. (I know there is other evidence for recognizing and preferring mom, via her face or even her smell, very early on; I just find it surprising that she smiles roughly as much for others as she does for us).

Also, even if I hadn't tested M's face preference to schematic stimuli early on, her smiles would be a good indicator of her recognition of pictures. M will give a big smile at a picture of a baby's face. (Before I saw this, I never understood why people gave us board books filled with baby faces.) It doesn't seem surprising now that babies recognize pictures, but people used to argue that there were "primitive peoples" (presumably tribes somewhere) who didn't recognize photos. Hence picture perception – the ability to recognize the content of pictures – would be a learned cultural skill, and so babies wouldn't recognize pictures. A beautiful study by Hochbert and Brooks (1962), in which they denied their own child access to pictures and then tested his recognition, nicely disproved this idea.

Rolling over. M has rolled over a few times, from prone (tummy) position to her back. Each time, she was interested in an object on one side of her, and she turned her head and body that way (rotating herself onto her side), then began to kick her legs. When she kicked especially hard, her center of gravity tipped over her midpoint, and she flopped onto her back. This was clearly not something she was expecting, viz. her look of complete and total surprise. The interesting thing is that she hasn't been able to reproduce this behavior in a week. This kind of motor exploration really looks like reinforcement learning, where the issue is assigning credit for the result: which of many different behaviors produced the rewarding outcome?

Vocalizations. M started cooing right around when she started smiling – a very adorable behavior. Now her vocalizations have differentiated a bit more: coos when she is in a good mood, squeals when she is starting to fuss. But the most interesting noise she makes is something we call her "cognition noise." There are several physiological measures of attention and cognition in infants, for example heart rate and pupil dilation. M presumably shows these, though we haven't measured. What we didn't bargain for  is that she actually shows changes in respiration and vocalization when she is concentrating. When we give her a new toy, she stares at it, grunts, and breathes heavily. It's almost like the fan coming on for a MacBook Pro when the CPU is working hard. Adorable – and a nice external measure of attention.