Bergelson & Swingley (2013)

A recent paper by Elika Bergelson and Dan Swingley at UPenn reports that 10 - 13 month old infants show some evidence of understanding common abstract terms like "uh oh," "bye bye" and "all gone." This finding is big news because previous research hadn't found evidence for these terms until substantially later. Although parents report children's knowledge of these words very early on, parent report measures may be unreliable (an idea ratified by the current paper).

Bergelson & Swingley tested children from 6 - 16 months using a standard eye-tracking paradigm that measures whether infants look more at a movie when they hear a word that matches it. This work builds on a previous paper from the same authors that used the same method to show that 6 - 9 month olds showed some evidence of knowing simple common nouns. In the current study, the 6 - 9 month group showed no evidence of knowledge, but the 10 - 13 mos were above chance and the 14 - 16 mos were quite robust.

As someone who is fascinated by the emergence of early language, I like this paper and its predecessor quite a bit. It takes a common method for measuring infant language comprehension and alters in a few ways that may make it sensitive enough to measure comprehension in very young babies. The researchers use a corneal reflection eye-tracker to get accurate moment-to-moment data on babies' gaze. In addition, rather than using standard audio recordings of words, parents themselves produce the words after hearing a cue on headphones. And finally, stimuli like "all gone" (peering into an empty bowl) and "hi" (waving to the camera) are paired throughout the experiment, and what the authors measure is not whether the infants look more than chance (50%) at the "all gone" movie when they hear the matching words. Instead they ask whether, whatever the infants' bias is initially, this bias is changed by hearing the target item.

One thing that I don't love about this paired-item design, however, is that it doesn't let us make inferences about individual words. I'd be very interested to see whether children understand "all gone" in particular. This common term is one of the first that has negative content ("all gone" = "not there"), and it's an interesting and open question whether very young children understand negation. Some new research from my lab has tried to look into this question using a related method with older children and found significant challenges in children's comprehension of negative sentences. So it would be nice to compare our results to the Bergelson and Swingley findings. But because in their study "all gone" was paired with "hi," we can't really know which one the children understood (and it seems likely that it might have been "hi" in this case).

The framing of the Bergelson & Swingley paper rejects a traditional sounds -> forms -> meanings approach to acquisition, where young infants are assumed to know nothing about what words mean when they are mastering basic skills of speech perception. I strongly agree with this theoretical shift and think it dovetails nicely with two points I've been trying to make in my work.

First, natural language has a frequency structure that stands alongside its hierarchical organization into phonemes, morphemes, and words, such that some sounds, morphemes, and words are much more frequent than the vast majority of others. The Zipfian distribution of natural language means that some units at each level should be much easier to learn than others. So it stands to reason that some word meanings might be learned early, because they are just so frequent (and meaningful in the infant's life) even if the majority are mastered quite a bit later.

Second, my collaborator Mark Johnson has been writing about what he calls "synergies" in language acquisition - cases where it's better to try and solve two problems at once rather than one at a time. For example, in one piece of work we did together, we showed that models that tried to segment word forms from continuous speech performed better if they tried to link those word forms to word meanings at the same time. The word meanings acted as anchors that helped in the segmentation process. What this means is that if infants are trying to learn word meanings, learn sounds, and segment word forms all at the same time, they may see more success than if they follow a kind of "staged" strategy (as was previously assumed).

Two other minor points:

1. Since the sample size was very asymmetric across age groups (34, 46, and 18 babies in the three age groups respectively), I wondered what the recruitment strategy was and how the authors decided to terminate data collection. Given this asymmetry, I also wondered whether it would have been useful to use a more sophisticated analysis (e.g. some kind of non-parametric curve fitting) to estimate the shape of the developmental curve and when it differed from chance.

2. The authors included a nice corpus analysis of what sorts of interactions were happening when the children heard particular abstract words, as well as the frequencies of the words themselves. (I was also a bit surprised here that the authors didn't cite our work on interaction feature coding). Given that they now have comprehension data for both concrete and abstract words, it seems like they should be able to enter a number of different social and frequency predictors into a regression together to see which ones best predict the age of word acquisition. Perhaps Elika Bergelson has done this in her thesis already.

Overall, the two papers by Bergelson and Swingley make a strong addition to the literature on early word learning. I'm very curious to see what kinds of uses their method can be put to in future. One thing I have been considering, for example, is trying to do this kind of method longitudinally (say once a month) with M, as soon as she gets old enough. I'd love to see the emergence of knowledge of word pairs over time - though of course we'd have to think about whether we were testing her knowledge or adding to it!

Blogging "sabbatical"

My daughter M was born a little more than a week ago. The new (and very welcome) addition to our family means that I have a little more time to update this blog, something I've already begun to do in the quiet moments since she's been born. I'll be taking the rest of the summer off from active research duties and so will try to update the blog regularly, interspersing updates on M's development into my standard research blogging.

Review: Democracy Despite Itself

I just finished reading Democracy Despite Itself, a book by Danny Oppenheimer and Mike Edwards. Oppenheimer is a social [edit] cognitive psychologist and Edwards is a political scientist (Danny is a good friend of mine; we've known each other since we sat across the hall from one another when I was an undergraduate research assistant and he was a graduate student).

The premise of the book is that, despite the myriad flaws of democracy, it nevertheless has a number of features that make it the most successful system of governance. Oppenheimer and Edwards split the book into two parts. The first part describes four flaws of democracy: 1) voters typically don't know about the issues at stake in elections, 2) voters are generally "irrational" decision-makers even when they do have some knowledge, 3) it's very hard to design a representative democratic system (because of biases inherent in e.g. districting, election dates, etc.), and 4) for the elected representatives, it can be very hard to know what voters really want. Nevertheless, as the second part discusses, democracies are very successful overall--in terms of peace and prosperity for their citizens--compared with other systems of government, mostly because they set up good incentives for voters and representatives.

This general account sounds very reasonable overall, and DDI is a fun, easy read. It's filled with straightforward and clear examples (often from sports) that make the phenomena easy to understand. Danny wanted to call his book "DemoCrazy," and that nicely captures both the thesis and the tone of the book.

Despite how much I liked the book, I want to make two slightly critical points. First, DDI spends only a short amount of time on how to solve the problems it outlines (or even whether they are soluble). This is fine for a short book, but at times I worried that the two-part organization--"democracy is surprisingly crazy but yet works surprisingly well"--made the authors equate different kinds of problems that should be distinguished. While the quirks of human cognition--like our susceptibility to framing effects--aren't something that we can correct, other problems can be addressed, such as flaws in redistricting. While, as Oppenheimer and Edwards point out, there are many different metrics by which to assess the fairness of a redistricting scheme, some redistricting schemes aren't fair. There not be one single perfect solution, but we can definitely do better.  If redistricting choices are made by partisan groups, then they are especially likely to be unfair.

Second, DDI doesn't distinguish between problems that cause bias and those that cause variance. Consider elections as a method for estimating what governance a group of people wants. (This is potentially problematic but let's go with it for now). Let's start with the simplest case. Imagine Alice, a single individual. Alice has perfect knowledge of the policy landscape. She is a rational decision-maker. Presented with an infinite array of possible representatives who vary in their stated policies, she can then easily choose the one whose policies minimize her regret.

Now move away from this world in a few different directions. Imagine that she has only a random sample of a few possible representatives in each election. And she has only partial knowledge of each representative's position (assuming facts are sampled at random). Now finally, imagine there are thousands of Alices, each making their own decisions in accord with their own interest based on their own partial knowledge of the issues. It's easy to see that each of these lead to higher average regret for Alice (variance). But there is no systematic bias in the system yet. This seems like a reasonable model of some of the issues that DDI discusses, especially problems of imperfect knowledge on the part of the voters and representatives.

But there are other problems that would lead to bias, rather than variance. Imagine that the election is rigged, Alice's district is gerrymandered, her access to information is systematically manipulated. All of these systematically change the long-run fairness of the system. Of course we would want to reduce variance if we could, but these kinds of biasing factors seem much more dire. Distinguishing between the two feels important.

Outside of these minor concerns, DDI is a clear and engaging look at the topic. It's very well written also. I recommend it to anyone curious about the intersection of psychology and politics.

Hyperlinks and pointing (a Clarkian take)

I just read a piece of blog text describing and linking to a new article. It went something like this: "In a new paper by Author X and Author Y in Journal Z, theory T was shown to be false." Something rubbed me wrong about the linking style in this sentence. Which hyperlink should I have clicked on? (It turned out to be the link to the journal).

Placing hyperlinks in text is a funny business. It seems like there are some reasonably well-accepted standards for what text you underline when you want to place a link (at least such that I had the feeling of a violation in the sentence above), but I can't find any explicit guidelines. Perhaps it's because the google keyword "hyperlink" is not very informative at all...

But here's my alternative hypothesis: I think we place hyperlinks with the same timing as we use pointing in spoken speech, and for the same function: to determine reference. So that would mean that the best place for a link to start is where you would start pointing in a face-to-face conversation. And there are very clear norms for how the timing of pointing works; folks like Herb Clark have studied this extensively.

Going back to my example above, imagine I was sitting in my office with a pile of issues of Journal Y on my desk and pointing to illustrate my sentence. Imagine the following examples:

(1a) A [point] new paper by X in Y shows Z.
(1b) A new paper by [point] X in Y shows Z.
(1c) A new paper by X in [point] Y shows Z.
(1d) A new paper by X in journal Y shows [point] Z.

My intuition is that 1a is most natural, with 1d far behind. 1b and 1c definitely are about reference to the author and to the journal. Similarly:

(2a) A new paper by X in Y shows Z.
(2b) A new paper by X in Y shows Z.
(2c) A new paper by X in Y shows Z.
(2d) A new paper by X in journal Y shows Z.

The same thing seems to apply. If the link to Y doesn't go to the location (but to the actual thing itself) then it somehow feels wrong. Another fun way that technologies can build on pre-existing psychological abilities.

Thoughts on preregistration

There is substantial debate about pre-registering studies (in psychology and the behavioral sciences) to decrease the possibility of false discovery through exploratory data analysis. I agree with a lot of what has been said about the value of preregiatration. Nevertheless, preregistration exacts a constant cost in terms of researcher effort. We need to think through the cost of doing a study and how that cost compares with the preregistration cost. 

Consider a couple of different cases (all of these from my work):

1. a mechanical turk survey that can be repeated with many variations for a small amount of money
2. a sequence of studies with typically-developing children at a local preschool, easily repeated but with a month or two of RA effort
3. a longitudinal, school-based RCT in a foreign country, difficult and costly to repeat at best

In the case of (1) I don't think I'd recommend pre-registering each individual study. Once the study has been repeated, tweaked, and varied, a pre-registered version can be re-run as a confirmatory replication. But pre-registering each variation creates a system where the cost of running small-scale studies is increased, to the detriment of the productive process of "poking around" that can be very useful at the beginning of a study set.

For (2), personally I would again conduct several studies, progressing from pilots to investigations, with only the last one pre-registered, again for purposes of convincing readers and presenting a strong confirmatory test.

Finally, for (3), I would definitely advocate preregistration. The analyses can be complex and the temptation to poke around post-hoc and discover something unpredicted is just too great. I wish we had pre-registered our study of this type. Here the effort is minimal relative to the cost of the study.

Dale Barr tweeted earlier today "Tukey (1980) said we need both exploratory and confirmatory. #prereg gives us an honest way to signal the difference" - I think this is exactly the right approach.

(Post adapted from a message I sent to the OpenScience list.)