(tl;dr: Some questions I'm thinking about, inspired by the idea of studying the broad structure of child development through larger-scale datasets.)
My daughter, M, started kindergarten this month. I began this blog when I was on paternity leave after she was born; the past five years have been an adventure and revolution for my understanding of development to watch her grow.* Perhaps the most astonishing feature of the experience is how continuous, incremental changes lead to what seem like qualitative revolutions. There is of course no moment in which she became the sort of person she is now: the kind of person who can tell a story about an adventure in which two imaginary characters encounter one another for the first time,** but some set of processes led us to this point. How do you uncover the psychological factors that contribute to this kind of growth and change?
My lab does two kinds of research. In both my hope is to contribute to this kind of understanding by studying the development of cognition and language in early childhood. The first kind of work we do is to conduct series of experiments with adults and children, usually aimed at getting answers to questions about representation and mechanism in early language learning in social contexts. The second kind of work is a larger-scale type of resource-building, where we create datasets and accompanying tools like Wordbank, MetaLab, and childes-db. The goal of this work is to make larger datasets accessible for analysis – as testbeds for reproducibility and theory-building.
Each of these activities connects to the project of understanding development at the scale of an entire person's growth and change. In the case of small-scale language learning experiments, the inference strategy is pretty standard. We hypothesize the operation of some mechanism or the utility of some information source in a particular learning problem (say, the utility of pragmatic inference in word learning). Then we carry out a series of experiments that shows a proof of concept that children can use the hypothesized mechanism to learn something in a lab situation, along with control studies that rule out other possibilities. When done well, these studies can give you pretty good traction on individual learning mechanisms. But they can't tell you that these mechanisms are used by children consistently (or even at all) in their actual language learning.
In contrast, when we work with large-scale datasets, we get a whole-child picture that isn't available in the small studies. In our Wordbank work, for example, we get a global picture of the child's vocabulary and linguistic abilities, for many children across many languages. The trouble is, it's very hard or even impossible to find answers to smaller-scale questions (say, about information seeking from social partners) in datasets that represent global snapshots of children's experience or outcomes. Both methods – the large-scale and the small-scale – are great. The trouble is, the questions don't necessarily line up. Instead, larger datasets tend to direct you towards different questions. Here are three.
1. How do you connect small mechanisms to big changes?
An individual child's vocabulary is made up of hundreds or thousands of individual words, each of which has its own natural history – how and when it was learned, what information was used, what inferences were made. For example, M figured out that "parchment" is a kind paper because Harry Potter was always writing on it. But this is true for any other piece of knowledge (or for that matter, any other skill) as well – it has its own learning history that is contributed to in different ways and to different extents by particular processes and experiences. These individual contributions are typically the object of study for small-scale experimental studies, but in larger-scale observations we only see the result of these – the accreted strata of experience as fossilized by learning.
The problem is that paleontology in this situation isn't straightforward. We don't have a good sense what it would look like if words – or for that matter, any other kind of skill or knowledge – were learned exclusively via a particular route. The best work of this type that I know about is a slightly-esoteric but cool line of computational investigations of word learning (example 1, example 2) that ask about what vocabularies look like – in terms of their growth, composition, and learning times – under different assumptions about the mechanisms in operation.
Relatively little work has tried to connect this kind of theorizing to empirical datasets, however. In one very recent preprint we've tried to take a first step in this direction by asking about the effects of different predictors on the composition of children's early vocabulary (e.g., does word frequency in the input predict which words are learned earlier, or does the conceptual concreteness predict better?). But lots of work is still needed to connect actual mechanistic proposals about in-the-moment learning mechanisms to larger-scale datasets that characterize what children's knowledge looks like.
Even if you have proposals about learning mechanisms, how do you verify that they add up to the kind of child you see in the aggregate measures?
2. Does development mostly hang together or is it many different things?
Piaget's developmental theorizing offered at least two things. The first is an account of how knowledge grows and changes – the relationship between assimilation and accommodation. This account feels very modern to me, as I wrote about a while back ("Was Piaget a Bayesian?"). The other part of the story was an elaborate theory about global, stage-based transitions in children's development. This second part, the stage theory – while on the whole still taught and tested more in textbooks of developmental psychology – has fallen into disrepute in terms of its empirical validity. My favorite critique is Gelman & Baillargeon (2003). But the particular stages posited by Piaget don't need to be right for us to consider the factor structure of development more broadly.
Another way of looking at this. My grandmother (who worked as a research assistant at the Yale Child Studies Center in the 50s and 60s) apparently used to say that kids "either walk or talk," meaning that they would either achieve one milestone or the other first. This is a multi-factorial view of development, in which language vs. locomotor development are two different capacities that are in fact anti-correlated.*** Actually it seems like walking and vocabulary growth are positively correlated. This is a small case study, but it raises the question of how the different features of global developmental progress relate to one another.
Intelligence is defined psychometrically via little g, the first factor in a factor analysis of many tests of cognitive ability. The empirical regularity is that g usually accounts for a substantial amount of variance across cognitive tasks – though that doesn't necessarily mean it's a unitary construct. One analogous question you could ask is about development in early childhood. Early language hangs together astonishingly well, but does early language relate to motor development, for example? There are some reviews that argue that it does, but I'm not aware of a comprehensive analysis of children's trajectories through both that dissociates shared variation due to age.
More generally, is there a little d, that – beyond age – explains global developmental advancement or delay? Statistically, there must be, but how much of the variance does it explain, and what capacities are most tightly related to one another?
3. What's variable and what's consistent?
Finally, how universal are developmental trajectories, across children and across cultures? Imagine having some arbitrary estimate of locomotor development that assigned a number on some (hypothetically) reliable and valid scale. We could ask about the variance of this measure for a particular age group, but that would be largely meaningless without any units or comparison. But by comparing that variation to developmental variation, we can reason about how consistent individuals' development is. This variation is argued to be small for stereoacuity of depth perception, for example, while it is much larger for vocabulary.
Neither of these cases make apples-to-apples comparisons, however. To be precise, units of variation would have to be defined in terms of the ratio of individual variance to developmental variance (as a function of either absolute age or percentage age). Using this approach, you could begin to ask, is variation across individuals larger for particular aspects of development than others? Or is variability itself standard across developmental phenomena?
One further addition is the application of these ideas to cultural variance in skills. Once we have comparable units for a particular skill, we can ask about the relative variability across individuals vs. variability across cultures. What proportion of total variance is due to cultural variability vs. idiosyncrasies of individuals' development? This variance-partitioning approach is in some sense a statistical answer to old questions about universals and variation in language (and other domains) of development.
Conclusions
Bigger datasets shouldn't lead us to abandon our questions. Nor should they lead us to forget basic statistical facts – e.g., the problematic nature of correlational studies or inferences from convenience samples. But in pursuing the kinds of answers they can give, they sometimes lead back in interesting ways to prior theoretical developments; some of these feel almost forgotten in our current emphasis on small-scale, tightly controlled experiments.
---
* That's just on the professional side. Being a parent has changed me profoundly as a person – I hope for the better.
** Harry Potter, of course, and Hiccup from How To Train Your Dragon.
*** I don't know if she'd endorse this view more generally – she passed away before I was born, and this anecdote is related by my dad.
No comments:
Post a Comment