#word-learning

[appliedfunctions]

I was searching for Star Wars memes, and I saw this beauty. The asymmetry between recognizing speech and talking sets off so many associations for me. 

It reminds me of the difference between receptive and expressive vocabulary. The number of words that we speak or write on a day-to-day basis is much smaller than the words we can read or comprehend. Which is the truer measure of vocabulary knowledge? Or maybe that question is a red herring, and it doesn’t make sense to partition our words into the ones we know well enough to use in a sentence versus the ones we know well enough to understand in a sentence. 

The meme also makes me think of Hinton’s work in deep learning: “To recognize shapes, first learn to generate images”. In a digit-recognition network, you train your model to recognize (to correctly label) a digit from a pixel image. Traditionally, you use supervised learning, so that the model starts guessing labels for words immediately and learns by minimizing its error rate. Hinton instead suggests that the model undergo unsupervised “generative pretraining” first and learn to generate those images. With this training, it can develop discover a useful set of general features and structure in the images. Once those features are in place, learning to label images become a matter of “discriminative fine-tuning” for the network. I still haven’t wrapped my head around the recognition/generation asymmetry in this domain, admittedly, but the meme reminds me of it. Maybe it’s a difference between seeing and imagining.

Lastly, the meme reminds me of P vs. NP, the question in computer science about different kinds of computational problems. The problems in P can be solved efficiently (P for polynomial time) like sorting a list or multiplying numbers. The problems in NP can have their answers verified efficiently (NP for nondeterministic polynomial time) like Sudoku, Minesweeper or subset sum. You don’t need to solve the Sudoku again to check the answer; just check each row, column and cell for duplicate numbers. The problems in P can have their solutions checked efficiently because they can be solved efficiently: Just solve the problem again and see if the solution matches the new answer. The million-dollar question is whether the family of problems that can be solved efficiently equals the family of problems that be verified efficiently. (The answer is probably not.) Here the production/recognition asymmetry translates into a difference between solving and checking and the question of whether there are problems that are easy to check but inherently hard to solve.

So, that Star Wars meme--it’s a real tapestry of meaning if you ask me.

[appliedfunctions]

Poster, code, and data for the poster I presented at 56th Annual Meeting of the Psychonomic Society (Nov. 2015 in Chicago, IL) . 

Next steps are to elaborate on the home language input factor structure and examine other ways to quantify lexical processing.

[appliedfunctions]

Vocabulary Growth Milestones

How do children's vocabularies grow over the first two years of life? Rescorla, Mirak, and Singh (2000) summarized findings from parental diary studies from the 70s and 80s as a series of milestones. A child's first words appear around 10-13 months, followed by a period of slow and steady vocabulary growth (on the order of 10 words/month) during 12-18 months. After a child has accumulated around 50 words, around 16-21 months, the "vocabulary spurt" begins and the child's vocabulary growth accelerates dramatically. Indeed, Huttenlocher and colleagues (1991) modeled vocabulary growth from 14 to 26 months as a simple quadratic function of age.

Importantly, there are individual differences in vocabulary growth rates. All children start out with the same number of words (zero), and there's very little variation in vocabulary sizes in children during the first year. On the first edition of the MacArthur Communication Development Inventory (CDI; 1994), the authors observed little variation in vocabulary sizes early on. Around 13 months, however, variation increases as the vocbulary percentiles begin to fan out.

We can reproduce this fanning-out finding and the other developmental phases using the Stanford Wordbank, a database of scores for the 2007 edition of the CDI. The figure below shows individual scores on the CDI Words and Gestures from 8 to 18 months.

The fitted lines in the figure show percentiles at each slice in time. The data are cross-sectional, so different children make up the scores in each time-slice. The topmost line tracks the average vocabulary size in the top 10% (90th percentile) of word learners in each time slice.

We can observe some trends mentioned above within this plot. First, the 90th and the 10th percentiles noticeably begin to drift apart around 12-13 months. This is the "fanning out" mentioned above; individual differences in vocabulary sizes begin to emerge around this point. Second, the precocious word-learners in each sample, shown in the top two lines, begin to show accelerating vocabulary growth, whereas the median word-learners are still in that slow, steady phase of word-learning.

The Wordbank also provides data on the CDI Words and Sentences, so we can track growth in vocabulary sizes from 16 to 30 months.

The same caveats apply as before: The data-points in each time slice comprise different samples of children. A further caveat applies as well: There are around 650 words on the CDI Words and Sentences inventory. We see a classic ceiling effect, so the curves begin to decelerate as they approach the ceiling. Nevertheless, the figure depicts the onset of accelerating growth in the 10th and 25th lower percentiles.

[appliedfunctions]

We wanted a task that would require the infant to show at least some evidence of semanticity, so it was essential that the task not just require the infant to listen to the form of a word but also require the infant to link the word to an object or event. But we wanted as well to be likely to capture infant sensitivities as close as possible to the 10-12 month age. At this age, word comprehension knowledge likely involves only recognitory understanding rather than referential understanding (Oviatt, 1982; Hirsh-Pasek and Golinkoff, 1996). Recognitory comprehension involves only associative knowledge, and thus simply requires that the infant learn the link between a word and an object. Referential comprehension requires an understanding of the symbolic link between a word and an object, entailing the notion that the word can stand for the object even when the object is not present. We thus developed a word-object associative-learning task in which the infant is required to learn the arbitrary association between a particular word and a particular object, and to show recognition of that learning by differential looking to a pairing that violates, versus one that maintains, the associative link that has been learned. Even if this type of associative word-object learning does not constitute full referential word learning, there is considerable consensus that such associative learning provides a necessary step toward referential word learning.

Werker, J. F., & Fennell, C. T. (2004). Listening to sounds versus listening to words: Early steps in word learning. Weaving a lexicon, 79-109.

Posting this paragraph because 1) I had never heard about the distinction between recognitory and referential comprehension until now and 2) I like how the authors describe what the baby has to learn and how he or she has to then demonstrate that learning in the lab.