April 04, 2005

Higher scrabble scores lead to lower test scores

The abstract from a paper by David N. Figlio, "Names, Expectations and the Black-White Test Score Gap", NBER Working Paper No. 11195 (March 2005):

This paper investigates the question of whether teachers treat children differentially on the basis of factors other than observed ability, and whether this differential treatment in turn translates into differences in student outcomes. I suggest that teachers may use a child's name as a signal of unobserved parental contributions to that child's education, and expect less from children with names that "sound" like they were given by uneducated parents. These names, empirically, are given most frequently by Blacks, but they are also given by White and Hispanic parents as well. I utilize a detailed dataset from a large Florida school district to directly test the hypothesis that teachers and school administrators expect less on average of children with names associated with low socio-economic status, and these diminished expectations in turn lead to reduced student cognitive performance. Comparing pairs of siblings, I find that teachers tend to treat children differently depending on their names, and that these same patterns apparently translate into large differences in test scores. [emphasis added]

Figlio used

test score, gifted classification and transcript data for every student in this Florida school district from 1994-95 through 2000-01. Because of confidentiality restrictions, I cannot reveal the identity of the school district, but I can report that my dataset includes information on 55,046 children in 24,298 families with two or more children.

Most notable about my dataset is that I can compare the outcomes of sibling pairs, as proxied by children sharing the same home address and phone number.

He modeled the socio-economic status of names from an independent data set:

In order to measure the socio-economic status of a name, I use birth certificate data from all children born in Florida between 1989 and 1996 to predict the probability that a baby’s mother will be a high school dropout. I decomposed every observed name into a series of phonemic components—combinations of sounds, letter orders, and punctuation, and then regressed these combinations against maternal dropout status to construct predictions of socio-economic status implied by a name. Four frequent attributes of low socio-economic status names are particularly striking: (1) the name begins with one of a number of prefixes, such as "lo-", "ta-", and "qua-"; (2) the name ends with one of a number of suffixes, such as "-isha" and "-ious"; (3) the name includes an apostrophe; and (4) the name has is particularly long, with several low-frequency consonants. The easiest way to characterize this fourth characteristic is to count the number of "Scrabble" points of the name—I consider a name to have a high Scrabble score if its Scrabble value exceeds twenty points.

This measure identifies about 12% of the children in his school sample as having low socio-economic status names. Figlio found that "there is considerable within-family variation in naming patterns. Moreover, ... families, both Black and White, are equally likely to transition from a low socio-economic status name to one that has no identified characteristics as they are to transition away from a name with no identified characteristics".

While confidentiality restrictions prevent me from describing the names that are extremely uncommon in the Florida data set, I can identify names given at least ten times in the data to describe a hierarchy of names’ expected socio-economic status, and present all regression results in terms of a range of observed names—first I compare two marginally common names, one given almost exclusively to White children ("Drew") and one given almost exclusively to Black children ("Dwayne"). Then I compare names along a hierarchy, from a name with one identified attribute ("Damarcus") to a name with two identified attributes ("Da'Quan") to a name with three or more identified attributes (none are observed with sufficient frequency to name here.) Almost no White children are given names with two or more observed attributes, but ten percent are given names with one of these attributes. Most are sufficiently uncommon to name here, but some names given to at least ten White children in my dataset include "Jazzmyn" and "Chlo'e" (not to be confused with "Chloë", which is associated with high socio-economic status.)

He uses national percentile rankings on nationally-norm-referenced tests, and regresses these against the equation. The results:

The upshot here is that while names associated with Black children tend to be associated with modestly lower test performance, the largest estimated negative relationships between names and test scores occur with regard to low socio-economic status. We observe virtually identical results regardless of whether I characterize names using a socio-economic status index or merely count the number of low socio-economic status attributes of the name.

In fact, none of the effects are enormous: the largest (statistically-significant) effects on test scores that Figlio cites seem to be about 1.5 in terms of "national percentile ranking" (which implies a scale of 100). However, he presents the quantitative results exclusively in terms of a hierarchy of paired name comparisons (e.g. "Drew" vs. "Dwayne" or "Damarcus" vs. "Da'Quan"), and it may be that there are larger effects across the whole spectrum of names.

[via Jeff Erickson at Ernie's 3D Pancakes, via Abiola Lapite at Foreign Dispatches. Also a WaPo article.]

Posted by Mark Liberman at April 4, 2005 05:59 AM