March 13, 2007

Ken DeRosa on the Reading First controversy

The most important public policy issue relating to language, in my opinion, is the worldwide controversy about how to teach kids to read. A two-year-old skirmish in this war was reported in last Friday's NYT by Diana Jean Schemo, "In War Over Teaching Reading, a U.S.-Local Clash", dealing with the decision of the Madison WI school district to forego $2M in federal Reading First funds, rather than to give up their preferred approach to reading instruction.

Schemo's article spun the story as plucky little David against nasty corrupt Goliath -- at least that was the impression I took away from it ("Reading Corruption", 3/9/2007). Schemo mentioned "a string of blistering reports" from the U.S. Education Department's Office of the Inspector General about conflicts of interest and inappropriate attempts to force schools to use specific curricula, and she featured the Madison school system's claims that they're sticking with their system because it works so well.

But when I looked into the "string of blistering reports", they seemed pretty mild to me. And Liz Ditz pointed me to some posts by Ken DeRosa at D-Ed Reckoning, arguing (in my opinion convincingly) that in fact, Madison's system has badly failed the kids that Reading First is designed to help ("Schemo gets pwned", 3/9/2007; "Madison Cooks Books", 3/12/2007).

DeRosa's conclusion was that the Madison administrators fooled Schemo. It seems just as likely to me that the NYT is taking sides in this controversy -- the wrong side, unfortunately -- and that their reporter and editors were looking for numbers to show good results from an anti-Reading First school system. Certainly no one at the NYT checked out Madison's numbers the way that serious reporters would routinely check out the Pentagon's budget or politician's reports of campaign contributions -- and there was no event that made it crucial to get this story out a few days earlier, without hearing from critics of Madison's program.

As for the broader controversy about the implementation of the Reading First program, Ken DeRosa sent me a sketch of the legal and political issues involved, which I found very helpful. It's reproduced below.

Here’s what you need to know to make sense of the Reading First (RF) scandal (at least in my view):

Let’s start with the statute. In order to get RF funding “an eligible local educational agency … shall use the funds provided under the subgrant to … Select[] and implement[] a learning system or program of reading instruction based on scientifically based reading research that … includes the essential components of reading instruction.” Section 1202(C)(7)(A) (p. 114).

So, to get RF funding an eligible school must adopt a reading program that 1) is based on SBRR and includes 2) the ECRI. In the statute these are defined terms. Section 1208(3) defines the essential components of reading instruction as: explicit and systematic instruction in (A) phonemic awareness; (B) phonics; (C) vocabulary development; (D) reading fluency, including oral reading skills; and (E) reading comprehension strategies.

Basically, the ECRI come from the National Reading Panel’s (NRP) meta-analysis. In particular, ECRI must include instruction in phonics that is “explicit and systematic.” This clearly knocked out whole language programs. And, according to DOE’s reading of the statute, also knocked out any program based on implicit or embedded phonics, i.e., balanced literacy.

Section 1208(6) basically defines scientifically based reading research as research that is scientific. Here’s the verbiage: “research that (A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant to reading development, reading instruction, and reading difficulties; and (B) includes research that (i) employs systematic, empirical methods that draw on observation or experiment; (ii) involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; (iii) relies on measurements or observational methods that provide valid data across evaluators and observers and across multiple measurements and observations; and (iv) has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.”

The intent of this provision is to clear out all the bad research that clutters the files of education research. Here is where things start getting interesting. Note how section 1202 permits reading programs that are “based on [SBRR]” as opposed to reading programs that have their own SBRR base. This is the camel’s nose under the tent.

There are only three reading programs that have a SBRR base.

1. Success for All (SfA)
2. Open Court published by SRA (OC)
3. Direct Instruction, i.e., Reading Mastery, also by published bySRA (DI)

In actuality, the SBRR for OC is for a program that is no longer in print. And, the SBRR for DI is about twice as large as the SBRR for SfA and OC for at-risk kids. By this I mean that DI has larger effect sizes and a larger number level 3 research studies and replicated level 2 research studies. Let’s call these programs the only programs with validated SBRR.

All the other publishers vying for a piece of the RF pie set out to make their programs look like they contained the ECRI and then tried to pass themselves off as SBRR programs, as permitted under the RF statute.

DOE basically permitted any program to get RF funding if it contained the ECRI and, in particular, taught phonics in as systematic explicit manner as phonics is taught in SfA, OC, and DI. This meant that whole language and balanced literacy programs would be excluded and that most programs receiving RF funding would have no real SBRR base. Arguably, they would be based on SBRR which is all law requires, however misguided.

I have yet to see any evidence in any of the released OIG reports that shows that DOE excluded any reading program that was permitted under the statute or permitted any program that was excluded under the statute. DOE rightly excluded any reading program based on whole language and balanced literacy and they seem to be the only programs that were excluded by DOE.

However, many reading programs were excluded by state DOE’s. This is were the exclusion of SfA comes in. SfA wasn’t excluded by the federal DOE, it was simply not included by many state level DOE in their grant applications. This is a critical distinction missing from most analyses. The exclusion of SfA occurred at the state level, not the federal level. By the same token so did DI get excluded at the state level. Together SfA and DI account for about 3% of the reading programs getting RF funding, even though they are the only reading programs having validated SBRR.

Also, bear in mind that most state DOE’s and whole language/balanced lit programs were trying their hardest to get funding for their programs by alleging that their programs had ECRI and were also based on SBRR.

For more info see Bob Sweet’s letter to WaPo editor.

I have a few posts up related to the first OIG report and the subsequent media coverage collected here.

I didn’t rehash the latest OIG report, but basically concur with your analysis. Merely showcasing 2 of the 3 reading programs with validated SBRR does not amount to mandating, controlling, or directing curriculum.

Posted by Mark Liberman at March 13, 2007 06:09 AM