March 01, 2006

Counting freedoms, Simpsons and percentages

[More here on the way the survey numbers were spun.]

According to a piece that aired today on NPR's Here and Now ("Freedom and The Simpsons"):

A new survey conducted by Chicago's McCormick Tribune Freedom Museum, which has yet to open, finds that only 28 percent of Americans are able to name one of the constituational [sic] freedoms, yet 52 percent are able to name at least two Simpsons family members.

If you listen to Robin Young interview Dave Anderson in the cited segment of the show, you'll learn that the "five freedoms" guaranteed by the First Amendment are religion, speech, press, petition and assembly -- and that only 1% of Americans surveyed could name all five of them, while 22% could name Homer, Marge, Bart, Lisa and Maggie.

But counting the Five Freedoms is confusing, after all: the Bill of Rights has 10 amendments, but the First Amendment covers 5 freedoms. And the wording of the First Amendment only mention 2 "freedoms" as such (speech and press), plus 2 "rights" (assembly and petition); and religion gets mentioned twice (no establishment of it, no prohibition of it), but only counts as one "freedom":

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

Then there are all sorts of other freedom-y things in the other nine amendments in the Bill of Rights, like the right to bear arms, the freedom from unreasonable searches and seizures, freedom from forced self-incrimination, freedom from cruel and unusual punishments. And then there are later amendments that have a lot to do with freedom, like the abolition of slavery in the 13th amendment. I bet that lots of people started listing things like those, and then realized that there are a lot more than five of them, and that some of them are more rights than freedoms, and ...

No, it's not surprising that Americans have trouble with the arithmetic of freedom, although it would be great if everyone could reel off religion, speech, press, petition and assembly with just as much facility as they can name Homer, Marge, Bart, Lisa and Maggie. A more surprising problem with arithmetic came in this passage:

Robin Young: Well, a- and when you say one percent, you mean you talked to a thousand people, only one in a thousand people knew all of the five freedoms?
Dave Anderson: That's correct.

[Update: There's a bigger numerical problem with this report: the museum's survey actually says that "72% were able to name at least one of these rights correctly, [but] this fell to only 28% who could name two or more,"; Robin Young renders this as "only 28% of those asked were able to name *one* of the freedoms, yet 52% could name at least *two* of the members of the Simpsons family".]

OK, this is kind of unfair, and I vacillated before posting about it. Robin Young co-hosts an hour-long radio show every day, day in and day out. It's a fine show, one that I often listen to. And everybody says dumb things from time to time, especially me. Several times a semester, someone comes up to me after a lecture and points out that I said something that didn't make any sense at all, and I realize that I swapped two words around, left out a not, or said log when I meant exp, or something like that. So I'm reluctant to beat up on Robin Young for defining "one percent" as "one in a thousand".

But what about Dave Anderson? Was he just disposed to say "that's correct" no matter what nonsense Robin Young attributed to him? Or is he also somewhat confused about the meaning of the statistics that he's peddling?

And Here and Now is not a live show. Presumably several people heard this piece before it aired. Does the fact that this blooper got through tell us something about quality of editing (or the degree of mathematical literacy) at NPR? No, let's just chalk it up to the fact that everybody makes mistakes, especially when speaking ex tempore.

[Wow. The confusion is spread wider and deeper than I thought. According to the UPI wire service story,

More jarring is that 22 percent of those polled can name all five characters -- Homer, Marge, Bart, Lisa and Maggie -- but just 1-in-1,000 people surveyed -- 0.01 percent -- were able to name all five freedoms.
...
The random telephone survey of 1,000 adults was conducted in January, and has a 3-point margin of error.

So this story makes an order-of-magnitude error in the opposite direction ("1-in-1,000" is given as "0.01 percent", though it's actually 0.1 percent) -- and also reports a rate of "1-in-1,000" for a survey that has a 3-point margin of error.

The press release from the McCormick Tribune Freedom Museum says "just one in 1,000 people surveyed (.1 percent)". So it's not their fault, except for featuring a 0.1 percent value from a survey with a nominal plus or minus 3.0 percent margin of error.

A longer form of the survey results is here. A quick read shows that there should be some very embarrassed people at Here and Now. The actual survey results were that 72% of the respondents could name at least one first amendment freedom, not 28%. The figure of 28% was for naming at least two of the five. The press release says that "only about one in four Americans (28 percent) are able to name more than one of the five fundamental freedoms granted to them by the First Amendment to the U.S. Constitution". This is misleading and tendentious writing, characteristic of the way that PR people spin the presentation of facts to promote the interests of their clients. In this case, the press (at least someone at Here and Now) swallowed it hook, line and sinker, and wound up mis-stating the true percentage by more than a factor of two (28% rather than 78%).

And the document confirms that the most of the people surveyed were able to list a large number of constitutionally guaranteed rights and freedoms -- the right to bear arms, the right to a trial by jury, and so on. They're just not keeping track of which rights and freedoms are covered by the First Amendment as opposed to other provisions. It's hardly fair for members of a profession that thinks 1 in a 1,000 is 1 percent (or maybe 0.01 percent), and that renders 72% as 28% when summarizing a document right in front of their face, to get all indignant about this public "ignorance". ]

Posted by Mark Liberman at March 1, 2006 08:27 PM