OK, all you spinmeisters out there, listen up. This is a small trick, but it's a good trick, and there are plenty more where this came from.
Suppose that your thing is X-ology, and you want to emphasize how ignorant people are about X, in order to publicize your efforts to educate them. So you do a survey where you ask a simple and important question that is actually tricky and confusing: say, what are the freedoms guaranteed by the First Amendment to the U.S. Constitution? Or what are the capitals of the states in the Pacific time zone? Or what are the features of a passive sentence in English? You ask a thousand people, and the results come out something like this:
| Number of correct answers |
Number of respondents |
Percentage | Cumulative percentage |
Reverse cumulative percentage |
0 |
274 |
27% |
27% |
100% |
1 |
436 |
44% |
71% |
73% |
2 |
197 |
20% |
91% |
29% |
3 |
70 |
7% |
98% |
9% |
4 |
20 |
2% |
100% |
2% |
5 |
1 |
.1% |
100% |
.1% |
Of course, we're talking about the recent First Amendment survey, and the percentages in the third column are the numbers reported in the press materials provided by the McCormick Tribune Freedom Museum. (I created the other numbers to correspond to those percentages..) Now, how to present this?
73% could name at least one freedom, but 73% sounds like a really good number for a poll or survey. Almost 3/4 of the people surveyed "got it". Headline: Only 73% Can Name 1st Amendment Freedom. No, that's not going to work. What to do?
There are plenty of low percentages in the table, but we need a "good bad number", so to speak. You could say that only 1 in 1,000 could name all five -- but Americans don't expect perfection on tests, and such an extreme result risks focusing attention on the weirdness of the question. No, we need a number around 30% to symbolize the pitiful state of Americans' knowledge. (When W's approval rating reached 34% in some recent polls, we saw headlines like "poll-axed" and "all-time low".)
However, we also need a statistic that has a catchy and persuasive description. We could say that only 9% got three or more answers right; but 9% is suspiciously low, and anyhow, three or more seems to be an arbitrary choice. The only number that doesn't sound arbitrary is one -- but we already saw that 73% got one or more answers right. The 29% that got two or more right is a good percentage, but again, two or more seems like an arbitrary and therefore meaningless threshold. So to frame the result in a way that makes it seem natural and meaningful, how about "more than one"?
Bingo: the first sentence of the press release reads:
A new McCormick Tribune Freedom Museum survey finds that only about one in four Americans (28 percent) are able to name more than one of the five fundamental freedoms granted to them by the First Amendment to the U.S. Constitution.
[Never mind the 28%/29% difference -- there's a round-off issue in here somewhere.]
And here's the cool part. It's easy to confuse "more than one" with "one or more"; and "one or more" is a lot commoner:
| Yahoo | MSN | LDC News | ||
| "one or more of the" | 48.4M |
23.7M |
4,987,631 |
1,223 |
| "more than one of the" | 1.49M |
0.964M |
222,649 |
166 |
| ratio | 32.5 |
24.6 |
22.4 |
7.37 |
So you can hope that even some excellent journalists, like Peabody-award winning Robin Young of NPR's Here and Now, will report your results like this:
... a new survey by the soon to open McCormick Tribune Freedom Museum in Chicago found that only twenty eight per cent of those asked were able to name *one* of the freedoms, yet fifty two percent could name at least *two* of the members of the Simpsons family.
Even if the press repeats your phrase "more than one" correctly, a large fraction of the public will make the same mistake that Robin Young did.
Of course, the museum's real genius here was to compare the first amendment poll results with the results of a question about members of the Simpsons family. But some careful spinning was still needed! The results stacked up as follows:
| Number correct |
Freedoms | Simpsons | Reverse cumulative |
Freedom | Simpsons |
| 0 of 5 | 27% |
35% |
|||
| 1 of 5 | 44% |
13% |
1 or more | 73% |
65% |
| 2 of 5 | 20% |
9% |
2 or more | 29% |
52% |
| 3 of 5 | 7% |
9% |
3 or more | 9% |
43% |
| 4 of 5 | 2% |
12% |
4 or more | 2% |
34% |
| 5 of 5 | .1% |
22% |
all 5 | .1% |
22% |
Some possible headlines that we didn't see:
Don't have a cow, man: 30% more draw a blank on the Simpsons than on the Constitution [35% vs. 27%]
Survey: 73% can name a First Amendment freedom / Only 65% can name a Simpson
Some of the other results would also have lent themselves to reverse-spin headlines. In unprompted free recall, 70% of respondents came up with "freedom of speech" as a right guaranteed by the first amendment. The best-remembered member of the Simpsons was Bart, who was named by 61% of respondents. But we didn't see this headline either:
Freedom of speech: more famous than Bart Simpson!
And the most shocking statistics in the report, in my opinion: only 51% identified Homer, and only 43% identified Marge. Barely half of Americans can remember that Homer is a Simpson?!? Fewer than half can remember Marge?!? I mean, talk about burying the lede. An emergency educational initiative in Simpsonology is clearly required.
[All kidding aside, I'm solidly in favor of the McCormick Tribune Freedom Museum's initiative, to the extent that I understand it. In fact, since we Americans have such a short, inspired and well-written constitution, and since I think that memorization of canonical texts is a Good Thing, I'd be in favor of encouraging everyone to memorize all 7600 words of the constitution and its amendments. But the museum's presentation of its poll was a classic example of the rhetoric of public relations -- not the most dishonest one ever seen, but not overly scrupulous either -- and the press swallowed it hook line and sinker. Or maybe the press was a willing collaborator in spinning the issue for the public, I'm not sure.]
Posted by Mark Liberman at March 3, 2006 08:24 AM