Mean media metaphor of the month: Jack Shafer's judgment on Judge Richard Posner's essay "Bad News":
Maybe Posner should stop composing his essays with a paint roller and switch to a Sanford Uniball Micro.
Courtesy aside, Shafer's criticisms are reasonable ones: Posner's piece links broad-brush conventional wisdom about lowered barriers to entry with mostly-unsupported assertions about increased sensationalism and polarization. However, Shafer ends his critique with an astonishing and gratuitous piece of quantitative idiocy, which significantly undermines his whole "let's draw rational conclusions from documented facts" stance.
First, let's set the stage. Here's Judge Posner's conclusion:
Thus the increase in competition in the news market that has been brought about by lower costs of communication (in the broadest sense) has resulted in more variety, more polarization, more sensationalism, more healthy skepticism and, in sum, a better matching of supply to demand. But increased competition has not produced a public more oriented toward public issues, more motivated and competent to engage in genuine self-government, because these are not the goods that most people are seeking from the news media. They are seeking entertainment, confirmation, reinforcement, emotional satisfaction; and what consumers want, a competitive market supplies, no more, no less. Journalists express dismay that bottom-line pressures are reducing the quality of news coverage. What this actually means is that when competition is intense, providers of a service are forced to give the consumer what he or she wants, not what they, as proud professionals, think the consumer should want, or more bluntly, what they want.
This is a plausible story, but as Shafer observes
The authentic media maven understands that newspapers have been "dying" since the advent of radio in the 1920s, with the number of titles dwindling steadily with the rise of every new media (television, cable, the Web) and their share of the audience shrinking.
(A linguistic aside: note that media, like data, is now firmly singular in general usage...)
Shafer persuaded me that Posner's essay combined fuzzy thinking with factual carelessness. But Shafer's take-down makes an astonishing claim in its conclusion, a quantitative assertion that a few seconds of common-sense reasoning will show to be several orders of magnitude off.
Posner reveals the sort of rigor he applied to this piece of hackwork in his conclusion, where he notes that a survey by the National Opinion Research Center recorded the public's confidence in the press declining from 85 percent in 1973 to 59 percent in 2002 "with most of the decline occurring since 1991." He writes:
So it seems there are special factors eroding trust in the news industry. One is that the blogs have exposed errors by the mainstream media that might otherwise have gone undiscovered or received less publicity. Another is that competition by the blogs, as well as by the other new media, has pushed the established media to get their stories out faster, which has placed pressure on them to cut corners.
How could blogs have played any role in eroding public trust by 2002 when almost nobody in the mainstream had heard of them? The press loves to seize on new trends, especially techno-trends, but the word "blogs" doesn't appear in a Nexis search of all U.S. newspaper and wire stories until 2000, when it was mentioned in 22 stories. In 2001, the word appeared in 67 stories. In 2002, the concluding year of the survey cited by Posner, it appeared in 359 stories. That's too few by a factor of about 100,000 to have had an impact on the public's view of the press.
Does Shafer really mean that for blogs to have an impact on the public's view of the press, the word blogs would have to appear in about 359*100,000 = 35.9 million newspaper and wire stories within a calendar year?
The version of Lexis-Nexis that I have access to won't give me a response if the size of the set returned is greater than 1,000. So as a proxy, I tried single-month searches, with the results as follows. All searches were done on Lexis-Nexis Academic, in the category of "General News", source "Major Papers", search terms "blogs" in "Full Text".
March |
April |
May |
June |
Sum March-June |
Full year |
Shafer's counts |
|
| 2001 | 1 |
10 |
7 |
4 |
22 |
50 |
67 |
| 2002 | 8 |
19 |
15 |
24 |
66 |
205 |
359 |
| 2003 | 78 |
57 |
58 |
66 |
259 |
883 |
|
| 2004 | 119 |
129 |
201 |
170 |
619 |
? |
|
| 2005 | 524 |
539 |
619 |
673 |
2,355 |
(I guess that Shafer has access to a "media pro" version of Lexis/Nexis that indexes a somewhat larger set of sources -- but his counts are within a factor of 2 of mine, and the exaggeration we're talking about involves a factor of 1,000 or so.
Shafer's basic point against Posner is obviously correct. To attribute to the influence of blogs something that happened over the period 1991-2002 is preposterous. But in his excess of indignation, Shafer does something that Posner doesn't -- he pulls a specific number out of nowhere that is roughly three orders of magnitude too large. Here's a reprise of this bit of froth:
In 2002, the concluding year of the survey cited by Posner, it appeared in 359 stories. That's too few by a factor of about 100,000 to have had an impact on the public's view of the press.
Again, 359*100,000 = 35.9 million. My Lexis-Nexis count for blogs in the March-June period of 2002 is 66, and for the same period of 2005 it's 2,355. That's an increase by a factor of 35.7, which is way less than 100,000. It's 2,801 times less, to be precise.
One way to read this is that blogs are not yet having an impact on the public's view of the press, and won't do so until there are 36 million newspaper and newswire stories a year that include the wordform blogs. But surely this is not what Shafer means. If that's the criterion, there can't be many developments that actually do have any impact on the public's view of anything. I mean, it might have seemed like there were 36 million stories about Michael Jackson last year, but there weren't -- checking Lexis-Nexis for "Michael Jackson" in June of 2004 turns up a mere 319 stories...
No, I think Shafer just pulled a big number out of the air. It wasn't a number based on careful sociological studies of the impact of media on public opinion, and it wasn't even a number that Shafer bothered to evaluate for common-sense plausibility. It was just a big-ass number. So if I were Richard Posner, I'd offer to stop writing my essays with a paint roller if Jack Shafer agrees to stop doing arithmetic with his rear end.
[Update: a couple of readers have suggested that maybe Shafer meant that the number of stories in 2002 was too low by an additive increment of about 100,000, not a (multiplicative) factor of 100,000 -- 359+100,000, not 359*100,00. Frankly, I don't see any evidence that he gave the matter enough thought to distinguish those two cases. In any event, this would be contrary to the ordinary-language meaning of the word factor, e.g. "A quantity by which a stated quantity is multiplied or divided, so as to indicate an increase or decrease in a measurement", as the American Heritage Dictionary puts it. And if you're beating up on someone for sloppy thinking, careless writing and poor factual support, and you want to avoid charges of hypocrisy, this is not a good mistake to make.
Even an additive increment of 100,000 is probably hyperbole, since extrapolation from my 4-month Lexis-Nexis counts for 2005 suggests fewer than 10,000 stories in major newspapers containing the word "blogs" this year.]
Posted by Mark Liberman at August 3, 2005 07:15 AM