A search engine is a tool, no better or no worse than any other tool, an axe, a shovel or anything. A search engine is as good or as bad as the man using it.
Or was it "a gun is a tool"? My memory's shaky on that one, but whatever it was I know for sure that Alan Ladd said it to Jean Arthur in Shane. And with no waiting period and everybody packing, it's a good idea to keep your head beneath the parapet when the hit counts start to fly.
Take the exchange that recently surfaced on the letters pages of The American Prospect. It began with a sentence in Todd Gitlin's July 5 review of Eric Boehlert's book Lapdogs: How the Press Rolled Over for Bush. In the course of describing how media reporting is skewed, Gitlin reported Boehlert as saying:
Outside The Boston Globe. . . the total number of media accounts that mentioned both [Bush's National Guard] absenteeism and Texas pol Ben Barnes' acknowledgment that he tried to sneak young Bush into the Guard: two. The number of accounts of the phony charge that Al Gore claimed to have invented the Internet: more than 4,800.
No way, said Alan Abramowitz, the Alben W. Barkley Professor of political science at Emory, who was impelled to write a letter to the editor to say that his own researches showed that the Gitlin/Boehlert claim was clearly wrong:
A Lexis-Nexis search reveals only 19 mentions of the "Gore-invented-the-Internet" charge in major American newspapers between January 1, 2000, and Election Day. Moreover, the point of several of these articles was that Gore had never made such a claim but that he had been a strong supporter of the development of the Internet. . . Gitlin's (and Boehlert's) claim that the media frequently and uncritically reported this accusation, like the accusation itself, appears to be greatly exaggerated.
Needless to say, that's an wildly and patently incorrect result. In a response to Abramowitz's letter, Gitlin replied:
My source for the "more-than-4,800" claim was Boehlert's Lapdogs (p. 160). Maybe I should have checked earlier. Strangely, when I did so just now, Lexis-Nexis turned up neither 4,800-plus entries, nor the 19 that Professor Abramowitz found, but 445.Actually, Gitlin's result isn't inconsistent with Boehlert's claim. Lexis-Nexis major papers includes only a small -- if influential -- portion of the American press (and around half the papers in the database are foreign ones). When you search severally in each of Nexis's four regional US News files, you come up with 973 hits for the period in question -- add the previous nine months (since the story hardly began at the beginning of 2000) and you come up with over 2000. In a more careful search at the Prospect's Tapped blog, Paul Waldman notes that the story gets a total of 4349 hits on Nexis's Allnews database over the 18 months prior to the 2000 election. And even that database doesn't include most local TV or AM radio talk shows, not to mention major blogs and Internet sites (the string "gore 'invented the internet'" gets 90 hits on townhall.com alone). In short, Boehler's "4800 media stories" is unquestionably a considerable understatement . And even if a good number of those stories involve refutations of the charge that Gore claimed to have invented the internet, the very need for them suggests that the damage was done. As Gitlin observed:
. . . lest we succumb to the fog of dueling Nexises, I submit that we recall Karl Rove's principle: When you're explaining, you're losing. Insofar as newspapers were saying that Gore was defending himself against a deceitful charge, he sounded, to some undecided population of voters, like an evasive braggart. That was bad enough.
Of course counts of media stories are only a rough indication of how widely diffused a story is, but even if we restrict ourselves to print, the contrast between Abramowitz's 19 stories and the actual figure of several thousand is pretty striking. But then anybody who lived through this period knows without having to check that the story was all over the place. Which leads me to ask, How could Abramowitz possibly have believed the number his search returned?
there's no way to tell exactly how Abramowitz managed to come up with the figure of 19. Waldman suggests that he must have searched only on the specific string "Gore invented the internet" in the Nexis Major Papers files, which is what I assumed too, but it turns out that even that very sentence gets 24 hits in that database for the first 9 months of 2000. Did he maybe use some other search string but leave the pulldown menu at the default "headline, Lead Paragraph(s), Terms" value rather than doing a full-text search? Did he enter the dates wrong? Did he screw up the search syntax, or enter a string that presumed that Nexis was using Google's search syntax, as a lot of people do these days?
Who knows -- there are an awful lot of ways to get this stuff wrong, and man and boy I've personally explored every one of them -- after 20 years of using Nexis, Dialog, and other news databases, I'm still doing searches and getting results that are implausible on their face, so that I have to give my processor a whack and try again with some other search terms.
From Abramowitz's failure to do just this, it's clear both that he's a newbie to Nexis and that he has an inordinate, or at least unwarranted confidence in his ability to find his way around with the technology. If that's especially odd in his case, it's only because he does quantitative research in voter behavior, so that you'd figure he'd have to have had the same experience using stat packages: getting some completely implausible result and having to go back and correct some setting -- or more likely, at this stage of his career, making a caustic remark and sending his graduate assistant to run the data again. I mean, who hasn't had this happen to him?Abramowitz isn't alone in this -- people are always trotting out search results in the service of this or that point that are absurd on their face, even if not a whole lot of them are the holders of named chairs in quantitative disciplines. Blame the googlization of the Web, which gives us all the illusion of search-engine expertise. Or blame people's tendency to believe quantitative results and claims even when they fly in the face of plausibility.
Whatever the cause, you can put it down as one more example that makes the case for universal instruction in information literacy -- even if it probably comes too late for the tenured classes. Posted by Geoff Nunberg at September 22, 2006 04:31 PM