Last Thursday, just before Geoff Pullum checked up on Mark Bauerlein and Geoff Nunberg warned about biased studies of bias, I got an interesting email in reference to my 10/31 post on the rhetoric of citation. I had described the mathematical model used by Groseclose and Milyo in their widely-referenced study, "A Measure of Media Bias", and complained that "as I read G&M's model, it says that we derive maximum 'utility' from citing the most extreme sources whose political polarity is the same as ours". I used a (small, unscientific) sample of blog entries citing Karl Marx to suggest that this is not always an accurate way to predict who cites whom: forcing everyone artificially onto a one-dimensional political spectrum, I found that citations of Marx were roughly twice as likely to be from right-wing blogs as left-wing ones.
The note was from a writer at The Economist, and began like this:
Just discovered the wonderful Language Log blog. I thought that a lot of what you said in "Marx: Red or Blue" rings true--people do very often cite those they disagree with. But the study you mention deals only with Congressmen/women and the media, and the phenomenon being observed is a very specific one: reference to think-tanks. These are of course not representative of the speech community as a whole.
Members of Congress, it seems to me without doing any research, very rarely cite opposing think-tanks. They often try to sneak by with no attribution at all - e.g. a Republican criticizing Kerry's health-care plan may say "A recent study found that it would cost twice as much as he says..." when the data come from Heritage or the AEI. If they did cite, they would surely cite their own guys--liberals would use Brookings, Republicans Cato, AEI or Heritage. It's very hard for me to imagine Ted Kennedy citing AEI or Trent Lott.
As for media outlets (I'm a reporter), you're right that opinion writers (which bloggers more closely resemble) will cite their opponents to dismiss them. But "straight" reporters will cite evidence they believe to be credible. And what they believe to be credible will (it seems likely) subtly be affected by their own viewpoint.
A conservative friend points out that the oft-cited Brookings is rarely called "a liberal think-tank" by reporters--because they don't think it's liberal, though conservatives do. AEI and Heritage are, by contrast, almost always flagged with a cautionary label like "conservative". A quick-and-dirty google news search confirms: of 3350 pages mentioning Brookings, only 322 also contain the word "liberal" (under 10%). The comparative figures for Heritage are 1420 mentions, of which 810 include the word "conservative" (57%). For the AEI the numbers are 1350 and 509 (38%).
So, in sum, I think the Groseclose and Milyo methodology you criticize might actually hold up fairly well in the specific instance of congressmen, reporters and think-tank citations.
I responded (in part)
I admit that I don't know what the distribution of affinities was between reporters and sources in the cases they covered [in the data for the G&M study]. My point was just that it's far from obvious that the mathematical form of their model corresponds to the psychological and rhetorical reality of the citation process.
and sent him links to Geoff Nunberg's earlier writings on media bias. He responded:
On reflection, I'll simplify my response to your post: the citation habits of reporters are different from the citation habits of bloggers or academics, so I'm not sure your Technorati search invalidates G and M's methodology. Bloggers and academics argue, and explicitly cite opposing evidence to knock it down. Journalists, who if they're lucky get 1000 words to make a point (and TV gets far fewer), are more likely to cite a single "trustworthy" source, and I'd not be surprised if most of those aren't left-of-center.
I'll accept his expert opinion about journalistic constraints and their consequences for journalistic rhetoric -- though we still don't really know what G&M's data looked like, in this respect. For example, my point about the many citations of Geoff Pullum's Dan Brown critique by Catholic writers (here, scroll down to the bottom of the post) still stands, as an example that is consistent with Lane's picture of what journalists do, but which would tend by G&M's methodology to identify Geoff (inappropriately) as a Catholic writer.
My correspondent added:
thanks for the Nunberg links - am enjoying digesting with my breakfast
which echoes my own opinion that it's a special pleasure to be able to follow the back-and-forth on such issues through a linked set of magazine articles, blog entries and so on.
Posted by Mark Liberman at November 13, 2004 09:43 AM