As you've probably read by now, some grad students at MIT ginned up an "Automatic CS Paper Generator", using a "hand-written context-free grammar to form all elements of the papers". As the authors (Jeremy Stribling, Max Krohn and Dan Aguayo) explain,
One useful purpose for such a program is to auto-generate submissions to "fake" conferences; that is, conferences with no quality standards, which exist only to make money. A prime example, which you may recognize from spam in your inbox, is SCI/IIIS and its dozens of co-located conferences (for example, check out the gibberish on the WMSCI 2005 website). Using SCIgen to generate submissions for conferences like this gives us pleasure to no end. In fact, one of our papers was accepted to SCI 2005!
This exploit has made it into news outlets via Reuters, and no doubt soon other services. The Reuters story suggests that the students were "surprised" to have a paper accepted at SCI2005 ("The 9th World Multi-Conference on Systemics, Cybernetics and Informatics"). For my part, I was surprised that only one of their papers was accepted.
Like everyone else on whatever lists the SCI/IIS spammers use, I regularly find my spam traps clogged with hundreds of transparently fraudulent invitations to submit abstracts to conferences to be held in somewhat attractive places, contingent of course on my willingness to pay a few hundred dollars to register. I've always assumed that these folks would be happy to take my $390 and accept a compilation of old shopping lists. Perhaps Stribling, Krohn and Aguayo submitted two papers under the same authors' names? That would explain why only one of their papers was accepted, since the other would bring in no additional revenue. And I really like the letter that they got when they asked to see the reviews on the basis of which the paper had been rejected...
Reuters makes the connection to Alan Sokal's Social Text hoax:
The prank recalled a 1996 hoax in which New York University physicist Alan Sokal succeeded in getting an entire paper with a mix of truths, falsehoods, non sequiturs and otherwise meaningless mumbo-jumbo published in the quarterly journal Social Text, published by Duke University Press.
But I wasn't aware that Social Text (which is still publishing) was (or is) generally regarded as a "fake journal" in the fields that it served, in the sense that SCI/IIS has always been widely understood to be a "fake conference". And the editors' motivation for publishing Sokal's parody was not financial -- the hope of making money by inducing him to register for a phony conference -- but rather ideological --- his submission (as he has explained it) "came from a 'conveniently credentialed ally'' (as Social Text co-editor Bruce Robbins later candidly admitted), flattered the editors' ideological preconceptions, and attacked their 'enemies'."
Reuters quotes the infamous Nagib Callaos offering some feeble excuses. (I was shocked to learn that Callaos actually exists, since I had always assumed that he was one of those people like Serenity Q. Oxbow from whom I often receive other attractive offers by email.)
Nagib Callaos, a conference organizer, said the paper was one of a small number accepted on a "non-reviewed" basis -- meaning that reviewers had not yet given their feedback by the acceptance deadline.
"We thought that it might be unfair to refuse a paper that was not refused by any of its three selected reviewers," Callaos wrote in an e-mail. "The author of a non-reviewed paper has complete responsibility of the content of their paper."
However, Callaos said conference organizers were reviewing their acceptance procedures in light of the hoax.
Asked whether he would disinvite the MIT students, Callos replied, "Bogus papers should not be included in the conference program."
It's normal (and not necessarily stupid) to see a random recombination of problems, models, algorithms and thematic settings in the papers accepted at serious conferences. There's a long tradition of poking fun at this process, exemplified by the call for papers for the " 1st Workshop on Unnatural Language Processing". But at a serious scientific or engineering conference, even the worst of these memetic recombinations makes sense at a certain level, and sometimes the process creates a new conceptual species that even deserves to prosper. The results of the MIT students' interactions with Nagib Callaos provide evidence that the SCI/IIS system, in contrast, really is the scam that it appears to be.
[Update 4/16/2004: I had completely forgotten an earlier prank originating at MIT, only slightly less spectacular, that also demonstrated the fraudulent character of the SCI/IIIS process: Prof. Justin Zobel submitted three nonsensical papers, all of which were accepted. One was created by selecting alternating sentences from two existing papers; another explained that "we have implemented a[n] ... algorithm ... the computational cost is high, and the method does not work at all. We believe that this method is not capable of being improved", and included discussions of the consequences of the authors' inebriation and their decision to "invent more promising results, an approach that we report on in the last line of the table"; the third was '[a] surreal collection of remarks about information retrieval, by myself and a colleague. Aside from the first page, many of the paragraphs make no sense, and much of the content consists of jokes and nonsequiturs".
All three were accepted. I learned from Zobel that my theory about why the second artificially-created paper was rejected must be false:
The organisers of the conference invite the contributors to pay the registration fee, with a separate fee for each accepted paper, and state it is not necessary to attend so long as the publication fee is paid -- a highly unusual practice. I have repeatedly requested the referees' reports, but there has been no response. The organisers have however rapidly responded to queries about the financial arrangements.
]
[And now, an even better submission has been revealed! I'm seriously tempted to borrow it and submit a copy myself!!]
[Update: more on the work of Prof. Callaos himself here]
Posted by Mark Liberman at April 15, 2005 07:00 AM