It looks like BBC News scored a big scoop on December 14: "Skype users to get lie detectors", 12/14/2006:
Callers using internet phone system Skype who might be tempted to tell a few porkies should beware - the user on the other end may have a lie detector.
Skype is to offer the KishKish Lie Detector, which is made by BATM, as an add-on for customers.
It analyses audio streams over a Skype call in real time and illustrates the stress levels of the other person.
But experienced Language Log readers will be able to guess what's coming -- could this be another credulous BBC reproduction of a press release?
BBC news seems to have been the first to carry this story -- but more than a month later, no other major news outlet seems to have gotten onto it. Google News has 28 hits this morning for {Skype lie detector}, but they're all from outfits like TG Daily, Israel 21C and the Sofia Echo:
Wolfgang Gruener, "Lie detector for Skype watches over voice communications", TG Daily, 12/21/2006;
Nicky Blackburn, "Forget the tall tales - Israeli lie detector keeps Skype users honest", Israel 21C, 12/31/2006;
"Bulgarians, Israelis Develop Lie Detector for Internet Calls", Sofia Echo, 1/8/2007.
Although Wired didn't carry this as a news story, Ryan Singel at Wired Blogs had a sensible response -- he tried the plug-in out a bit, and asked for help in testing it further ("Help 27B Test Skype Lie Detector", Wired Blogs, 12/18/2006):
How well does it work? I don't know. A test call to "Clinton Denial" seems to show that Clinton was lying when he said he never asked any one to lie for him. Like I needed software to know that. But another call to the Skype Test Call seems to show that the woman thanking me for using Skype is lying. Does she really not care.
So, help me out folks. Make my internet phone jingle (my handle is Ryan Singel) and I'll ask you a series of questions after we chat for 15 seconds or so so a baseline can be set. I'll ask a 5 to 10 questions and you should lie or not lie accordingly.
I look forward to the report of the results -- but the information on the KishKish Lie Detector™ site indicates that this is an implementation of the idea of microtremor-based "voice stress analysis"; and thus a test of its effectiveness is likely to be a replication of a long series of (mostly negative) results that began several decades ago. KishKish tells us that
Voice Stress Analysis (VSA) is a type of lie detector which measures stress in a person's voice. The use of Voice Stress Analysis (VSA) as a lie detector became popular in the late 1970s and 80s. In the 90s the first Computerized VSA (CVSA) systems came to out to the market. The CVSAT is now the truth verification device of choice in the law enforcement community as the number of law enforcement agencies utilizing the CVSAT continues to grow dramatically, proving the viability of the system for twenty-first century crime detection. The CVSAT is also being utilized by the US Military in the global war on terrorism.
Now KishKish Lie detector offers you a tool to detect the stress level of the person you communicate with over Skype. With the use of KishKish Lie detector you can monitor in real-time the stress level of the person you talked with. This allows you to gage the level of stress and modify your questions in real time. You could also use our KishKish SAM VSA that allows you to record the call and analyze the stress level off-line.
I discussed "voice stress analysis" in a blog post a couple of years ago ("Analyzing voice stress", 7/2/2004), and I won't go over the same ground in detail again here. Some highlights:
I've been amazed by this work for almost three decades. What amazes me is that research (of a sort) and commerce (at a low level) and law-enforcement applications (here and there) keep on keepin' on, decade after decade, in the absence of any algorithmically well defined, reproducible effect that an ordinary working speech researcher like me can go to the lab, implement and test.
Well, these days there's no need to go to the lab for this stuff -- you just write and run some programs on your laptop. But that makes the whole thing all the more amazing, because after 50 years, it's still not clear what those programs should do. I'm not complaining that it's unclear whether the methods work -- that's true too, but the real scandal is that it's still unclear what the methods are supposed to be.
Specifically, the laryngeal microtremors that these techniques depend on haven't ever been shown clearly to exist, as far as I know. No one has ever shown that if these microtremors exist, it's possible to measure them in the pitch of the voice, in a way that separates them from all the other phenomena that modulate the pitch at similar rates. And that's before we get to the question of how such undefined measurements might be related to truth-telling. Or not.
In the absence of any clear recipe for replication, all that people can do is test the commercial devices, just as Ryan Singel at Wired Blogs set out to do. When the tests are carefully done, they generally appear to give negative results -- for example, Troy E. Brown et al., "Ability of the Vericator™ to Detect Smugglers at a Mock Security Checkpoint", DoDPI03-R-002.
As I observed,
How can I make you see how amazing this is? Suppose that in 1957 some physiologist had hypothesized that cancer cells have different membrane potentials from normal cells -- well, not different potentials, exactly, but a sort of a different mix of modulation frequencies in the variation of electrical potentials between the inside of the cell and the outside. And further suppose that some engineer cooked up a proprietary circuit to measure and display these alleged variations in "cellular stress" (to the eyes of a trained cellular stress expert, of course), and thereby to diagnose cancer, and started selling such devices to hospitals, and selling training courses in how to use them. And suppose that now, almost half a century later, there is still no documented, well-defined procedure for ordinary biomedical researchers to use to measure and quantify these alleged cell-membrane "tremors" -- but companies are still making and selling devices using proprietary methods for diagnosing cancer by detecting "cellular stress" -- computer systems now, of course -- while well-intentioned hospital administrators and doctors are occasionally organizing little tests of the effectiveness of these devices. These tests sometimes work and sometimes don't, partly because the cellular stress displays need to be interpreted by trained experts, who are typically participating in a diagnostic team or at least given access to lots of other information about the patients being diagnosed.
This couldn't happen. If someone tried to sell cancer-detection devices on this basis, they'd get put in jail.
But first, BBC News would publish a story about each new "cellular stress analysis" product, without any indication that there might be any history of concern about what it does and whether it works.
Let me emphasize that I'm open to being persuaded on all these points. As I wrote back in June of 2004:
I'm not prejudiced against the "microtremor" theory -- I'd love to have another measurement dimension for speech analysis. I'm not prejudiced against "lie detector" technology -- if there's a way to get some useful information by such techniques, I'm for it. I'm not even opposed to using the pretense that such technology exists to scare people into not lying, which seems to me to be its main application these days. But when a theory about quantitative measurements of frequency-domain effects in speech has been around for half a century, and no one has ever published an equation, an algorithm or a piece of code for making these measurements, and willing and competent speech researchers (like me) can't create reliable methods for making such measurements from the descriptions we find in the literature... something is wrong.
If someone at KishKish will describe the algorithm to me -- I wouldn't say no to some Matlab code, but I'd be happy with a clear description of the analysis algorithm -- I'd love to try it out, and I'll praise it to the skies if it works, even at the level of giving a reliable measurements of something you could call a "laryngeal microtremor".
And if someone at BBC News will explain to me how what they did on December 14 wasn't just parroting a press release without any attempt to analyze its credibility, in a way that suggests that their standards are lower than those of every other major news organization in the world, I'll apologize for calling them a disgrace to journalism.
[Update: checking Yahoo News shows that Agence France Presse ran this story on January 8 in an even more credulous form (Sophie Nicholson, "Nothing but the truth with Israeli Internet lie detector"). AFP interviewed Zvi Marom, the CEO of BATM, the company responsible for the KishKish product, and Paul Amery, director of Skype Development; but again, there was no attempt to look into the background of this non-innovation. So to be fair, I guess that I have to admit that the BBC is not uniquely untrustworthy as a source of information about things like this, though they were still first out of the misinformation gate by more than three weeks. I didn't learn about the AFP story from Google News, by the way, because AFP sued Google last year to prevent their stories from being indexed: "France defies Google", 3/19/2005.
Yahoo News also informed me that Nate Anderson at Ars Technica found that the KishKish accused his mother of lying about Christmas dinner: "Skype stress detector calls my mother a liar, 12/22/2006. His sensible response: "This might not be the sort of program you want to base major life decisions upon".
Isn't Skype at some legal risk for disrupting friendships and family relationships, if trusting users actually take this stuff seriously? ]
[Update -- Michael Katsevman writes:
Love the blog, loyal reader, etc. :)
I find it very amusing that the source of the company's name KishKish seems to me to come from an Aramaic proverb that's fairly common in Israel: "Istra balagina kish kish karya" which translates to "A coin in a clay vessel makes the sound 'kish kish'". In meaning, it is roughly equivalent to "a storm in a teacup" and "much ado about nothing".
If that's not delightfully appropriate, my sense of irony must be broken.
] Posted by Mark Liberman at January 17, 2007 07:15 AM