Linguists face the same types of problems. We can gather data and experiment with it in our laboratories, but often we can't be very sure that the results will reflect what happens under natural conditons. For example, much of the research on deception is based on experiments in which people (usually undergraduate students) are asked alternatively to lie and to tell the truth for a stimulus given to subjects who are to identify deceptive statements. Not surprisingly, the subjects aren't very good lie detectors. It would seem reasonable for researchers to produce a stimulus based on naturally occuring data taken from sources such as police interrogations or court trials, that eventually was proved to be untruthful. No pretending here. From this a stimulus could be administered to a sample population (preferably not just undergraduate students) to see if they could detect deception that is found in the real world.
Linguists who study naturally occurring language face serious handicaps when their results are compared with the much neater findings produced by controlled experiments. One the problems that forensic linguists (and other expert witnesses) face when judges consider whether or not to permit them to give testimony at US trials grows out of the standard for admissibility established in 1993. The US Supreme Court new rules in Daubert v. Merrell Dow Pharmaceuticals, Inc., now commonly referred to as Daubert. Before that decision, the courts relied on the 1923 decision of Fry v. United States, which specified that qualification of experts was to be based on what is called the general acceptance test. This test required that proposed witnesses be qualified experts in their field, that their testimony should be relevant to an issue in dispute in the case, and that the the scientific theory or technique should be generally accepted by the relevant scientific community.
In replacing the Fry general acceptance test with the Daubert reliability assessment, judges were expected to make an independent judicial evaluation of the reliability of the proposed testimony. The Court suggested four factors to assist judges as they made their inquiries:
For a few years there was considerable confusion about the scope of Daubert. Did it apply only to scientific testing and anaysis, or did it apply to all forms of expert witness testimony, including such groups as engineers, auto mechanics, and therapists? In 1999 the US Supreme Court ruled in Kumho Tire v. Carmichael that the Daubert reliabililty test applied to all forms of expert witness testimony, even to palmists and astrologists.
The Daubert factors can be met by many kinds of linguistic analysis. The scientific nature and credentials of our field meet factor 1. Publications and peer reviews help with factor 2. Factor 4 is not a serious problem because linguists get research funding from the National Science Foundation, a good idicator. NSF is clearly regarded as scientific and most universities have linguistics departments or linguists working in allied departments these days. But factor 3 poses a serious problem for one type of linguistic expert witnesses. It favors experimental research and many social scientists engage in more ethnographic or qualitative studies.
In contrast, the hard sciences create a body of knowledge resulting from the performance of replicated controlled experiments in laboratory settings. Many linguists also do this, but Daubert's factor 3 doesn't quite fit linguists who gather and use naturally occurring language data rather than data acquired from controlled experiments. That type of data is relatively impossible to replicate or convert into experimental conditions. In the forensic context, such data usually takes the form of tape-recorded conversations used as evidence in criminal trials.
I've worked on many such cases, but one that was the most frustrating was a case in which there were two tape-recorded conversations in evidence. The prosecution claimed that one of these was a genuine conversation (the target did not know he was being recorded) but that the other one was a fake, staged conversation between the same two men. My task was to address the question of whether or not the allegedly faked conversation was actually staged.
Even linguists who study conversation as their specialty haven't given this topic much thought. I couldn't find literature on that subject so I couldn't tell the court that my findings had been subjected to previous peer reviews and publication (factor 2). Nor could I say that there was any known or potential error rate for this kind of analysis (factor 3). I had compared the known real conversation with the alleged staged one and concluded that in terms of phonology, morphology, syntax, discourse structure, speech errors, pause lengths, and other things, there were no substantive differences between the two tapes. I could not testify that the intentions of the speakers were, indeed, to produce either a real or staged conversation, for that would go beyond the scope of linguistics (or any other field, as far as I know). But I could say that there was no linguistic evidence that would mark the two conversations as different. This didn't satisfy the judge who, for all I know, may have been predisposed to disallow my testimony anyway. She stuck rigidly with the Daubert factors and rejected me as an expert witness.
So when I read Cheyney and Seyfarth's complaint about the lack of research on chimpanzees in a wild, natural context, I couldn't help thinking that it parallels the Daubert situation facing expert witnesses these days. Sometimes experimental research, for all its virtues, can't tell us all that we really need to know.
Posted by Roger Shuy at October 10, 2007 12:45 PM