Language Log

Fun quiz of the day

Fev at Headsup: The Blog is running a quiz on science journalism. He starts his post this way:

Hey, I know. Let's have a quiz! Here's a story, with heds, as it appeared today on the Web site of a major metropolitan daily. There's a brief quiz (true-false and multiple guess) at the end. Post your answers as comments or send 'em to the address at right.

Extra credit: One point for finding a copy of the abstract by first-edn deadline (let's say that's 9 p.m. Saturday in whatever time zone you're in). Two points for finding the article itself by that deadline.

I'll post answers when I'm done pounding nails into my skull.

Media erode men's self-esteem
Study links TV to risky behavior, aggression

Well, I'm not going to make Fev's extra-credit deadline, because I didn't read his email inviting me to participate until half an hour ago. But I'm going to take his quiz anyhow, since it's a sort of perverse comfort to find that science reporting is just as bad when it's not about speech or language. And at the end, I have a question for him -- and I invite the rest of you to offer your own answers in the comments to his post.

What had Fev pounding nails into his head is this section of a (widely reprinted AP wire) story:

In the past, research has understandably focused mostly on women, and the dangerous eating disorders that can stem from body-related emotional issues. And when looking at men, researchers asked the wrong questions, Schooler argues.

"Asking men about just weight or size misses the boat," Schooler, a research associate at Brown University, said in a telephone interview. What men are more concerned about, she says, are other "real-body" factors, like sweat, body hair and body odor.

In a study published last spring and recently featured in Seed magazine, Schooler, then at San Francisco State University, and a colleague looked at 184 male college students. The more media these young men "consumed" - especially music videos and prime-time TV - the worse they felt about those "real" aspects of their bodies, the researchers found.

Further, they found that such negative feelings impacted their sexual well-being, in some cases leading to more aggressive and risky sexual behavior.

The study in question was: Deborah Schooler & L. Monique Ward, "Average Joes: Men's Relationships With Media, Real Bodies, and Sexuality", Psychology of Men & Masculinity, 7(1) 27-41, January 2006.

1) This study measures "self-esteem" in men.
a) True
b) False

Dunno. "Participants completed several survey measures", including the "Body Esteem Scale" and the "Body Comfort/Body Modesty Measure". These questionnaires surely measure some aspects of self-esteem. On the other hand, the subjects were 184 students from "two undergraduate psychology classes at a large midwestern university", minus five who self-identified as "gay", which might be regarded as a less-than-representative sample of "men".

2) Consider the image in the lede. Did the study find that men who read lots of fitness and porn magazines have:
a) More positive body esteem?
b) Less positive body esteem?

Here's the table with the results of their multiple regression:

Neither reading Playboy and Penthouse (their proxy for porn magazines), nor reading Men's Health and Muscle and Fitness (their proxy for fitness magazines) had any statistically-significant effect on the "Own Body Comfort" index. All of the predictors they used were only able to account for 17% of the variance in this index. The most effective predictor was body mass index, unsurprisingly.

As for the two BES scales, reading porn mags had a significant positive connection with both; reading fitness magazines had a marginal positive connection with the "upper body strength" scale and no connection with the "physical condition" scale. All of the independent variables combines explained only 3% and 5% of the variance in those measures, respectively.

3) How many times would you estimate the words "aggression" or "aggressive" appear in the study?
a) None
b) 2-5
c) 6-10
d) 11 or more

We don't need no stinkin' estimates. By actual count, the number is zero.

4) Findings from this study can confidently be generalized to:
a) Men
b) Men aged 18 to 35
c) Straight male undergraduates in introductory and general psyc classes at Michigan State
d) None of the above; generalizing from convenience samples is inappropriate

I'd go with (c), with the caveat that the method of recruiting subjects (offering extra credit in psych class) might have created some bias. If I weren't such a positive and easy-going kind of person, I might go with option (d).

5) How would the researchers characterize the "real body comfort" measure used here?
a) Well established in a number of social science fields
b) New and not yet broadly established as valid
c) Well known in experimental psychology but new elsewhere
d) Hypothetical and used here for the first time

According to the article, their "body comfort" scale is based on a weighted combination of answers to certain questions taken from a recent study of "women's reproductive attitutes" (Merriwether, A., & Ward, L. M. (2002, August). Comfort in our skin: The impact of women's reproductive attitudes. Poster presented at the annual meeting of the American Psychological Association, Chicago). This seems to be the first time it has been used. That's not a bad thing, in my opinion -- but it didn't exactly shine, in this case, in terms of any discovered relationship to anything else. They'd probably have gotten more statistical juice out of asking about hobbies and recreational activities.

6) Among subjects in this study, comfort with "real body" factors predicts:
a) More sexual assertiveness and less sexual risk-taking
b) Less sexual assertiveness and less sexual risk-taking
c) Less sexual assertiveness and more sexual risk-taking
d) More sexual assertiveness and more sexual risk-taking

Here's the table.

The answer is "none of the above", as far as I can tell: "own body comfort" and "comfort with women's bodies" were both significant predictors sexual assertiveness, but neither was significantly predictive of sexual risk-taking. But note that all predictors combined accounted for only 16% of the variance in "sexual assertiveness" and for less than 2% of the variance in "risky sexual behavior". In a sample that size, you could probably predict 2% of the variance in risky sexual behavior from the subjects' astrological sign. (Or from a table of random numbers.)

7) The only significant demographic predictor of risky sexual behavior in this study is:
a) Religiosity
b) Being Asian
c) Own body comfort
d) Comfort with women's bodies

The answer is (b), being Asian. But it's not much of a predictor, since all the factors together accounted for only 2% of the variance.

8) The study found a causal relationship between negative feelings about the "real body" and aggressive sexual behavior.
a) True
b) False

False. The study asked about "sexual assertiveness", not aggression, but in any case, it found that greater "own body comfort" slightly predicted greater "sexual assertiveness".

9) The study found that men are more concerned about "real body" factors than about weight or size.
a) True
b) False

The answer is (b). The question is not addressed directly, but there (for example) participants estimated their "comfort with own real body" at an average of 5.41, whereas their "body esteem" factors were 3.43 for "upper body strength" and 3.46 for "physical condition".

10) The strongest predictor of whether men in the study were comfortable with their own bodies was body-mass index.
a) True
b) False

True -- see the table reproduced above.

Now I've got a question for Fev.

The science journalists whose work is typified by this story are

a) stupid
b) lazy
c) cynical and dishonest

This is not a rhetorical quiz-question -- I'm genuinely puzzled. It's clear that such people can read and write and think -- so why do they appear to misunderstand so completely? And I'm sure that you can't become a successful journalist without years of hard work -- so why don't they bother to take a few minutes to look up original articles and read what they say?

The only answer that I can come up with is (c) -- they think that the real story would not be nearly as interesting to their readers (and perhaps to their editors?) as the crap that they make up is.

Or am I misunderstanding something here?

[Update -- on reflection I need to add two more possible answers:

d) gullible
e) math phobic

In some cases, scientists (or the PR agents of their institutions) may promote research in very different terms from those in which they describe it in their formal publications (which in any case may not exist at the time of the promotion). And maybe journalists tend to take them at their word, thus exhibiting the relevance of answer (d). That's clearly what happened in the infamous "modern electronic communication is worse for you than marijuana" case, Of course, a certain amount of (a), (b) and/or (c) is also needed on the part of the journalists and editors involved.

And also, I do realize that some people freeze up and stop thinking whenever they see a mathematical symbol or term (even something routine like r² or p<.05). In this state of intellectual desperation, if forced by circumstance to pretend to understand what's going on, they clutch reflexively whatever simple-minded description comes most quickly to hand. This certainly happens sometimes on exams, and perhaps some journalists have a similar experience when assigned to a science story.

In any case, I blame myself. At least, I blame the profession of college teachers that I belong to. We seem to be turning out many students who can't understand relatively simple sorts of modern science, but lack the ethical gumption to admit it. Unfortunately, this ignorance and pretense combine with darker motives of sensationalism and pandering to stereotypes, creating a perfect storm of misinformation.]

Posted by Mark Liberman at October 7, 2006 11:56 PM