Further to our discussion last week, I emailed Add Health about the nature of the data that Kanazawa used. They sent me a statement this morning, which has obviously been sent to many others who enquired. Here are some excerpts:
"The data Kanazawa used for his research were drawn from the National Longitudinal Study of Adolescent Health (Add Health), a congressionally-mandated study funded by the U.S. National Institutes of Health. Add Health data are available in two forms: a “public use” data set, which includes data from a subset of participants, and a “contractual” or “restricted-use” data set, which includes the full set of variables and participants. The “restricted-use” data are available to researchers who have appropriate research credentials (e.g., post-graduate degree) and an Institutional Review Board in their research institution that ensures their use of data security procedures required by Add Health to protect data and participant privacy and confidentiality. Kanazawa applied for and was granted access to these restricted data, as have thousands of other researchers . . .
"Kanazawa based his blog post on data derived from interviewer ratings of the respondents that were recorded confidentially after the interview was completed and the interviewer had left the interview setting. It is a widely-used and accepted survey practice for interviewers and researchers to include such post-survey completion remarks. These remarks provide both an additional observation about the respondent and data on the context of the interview for researchers to assess data quality. . .
"Interviewer ratings of respondent attractiveness represent a subjective “societal” perception of the respondent’s attractiveness. We included these items because there is a long line of research evidence that indicates that perceived attractiveness is related to important health and social outcomes, including access to health care, health education and instruction, job search, promotions, academic achievement, and social success in friendship and marriage. For example, males who are rated more highly attractive tend to have higher wages, shorter periods of unemployment, and greater success in the job market . . .
"Because the interviewer’s perception is subjective, researchers need to account for the characteristics and life experiences of the interviewer in interpreting their ratings. A wealth of research on perceived attractiveness (that is, as perceived by others, not oneself) has shown that such ratings vary according to the characteristics of the rater. For example, a male interviewer might rate a female’s attractiveness according to different criteria than a female interviewer rating the same female’s attractiveness. Other interviewer characteristics that are important to take into account are age, race, ethnicity, education, geographic location, and life experiences, in general."
Though some of us may have sussed out the main points, especially those contained in the final paragraph above, this statement clearly demonstrates the irresponsibility of Santoshi Kanazawa's "research". As somebody who was granted access to the restricted-use data-set, he would have been aware of the nature of the attractiveness rating. Yet, he gave no indication of this in his blog post.
Add Health Director, Dr. Kathleen Mullan Harris, a professor of sociology at UNC, Chapel Hill drove the point home in an interview with NPR, quoted in the email sent to me: "He's mischaracterizing the objectiveness of the data — that's wrong. It's subjective. The interviewers' data is subjective."
It is not unreasonable to ask that LSE investigate Kanazawa, and take appropriate action. I leave to others who are more experienced in this kind of action how this request could be prepared and communicated to LSE.
(I will cross-post this on It's Only a Theory.)
Recent Comments