Measuring Confidence, Accurately

University President Lawrence H. Summers was formally rebuked by the Faculty of Arts and Sciences (FAS), after a well-attended faculty meeting one week ago produced two votes of “no confidence”—or a “little confidence” or “some confidence but not a lot”—in Summers’ leadership. A poll put forth by the Graduate School Council, which closes this afternoon, purports to do the same for students of the Graduate School of Arts and Sciences (GSAS) using the same questions asked in the FAS measures. But when the votes are tallied, what will we have learned? Unfortunately, not much. The unscientific nature of the polls, and a startling lack of attention paid to the wording of the measures, casts real doubt on their actual meaning.

Neither motion is worded to be friendly. The first, “The Faculty lacks confidence in the leadership of Lawrence H. Summers,” is loaded with acquiescent response bias—the tendency of people to automatically agree with any statement presented to them, regardless of its content. How different would the results be if a Summers supporter had called the confidence motion—rather than Professor of Anthropology and of African and African American Studies J. Lorand Matory, a self-acknowledged opponent of Summers’ leadership—and asked the Faculty if they had confidence? The results would certainly be different, but at this point it’s impossible to know by how much.

To make it worse, the second vote—phrased to be a compromise—is 66 words long and so brimming with compromise that it would take a herculean effort to disagree. It states that Summers should be rebuked for his comments and style of leadership, but that the faculty “appreciates his stated intent” to fix these problems. It’s all too easy to agree with any one of these three ideas and be lukewarm about the others, yet still vote in favor of the proposal.

I guess I shouldn’t be surprised at the way this is being handled; after all, determining Summers’ standing is by definition a political process, not a scientific one, and that phrasing questions in ways designed to elicit a desired response is part of the game.

What sets this apart from the typical vote of confidence is that there is little price to pay for a non-binding statement. In most legislative assemblies, votes that bring down the government are typically those that defeat budgets or other bills essential to the functioning of society, since there is true meaning, and risk, in halting the operation of government. In this case, the Faculty—with no formal authority over Summers’ job security—had no other option but to fire off a loud cannon shot, in one direction or the other (it’s hard to tell which one). In the same way, it will be impossible to tell exactly how the GSAS student body feels about Summers’ leadership when the votes are counted tonight.

If the Faculty wants to take a more accurate look at Summers’ leadership, there must be a properly designed survey to take the Faculty’s temperature (I declined to respond to the GSAS survey because it has the same bias problems as the faculty questions). Here’s one quick way to do that: for every survey sent out, randomly determine which statement is presented: the vote of confidence, or that of no confidence.

And while we’re at it, let’s ask questions about the state of women and hiring practices—the real issue that needs debate—and reverse their wording at random as well. I’m curious to know what percentage of Harvard faculty would describe their working environment as unfriendly due to gender issues, or how many men or women changed or intend to change their working hours to spend time with their families. I’m sure I’m not alone.

Politics tramples over rational debate all the time, so I shouldn’t be surprised when it happens here. But I know that if we as a community are looking for real solutions to problems, we can do a whole lot better in the way we ask our questions.

A.C. Thomas is a graduate student in the Department of Statistics.