Star Ec Prof Caught in Academic Feud

Nearly five years after she published a landmark paper on school choice, Professor of Economics Caroline M. Hoxby ’88 is engaged in a fiery academic dispute over the paper’s integrity.

Princeton assistant professor Jesse M. Rothstein ’95 set off the public feud with Hoxby last spring, after he published a 69-page challenge to her 2000 paper asserting that he could not replicate Hoxby’s results. Rothstein additionally alleged that Hoxby—a well-known and influential figure in economics—delayed releasing data that would have allowed others to evaluate her conclusions.

In March, Hoxby responded by publishing a new defense of her scholarship. And in a recent interview, she speculated that the dispute was part of a larger issue of “race and gender bias” in higher education. Hoxby is black, while both Rothstein and his dissertation adviser at University of California, Berkeley, Professor David Card, are white.

In the contested 2000 paper, “Does Competition Among Public Schools Benefit Students and Taxpayers,” Hoxby argues that the ability to select among schools within a district positively influences the quality of schools.

Over the last two years, Rothstein has amassed a case that he believes invalidates the central findings of Hoxby’s paper. He set the economics field abuzz when he publicly challenged Hoxby’s data and findings in his March National Bureau of Economic Research (NBER) working paper. In addition to disputing Hoxby’s results, Rothstein implied that Hoxby’s initial refusal to share data from her paper with him violated American Economic Review (AER) policy.

AER editor Robert A. Moffitt, who declined to comment specifically on the Hoxby-Rothstein debate, said that AER’s policy about data availability had been in effect at the time, although not strictly enforced. In an e-mail, Moffitt wrote that Hoxby’s paper was published “prior to the strengthening of our data availability policy,” which occurred last March.

Hoxby published her response alongside Rothstein’s NBER working paper, dismissing his accusations.

To this day, the Hoxby-Rothstein quarrel—deemed unprecedented by some economists—continues to be a topic of water-cooler conversation in academic circles and on Internet blogs.

And the stage for the next round of dueling has already been set: Hoxby and Rothstein have each submitted their papers for publication in AER, a Supreme Court of sorts in the hierarchy of economics journals.

Both papers are still pending publication, but what began four months ago as a comment on a single paper has burgeoned into a full-fledged war of words, extending beyond the boundaries of Cambridge and Princeton and into the pages of the nation’s top economics journals.

DUELING PAPERS

School choice and the debate over school vouchers and charter schools have become contentious political issues in recent years.

Advocates for choice say it promotes healthy competition, forcing lower-performing schools to improve in order to attract students. Opponents say implementing a system of choice would worsen the public educational system, siphoning money from struggling schools into funds to pay for school vouchers.

Hoxby—who has consistently defended school choice—lent more fuel to supporters of a voucher system with the publication of her 2000 paper. The paper concludes, among other things, that students in areas with many competing school districts perform better on tests than students from areas with fewer school districts.

In making her case, Hoxby used the number of rivers and streams in a metropolitan area as a predictor of the number of districts in that area. She then employed a technique called instrumental variable regression to isolate the effect of school district size on test scores.

Employing this particular technique meant that Hoxby’s results depended crucially on her measurement of the streams. It is these measurements, and Rothstein’s own measurements of the streams, that are at the center of the academic dispute between the two economists.

Rothstein has said that his attempts to replicate Hoxby’s results using her streams data has proved unsuccessful, which has led him to argue that Hoxby’s conclusions are overstated.

In his 2004 working paper, Rothstein argued that Hoxby’s analysis contained a series of errors in data and computer code, casting her results into question. He contended that by using different models, the effect of school choice remained near zero—a discrepancy that would invalidate Hoxby’s conclusion that competition has a positive impact on school quality.

“What worries me most about the results is that they are so sensitive,” Rothstein said. “You get such differences in the results when you make minor changes.”

In her response to Rothstein, Hoxby denounced each of Rothstein’s major claims, calling his arguments “wrong”, “misleading”, and “not credible,” even venturing that Rothstein “may suffer from…confusion.”

“The comment is not a replication, and each of its points is incorrect,” she wrote in her March paper.

‘IDEOLOGICAL BIAS’?

Hoxby said she believes that Rothstein’s interest in her work—and his working paper—have been instigated by the “ideological bias” of Rothstein’s dissertation adviser, Card. She said she believes there may be a larger issue of prejudice underlying the entire debate.

“I think there is quite a lot of race and gender bias going on here,” Hoxby said in an interview with The Crimson. “I think it’s really a pity. I think there are a few people being very unreflective about their behavior.”

Neither Rothstein nor Card would directly address the accusations of bias.

“I won’t dignify…[those] questions with responses,” Rothstein wrote in an e-mail. “My work speaks for itself.”

Princeton economics professor Cecilia E. Rouse ’86 completely dismissed Hoxby’s accusations.

“Quite frankly, I think it’s pretty outrageous,” Rouse said. “Everything I know about Professor Rothstein suggests that that was not at all his motivation.”

DATA DEBATE

Aside from the content of the papers, Rothstein’s implication that Hoxby violated a long-standing AER policy by initially refusing to publicize her data has sparked the interest of some in the field.

Hoxby said in her response that she was unable to release the data due to the “restricted-access” nature of the information she was working with.

AER editor Moffitt said that “the data availability issue for that paper has arisen only because Professor Rothstein has attempted to replicate Professor Hoxby’s work.” He said that AER has granted exemptions to its policy in the past if confidentiality restrictions trump an author’s ability to release data. In those cases, Moffit said, AER asks the author to “assist other researchers in obtaining the data directly from the data provider.”

In 2004, Hoxby made a CD version of her data available to researchers through the National Center for Education Statistics. The CD contained both the extracted data and raw data from the 2000 paper, and Hoxby said Rothstein was the first to receive a copy.

Hoxby said that restrictions on the availability of her data precluded her from making it available sooner.

“Sometimes people would ask me for the data, I would say I couldn’t give them the data because of the restricted access,” she said. “I don’t have the choice about that.”

In September, Rothstein sent Hoxby an e-mail, claiming that the data on the CD seemed different from the data used for her original paper.

When asked about the e-mail, Hoxby only said that she found Rothstein’s letter “unprofessional.”

“I don’t even know what [Rothstein was] talking about,” Hoxby said. “He was complaining that he had a different CD. That’s when I said to him it is fine to ask me about these things, but you have to copy the AER editor.”

“I’ve really gone above and beyond the call of duty with this,” she added.

After that e-mail, Rothstein said he stopped corresponding with Hoxby.

An economics professor at a major university, who spoke to The Crimson on the condition of anonymity, said that he had similar difficulty obtaining data when he tried to replicate the results of Hoxby’s 2000 paper three years ago.

“She explicitly said she gave [the data] to no one—not even big shots in the economics field—and that I should I not take it personally, but she was not going to release that data,” said the professor.

PROFESSIONAL DIALOGUE?

Professors say it is not uncommon for authors to publish opposing papers in economics journals. But many in the field say that the tone of the Hoxby-Rothstein dialogue—and Rothstein’s decision to challenge a star of the economics field in his second year as a professor—sets it apart from past exchanges.

“[The debate] does seem to be particularly personal and particularly combative in a way some people might describe as being a little unprofessional,” said Brigitte C. Madrian, an associate professor at the Wharton School of the University of Pennsylvania who has known Hoxby for 15 years.

Casey B. Mulligan ’91, a professor in the University of Chicago’s economics department, said that he found Hoxby’s work convincing, but added that he was taken aback by what he called “irrelevant rhetoric” in her response.

“Some people didn’t make it to the data because they found the rhetoric so offensive,” he said. “I’m hoping for her sake that she cuts that out of the [AER] version and sticks with the facts. It seems like passing notes in gym class.”

Hoxby has defended her strong language, arguing that her anger is a reasonable response considering the nature of Rothstein’s accusations. She has alleged that Rothstein is “out of line” and “in a lot of trouble at this point.”

“That’s unprofessional in economics and we don’t engage in that type of behavior,” she said.

NEXT STEPS

The dispute will most likely be resuscitated sometime over the next year, after the publication of both papers in AER. Its long-term effects on education policy—and the likelihood of any vicissitudes in either professor’s professional standing—are hard to predict, economists say.

Hoxby said she is “not worried” about any possible repercussions and said the dispute involves “a very specific paper and very specific claims.”

Rothstein said he looks forward to the imminent publication of both papers in AER and believes that the journal’s policies will permit a more productive exchange. Unlike NBER working papers, submissions to AER are reviewed and edited by a number of economists prior to publication.

“I am confident that the AER editorial process will ensure that the tone of the exchange remains professional and scientific, and that untrue claims are not printed,” Rothstein wrote in an e-mail.

Madrian said she believes the fierce back-and-forth between Hoxby and Rothstein will have no tangible impact on Hoxby’s professional standing in the end.

“Caroline already had a bit of a reputation as a tough cookie,” she said. “I don’t think this particularly changes her personal reputation. It might make more people familiar with the wrath of Caroline Hoxby.”

—Staff writer Javier C. Hernandez can be reached at jhernand@fas.harvard.edu.