EdX Users Cheat Through MOOC-Specific Method, Study Says

Roughly 1 percent of certificates granted by Harvard’s and MIT’s school-specific edX platforms were earned by users engaging in a form of cheating never before possible in traditional brick-and-mortar classrooms, according to a working paper released by researchers from the two schools.

The paper—authored by MIT Ph.D. candidate Curtis G. Northcutt, MIT professor Isaac L. Chuang, and HarvardX research committee co-chair Andrew D. Ho—studied a previously unexamined form of cheating known as "copying answers using multiple existences online," or CAMEO.

Users engaging in CAMEO register for a given course with multiple accounts, one of which is designated the “master” account. They then use non-master “harvester” accounts to click through assessments as quickly as possible until the grading tool provides correct answers. The CAMEO user then inputs these “harvested” answers to gain certification for their “master” account in a significantly shorter amount of time than is generally possible.

“The strange thing is this is a kind of cheating you can’t normally have on campus,” Chuang said. “The uniqueness of this was really quite interesting and startling.”

An algorithm that cross-checked IP addresses with time-stamps of answer submissions found that 1,237 certificates across 69 HarvardX and MITx courses were obtained using CAMEO, amounting to 1.3 percent of certificates granted in those courses.

The study found that the CAMEO prevalence was greatest in the social sciences, government, and health categories, where 1.3 percent of the certificate issued were obtained by cheating, and lowest in the computer science category, where only 0.1 percent of the certificates were associated with CAMEO use.

Chaung attributed the disciplinary disparity to the differences between assessments in humanities and engineering courses.

“Randomized questions are easier to ask when you have numeric answers,” Chuang said. “It’s more difficult to ask different questions in humanities and social science scenarios.”

The study also found that CAMEO users are more likely to be young, male, and international.

“The between country or between group differences might have a lot to do with the perceived value of the [course completion] certificate,” Ho said, emphasizing the need for further research to conclusively establish the causes of the disparities.

According to the paper, the researchers’ detection algorithm was designed “with highly conservative cutoffs intended to reduce false positives,” and Chuang and Ho said that the algorithm may have missed CAMEO users who space out their answer input over a longer period of time or make use of IP anonymizers.

They also acknowledged that CAMEO use may become more refined as detection methods become more prevalent.

“In many ways, it can be an arms race,” Chuang said. “For any detection mechanism we create, we suspect that there will be gradually more sophistication [amongst CAMEO users], but, you know, the idea is not to go there if at all possible.”

The study also includes suggestions for course designers of ways to avoid CAMEO, including withholding answers until after an assignment is due and writing questions with randomizable elements so that each user receives a different version of the question, but Ho and Chuang said that these changes could entail significant pedagogical trade-offs for instructors.

The publication of the study, which points out that CAMEO is in violation of edX’s honor code, coincided with the rollout of Harvard College's own honor code and ongoing conversations about academic integrity at the University.

In response to the study, HarvardX administrators wrote in a statement that “HarvardX will take appropriate corrective action in response to violations of the edX honor code, which may include dismissal from the HarvardX course; revocation of any certificates received for the HarvardX course; or other remedies as circumstances warrant.”

HarvardX spokesperson Michael P. Rutter declined to provide further details on how CAMEO cases might be adjudicated or how certificates, all of which are distributed electronically and many of which do not bear the legal name of the recipient, would be revoked.

—Staff writer C. Ramsey Fahs can be reached at cfahs@college.harvard.edu. Follow him on Twitter @ramseyfahs.