Harvard Continues With College Board's 'Adversity Score' Following Criticisms and Changes to the Program

The Harvard College Admissions Office will continue to participate in the College Board’s effort to provide colleges and universities with information about applicants’ neighborhoods and secondary schools after changes were announced in August following criticisms.

College spokesperson Rachael Dane wrote in an emailed statement that the University still participates in the program, while working to determine its efficacy.

“We remain a member of the program and continue to evaluate its relevance,” she wrote.

The College Board’s Landscape program — which was previously known as the Environmental Context Dashboard but often referred to as the “adversity score” — came under fire when it was first announced earlier this year. Some experts feared it was reductive of student experiences and could negatively impact students it was designed to help. They also criticized the lack of transparency in the score’s calculation.

"The idea of a single score was confusing because it seemed that all of a sudden the College Board was trying to score adversity. That's not the College Board's mission," College Board CEO David Coleman told National Public Radio in late August. "The College Board scores achievement, not adversity."

The Harvard Admissions Office was one of 50 colleges and universities nationwide to use the Dashboard in its regular admissions cycle in 2019 for the Class of 2023.

While the Dashboard used a range of neighborhood and high school demographic information — excluding race — to come up with a single composite score between 1 and 100, the Landscape presents two such scores: one for an applicant’s neighborhood and one for their school. It also comes with a clearer explanation of what components comprise the score.

The neighborhood score includes college attendance, household structure, median family income, housing stability, education level, and crime, while the school score considers factors such as senior class size and the average number of Advance Placement tests each student takes.

“We listened to thoughtful criticism and made Landscape better and more transparent,” Coleman said in a press release. “Landscape provides admissions officers more consistent background information so they can fairly consider every student, no matter where they live and learn.”

Casey D. Cobb, a professor at the University of Connecticut and a fellow at the National Education Policy Center, said the original and modified programs are similar.

“The Landscape isn’t much different than the original Adversity Index, other than [that] the College Board [is] reporting separate scores for the high school and neighborhood indices, and students now can receive their score along with colleges,” he said. “I think the biggest change was the name. They switched it to Landscape to deflect criticism that the scores validly measure adversity.”

He said, though, that he believes the real issue is not the score itself, but how colleges use it.

Madhabi Chatterji, a professor at the Columbia University Teachers College who studies evaluation in education, said she is concerned about the College Board’s willingness to present institutions like Harvard with such a new and untested feature.

“It’s great that they are trying to provide contextual information to admissions officers in various colleges, but they have to back it up with evidence of validity and reliability and long term studies that show positive consequences for both the individuals who are admitted as well as for admissions officers at colleges,” she said.

Given the lack of evidence, Chatterji urged Harvard admissions officers not to rely on the Landscape scores.

“Without adequate research studies, the amount of weight that colleges put on this data has to be cut back,” she said. “They have to approach it cautiously.”

Cobb recommended that the Admissions Office develop a common understanding of the Landscape prior to using it, and suggested performing “before” and “after” institutional analyses.

“As any college or university, those that use these data should be explicit in how they are treating it in the admissions decisions,” he said.

The College Board’s “Appropriate Usage Guidelines” for Landscape — which are posted on its website — state that it is “to be used only as supplemental information” to the rest of the college application and should never be the “primary or sole determinant” of admissions decisions.

He also noted that Harvard is viewed as an “exemplar,” and could influence how other schools make use of the Landscape scores.

The College Board anticipates that between 100 and 150 colleges will participate in the pilot of the Landscape feature this year, before it becomes broadly available for free to all colleges and universities in fall 2020, according to its website.

—Staff writer Camille G. Caldera can be reached at camille.caldera@thecrimson.com. Follow her on Twitter @camille_caldera.