Course Guide Under Fire

It's one of the best read books on campus, but it's not by Shakespeare, Tocqueville, or Lipsey and Steiner.

Students write it. Faculty egos rise and fall with it. Junior professors' and graduate students' careers may hinge on it.

Established in 1974 as an aid for undergraduates and a source of feedback for instructors, the CUE Guide has become more than just another registration handout. Following a string of policy changes and a new wave of Faculty criticism aimed at the book, Harvard's official course critique is at the center of a debate that goes beyond the ritual of shopping for courses.

In recent weeks, members of the Faculty have been reexamining the Guide's influence, questioning the stock students and professors place in the annual course evaluation book. The scrutiny follows concerns raised earlier this year, when a series of letters from professors challenging the Guide's reliability prompted the student-faculty Committee on Undergraduate Education (CUE) to begin a formal review of its namesake.

At the outset, the review focused on complaints that the book had become overly subjective. Critics charged that the book often failed to faithfully reflect student opinion and had come to resemble The Crimson's tongue-in-check Confi-Guide. Several science professors voiced additional objections, complaining that the CUE's general questionnaire did not allow for an accurate assessment of science courses because it focused on reading assignments instead of problem sets and labs.

After months of discussions with Faculty officials and committee members, the publication's editors two weeks ago detailed a series of changes adopted to satisfy the critics. But even as the editors announced their reforms, members of the committee raised new concerns, questioning whether the Faculty misuses the only formal assessment of its academic offerings in two particular ways.

First, several committee members expressed concern that the CUE Guide evaluations are accorded too much weight in several departments which use them when deciding whether to promote junior faculty.

Some committee members also objected to the University's use of CUE evaluations to assess the performance of teaching fellows. Among other consequences, graduate students are known to have been removed from teaching positions on the basis of low CUE ratings, committee members reported.

Associate Dean for Undergraduate Education Steven E. Ozment, who chairs the committee and oversees the guide, indefinitely suspended that practice last month, and it is currently under review.

But beyond that particular element of the CUE controversy, discussion continues about the quality and value of the CUE Guide and its role at Harvard.

Behind the Guide

While course review book, were already fixtures at other schools 11 years ago, Harvard students still had to rely on word of mouth. Then, someone in Massachusetts Hall had a brigh, idea.

"President Bok has said the CUE Guide is one of the best ideas is had," says Ozment.

This year's Guide is 641 pages long and was budgeted at $65,000. Funded by the Faculty, the book is run by a salaried staff of three student editors and ten writers who labor through the summer in the Guide's Vanserg office.

The CUE attempts to evaluate all courses with enrollments of more than 14 students, excluding Expository Writing, tutorials, and seminars. However, not every course that meets the enrollment cutoff appears in the book. According to Faculty policy, the Guide cannot evaluate a course without the instructor's written permission. Each year, between 15 and 20 percent of all eligible professors withhold their approval, says CUE Editor Barbara S. Okun '86.

Some professors deny the Guide permission because they distrust it: others simply are unwilling to sacrifice 20 minutes of class time so their students can complete the Guide's written questionnaires, Okun adds.

The Guide bases its evaluations on students' responses to a standardized survey. The survey, which is distributed in class toward the end of the semester, asks students to rate the performance of professors and section leaders on a seven-point scale. It also asks students to rate the degree of difficulty and time consumption of the course work. Several essay-style questions seek personal comments to balance the objective data.

The reams of data are fed into a Science Center computer, which spews out neatly processed statistics for each course.

Drawing heavily on the individual comments, CUE writers prepare descriptive essays to accompany the numbers. Last year, Okun says, each piece was edited three times before being cleared for publication.

The Guide also includes information provided by the instructors about the reading lists and course requirements.

"We try to be objective. We combine what professors say and what students say to give a balanced view of the course," Okun says.

Although the Guide is a University publication and officials help shape its editorial thrust, the actual writeups are free from official censorship, Okun says.

The finished product is distributed free of charge to professors and students at the beginning of each year.

The Guide Applied

As the only source of critical information about Harvard courses and teachers, the CUE Guide is put to a variety of uses.

Once professors have submitted grades to the Registrar's Office, the Guide sends them reports of their ratings.

As a result of a Faculty directive issued last year, the Guide also sends professors copies of the original student evaluation forms. Professors are requested to share this material with their teaching fellows.

Some professors say they get the greatest benefit from seeing students' individual comments. "I don't get a great deal out of the numbers. I learn from thoughtful comments about the course," says Michael B. McElroy. Rotch Professor of Atmospheric Science.

McElroy also notes that the CUE offers professors a different perspective on their colleagues. "The CUE Guide allows me to get an idea of how serious my colleagues are about their teaching," he adds.

While professors use the CUE Guide to fine-tune their teaching and compare themselves with their colleagues, students are its main consumers. When it's time for undergraduates to shop around, sitting down with the Guide and the course catalog is standard procedure for more than a few.

Many appear to act on the basis of what they read. Course enrollments have jumped from 40 to 400 in a single year following favorable reviews, says CUE Systems Analyst Richard J. Kelly '85, who supervises the processing of data from the evaluations. Kelly adds that the Registrar's Office requests an advance copy of the Guide so officials can take it into account when planning room assignments.

But the recent controversy surrounds other applications of the Guide--applications for which it was never intended, some professors say.

Among the recipients of CUE data is the Office of Instructional Research and Evoluation, administered by Dean K. Whitia. The office oversees teaching within the Faculty of Arts and Sciences.

Until recently, Whitla's office used results of the CUE survey to status the performance of graduate student teaching follows. Section leaders scoring below a 3.0 on the CUE ratings were routinely removed from teaching positions or "invited" to brush up their teaching skills at the Harvard-Danforth Center for Teaching and Learning, Whitla says.

That policy resulted in the reassignment of "a small number" of graduate students whose instruction fell below par, Whitla adds. "Fortunately, we land the CUE to help bring them to our attention," he notes.

Members of the Committee on Undergraduate Education say CUE evaluations are also weighed heavily when some departments consider junior faculty promotions.

Promotion decisions require an assessment of professors' touching qualifications; the CUE Guide offers a convenient--if not the only--formal critique of their performance, says Mckay Professor of Computer Science Harry R. Lewis, a CUE represintative.

"In my department, [CUE evaluations] are weighed quite heavily," says Professor of Anthropology Sally F. Moore, another committee member. She adds that other universities often consult the CUE Guide when they consider offering posts to Harvard porfessors.

Use and Abuse

At a recent committee meeting. Moore and Orment condemned the Faculty's use of the Guide as a basis for decisions that affect the careers of teaching fellows and junior professors.

They assert that the Guide is not an authoritative measure of instructors' teaching abilities.

"It's become too big for its britches in some ways, and it concerns me," Moore says. "It's use can be devastating for some junior faculty," she adds.

Any department that relies on CUE evaluations is a "very lazy, shiftless department." Ozment says, urging that senior professors take the time to observe their younger colleagues' teaching firsthand.

Orment, now in his first year as associate dean, last month suspended the use of CUE ratings to word out weak section leaders.

Unrestricted usage of the CUE Guide creates undue anxieties among teaching fellows and non-tenured professors, he says. "The CUE Guide is not a letter from President Bok. The University has a responsibility to put it in perspective, and I'm not sure we've doing that right now."

But putting the Guide in its proper perspective doesn't necessarily mean prohibiting its use altogether, says Whitla. "It's a very useful source of information. It's one of the things that should go into consideration."

But Whitla adds that he does not oppose Ozment's decision to suspend Harvard's use of the guide to evaluate teaching fellows.

"It's quite appropriate that the system we've been using be reviewed," Whitla says.

Part of the problem, professors say, is that the Guide was designed primarily for student use. It was never conceived, they add, as the answer to the Faculty's tenure dilemmas.

When members of the Committee on Undergraduate Education addressed the Faculty's use of the Guide at a recent meeting with the CUE Guide staff, the issue took the editors by surprise. They said they did not realize the Guide was used to formally evaluate instructors.

"Obviously, the CUE Guide is written with a particular audience in mind," says Lewis. "But popularity among students is not equitable with quality in teaching. You have to read and interpret the evaluations very carefully.

Okran, the book's editor, to say whether she thinks the Faculty should refer to it when teachers' jobs are at statute. "People rely on the CUE Guide out of necessity,' she says. "Everyone wants it," irritation, and information is hard to get."