Warped Standards

IN JANUARY, 1970, Bowdoin College announced that it no longer required Scholastic Aptitude Test and Achievement Test scores from applicants for admission. The decision was widely understood to be one college's reaction against the dangerously heavy emphasis placed upon standardized tests in admissions offices throughout the country and few other colleges saw fit to follow suit. In the subsequent floor of correspondence from students, parents, teachers and administrators who had heard the news, Richard Moll, then director of Bowdoin admissions, received a note from an admissions colleague in New York: "Bowdoin will never amount to anything anyway, because your zip code number is too low."

Bowdoin's unprecedented break with the great American tradition of standardized testing has turned out to be a freak. Elsewhere in the country, nothing much changed; and down in Princeton, N.J., Educational Testing Service (ETS) was expanding its operations into a host of new fields. Apart from some 1.5 million pre- and elementary school children just entering the mill, 20,000 teenagers applying to exclusive private schools, 1.8 million high school students heading for college, 300,000 applicants to graduate schools, 74,000 considering business careers, and 120,000 hoping to be lawyers--all of whom will have to sharpen up their number 2 pencils and lay their futures in the hands of the country's biggest and most successful standardized tester--there are millions more.

ETS is currently involved in testing teachers, foreign service officers, CIA candidates, gynecologists, hospital finance managers, podiatrists, furniture warehousemen, stock brokers, architects and Peace Corps volunteers, to mention just a few. Not long ago ETS developed a "racially unbiased" test for people applying to the Philadelphia police force. The ever-broadening sphere of ETS influence, and the importance of ETS test scores in determining who does what in this country, is mind-boggling.

ETS makes no other claims for their tests than that they predict how well an applicant will perform at the institution to which he is applying. And according to an article by Steve Brill in the October, 1974 issue of New York Magazine, entitled The Secrecy Behind The College Boards, "ETS admits that aptitude and achievement cannot be measured in terms nearly as specific as the score that is recorded." An SAT score of 600, says Brill, means only that there is a two-in-three chance that your true score would be somewhere between 570 and 630; there is a one-in-three chance that it would be below 570 or above 630. According to an ETS booklet, a 72-point difference between two students' scores on the SAT math section, and a 66-point difference on the verbal section, is so statistically insignificant that "it cannot be taken seriously." For the Law Boards, a difference of 67 points cannot be taken seriously.

BUT WITH MORE than 100,000 applicants for 37,000 places in law schools each year, someone must be taking those differences seriously.

In spite of nagging questions about the reliability of ETS's three-figure quantifications of people's "aptitude," over-worked, understaffed admissions boards and personnel managers continue to rely upon test scores for an easy, quick, "objective" standard. In some parts of the country, state law now sets specific SAT and LSAT cut-off scores for admission to state universities. In the 1971 Federal court case of Baker et al. v. Columbus (Miss.) Municipal Separate School District et al., the court established that the Columbus school authorities' use of the National Teacher Exam (an ETS test) cut-off score of 1,000 for hiring and promotion created a racist classification. While 90 per cent of the white graduates of Mississippi colleges scores 1,000 or higher, 89 per cent of the black graduates scored below. The court ruled that the racial classification that this created (albeit inadvertently) constituted discrimination under the Fourteenth Amendment and that "there is no convincing evidence in the record showing any relationship between 1,000 on the NTE and effective classroom teaching."

Although the Mississippi ruling was not that the tests themselves were inherently prejudiced, the question of racial, sexual and cultural bias in ETS tests has been coming up for years. Brill cites an ETS study that found a direct, consistent correlation between seven categories of family income and SAT scores: students from wealthy families have higher median board scores than middle-income students, who in turn score higher than low income students. Brill goes on to say that "other ETS data, which ETS Executive Vice President Solomon said could not be made public because 'it would be misinterpreted,' show that students from the Northeast do best on the tests and Southerners do worst, and that males do better than females (on the math College Boards, at least)."

On a lesser scale, cheating and coaching call into question the validity of test scores as a reliable standard of measurement of "aptitude." For $150, the College Skills Center in New York offers a thirty hour course in vocabulary and reading comprehension and claims to improve students' scores by an average of 50 to 100 points. And rumor has it that for something upwards of $200 you can rent a Harvard Law student to take your law boards for you. ETS itself acknowledges about 2,000 cases of cheating each year--that it knows about.

BUT BY FAR the most common complaint concerns racial bias. Mean SAT scores for blacks, according to an August, 1975 article by Susan Schwartz McDonald in the Philadelphia Inquirer, average about 100 points lower than mean scores for whites. Brill claims that he found a study that showed a gap of 133 points between the median scores of black and white males on Law Boards. Some, including the folks at ETS, argue that the lower mean scores simply reflect inequalities in the educational system--differences in previous training. Executive Vice President Solomon insisted to Brill that the tests "have actually opened doors" to minorities and poor people. "With a national test you can compare the rich and the poor. If you do well, your score will show it, whether you're white or black or brown." And in a moment of candor, ETS President William Turnbull told Brill that criticizing ETS on the basis of racial bias "is like criticizing the Toledo Scale Company because some people are fat."

But City College Psychology Professor Lawrence Plotkin argues that "ETS tests operate to exclude blacks, poor people, Chicanos and other minorities from professional training." Which isn't surprising when, for example, millions of Spanish speaking Americans have their "aptitude" tested in a language that isn't spoken in their homes.

Along with the questionable influence of standardized test scores upon equal education and equal employment opportunities, it is worth considering the effect of the system on a much broader scale. Banesh Hoffman, author of The Tyranny of Testing, writes:

Few people realize how far-reaching are the effects of the current emphasis on multiple-choice tests. These tests have become the dominant factor in educational research; they furnish the yardstick--indeed the very definition--of "progress."

Of course it is possible that the size of the American educational system and the need for a standard measure of comparison necessitate some form of standardized testing (though whether it should be centralized, and so all-pervasive, is another question). But where the tendency to overemphasize and abuse test scores is so strong, the issues of test reliability, bias, validity and misinterpretation are critical. ETS, being accountable only to its board of trustees (who elect their own successors), has rarely been eager to get involved in making sure that the scores from its tests are used properly. Although proposals for curbing, or at least monitoring, the power of ETS have gone so far as to suggest strict government regulation, it is, for the time being, up to schools and employers to take a critical view of the dubious significance of test scores in measuring or predicting anything other than a student's ability to take tests.

The still primitive state of the art of psychometrics, statistically measuring psychological characteristics, is no excuse for overreliance upon a defective standard of measurement. As Hoffman concludes in The Tyranny of Testing:

Let us not seek to replace informed judgment, with all its frailty, by some inexpensive statistical substitute.

No one expects ETS to throw out its own system and start all over again. Its near-monopoly of the testing market (hardly surprising when most similar companies don't enjoy ETS's tax-free status as a "non-profit educational institution"), and the subsequent absence of competition, have removed the usual pressures that normally regulate corporations in a market economy. Its "client organizations" like the College Board--devoted offspring of the mother organization--are no more likely or able to force changes on ETS than college admissions offices. Everyone seems to have accepted the ETS line--"Well, it may not be perfect, but there has to be some objective standard. Got any better ideas?" And, in the most helpless position of all, is the consumer, whose future will probably be decided by this product that he never even wanted to buy in the first place. A rotten deal. Maybe that $200 to rent a law student isn't so high after all.