Professors Propose New Rankings

A study released last week by a team of researchers, including two Harvard professors, proposed a new ranking system for colleges based only on the preferences of high school students.

The study aims to reduce the influence of rankings like that of U.S. News and World Report, whose evaluations depend on admission rates, student-faculty ratio and other data provided by colleges.

Entitled “A Revealed Preference Ranking of U.S. Colleges and Universities,” the study was released as a working paper for the National Bureau of Economic Research (NBER). Harvard tops the new list—as well as the U.S. News list—followed by Yale, Stanford, CalTech and MIT.

The four authors—including Professor of Economics Caroline M. Hoxby and Professor of Public Policy Christopher N. Avery—ranked 105 colleges using information on 3,240 students provided confidentially by high school college counselors nationwide. The study focuses on the decisions the students made in choosing between different schools in 2000.

“We were interested in how colleges would rank if you took a more student-based approach,” Avery said.

Director of Undergraduate Admissions Marlyn McGrath Lewis ’70-’73 wrote in an e-mail that the new method “is simply another way of measuring customer preference.”

“Unlike some other kinds of rankings, this method does not claim to introduce considerations of ‘value’—i.e., elements of substantive worth, as defined by the one doing the measuring,” Lewis wrote.

The paper criticizes U.S. News and other rankings for having unduly influenced the college admissions process, characterizing the admission and matriculation rates used in the magazine to measure selectivity as “crude proxies.”

“They induce colleges to engage in distorted conduct that decreases the colleges’ real selectivity while increasing the colleges’ apparent desirability,” the paper asserts.

For example, a college may encourage “marginal” students to apply only to reject them, thus increasing the college’s apparent selectivity.

“When the admission and matriculation rates are given a lot of weight in rankings, college admission officers feel obliged to play a lot of games in admissions,” Hoxby said.

There are several discrepancies between the NBER rankings and the most recent one by U.S. News.

Washington University in St. Louis, for example, ranks 11th in U.S. News, but 62nd in the NBER study.

Avery pointed to such discrepancies as a testament to the strength of the system he and his co-authors proposed.

“It shows that U.S. News is valuing something that is not being valued by students in the same way,” said Avery.

The magazine’s reaction to any new ranking system is “Welcome to the club,” according to Richard Folkers, director of media relations for U.S. News.

Both Avery and Hoxby said that they and their co-authors—Professors Mark E. Glickman of Boston University and Andrew Metrick of the University of Pennsylvania—did not necessarily set out to create a replacement for U.S. News’ rankings.

“What we are anxious to do is to make everyone recognize that the rankings out there aren’t transparent, are manipulable, and are frequently thought to be science in a way they’re not,” Avery said.

Hoxby stressed that students should not accept the weighting U.S. News uses in its system. But students should not exclusively use the NBER rankings either.

“If you’re a student I wouldn’t advise you to just look at what other students prefer,” Hoxby said. “Look at several indicators for a college and put your own weight on things.”