As it investigates the accuracy of the data it’s submitted, Columbia University has decided to not … [+]
Last week’s revelation by Columbia University that it would not participate in U.S. News and World Report’s upcoming college rankings because of unanswered questions about the accuracy of its data is reverberating through the halls of higher education. One of the questions that decision has prompted is whether other institutions might reconsider their own participation in the annual rankings scheme.
Here’s what happened at Columbia and why other colleges would be wise to not merely scrutinize the reliability of the data they submit to U.S. News but perhaps reconsider whether they’ll continue to participate in the rankings at all.
In February of this year, Michael Thaddeus, a professor of mathematics at Columbia, posted a lengthy critique of the U.S. News rankings and many of the data that Columbia had submitted for the rankings, which placed it second (tied with Harvard) among the nation’s best universities for 2022.
Thaddeus compared institutional data to the numbers Columbia had sent in and questioned the accuracy of several of the self-reported figures, including the number of classes of various sizes, the percentage of faculty with terminal degrees, the percentage of classes taught by full-time faculty, the student:faculty ratio, and the amount Columbia spent on instruction.
Initially, Columbia defended its data. But on Thursday of last week, just before the July 1 deadline for submitting new data to U.S. News, Columbia changed its tune and, in a statement by Provost Mary Boyce, said it would not submit data this year. That statement indicated that the university had “embarked on a review of our data collection and submissions process.”
The statement continued, “Columbia has long conducted what we believed to be a thorough process for gathering and reporting institutional data, but we are now closely reviewing our processes in light of the questions raised. The ongoing review is a matter of integrity. We will take no shortcuts in getting it right.”
The university committed to publishing a Common Data Set this fall that would include much of the same information U.S. News covers. That’s an interesting development in itself, given that Columbia – unlike most universities – has not been making its Common Data Set public. Columbia’s statement also did not indicate whether or when it would participate in the U.S. News process again.
So much for Columbia. What about other colleges and universities? Will Columbia’s decision to not participate lead more schools to question the veracity of their data and perhaps cause them to step away – at least temporarily – from the U.S. News rankings.
It might – and it should – for at least a couple of reasons. First, you can be sure there are more faculty or staff like Michael Thaddeus out there who are now inspecting their own colleges’ data with a careful eye toward uncovering mistakes, discrepancies and fraud. And chances are good that at least some of those sleuths will uncover patterns that are suspect, casting doubt on whether their institutions fudged the data to make them look better and rank higher than they deserve.
The damage to a college’s reputation when it’s revealed to have deliberately or even unintentionally submitted inaccurate data is substantial, and as many university officials will acknowledge – at least privately – the process by which institutional ratings data are internally reviewed/verified is far from robust. Columbia’s situation illustrates the dilemma – the risk of being accused of, or admitting to, providing fraudulent data likely outweighs the risk of not participating in the rankings.
Second, the problem of universities submitting false data has been discovered often enough to raise serious questions about the value of the rankings. As examples, in just the past decade,
- The University of Southern California withdrew the Rossier School of Education from U.S. News & World Report’s 2022 rankings after discovering “a history of inaccuracies” in the research and student enrollment data it had reported for at least five years. In response, U.S. News said it would require the university to provide a letter certifying the accuracy of Rossier’s data submissions for the next three ranking cycles in order to be included.
- Moshe Porat, the former dean of Temple University’s Fox School of Business, was found guilty in 2021 of wire fraud and conspiracy to commit wire fraud for his role in falsifying data to boost his school’s rankings.
- In 2019, U.S. News reported moving five universities to its “unranked” category because they had misreported data used to calculate their rankings. The schools were the University of California—Berkeley, Scripps College, Mars Hill University, the University of North Carolina—Pembroke and Johnson & Wales University.
- Other recent, prominent “misreporters” cited by U.S News include the University of Oklahoma, the University of Missouri, the University of Virginia, and Washington University (Saint Louis).
- Emory University admitted in 2012 that its administrators had knowingly misreported information about incoming students’ SAT and ACT scores, class rank, and grade point averages for a number of years.
- That same year Claremont McKenna College revealed that one of its Vice Presidents had been misreporting admissions data for several years.
No one believes that the universities that have been caught are the only culprits. But how widespread is the problem of dubious or doctored data? That’s unknown, and most institutions are not motivated to dig too deeply into their reporting methodology to find out if they might be cheating.
U. S. News claims that only a “very small proportion” of ranked universities have been found to have submitted false data, but in February of this year it listed a total of nearly 50 institutions that it had unranked in various categories since 2018 as a result of misreported data. At some point, a few more college leaders might conclude this is a club to which their schools don’t need to belong.
The college ranking industry has been criticized for years, often on the grounds that “gaming” or cheating the system is far too easy and common. For example, Colin Diver’s Breaking Ranks: How the Rankings Industry Rules Higher Education and What To Do About It is a spirited, often witty critique of the college ranking industry, particularly the“gorilla” in the neighborhood, U.S. News.
Driver is the former president of Reed College, which made news in 1995 when it chose, under the leadership of a previous president, to no longer participate in U.S. News’ rankings. That stimulated hopes among critics of rankings that others would follow suit. A few schools have done so, but the vast majority of U.S. institutions continue to play what many see as pernicious game.
Maybe Columbia University’s decision to skip next year’s rankings will embolden other college leaders to take a second look at their future participation. They know there are more Michael Thaddeus wannabes out there, maybe even on their own campus.
