Education

Comparing The Major College Ranking Systems: How Methodology Matters


The major college ranking systems have all published their much-anticipated lists of the nation’s best colleges for this year. These lists continue to influence students, families, college administrators and the general public, even during a pandemic that has arguably rendered many of the ranking dimensions much less relevant than in the past.

How do the methodologies employed by the various systems compare? And how similar are their rankings? Here’s a brief primer on four of the most influential approaches.

U.S. News & World Report

U.S. News & World Report’s 2021 Best Colleges rankings ranks more than 1,400 colleges and universities granting baccalaureate degrees. Now in its 36th year, it’s the dean of all ranking systems.

U.S. News revises its methodology over time, and this year it again adjusted the weights assigned to 17 measures of “academic quality” grouped under six categories. The major changes from last year were:

  • The addition of two measures of student debt.
  • More weight placed on its “outcome” measures from 35% to 40%.
  • A corresponding reduction in the weights given to SAT and ACT scores, high school class standing, and alumni giving.

The 2021 ranking categories and their weights for National Universities are as follows:

  • Outcomes (40%) includes student indebtedness, graduation and retention rates, social mobility (the enrollment of Pell grant students and their graduation at a rate that’s close to the rate of non-Pell Students). 
  • Faculty Resources (20%) is composed of class size, student:faculty ratio, average faculty salary, proportion of faculty who are full time, and who have earned terminal degrees in their discipline.
  • Expert Opinion (20%) reflects reputational ratings by presidents, provosts, and admissions deans.
  • Financial resources (10%) involves spending per undergraduate student on academics, such as instruction, student services, and research.
  • Student Excellence (7%) taps ACT/SAT scores and high school class rank.
  • Alumni giving (3%) is the percentage of bachelor’s degree graduates who donate to their institution in a given year.

Washington Monthly

Washington Monthly’s 2020 rankings are promoted by its editors as “the socially conscious alternative to U.S. News & World Report.” It ranks schools based on three equally weighted criteria that emphasize contributions to the public good, rather than exclusivity, reputation, or wealth. A detailed description of its methodology can be found here.

Each of the three criteria is measured with several components.

  • Social mobility is made up of overall graduation rates, the difference between actual and predicted graduation rates (based on the makeup of the student body), the percentage of students receiving Pell grants, the percentage who are first-generation college students, college affordability, adjusted median earnings ten years after graduation, and student loan repayment rate. 
  • Research is based on five criteria, including total institutional research spending, the number of science and engineering PhDs awarded, the number of undergraduate alumni who go on to earn a PhD, faculty research awards, and the number of faculty who are members in the National Academies.
  • Community and national service consists of five composites, including the percentage of students in campus ROTC programs, the percentage of alumni in the Peace Corps, the percentage of work-study grant money spent on community service projects, whether the institution provides a match to students receiving Segal AmeriCorps Education Awards, and the extent to which a campus promotes and supports student voting.

Money

Money’s best colleges for 2020 is the seventh edition of its rankings. Its methodology is based on 27 factors in three categories:

  • Quality of education (30%), which includes six-year graduation rate of both transfer and first-time students (30% of this category); value-added graduation rate (30%) – the difference between a school’s actual graduation rate and its expected rate, based on the economic and academic profile of students; peer quality (10%) measured by standardized test scores of entering freshman (5%) and “yield” rate (5%); student-to faculty ratio (10%); institutional financial troubles (10%); Pell Grant recipient outcomes (10%).
  • Affordability (40%), including net price of a degree (30%); net price paid by students in different income brackets (20%); student debt (20%); ability to repay debt (15%); the school’s performance on student loan repayment and default measures after adjusting for economic and academic factors (15%).
  • Outcomes (30%), consisting of graduates’ earnings, as reported by alumni to Payscale.com (15%); earnings adjusted by majors (20%); College Scorecard employment outcomes (30%); earnings ten years after college entry (10%); value-added earnings (15%); and a social mobility index (10%), which shows the percentage of students moving from low-income backgrounds to upper-middle-class jobs by their mid-30s.

Wall Street Journal/Times Higher Education

The WSJ/THE 2021 rankings are based on four categories of measures.

  • Resources (30%) which includes finances per student (11%); student:faculty ratio (11%); research papers per faculty (8%).
  • Engagement (20%), for which most of the data are gathered through the THE US Student Survey, tapping measures of student engagement (7%); whether students would recommend the school to others (6%); interaction with teachers and students (4%); number of accredited programs (3%).
  • Outcomes (40%) which is based on graduation rate (11%); value added to graduates’ salary (12%); debt after graduation (7%); academic reputation (10%).
  • Environment (10%) which attempts to asses whether all students are provided a good learning environment. It includes proportion of international students (2%); student diversity (3%); student inclusion (2%); staff diversity (3%).

One oddity in the WSJ/THE rankings is that it cancelled this year’s student survey “because of the coronavirus pandemic and subsequent difficulties for institutions in the US.” As a result, “the data for the three student engagement metrics has not been updated: we are using the scores obtained by institutions last year.” It’s a questionable approach: new year, old data.

Comparing the Results

So what difference in rankings do the different methodologies make? Here are some observations.

  1. At the top, there is considerable consensus, not surprisingly given the fact that all the ranking systems give substantial weight to factors like graduation rates, faculty quality, student debt, and graduate earnings. Among national universities, six institutions were ranked in the top 20 by all four systems: Princeton, Harvard, Yale, MIT, Stanford, and Duke. Five more made three of the four top-20 lists: Columbia, University of Pennsylvania, Vanderbilt, Rice and UCLA.
  2. Private universities dominated the U.S. News and the WSJ/THE top-20 rankings; only one public university received a top-20 placement – UCLA which slipped in at number 20 on the U.S. News list.
  3. Public research universities fared much better in the Money and Washington Monthly rankings, with eleven universities making the top 20 in both.
  4. The University of California enjoys a dominant position among top-ranked public institutions. University of California campuses make up five of the eleven public schools in both Money’s and Washington Monthly’s top-20 lists.
  5. The public-private discrepancy is one of the most significant consequences of the numerous methodological differences. U.S. News and WSJ/THE give considerable weight to institutions’ financial resources and reputation, two factors that favor wealthier universities and are either minimized or eliminated by Money and Washington Monthly. Conversely, within their systems, the latter two sources put greater premiums on institutions’ affordability, their impact on the social mobility of graduates, and their promotion of various forms of public service.



READ NEWS SOURCE

This website uses cookies. By continuing to use this site, you accept our use of cookies.