Home | Letters to the Editor | www.gmac.com
To print this story, please select “Print” from the File menu of your browser.
Selections Interview with Bob Morse, Director of Data Research, U.S. News & World Report
Selections: Why did U.S. News begin ranking business schools?
Morse: Because at that point in time, I think outside of the field of education, business is the largest graduate discipline. We came at it to provide a consumer service. We felt that with so many people going to business school . . . and there was a desire by applicants to have information to help them make a choice.
Selections: You started in 1990?
Morse: Actually, the first year we did it was 1987. We actually beat Business Week. In 1987, we did it by reputation only. And in 1990, we started doing it in a way that is broadly similar to the way it is done today. Not exactly, but broadly similar.
Selections: What was the methodology for your first survey that was based solely on reputation?
Morse: Asking the business school deans. Of course, the Wall Street Journal and Business Week surveys are 100 percent reputation.
Selections: In 1990, how did you develop the methodology—including the weighting system—for your graduate business school rankings?
Morse: I did a lot of research. I personally visited a number of business schools, and there were a number of business school deans who came into our office. And we approached it from the premise of having a multivariable ranking system like we did with the “best colleges” ranking. We thought we could do something . . . not similar because the methodologies are very different. We did try to emphasize that if there were outcome measures—and we would define placement as an outcome measure or the opinion of recruiters. We tried to develop a methodology that was geared to the specific graduate disciplines.
Selections: How did you determine how to weigh the various elements in your ranking?
Morse: It was somewhat of our own judgment and talking to business school deans. We wanted reputation to have some weight and we wanted the statistical indicators to have some.
Selections: I know the schools supply a good deal of information for your survey. I’m wondering how you ensure that the schools are supplying accurate data and that everyone understands the questions in the same way? Have you taken measures to audit the information that schools are sending you?
Morse: At the beginning, the standards of business data reporting were significantly less evolved than they are today. There is the MBA Career Services [Council], which is two or three years old. And the AACSB [Association to Advance Collegiate Schools of Business] has now standardized information.
Selections: So, you work with these organizations to create standards for the way this data is reported? Or go by the standards that they create?
Morse: We are doing that. I would say we are the leaders. We were the first publication to implement the career placement standards. And I think we were the first guides to implement the definitions for the acceptances in GMAT scores. I don’t remember each data point. But we have implemented those in our surveys. And I think that we said that if this auditing does go into effect, then we would consider “asterisking” a school [noting in U.S. News that the school submits audited data].
We’re not working with the other organizations. If the other organizations develop standards and schools say their data is audited, versus generally accepted accounting practices, then we would indicate that. But we ourselves aren’t working to develop those standards.
Selections: What major changes have you made to your rankings methodology to make the U.S. News survey more transparent and accurate?
Morse: Our survey is very transparent. The others are very nontransparent. I think anybody you’d talk to would say that. Haven’t they?
Selections: I’ve been talking to all of the publication editors in charge of the major rankings, and certainly everyone so far has been standing behind their own methodology and their own way of doing things. Maybe you can talk about why you believe your survey is more transparent than the others, and if there are any specific examples that highlight that.
Morse: Well, I guess for a couple reasons. When you [say] transparent, you mean the most understandable to the public. That, if you were a student and you read our methodology, then you would have fewer questions about how it is done than the others. I would say that the way we publish the data using the actual scores . . . we publish the actual scores and the actual data that are used in the ranking. And we say, “This data point is weighted this,” and I think people understand. If you were to ask any dean, they would say that ours is the most transparent by far. I think people understand when you give a salary or the percent of people employed or a GMAT score and you say the schools that have the highest reputation and score the highest in these things are the best schools, people understand that. And when you give the weights, I think that is more understandable. And we give an overall score. Business Week does not give an overall score. It only gives a rank of its two reputation surveys. It says that in some way, it takes both of the responses, so that “if you do this in ‘recruiter’ and this in ‘student’ and one is weighted this and one is weighted that, and we’ve adjusted for historical, and we’ve made sure yadda yadda yadda.” I also think it is clear that Business Week has had trouble with people understanding that theirs can be gamed. Otherwise, they wouldn’t have had to weight previous years surveys so much. That indicates that they are dealing with gaming. It would be interesting to see what they have to say about us, but you can argue that it’s counterintuitive to say you are doing a consumer survey—which is an important thing to do, and I’m not arguing that is not an important thing to do—but then say that the consumers who you are counting may not even be experiencing the same education or the school may have changed in response to bad scores, and you’re still counting their grades in the survey. The students know that the value of their degree determines salaries in the marketplace. Let’s not be naïve.
If I could make one comment about the Wall Street Journal’s survey. Their ranking is not for students. Their ranking is for business people. I got an MBA from Michigan State, so I will say that before I say this. If I’m a student, and I get into Harvard, Stanford, Duke, and Michigan State—if you’re a student, and Wall Street Journal is telling you to choose Michigan State . . . I believe it’s correct that in the Wall Street Journal survey, the three schools I named rank significantly lower than Michigan State. And you’re also telling the students who are using this ranking that you are smarter to go to a school where the salary compensation is at least 50 percent to two-thirds lower. That flunks the reason why students choose schools. Now maybe it is a good tool for recruiters, if you believe that . . . and I’m not saying it’s not true that the Michigan State student is going to make just as good an employee as a Stanford student. That is a different thing [from] whether you’re better off as a student going to a school where the compensation packages are at least twice as much. And I’m not even getting into the subtleties of their methodology, which only uses one indicator.
Selections: Why do you rely on input from deans and department heads for the academic reputation portion of your survey? Why not speak to students?
Morse: Because we personally think that business school deans and program directors know about what is going on in their . . . It’s a small universe of schools. They certainly rub shoulders with each other, they are competing over faculty, they are competing over research proposals and other things, and they are aware of what is going on in the field. We feel they have a high level of knowledge of what are the better programs. We would argue that reputation is probably more important in business schools than in medical schools, as an example. You can argue that with business schools, because there isn’t any licensing involved, the reputation of the school is what is determining the value of a degree in the marketplace. It’s not by what students say. We feel the academics have something important to say about it.
Selections: Do you think using input from deans and department heads opens the door to bias?
Morse: Well, there are so many people who respond, and they are rating on a scale of 1 to . . . unlike Business Week . . . and I think you need to talk to the Wall Street Journal. I know you are not going to get a straight answer from them. It will be interesting whether you think you have. How did they respond? Did you ask them something like, “Why should a student choose to read your survey?” I’m not even talking about a recruiter or an executive but a prospective student.
Selections: I spoke with Joy Sever at Harris Interactive and she says students would be able to see from the Wall Street Journal survey where recruiters are happy doing business, where they feel that they get good results from their recruiting efforts.
Morse: Even though you would earn a lot less. I think that flunks the real-world test of what happens.
When the people respond to our survey as either an academic or a recruiter, they have a list of all of the schools, and they are judging them against each other. It’s my understanding that when the students rate the schools [in the Business Week survey] they are only rating their school, and they haven’t experienced what is going on at the other schools. And Business Week takes those responses and makes comparisons. That to me . . . at least in ours, where the academics are judging the schools against each other, you are getting a relative scale. Whereas Business Week is assuming the same rating system . . . We have enough votes and the scale is so small that not one voter can cause a change in the overall score of the school in the reputation survey.
Selections: In what ways have you seen the business school rankings in general, and the U.S. News & World Report rankings in particular, impact the schools and management education?
Morse: I would say the thing that I am most proud of with the rankings, and this came out at the AACSB meeting in New York this year—I was on a media panel. I think the schools believe—even though it makes them madder than the dickens—that U.S. News and maybe Business Week to some degree—have been responsible for causing data standardization. And if it wasn’t for U.S. News—given that we are the only ones to use the data in the ranking; the other ones just publish it, right?—then there wouldn’t have been the standardization of the placement data, and there wouldn’t be move by GMAC to try to implement auditing standards and put in rigorous definitions and schools signing up for it. I think we have caused that. Or, as a result of the rankings and the need for better consumer data as a result of that, the schools have realized that they have had to do that. And it’s also helping the MBA education at least to seem more professional by having better data. I also think the schools have cared more about the students. They have taken the concept of the student being a consumer. They have probably put in more resources and taken more care about the placement. And even though sometimes it doesn’t seem that way, some of the top schools—I’m defining U.S. News top schools, not Wall Street Journal top schools, since they are very different lists—but Harvard, Stanford in particular, which had been aloof in terms of consumer data. I think they have now come a long way, and maybe they are not the same as a Duke or an MIT, but I think they have moved up a long way. I’m not saying they are exactly the same as the other schools, but the gap between openness about their data is now almost closed, and I think that is because of the rankings. I also think business education—which actually isn’t because of the rankings—but I think business education changes more quickly than any other discipline and responds to changes in the marketplace more quickly than in any other field. And that is not commonly understood. But that has nothing to do with the rankings.
Selections: I see the benefit of getting this information about the schools out to prospective students and recruiters and for having standards for how the data are reported. But what is the benefit of putting a ranking number next to the schools?
Morse: The benefit to whom? I think of any field in higher education that understands assessments, ratings, judgment, and competition, it is business schools. I think that the academics in the schools—that while they may complain about the rankings, they’re used to competing, and the business world is like that. I think the business schools are more geared to this. I know there has been a proliferating of these rankings, and maybe it has become confusing, but that is hard to determine. We touched on before that there isn’t any licensing degree and the reputation is important to the value of the degree. At least for the full-time programs, students have taken themselves out of the workplace, and it’s probably costing them over $100,000 in earnings, not even getting into the cost of tuition. They want to know what are the best schools and where they have a chance to get the highest-paying jobs. This is important to students, and our particular rankings give them that.
Selections: How do the business school rankings impact magazine sales at U.S. News?
Morse: The newsstand sales aren’t very big. U.S. News is basically a subscriber-based publication. I think 98 percent of the sales are from subscribers. And newsstand sales in any given week are about 2 percent. So, while it may result in a gain, I don’t have the exact numbers. The data supports three things: the weekly magazine, the grad book—and, of course, we do undergraduate business school rankings and no one else does those—and both sets of data support the Web site. We’ve seen that 70 to 80 percent of traffic to the U.S. News Web site goes to the college and grad school area. So it is the main magnet for the Web site.
The ranking has created an important brand for U.S. News, and the ad pages are larger for the rankings issues than they are for other issues.
Selections: Some people argue that above all, U.S. News and the other publications publish rankings to sell magazines.
Morse: If you define selling magazines as people buying a copy, that is not where the economics are. The economics are in creating a brand and having increased ad pages. I don’t know the exact economics for the guide book.
Selections: There are stories about administrators at different schools being fired or reassigned after their schools have fallen in the rankings. Have you heard this, and what is your response to that?
Morse: I guess I have heard that somewhat for Business Week. I don’t think it is the right response. I think the schools have to use our rankings . . . I think Business Week’s are more consumer-oriented. In some cases, schools fall in our rankings because of bad placement resources. Because of the data we use. It shows things that are really happening. It’s probably not the person’s fault. That is not the reason that we are doing them, to get people fired. It’s an incorrect response to fire one person because more than likely, it’s not one person’s fault. They’re just creating a scapegoat for the problem. It’s probably unlikely that it is one person who has caused this.
Selections: Are you concerned about the proliferation of rankings by other publications?
Morse: I just can’t get away from the Wall Street Journal survey. And I’m not saying that Stanford and Harvard and MIT have to be ranked number 1 or 2, like they are in ours. I think these schools saw the Wall Street Journal’s system produced the rankings versus a weakness in the school.
Selections: Yes, it did seem to surprise a lot of people that Stanford was ranked number 45 in the Wall Street Journal survey. Yet, when the recruiters were asked where they would want to go to business school were they to get an MBA now, Stanford was ranked number 2.
Morse: I know. That is what I am saying. If I had a choice between Michigan State and Stanford, it would be highly unlikely that I would choose Michigan State. I think that is proven by Stanford’s and Harvard’s and MIT’s yields. Their yields [of applicants who accept offers] are in the upper 70s or 80s. They are getting almost everybody who applies.
In terms of the proliferation, I said I think that students are smart enough to ignore the Wall Street Journal. Time will tell whether I am right on that. Again, I think the Wall Street Journal’s rankings are not for students. I don’t think their rankings are appealing to the main motives of students for choosing an MBA. I think the main motive is to improve yourself financially as much as you can. You go to the school, generally speaking, where you get the highest salary after you graduate. I am not saying that happens in all cases, but on average. And the Wall Street Journal rankings don’t capture that, generally speaking. I don’t know whether students are smart enough to understand that the Business Week rankings are saying something different than ours. I think they are, but I’m not sure. I’m not sure how people relate to the Financial Times. I think if you’re smart enough to get into one of these schools and the acceptance rates are pretty low, then you’re probably smart enough to understand the differences and pick out the information from each ranking that is important to you, and you’re able to distinguish the differences.
Selections: Do you consider these other rankings to be competition for U.S. News?
Morse: No. Because we are doing something different. I think the Wall Street Journal’s rankings are being looked at more so by businesses and businesses recruiters. I think students will find them illogical. I think Business Week is the leading weekly business publication, and their rankings are highly sought after. We are the only ones publishing a multidiscipline ranking yearly, and we are dedicating much less space for each discipline. Where people are looking at ours, the consumer is broader than just prospective business school students. Since we are producing a different kind of product, I don’t think the other publications are taking anything away from us.
Selections: What is the most significant challenge you face as you put out these business school rankings each year?
Morse: The amount of data that we have to gather and the deadlines that we have to do it under. Because we are operating under inflexible deadlines and under short turnaround time. And then making our best effort to resolve data problems.
Selections: How long do you have to collect the data?
Morse: Because we are collecting fall entering class, we can’t start collecting until October 15. So, it is from October 15 through the middle of January. We publish the survey in the beginning of April.
Selections: What do you see for the future of rankings?
Morse: Well, we will continue to have our rankings being based on data—at least to a significant degree. And I think there will be a movement, which will benefit business schools, to implement these standards. I think this will have the effect of making the field more professional, because it gives consumers more confidence in the data, and I think the data will get better as time goes on. I think some kind of accounting or auditing standards will be implemented. I’m not sure how long it is going to take. I’m not sure how many more publications will do them, so I don’t know if there will be more.
I think the main players are in the field. This is a guess, but I don’t think American students are consulting the Financial Times to the degree that international students are. I also don’t know if international students consult the Financial Times more. I think they probably don’t, because I think they are going to U.S. publications that are closer to the market.
Selections: You haven’t opened your survey to any international schools, correct?
Morse: Correct. We have our reasons. One is that we do not market—or only in micro, micro ways—our print product overseas. I think both Business Week and the Wall Street Journal have international editions. And it is true that a significant proportion of business school students come from overseas. That is true. But they can get access to our data for free on the Web. And I think they do. We’re probably missing a business opportunity by not marketing the product overseas, but we don’t. And it would actually be harder to use our system of GPAs if some of the schools aren’t using the GMAT . . . some of our ways would be harder to translate internationally.
Selections: How do you go about choosing the schools you include in your ranking?
Morse: We use the AACSB [Association to Advance Collegiate Schools of Business] accredited schools. We look at all of those schools. I guess that is another main difference between U.S. News and Business Week. They limit theirs to 30 or 50 and don’t open it up. I’m not sure about the Wall Street Journal, but theirs is a moot point since some of the recruiters don’t visit the lower schools. Then, in essence, their system makes it a moot point.
Selections: Whom do you consider to be your main audience for your business school ranking?
Morse: Prospective students—that is the main audience. The general public is second, and academia is third. Academia is very interested in our rankings.
Selections: What are some of the most common mistakes you see the schools making with regard to the rankings?
Morse: I think if you’re a cusp school, meaning you’re 48 or 50, and you make a big deal about being ranked. If we’re doing a top-50 cut-off, and you do a big marketing campaign and then fall off the list . . . because that is an area of the ranking where schools fall on and off of. The 20th school isn’t going to fall off the ranking. They may change a few places . . . So, I think if you’re at the cusp of ours, and you market yourself as being ranked, you may get questions the next [time] about why you disappeared. I would call that a bad PR choice. Is that what you’re trying to get me to respond to?
Selections: A better question might be, how should schools be using the information published in your survey?
Morse: Two things. One is they can use it for peer analysis. I think we’re the first publisher of this data. So, it’s the first time schools have the ability to see the most recent admissions and placement data of their competitors. So they can use it as peer analysis. And I think if a school does fall, it can be a painful jolt and they can decide to force a correction. I think they need to be careful about how they publicize it.
The reason our survey is so transparent is it’s more easily criticized.
Selections: The two criticisms I heard were that deans don’t know what is going on at their schools, let alone other schools. The other is that schools lie when they submit their data.
Morse: I didn’t say they lie, but I think the data has improved. I haven’t seen any evidence of some [smoking] gun. The standards are a lot higher and the data are a lot better. Is it perfect? No. I guess each of the rankings has its own weaknesses.
© Selections: Fall 2001