Home | Letters to the Editor | www.gmac.com
To print this story, please select “Print” from the File menu of your browser.
Selections Interview with John Byrne, Senior Writer at Business Week
Selections: You were involved in creating Business Week’s first business school ranking in 1988. Why did Business Week begin ranking business schools?
Byrne: I had recently come over from Forbes magazine, where I actually began writing about graduate education. The initial impressions I had about MBA education back then—and we’re talking about the mid-1980s, ancient history—is that it was typical academia. No one thought they had customers that they had to be responsible to. This is an interesting issue even today, oddly enough, where that argument is frequently made at universities. I thought that if it had any legitimacy, it certainly had less legitimacy in a school of business, where the graduates were immediately employable in companies where they were expected to perform very quickly. Back in the mid-1980s, the most common criticism of the elite MBA [programs] was that [recruiters] would get these remarkably smart people into their companies, but [the graduates’] aspirations were so high and their interpersonal skills so low that there tended to be a good deal of dissatisfaction with them. That was particularly in corporate America. Less so in the field of investing banking and consulting, where there seemed to be a greater tolerance for arrogance. And the schools seemed to not care very much. With a few exceptions, I should say, the schools didn’t seem to care very much about the attitudes or the perceptions of the customers: the people who actually buy their product.
I felt two things. I felt there was no marketplace, really, to make the schools even pay attention to the demand. Why? You could say the marketplace existed because you could say they [the corporations] could simply not hire, but that is not the way it is. What was happening is, the people who were going to the [MBA] programs were remarkably talented, but they needed a little more seasoning. They needed more opportunities to work with people and through people to understand how important that was in the workplace.
The other thing you had going on at that time in the mid-1980s is the students were treated just as students, and there was a very heavy bent toward research—even though business schools are professional schools. This was at the expense—at the severe expense—of teaching quality. So when you talked to MBAs about their experience, they were still generally very positive about it, but they had very severe concerns about the quality of the teaching and the lack of attention to detail in these institutions that affected both the learning and the environment for the student. And you had many of the professors who would speak with derision about their obligation to communicate knowledge.
So what I thought is, one thing that a ranking would actually do is to create a market where none had existed. Create a market where schools could be rewarded and punished for failing to be responsive to their two primary constituents: the students and the corporations. I’m not talking about inexperienced kids. Here, we’re talking about people who have been out in the workplace already and have come back and [are] paying a lot of money, and they should be discriminating. And the schools should be forced to listen, but they weren’t. The corporations were hiring these people and were having problems getting them to, number one, stick and, number two, get along with other employees, and getting a real payback from the cost of hiring.
So, I thought that if we were to create a market, what we would have to do is to survey these two different and two very important constituents of the business school community. I went out and found lists of the latest graduates from the top programs. I went out and did a study on who were the major recruiters of MBAs.
Selections: When you say the top programs, how many schools did you start with and how did you determine which schools to survey?
Byrne: The first survey appeared in November of 1988. And I don’t remember exactly how many schools we initially surveyed. I think we basically focused on around 30 schools to get the top 20.
Selections: And this was looking at the “top schools” based just on general reputation? Were there other rankings at that time to guide you to the top schools?
Byrne: There have been rankings forever. Most of the rankings before this were driven by faculty research and were driven by a number of professors. And U.S. News & World Report [rankings] came out in around 1990. The first survey they came out with was so embarrassing, because they actually included Princeton in the list. Why? Because—I forget how they arrived at that—but I remember Princeton was on the list because whoever they asked simply named Princeton because of its overall reputation, not knowing that that school didn’t have an MBA program.
Selections: The first Business Week ranking came out in November 1998, and that is the survey in which Kellogg came out on top, correct?
Byrne: Yes. There was a lot of surprise about that. But there wasn’t any surprise to those who were listening to number 1. The problem was, no one was listening to customers except Kellogg. For example, the most common criticism of MBAs in corporate America was that they couldn’t get along with the workforce and they were very weak in communication and interpersonal skills. Well, guess what, Kellogg had listened to that criticism years before and had become the only school at the time to interview 100 percent of the applicants. So the fact that they were able to graduate, number one, people who had better interpersonal skills, was no accident, and secondly because they didn’t have at the time this elitist image that some of the long-time traditional leaders with MBA education had, their graduates were more easily embraced and more eagerly received than traditional MBA graduates, because that is what they were trying to address.
I should mention one other example about Northwestern. Northwestern was also one of the first schools to make their students work in teams. What would be helpful in terms of history here, one time at a conference when Dean Jacobs mentioned the culture of the school and the emphasis on teamwork, another dean from a very prominent school turned to him and said, “At my school, we call that cheating.” For students to work together on projects, to help each other—“We call that cheating.” That was sort of the mindset at the time. It is important to note that because today, teamwork is taken for granted. We accept that that is the way it should be, and everyone does it. Back in the late ’80s, only Northwestern was doing it, and it made a very big difference. It is why they turned out to be number 1.
Remember—and this is another important thing—you can argue about what is best. Some may say “best” doesn’t necessarily mean that the corporations that hire the MBAs are happy with the MBAs they hire, and that the graduates think they have gotten the return on their money and are happy with their education. “Best” means we have a faculty that is leading edge in their field and our faculty publishes in all of the “A” journals, and if you want the latest, smartest, most intellectual faculty, then you probably wouldn’t go to Northwestern. I would argue that that is not what “best” is at all. In fact, if you were that strong in research, you were probably a lousy school for both corporations and students. And that is what made it so controversial. And I have to say what I liked about this work, what made me passionate about it, was the sense that you could create a marketplace that had not previously existed, and then schools had the option of saying, “We think this is garbage, and we are going to ignore it.” Or they could respond to it. In the beginning, there was a lot of controversy and arguments about whether we should pander to students and corporations: “Why should we do that? That is beneath us. We are educators. And we shouldn’t have to treat our students as customers, and we shouldn’t have to treat those companies that come and hire our MBAs as customers.” That is really ridiculous. That is what a lot of schools thought at the time. And to this day, frankly, the most heated rhetoric about rankings is also tied into this very point. You still have professors and you still have deans—primarily at heavy, heavy research schools—who still think this is terrible.
Selections: I’ve heard people argue that they believe the curriculum at the schools is being watered down because the faculty is trying to make the students happy.
Byrne: I think that is completely bogus, because basically, the people who go into these programs—especially in the last decade—the amount of talent that these schools are able to attract is frankly mind-boggling. And if you look at the great diversity of the students, and their backgrounds, professional and personal, it is truly awe-inspiring. These people are very discriminating, and if, in fact, they went to a school with watered-down curriculum or a place where they could get an easy A and where there was not a lot of hard work, guess what? They [the schools] would get graded down for that, particularly in our survey. Because we do, in fact, ask specific questions about workload. And if, in fact, the workload is perceived to be too light, the school gets marked down for that. People who say things like that are basically selling their students short. Now, has some of that occurred? Probably. But if it has occurred in a way that truly weakens the program, oh, we would pick it up in the survey.
The beauty of the survey is this: It allows us to reach thousands upon thousands of the latest crop of MBAs and ask them, really, the most important questions you can about their educational experience. And as a reporter, when you gather that much information from that large a group of schools and from over that large a group of backgrounds, boy, you have your hands on stuff that is really valuable. The surveys often are a starting point, incidentally, for deeper, more thoughtful reporting. Because when those surveys come in, we often use them to go back to these students personally and interview them. And that is where the richness really comes out, and allows the magazine to spot trends long before they can be spotted elsewhere.
The other thing this system did, in terms of creating a market, is we sent one questionnaire that went to the students of all the schools and one questionnaire that went to all the corporate recruiters. And for the first time there was—by accident, frankly—a system where you could literally measure schools on everything from how well the placement office worked to how well the students performed to the communicating ability of the students to how well the culture of the school was suited to the infrastructure. You had a standardized form to measure each school against each other. And, ultimately, that added a lot of value, because schools could really benchmark each other in ways they couldn’t before. Also, it allowed us and others to more clearly differentiate one school from another.
Selections: I know there has been some concern regarding coaching of students. Have you seen any evidence of that? Are there steps you have taken to prevent that?
Byrne: We have seen evidence of that. It has occurred occasionally and sporadically over the years. We have taken a number of steps to prevent it and correct it when it does occur. In the early years, there was no problem with this. Because it wasn’t that established. In fact, when I first surveyed the graduates, they didn’t even know what we were doing. No one knew I was creating a ranking, because I didn’t go running around telling everyone. In fact, I didn’t know what it would be. I didn’t even know it would be used for a ranking. I wanted to know what was going on at each school and how value was perceived by the graduates. Because no one had bothered to ask in any systematic fashion. Then it evolved into that [the rankings].
One of the first things we do is we have a professional statistician look at the results of the survey as they come in, and to do all sorts of analysis on the results. The analysis is primarily designed to basically cause red flags to be run up the flag pole and to spot irregular patterns in the way the surveys are filled out. Basically, the statistician can signal where the statistician thinks there might be a problem. And where there is, we can begin to investigate. That is number one.
Number two. Every survey has an open-ended question that allows graduates to just speak out on any subject whatsoever. When you have people who are 28 years of age and are discriminating people, if there was any sort of campaign, inevitably they tell us. You would be shocked at what they tell us. We find out everything from an affair with the dean to. . . They report everything. In every school, you have cheerleaders, and in every school, you have people with an ax to grind. You will find out unbelievable things. You will find out about the professors who just don’t give a damn—by name. You’ll find out about the placement offices that drop the ball. And you’ll find out if there was any sort of effort at all on campus to rig the results.
Besides all of that, I believe we also do not use one set of numbers for the full ranking. Meaning that we dilute the effect of any one year’s survey. We believe the quality of the school shouldn’t change too dramatically from one survey year to another survey year. If for any reason we don’t catch it with those other two—the study by the statistician as well as the comments made by graduates—either of which leads to an investigation by us if there is a problem. I believe we have thrown out schools. Then the results are some what mixed together so that if there is something going on, that lessens any kind of impact.
The other thing that we have done is, basically, we have issued a set of guidelines. For the last six years or four years, we have issued guidelines to schools, basically telling them that they are forbidden from communicating with the graduates about the survey in any way that might suggest gaming. And if they do so, we will throw them right out. And not only throw them out, but we will probably write a story and humiliate and embarrass the school in the magazine. That really does the trick.
Selections: How did you and Business Week develop the methodology—including the weighting system—for your rankings?
Byrne: In the early days, what happened is I went out and I kind of did it. I have to tell you what a labor of love this was. I actually made up all the surveys on my Macintosh at home. My wife and I stuffed envelopes in front of the television set and sent out thousands of things. I used every connection I had to make the corporations respond to the survey. Oftentimes begging them on the phone and in person to do so.
Selections: It’s unbelievable to hear that this is how the Business Week business school ranking survey started, given how influential it has become.
Byrne: Yeah, my wife and I would sit on the floor at night and literally fold surveys and put them in envelopes, address the envelopes, and send them out. And when they came in, they were not counted by computer. They were counted by hand. Because I really believed in the project. I loved the idea. I thought it was different. I thought it was important enough to devote this kind of time and energy to it.
We have a relationship with Harris [Interactive], the longtime pollster. And we hired a guy on their recommendation. His name was Matthew Goldstein. Matt was president of a company called the Research Foundation in New York. He had been a well-published academic in statistics. In fact, he coauthored four books on statistical math. He had been a consultant to AT&T and General Foods, among others. So, he helped me create the methodology for the whole thing.
Selections: So, when answering questions, students and recruiters answer with a measure of 1 to 10, correct?
Byrne: Yes. The questions are pretty interesting, if you look at them. In the early guidebooks that we published, we would literally show the results. We have been much more selfish with the data in recent years, because we don’t want people to know, frankly. The reason is to prevent gaming. But in the early days, we published the whole survey in these guidebooks. We conducted a couple of focus groups with graduates. I sat around in rooms here at Business Week, bringing a bunch of people in and asking them about their experience. Asking them what they thought other people should know. And the questions reflected my own prejudices and biases about what business schools should or shouldn’t be. The questions were always based on a 1 to 10 kind of thing. The questions really get at the heart of what a quality education is.
Selections: How do the rankings affect Business Week’s magazine sales?
Byrne: When we first came out with the ranking, I believe the first issue was the best-selling issue on the newsstand for that year. The second year, it was among the best. Traditionally, it is. But you can’t make much to do about that, and I will tell you why. The vast majority—and when I say “vast majority,” I mean over 90 percent of our circulation—is by subscription. So our newsstand sales for this magazine are fairly unsubstantial. So even when you talk about the best-selling issue on the newsstand, you’re talking about relatively modest numbers. I think that over 95 percent of people get this magazine by subscription. So, if there is 5 percent leverage, that is pretty little.
The truth is that in the early years, the project cost more than it made, for sure. And in terms of time. . . I mean if we had to justify doing this on the basis of time as an expense, certainly in the early years, I can say my editors would have prevented me from doing it. But I basically wouldn’t let them know how much time I was putting in on this, because I would have been discouraged.
So, I used my home time and my home Macintosh to print the surveys out, and the Xerox machine to reproduce them. My family, besides my wife, but some of my kids, would sit around the dining room table, stapling surveys together. I am serious.
Selections: What impact have the Business Week rankings had on management education?
Byrne: Overall, the impact has been good. But there has also been a negative impact. Let’s talk about the positive stuff first. Number one, the most important difference is this: There is more information in the marketplace today for people to make more informed decisions about where to recruit MBAs and for potential applicants to make a decision about what school suits them best. That is really invaluable, and I think that is the most positive contribution the rankings have had. Why do I say that? Because maybe the criticism would be, “Well, that is a lot of baloney.” Because what it does is, if one school is rated higher than another, willy nilly, the applicants will just apply there. That is not true. What the real yield of the survey results in is the ability to differentiate the schools from each other, which leads applicants to know more about each school and how different they are and to make the best match-up that can possibly occur. That is the real number-one value.
The number-two value is this: I do think that the survey undermines traditional prejudices in these educational institutions to not respond to the real concerns of the students and the companies who recruit them. It [the survey] completely undermines that. As a result, schools today are far more responsive to the marketplace than they have ever been and the schools are much better for it. The output at almost all of the schools—even those that won’t acknowledge that they have responded to the marketplace—is much better. I think there is almost total universal agreement on that. I have to say that one of the reasons is schools were fearful of having a poor showing in the survey. So, they went out and did their own focus groups with students. They listened to them more closely. They responded to their concerns. They went out and spoke more deeply with the major recruiters and brought them into the fold in a way that they had never done so before, and the educational experience has benefited greatly from that.
The third very positive thing about the survey is that there were basically, in the beginning, schools that put tremendous focus on teaching, and there were probably only a handful of them. And then there were schools that put tremendous focus on research and that was really the vast majority of them. Some schools were very successful at one or the other, and some schools weren’t. I think what has happened is there is much more of a “middle of the road” approach to education today in business management. Schools, because they were so heavily criticized—more often than not by us—for the emphasis on research and how it hurt management education, have taken teaching much more seriously. There are a number of schools that now factor the quality of teaching into their tenured promotion decisions. That was not true at all in the ’80s. Never. And there are some deans who in fact have been able to push through reforms that would have been impossible by basically using us as the foil. That has been helpful, because it has allowed people to change places that resist change pretty heavily.
Selections: Do you think this has had a negative impact on research?
Byrne: Yes. Thankfully, it has had a somewhat negative impact on research, but not negative enough. I wish it had a far more aggressive negative impact, because I feel there is still way too much negative research that goes on that is a total and complete waste of time for the professors, the schools, and I know for the corporations and the students. Total waste of time. The stuff that is being done is garbage today—not nearly as much as was done in the ’80s. But a tremendous amount of waste and a lot of stupidity out there. And I wish that there was a way for us to more aggressively get at it and ridicule it and force it out of the system, because it is a detriment to everyone. If you do research and it is read only by six people in the world, and it is the 110th study on discounted cash flow, why should anyone be doing that? It is insane. That research should not only not count for something, it should be a liability for the professor to do it. If you go before a tenure committee and you’ve done that kind of useless research, you should get an X. Not an F, but an X, meaning “get the hell out of here.” That is how strongly I feel about it.
Selections: I was just looking at some research myself that shows the business school rankings have changed management education by placing a greater emphasis on school image rather than on substance or true school quality. What do you think about this?
Byrne: I think that is true, too. And that gets us to a negative impact of the surveys. During the early goings—and even today, to some extent—there was the feeling that “All we have to do is hire a public relations agent to go out and get some good press for us, and that alone will do the trick. We don’t really have to do anything internally to change the nature of how we deliver the education or how we please our students. All we need to do is basically get some good PR and that will make a difference.” Well, that won’t make a difference at all in our survey.
Selections: What differentiates the Business Week survey from U.S. News & World Report’s survey and the other MBA rankings out there?
Byrne: I think U.S. News, because they survey deans—and really, quite frankly, my feeling is that most deans really don’t know what the hell is going on on their own campus, let alone what the hell is going on on other campuses. That is the unfortunate truth. Again, that is less true today than it was back then. Because back then, everyone was so internally focused and everyone was so in total disregard of the marketplace that they didn’t give a crap about what other schools were doing. They didn’t see them as competitors. Today, that is less so. Today, schools do see a competitive landscape.
Selections: And a large part of that is because of the rankings.
Byrne: That is exactly right. That is a good thing, but it is also a bad thing, because what it does is sometimes schools respond and do things for the wrong reasons because of image or reputation. Those kinds of things, which really aren’t very productive. And U.S. News, if you’re a dean at a school, you still probably don’t know much about what is going on at other schools today, so you may be more reactive to publicity that a school gets, so you may mark a school a little bit higher, particularly for the second-tier schools that you are even less familiar with. And that is a result of the “image thing.” That is the U.S. News & World Report problem.
The other problem with the U.S. News survey is that they rely on statistics supplied by schools. I can tell you the schools lie. And GMAC knows the schools lie, because they [GMAC] have tried to create standardized reporting, and the only reason they have tried to put standards on this is because they know that schools have exercised wide and disgraceful discretion in how statistics are reported, to make themselves look better. We rely on no statistics from the school at all. Never have. I don’t think a ranking should include how much a student makes and make that a key component of how high or how low that school should be. That doesn’t make any sense to me.
One of the reasons why the Business Week survey resonated goes back to that marketplace idea. We created a standardized form that allowed schools to compare themselves to each other on lots of different measurements. That is completely different from what the U.S. News & World Report approach was. It is completely different from what anyone else does.
Selections: To whom do you consider yourself accountable in the world of rankings?
Byrne: I think we are basically accountable to the readership: our customers. I have always maintained that the business schools are accountable to their customers, even though they didn’t believe that in the ’80s. We are accountable to our readership, our customers. I have to tell you that there is a very high standard of ethics and a very strong emphasis on fairness in our magazine. Do we get complaints? Yes. Do we respond to complaints? You bet. Do we get more applause than hate mail? Yes.
Selections: Are you concerned about the proliferation of rankings by various publications?
Byrne: Yes. I am concerned for a number of reasons. The first one is the more that there are, that sort of dilutes your own uniqueness and your own attempt to differentiate yourself—the magazine—in a very competitive marketplace. I don’t necessarily take the view that everyone jumping on the bandwagon is a form of flattery. I worry about it, because I think it allows the schools to just pick and choose whomever they get the best ranking from and then dilute the impact of a ranking or a couple of rankings and therefore weaken this marketplace that has been created by them. The more that there are, the less influential any one of them can be.
The second thing I worry about is this: When I first developed this stuff at Business Week, there was a strategic vision that I had for the magazine. I thought that the future generation of leadership would come from MBA schools, and I thought that if we could give them information that would help them to make one of the most important decisions in their lives, that is going to be very good for Business Week, because we are going to introduce them to the magazine at an age where we want to do that, and maybe if we’re lucky, we may be able to keep them. And as a result of all of the data collection we have done, we have—well we could always do better—but I think we have really made a concerted effort to cover business schools in a way that no one else has. If you go onto Business Week online, the data is extensive. It is unbelievable how much stuff we have on there. It is just incredible. No one even comes close, in terms of Q&As with deans and placement directors, admissions directors. Everything imaginable to help those two constituents I mentioned again and again. Even in the magazine, the rankings are an annual occurrence, because we do MBA rankings one year and executive education rankings the next. But all throughout the year, we are trying to do as many stories on management education as we possibly can. I think the goal is to have at least one story every two or three months in the magazine. That is to say that, number one, we believe this is important for us, and we believe it is important for business, and there is a great amount of interest in this stuff. We want this to be a so-called franchise for us. We want this to be a differentiator for us. That is another reason why I worry about everyone else jumping on the bandwagon—because that also erodes our ability to differentiate our magazine and the online site, because the more you have, the less unique you become.
Selections: What do you think the future holds for Business Week’s business school rankings?
Byrne: I think every time we do them, we re-examine what we’re doing. We try to make them deeper, bigger, smarter. We try to learn from all of the mistakes that inevitably get made. So, we are constantly changing everything here, too. I don’t know what can happen. We’ll get better data than we’ve ever had. We’ll scour it more closely than we ever had. We’ll get stories and ideas from that data that will have impact. And we just won’t let go of it, I don’t think.
Selections: So you don’t think the rankings are going anywhere?
Byrne: I don’t think they are going to die. If anything, I think they will be expanded to a greater universe of schools. And it is conceivable that while we’ve only ranked graduate programs, maybe we should be ranking undergraduate business programs, as well. That could be an area where we could make a big expansion. As well as internationally.
What I want to convey—because this is the absolute truth—is, I thought of this not because, “Oh, golly gee, maybe Business Week could sell a few more magazines.” I thought of this because it was a hell of a good idea and I was remarkably passionate about it. And I still am passionate about it. It is something I love and care about a lot. And we do have a great responsibility here, because it does have an impact, and that is why we do it. Okay. We do it because I sat around on the floor stuffing envelopes 13, 14 years ago. We do it because even today, I have a hand in doing it and helping it along and trying to rethink it every time we do it, and trying to make it better. And we do it because, damn, I think it helps people. I really do. Whatever the negative impact is that results from the survey, it is far outweighed by the positives.
I’m very passionate about this. If you were here right now you would see me pacing around my office with my arms flailing.
© Selections: Fall 2001