Home | Letters to the Editor | www.gmac.com
To print this story, please select “Print” from the File menu of your browser.
Selections Interview with Parminder Bahra, Senior Adviser on Education Projects, Financial Times
Selections: What is your role with the Financial Times rankings?
Bahra: One is to help with the methodology and to put together questionnaires. Second is to actually compile the information and put together the rankings themselves.
Selections: The 2001 rankings were the third Financial Times MBA rankings, and the rankings are annual?
Bahra: Yes. We started in 1997.
Selections: What are the goals behind the Financial Times business school rankings?
Bahra: Ultimately, to provide a basis on which we can compare MBA programs and the institutions that run them on an international setting. And it is ultimately to provide information for users. Also, with the rankings goes a series of articles. So, we like to think that existing or current MBA students and alumni or graduates might also find things of interest here, as well as faculty and other members of the school.
Selections: Whom do you consider to be your main audience for the rankings?
Bahra: Potential MBA students. Having said that, we have the rankings themselves, which appear as a survey once a year. But we also have a Monday page in the paper, which also provides information for not only potential MBAs but also the graduates and faculty and members of the business school community.
Selections: How are the Financial Times rankings different from the other MBA rankings out there, particularly those published by Business Week, U.S. News & World Report, and the Wall Street Journal?
Bahra: Each ranking addresses different issues. You can change the angle of a ranking according to the questions you ask. If you look at the different rankings across the board, they are all different—some quite radically different than others; some are quite close. There is an element of the history and the legacy of rankings. For instance, when the Business Week ranking was only U.S.-based, it was easier to run a survey of employers. I think if you’re trying to measure the reaction to business schools in the context of employers internationally, it becomes more difficult. In that sense, we provide a series of criteria that are slightly different, and when they are put together, you get a slightly different ranking. And it is up to the reader to come to a decision as to how to best use the information there.
Selections: Is there one survey that you feel you are most closely aligned with?
Bahra: I think it kind of changes. There are certain criteria that we have seen other publications bring on board. Also, because we have a single international ranking, I think that makes us somewhat unique. Where the ranking has already existed for the U.S., it is then difficult to apply it internationally without changing the methodology in a big way. In that sense, it is difficult to say we are quite close to one or the other. I think there are certain criteria that are universal, for instance, salaries. Everyone knows that the purchasing power of the MBA is pretty much determined through the salary. I think that is where you will find a great deal of uniformity across the business school rankings put out by the different publications.
Selections: Creating and conducting rankings seems to require a lot of time and effort. How does the Financial Times benefit from this time and effort?
Bahra: Ultimately, it is to be seen and involved in the business community. It is a financial paper, so it has to be seen to be operating in areas where finance and business are of interest. I think it is an extension of the work that we have done in the past. It just seemed to make a lot more sense to have a slightly higher profile, and rankings are one way to increase your profile within an area. And that is one major part of the rankings—to raise our profile in this community and, hopefully, to be accepted. Given the feedback we get, we do feel very much as part and parcel of this community, and the information we provide is of use.
Selections: You said that prospective MBA students are the main audience for your rankings. How should they be using the information provided in your survey?
Bahra: We produce the tables themselves that list all of the individual criteria. And we have a few tools on the Web that can help narrow down choices of schools a prospective business school student might want to look at. But they ought to look at individual criteria, and students should consider the reasons behind what they are doing: Why they want to get an MBA; the areas they expect to specialize in; what they expect to get out of it. I think some of these criteria will indicate the schools that are good in the areas they are interested in, schools that they might want to look at further. Our ranking alone ranks 100 schools. If you were to find out information on 100 schools, it would take you the best part of a couple years. It’s a bit like narrowing down and fine-tuning your search for business schools.
Selections: How should the schools and the business community, including the companies that recruit at these schools, be using the information published in the Financial Times survey?
Bahra: Again, there are certain criteria that potential employers might want to look at. Say, for instance, the research criteria. They might want to look at the research pedigree of a particular department or particular school or particular program. And it could be an indication of the cost of graduates from particular schools, as well. Although, I think there are probably other data series and other information that is more accurate that these institutions could use to gauge that aspect. But again, I think it is ultimately to look at the areas of specialization of particular schools. Especially if you’re a North American employer, you may not know a great deal about the European business schools. It’s an area where we could provide some new information for companies that might not have that information readily at hand.
Selections: Do you see the schools being able to use the information to see where they are doing well and where they might need to improve?
Bahra: I think very much so. One of the requests that we get from business schools is for a more detailed breakdown of information published in the survey. That is now something that we are trying to put together in a more formal way, so we can provide the business schools with a lot more detail about and an explanation of the data. I definitely think it is of use to the business schools. Obviously, some of the data on the other business schools is confidential, but some of the aggregates that we provide them with, information on the movement of alumni or graduates, is definitely been of use. We have a criterion called “international mobility,” which looks at the movement of graduates before, during, and after the MBA. I think that is the sort of information that a business school just wouldn’t have access to, short of actually doing a report on the level of our survey, which is about six months’ work. That kind of information is very difficult to gather. And then to actually place it in an international setting so that you can compare it with your neighbors or your major competitors. . . Without a doubt, that kind of information would be of use to the business schools.
Selections: How did the Financial Times develop the methodology—including the weighting system—for its survey, and why?
Bahra: I was brought in just shortly after the initial round of discussions that Della Bradshaw [editor of the Financial Times business school rankings] had. So, when it comes to the 1997 ranking, the very first, I probably didn’t have as much of an input into that as Della did. But subsequently, there has been a degree of fine-tuning. We are now three years into the survey. The first round of discussions took place between Della and some of the senior faculty and the deans of the business schools. The deans and senior faculty were asked to put forward suggestions for the ranking criteria and areas of importance. These were then narrowed down to fit a ranking from a statistical or methodological perspective. It’s one thing to have a series of questions that you would like to ask, and it’s another thing to be able to capture the results in a numerical format. So, we had to basically find the criteria that we could measure. So, the criteria now used reflect those that are measurable. There is a whole host of other criteria that we would love to capture, but it’s not always the case that we can capture those.
There is continuous dialogue with the business schools, the faculty and the deans, as to the different criteria.
Selections: I know that research is one of the main elements you look at. It’s career progression, diversity, and research, correct?
Bahra: Yeah.
Selections: The focus on research is interesting, given that the other surveys do not put much, if any, emphasis at all on research. How do you define research quality, and why include a measure of the research quality of a school?
Bahra: Our research rating looks at the publications of faculty within each department. We have a selection of 35 journals, and we look at a number of the publications that each of the faculty has had over the last three years. We look at it partly because the business schools—the faculty and the deans—suggested that this is something that is very important. So, it’s not something that we just plucked out of the air, as it were. These are suggestions from the business schools themselves. Ultimately, if you want to look at the MBA experience, there has to be some element of the newest ideas that are coming out. We want to capture the sense of an individual being immersed in a department as much as get the actual qualifications of the MBA. And these students are being taught by faculty members that are on the cutting edge of research. There may be an element of assumption here, but we would expect that this would be passed through to the teaching, as well. We admit immediately that this is a very difficult thing to measure. But as a proxy, it is a fairly accurate measure. It is a measure also that, as far as we can see, the majority of business schools are happy with, although there may be questions about the selection of journals. The Americans say it is pro-European, and the Europeans say it is pro-American. So, we kind of feel like we are between a rock and a hard place.
Selections: I understand you now include a PhD rating. What is that, and how does that factor in?
Bahra: The PhD rating is looking at the PhD students. Again, this is a good indicator of the research orientation or the degree of research that is taking place within a department. We look at the output of PhD students and look at where they are going after graduation. PhD graduates that are going on to take full-time positions in leading business schools is a good indication—it’s a good proxy—for the research activities within a department. It’s a five-percent weighting. Again, we look at the number of doctoral graduates for the last three academic years. And we give additional weighting for those students that actually take up faculty positions with one of the top 50 business schools from one of the previous year’s rankings.
Selections: Have you received any feedback on this from the smaller schools that might not have a doctoral program? Has there been any feedback that this tends to favor larger, research-oriented schools?
Bahra: To a degree, yes. There are obviously smaller business schools or those schools that don’t have doctoral programs that would prefer not to have that included. I think, again, it is an indication of the research activities that take place in a department. If you don’t have a research program for PhD students, then I think it’s an indication that the department may, compared to the bigger schools . . . the research environment is perhaps not so much geared toward the generation of new ideas.
Selections: Why does the Financial Times place so much emphasis on salaries?
Bahra: Ultimately, because it is such a good indicator of the value or the purchasing power of an MBA. The price mechanism, or the salaries, is fairly comprehensive. We feel that if you are going to embark on an MBA, you’re looking at investing anything up to $100,000 to $150,000. It [salary] has to be an important aspect of the decision-making process. Also, it is a very good indicator of the value of the degree from the various programs in the business community.
Selections: I know you go into this in some detail when discussing the methodology on the Financial Times Web site. But can you talk about the ways in which you standardize the salaries for comparison? Because you are looking at salaries across the world, where there can be quite a bit of variation in salaries, how do you make sure you’re comparing apples to apples?
Bahra: The first thing we do is to take all of the salary data and convert it to purchasing power parity in U.S. dollar rates. Purchasing power parity is a World Bank series that takes into account different price levels for living standards within countries. We take out any price differences. That is the first thing we do. Then, in the table itself, we publish two salaries. One is the salary today and one is weighted salary. The salary today is a straightforward salary. Where the sample is big enough, we will take away 15 percent of the figures on either side of the distribution. And that is to get rid of the extreme figures, getting rid of salaries such as those of students earning $3 million or $4 million. That would skew the figures somewhat. Where the sample is quite small and we cannot do that, we will put in a median value. The one that we actually use in the rankings is actually the weighted salary, where we take into account the sectoral salaries. What we do is divide the salaries into five sectors: IT, consultancy, finance/banking, telecom, and the consolidated other, which is all of the other salaries put together. We then calculate the average salaries within each of those sectors. We then take the global percentages for people who were in each of the sectors. This is much easier to explain with a diagram.
The salary data is supplied through the questionnaires supplied by each graduate. We ask the student to return salary figures for just prior to the MBA, upon graduation, and now.
Selections: What information do the schools supply themselves?
Bahra: There are three separate questionnaires that the schools complete. They provide us with information such as the employment after three months; the number of international students, faculty, and board members; the doctoral rating; and the number of faculty with doctorates.
Selections: Are there any steps you take to audit that information or make sure that everyone is supplying standardized data?
Bahra: This year, we will be looking into auditing the figures and maybe getting one of the consultancies to check with the business schools and actually audit the information formally. At the moment, the process is that we do some comparisons with previous years, just to make sure that they are in line. If any figures look like they could possibly be out of line, then we will go back to the business school to check to make sure it is correct. That is our formal check at the moment. But there is no formal auditing procedure at the moment. We are looking to introduce it in the next year or so.
Selections: You’ve been publishing your survey for three years now. What kinds of reaction are you receiving from schools, students, and the international business community?
Bahra: From the business schools, it depends where they come [in the rankings]. The higher they are, the happier they are. We’re definitely getting a lot more coverage than in the past. Definitely, a lot of third-party media organizations are interested in our rankings and are using and mentioning them. If I were to have done a search on the Web in Google two years ago for the Financial Times ranking, there wouldn’t have been a great deal of hits coming up. But if I do it now, there are many third-party organizations mentioning the Financial Times rankings.
We will get suggestions for improving or tightening up the table. Again, we’re only three years into the survey. There is still a degree of fine-tuning that must go on. Our research rating has changed somewhat from the first year, when we ran a peer-review-type rating. We asked the business schools to submit their top two papers in a particular area, and we got faculty at business schools to rate them. That proved to be not as successful as we would have liked.
Selections: Why is that?
Bahra: I just think it is very difficult to control. One, trying to get faculty to make a judgment on one or two papers is difficult. And it’s probably not the best way to make a judgment on a particular department, because most departments would want to submit six or seven papers. Most national research rankings are based on many articles, not just one or two. To ask faculty to look at 10, 15, 20 articles and rank them is, one, unfair, and, two, they don’t have to do it. I don’t think they were willing to spend that much time on it. So we looked at the research rating again, and that is why the new system has come about where we look across the journals. The first attempt didn’t work.
Selections: What are some of the other changes the Financial Times has made to make the business school rankings more transparent, accurate, and fair?
Bahra: This year, we included the international board or faculty. We looked across the board of governors or the boards within the business schools. We feel that, in a way, determines the learning environment of the program. Again, these are suggestions put forward by the business schools. There is some criticism . . . it’s actually more of a point that is made that diversity reflects the European business school strengths much better than they do the U.S. strengths. I think there is an issue there. I just happen to think that the European schools do better because they are from different, individual nations. So, the schools will provide a greater international influence because the nature of people moving from one country to another within Europe.
Selections: What are some of the biggest challenges you face in conducting this international ranking?
Bahra: The biggest issue is finding criteria that you can use internationally. As I said earlier, there is a criticism from American business schools that the diversity aspect is biased toward the European business schools because in America, you might consider a West Coast or East Coast school. Whereas in Europe, you may consider a school in the U.K. and a school in Spain. Using our methodology, looking at two European schools counts for more. So, finding criteria that you can actually employ internationally is one of the trickiest aspects.
The actual physical process of putting together the survey is much easier now than it was three years ago, and that is primarily because we use the Web a lot more. For the last three years, we have been entirely paper-based. So, everyone received a paper questionnaire. But we are moving almost all of our operations toward making them electronic. The actual procedure is actually getting much easier. And I think the business schools, as well, are much more geared toward having information ready for the surveys. I know from having spoken to so many business schools that it is something that takes up so much of their time. They have so many requests. But equally, they are much sharper now, in terms of getting their information together. Bear in mind that most of the publications ask them for different information, because they have different criteria. I’m quite impressed that they can actually do that. But that is something that is actually making our lives a lot easier.
Selections: There’s no denying that the business school rankings have had an impact on management education. How do you see the Financial Times rankings affecting business schools and management education?
Bahra: That’s a tricky one. But I think it’s probably from the international perspective. The fact that INSEAD and the London Business School are talked about in the same breath as the top American schools. The point of running an international survey is to understand the marketplace from an international perspective, and I think that now American employers and American business schools are more aware of what is going on in Europe and vice versa. In that respect, I think that is what we have added to the community.
Selections: As I’m sure you know, there has been some criticism of the rankings throughout academia. Some argue that the rankings have actually changed the nature of management education by placing a greater emphasis on school image and reputation rather than on true school quality. Do you feel this is true, and do you feel the Financial Times has contributed to this?
Bahra: I think it has always been there. We’re not necessarily adding or taking away anything. I think there has always been that need or that desire to form some sort of ranking. And I think it has always been there in one shape or form. And there is a lot of criticism of ranking tables, but I remember that 15 years ago in the U.K., there was one criterion that people looked at when they wanted to look at the performance of a university, and that was the number of students to the number of staff. Now I happen to think that 15 years ago, that single criterion was probably more misinforming than our 20 criteria today. And if anything, it is much better to have 20 criteria today and be able to make your own judgment and assessment than having just a single criterion. As to whether people are taking the rankings more seriously or less seriously or whether the rankings are changing the focus of the business schools, I think that is possible. But I think there are many other factors that will change a business school. I don’t think rankings are the only thing changing the focus of the business schools or their directions. Ultimately, the rankings should be there. The rankings can only collect information on what is asked. If a series of questions that you are interested in are not asked, then that table is probably not as relevant to you as the next table. I think people need to recognize that. I don’t think there are any right or wrong tables or that anyone’s tables are any better or more correct than others. It’s up to the individuals reading the tables to look at the criteria being offered. They should be making decisions on that basis. I’m fairly confident that a dean won’t change the direction or his or her vision for the business school on the basis of one set of bad rankings.
Selections: What is the purpose of—or the need for—publishing ranking numbers at all? Why say this school is number 1 and this school is number 2 and so on? And, really, how much difference is there between schools that are just a few ranking numbers away from each other?
Bahra: I think that is a valid point. If you look at the distribution of points throughout the table, you’re right. Outside of the top 10, a small number of points will move a school many places in the ranking. In a way, I think people like to have a ranking of one to 100. If I were speaking as a true statistician or a true researcher, I would say that given the margins for error within the calculation of figures, then we could probably have a system where we just tier schools: first tier, second tier, third tier. But I think the ranking provides a degree of interest to the readers because there is this difference. From my perspective, I don’t think there would be a huge difference in using tiers. But I think the current system provides for a more interesting read because you have that kind of level of granularity. It also provides a degree of comparison for the schools. If you just had tiering, then if you were comparing one school in one tier to a school in another tier, it may not be quantitatively as obvious.
Selections: How do you decide which schools to focus on in your survey?
Bahra: The minimum requirements are that the school is accredited and the program has been running for at least three years. We put together a list of schools that we consider to be the top schools. Again, they were put forward to the deans and senior faculty at the business schools. We asked around to see which countries were seen as the most important countries for business schools, and we put together a list from there. Now, it is getting bigger and more programs are being accredited. We are always looking at new schools. Initially, it was just a case of asking the business school community which schools they thought were the top schools.
Selections: So, if a new school becomes accredited this year, you would then look at this school and decide whether to include it?
Bahra: Yes.
Selections: And you would get input from deans and senior faculty on whether that would be a good school to include?
Bahra: That is right.
Selections: To whom do you consider yourself accountable in the work of producing rankings?
Bahra: I suppose ultimately it should be the business schools, in the sense that it is only with their cooperation that we can actually put this table together. If this business schools felt that this wasn’t worthy of publication, then they could always withdraw their resources and services. So as long as we can keep the business schools’ confidence on board, then I think we are okay. We can carry on running this and they will cooperate with us. They could always choose to withdraw their services and the information they provide.
Selections: How are the results of the rankings reported?
Bahra: In the newspaper and on the Web.
Selections: Do you also produce a book with the rankings information?
Bahra: Not at the moment. It may be something that we put together in the future. But it’s not something that we are doing at the moment.
Selections: Do you have information on how the rankings impact newsstand sales for the Financial Times?
Bahra: I don’t know about paper sales. But I do know that when we put the information out on the Web, it makes a huge impact in terms of the amount of traffic that we generate. I’m sure it has an impact on the paper sales. We probably get as many requests for information on back issues as we have people coming on the day the rankings are published and buying the paper. It’s definitely something that has a degree of inertia. We will still get requests for information in more detail.
Selections: How do you feel about the proliferation of rankings? Do you consider the other rankings to be competition?
Bahra: It’s competition in the sense that they will always make us think about our criteria and whether they are the best criteria that we can put together. And whether we are capturing the data in the most accurate fashion. There are always new ways of capturing information. And there are always new sets of data coming out. It’s competition in that sense. Ultimately, so long as the methodologies are secure and the information that is being put out is well researched, we have no objections and ultimately welcome the other surveys. Just because they provide more details for everybody. The other rankings keep us on our toes. Occasionally, we’ll see criteria that we haven’t seen before, and I will have to go off and start thinking about whether it is something we would want to incorporate.
Selections: What do you think the future holds for business school rankings in general and the Financial Times rankings in particular?
Bahra: Because of the proliferation of rankings and proliferation of the information that is required, I think I do see a time when there will be a formalized set of information provided by the schools. I think that because of the work required by the business schools, if there are too many people requesting information, it may be that they will say, “Here, we’ll provide you with this series of data, and do with it what you can.” It’s close to what pretty much everyone asks for. But how far away from that they are, I’m not sure. They seem to be able to accommodate our requests. I know we ask for more and more information every year, because we are trying to find new things to talk about. From my perspective, it would almost make sense to have a central organization to collect and coordinate the information. How that would happen, I don’t know. I don’t know who would be willing to stick their neck out and say, “Okay, we’ll look after the coordination.”
© Selections: Fall 2001