The article was written by the eminent statistician Frank Schmidt and his students at the Tippie College of Business, University of Iowa—and yes, we at GMAC love the title. This article offers useful insights that deans, admissions personnel, and statisticians should find helpful. It also underscores the value of participating in the GMAC Validity Study Service (VSS).

One of the article’s key concepts is that the methodology for evaluating a test’s validity must match the desired question. For admissions decisions, the question should be “what is the correlation between the GMAT and outcomes for the applicant pool?” The entire applicant pool is rarely admitted, though. The subsequent correlations based on enrolled students are not only restricted in range, they are also distorted and seriously biased downward. This restriction-in-range issue is commonly taught in statistics classes, and I think it is something that all test users should know. Frank’s article provides some excellent graphs to illustrate the problem.

The solution is to make a statistical adjustment to the observed, enrolled-student correlation—using the ratio of the variance of scores for the enrolled students to the variance of scores for the applicant pool. Frank’s refinement incorporates a true score estimate in computing that ratio.

In the AMLE paper, Frank and his students reanalyzed the data from a meta-analysis of GMAT validity conducted by Nathan Kuncel and his colleagues (also published in AMLE), using their new refinement to adjust for restriction in range. (In the interest of full disclosure, I must add that Frank and Nathan served on the Council’s Management Education Research Institute’s (MERI) review board and Frank currently sits on the GMAT Technical Advisory Committee.) For Nathan’s data, the average correlation between the GMAT Total score and first-year grades increases from .47 to .51. Applying Frank’s technique to our own more current data published in Educational and Psychological Measurement, the average multiple correlation for GMAT Total score and undergraduate GPA increases from .53 to .56. Under the reasonable assumptions of Frank’s method, the higher values are the right values for statisticians to report.

Each of these high averages, however, represents hundreds of different individual validity studies. In some programs the Quantitative score out-predicts the Verbal; in others, the opposite is true. In some programs work experience and undergraduate GPA add greatly to the prediction; in others, they do not. Here is where the GMAC Validity Study Service adds important value. Participating in this free service will help you better understand the relative strengths of different admissions criteria, including the GMAT, and which combinations of criteria work best for your program. You provide GMAC with your enrollee data, including your own set of predictor and criterion variables. GMAC analyzes your data—and that of all test takers who sent their score reports to your school. Finally, GMAC produces a tailored VSS report that is designed by admissions personnel and psychometricians. The report provides a wealth of useful data and charts. It also makes it easy for you to articulate why you turn down applicants, establishes evidence against charges of bias, and helps inform your future data-oriented decisions.

You’ll find a typical case study on the GMAC VSS website that involves the Max M. Fisher College of Business, The Ohio State University. Graduate programs director David Smith and associate director Rob Chabot comment on how the small amount of time they invested in the Validity Study Service provided a big payoff. They were particularly impressed by the VSS chart showing how the GPA and undergraduate GPA work together in their program. Diverse VSS participants have been pleased to document the quality of their special admit decisions, or were intrigued by the differences they observed in the contributions of various admissions criteria. The most common feedback is that the reports reinforce some gut feelings, and identify others that are simply wrong.

I urge you and your staff to seriously consider participating in the GMAC Validity Study Service, and remind that such validity studies should be done every three to five years. For more information, see www.gmac.com/vss.

Lawrence M. (Larry) Rudner, PhD, MBA is vice president of research and development at the Graduate Management Admission Council. He can be reached at lrudner@gmac.com.