Every day, requests for our research reminds us that deans and other school professionals rely on hard data from GMAC to evaluate incoming classes, stakeholder perceptions, recruitment efforts, and to help inform a wide range of strategic plans. One piece of hard data that deans and admissions personnel find particularly useful is a quantitative analysis of the admissions process.
The GMAC Validity Study Service (VSS) can identify the relative contribution of different admissions criteria toward predicting first year academic success. This customized service, free to GMAT®-using schools, allows programs to identify their own predictor and criterion variables. You can, for example, evaluate your at-risk judgments and identify for your program the relative importance of undergraduate grade point average, work, and GMAT scores. VSS is easy to implement, and turnaround time is quick. For details, see www.gmac.com/vss.
Nothing supports decision-making like hard data. Absent such data, however, one can often derive meaningful insights via validity generalizability theory―the notion that the results of many validity studies will generalize to a local program. Evoking validity generalizability theory is moving from hard data to semi-firm data—like going from grated cheese to a Gruyere.
An enormous amount of recent validity data supports the use of the GMAT® across business programs. In fact, seven studies have been published in peer-reviewed journals in the past four years, including a meta-analysis involving 273 business programs and 41,338 business students who took the GMAT® in recent years. From these studies, one can safely conclude that the GMAT® works very well across the entire ability spectrum, especially where quality data is most needed: for examinees in the middle of the ability distribution.
To evaluate validity generalizablity, one has to determine if the studies involved programs that are like yours, how recently they were conducted, and whether we are talking about the same test. Studies in other programs using the older paper-and-pencil form of the GMAT, for example, may not generalize to programs that use today’s advanced computer-based form of the test.
Recently, concordance tables have been offered as a way to link to hard validity data. The idea is to map an average or a range of scores on one test to an average or range of scores on another and then to use the corresponding scores interchangeably. This logic has a major fallacy along with built-in disadvantages that can cause misinterpretation. Concordance is about averages, not individuals; admissions is about individuals. Assuming a good concordance study was conducted―equally motivated test-taking, for example―such a table might be useful for research where one often considers averages. However, as with any average, there is a wide range of underlying scores, and a wide range of scores on the first test properly correspond to any given score on the second.
The National Council on Measurement in Education, the leading organization in psychometrics, assigned a title to its 2007 presidential address that cuts to the heart of the problem: “The Concordance Table: An Invitation to Misuse Test Scores.” Substituting an average for a real score is not fair to anyone. Indeed, this would be moving from grated cheese to Swiss cheese―full of holes.
The bottom line is that it’s best to have hard data specifically for and about your program. Request a validity study and see for yourself how well the GMAT contributes to your admissions process. It’s the kind of hard data that is available to you at no charge and is something that no validity generalizability theory or concordance table can match.
Lawrence M. (Larry) Rudner, PhD, MBA is vice president of research and development at the Graduate Management Admission Council®. He can be reached at lrudner@gmac.com.