Introduction

Medical school admission processes need to demonstrate the validity of their scores’ interpretations, since they are high-stakes assessments.

Objective

To apply an integrative model using Messick’s and Kane’s frameworks to obtain validity evidence for a medical school admission process.

Method

An analytical observational study was performed at the Autonomous University of San Luis Potosí Faculty of Medicine in Mexico, wit h the 2013 and 2014 cohorts. The admission process included a psychometric test (PT), a general knowledge test (KT), and EXANI-II. The model was applied in two steps: 1) uses and interpretations argument, and 2) validity argument, which has three stages: instrumental validity, interpretation and decision verification, and action usefulness. Hypotheses were established at each stage using Kane’s inferences, which were supported using Messick’s sources of evidence.

Results

The results of 1,373 applicants from 2013 and 1,554 from 2014 were analyzed, as well as the academic performance of the 145 students admitted in each cohort. A general intelligence factor (g) was identified as influencing the results of admission process. KT explains 15% of the variance in the first-year grades (2013), PT and EXANI-II have no impact on first-year’s academic performance (2013 and 2014). There was a correlation among some subcomponents that measure a similar construct. There was a difference in Biology results (KT) between men and women. Students’ degree of satisfaction with the admission process is adequate.

Discussion and conclusions

The sources of validity evidence provide relevant information to support and improve the admission pro cess, more data and an interdisciplinary team to retrieve more sources of quality evidence are needed. It is recommended to use welldefined frameworks in these processes and plan data collection and retrieval using these models.