Investigating the Parameters Accounting for Language Proficiency: the case of an English Language Test used for Academic purposes

Abstract

In language testing it is essential to understand the validity of test scores (Kane 2013 & 2016). However, the focus on whether test items/tasks lead to the target language use expected (Bachman and Palmer, 1996) has been somewhat limited especially in validation studies undertaken by examination boards. The present study explores the (content) validity, e.g. how different linguistic parameters account for language proficiency in high-stakes international English examination for Academic Purposes (the Pearson Test of English Academic - PTE Academic). Various measures of proficiency were taken into account for Writing and Speaking. Results showed that vocabulary mainly accounts for language proficiency and can be used as a predictor variable for the Writing and the Overall Scores in the test. Fluency could also predict some of the variability in the Speaking Scores. The paper contributes to ongoing research on how various language measures can discriminate between levels of proficiency and proposes a statistical model (regression analysis) that can predict speaking and writing scores. Our research intends to provide feedback to test developers or other stakeholders regarding the PTE Academic and offers research and methodological recommendations for the study of content validity of high-stakes tests in other contexts.