Understanding performance in test taking: The role of question difficulty order

Lina Anaya, Nagore Iriberri, Pedro Rey-Biel, Gema Zamarro

Producció científica: Article en revista indexadaArticleAvaluat per experts

3 Cites (Scopus)


Standardized assessments are widely used to determine access to educational resources with important consequences for later economic outcomes in life and to evaluate and compare educational systems. However, many design features of the tests themselves may lead to psychological reactions influencing performance. In particular, the level of difficulty of the earlier questions in a test may affect performance in later questions. How should we order test questions according to their level of difficulty such that test performance offers an accurate assessment of the test taker's aptitudes and knowledge? We conduct a field experiment with about 19,000 participants in collaboration with an online teaching platform where we randomly assign participants to different orders of difficulty and we find that ordering the questions from easiest to most difficult yields the lowest probability to abandon the test, as well as the highest number of correct answers. We obtain similar results when exploiting the random variation of difficulty across test booklets in the Programme for International Student Assessment (PISA) for the years 2009, 2012, and 2015, which provides additional external validity to the experiment. We conclude that question difficulty order in tests has important policy implications for optimal test design and performance. It additionally may have important implications for ranking candidates, as well as for the evaluation and comparison of educational institutions and systems.

Idioma originalAnglès
Número d’article102293
RevistaEconomics of Education Review
Estat de la publicacióPublicada - d’oct. 2022
Publicat externament


Navegar pels temes de recerca de 'Understanding performance in test taking: The role of question difficulty order'. Junts formen un fingerprint únic.

Com citar-ho