A non-parametric test for comparative evaluations of students’ performances

Comparing student performance is an effective way of educational evaluation. One crucial tool for evaluating the efficacy of educational practices concerns a statistical analysis applied to student performances before and after a course (Dunlosky et al., 2013). This analysis serves as a means to gauge the impact of instructional strategies on students’ learning outcomes and to inform pedagogical decisions. The relevance of such statistical examinations lies in their ability to provide empirical evidence of the effectiveness of specific educational interventions. By comparing students’ scores before and after a course, it becomes possible to assess not only the extent of students’ academic growth but also the influence of specific instructional techniques and other types of factors on their learning trajectories (Cook and DeMets, 2007). In the literature, some techniques related to data mining or algorithms have been applied to assess the level of education and predict students’ performance (Zhang et al., 2021; Tomasevic et al., 2020). In this work, the approach is based on a non-parametric inference in an experimental framework where the response variable is the student’s score variation (score after the course minus score before the course). The goal is to investigate the main and interaction effects of crucial factors such as the type of school (public or private) and the didactic approach (traditional or experimental). The problem is defined in the framework of a regression analysis and the solution is based on the permutation approach. The proposed test, unlike the parametric approaches, does not require the assumption that the distribution of the response follows a specific family of probability laws. Such a test is powerful, especially (but not only) when the typical assumptions of the parametric approaches (such as the normality of data) are not satisfied and the parametric tests are not reliable (Pesarin, 2001). In other words, the probability of correct decision in the test of hypothesis is usually higher than that of the parametric competitors. Furthermore, this method is more flexible and robust concerning the parametric tests and it could be considered in contrast with (but not only) stepwise regression (Harrar and Bathke, 2008). A permutation test on the goodness-of-fit of a multiple regression model is applied. Such a test is conceived as a multiple test on the significance of the single regression coefficients (main effects and interaction effects), and the proposed solution is based on the combination of the -values of the partial tests. The empirical studies in which this nonparametric methodology has been successfully applied are several and heterogeneous (Bonnini et al., 2023; 2024a; 2024b; Alibrandi et al., 2022). The rest of the paper is organized as follows. Section 2 presents the statistical problem and the methodological proposed solution. The case study is described in Section 3 and Section 4 contains the overall conclusions.