Verification of traffic micro-simulation model calibration procedures: analysis of Goodness–of-Fit measures

The problem of calibrating traffic simulation models comes within the framework of “no free lunch” theorems: solutions to the methodological issues arising when setting up a calibration study cannot be posed independently. In other words, the choice of the algorithm to use, for instance, depends upon the parameters to calibrate, upon the measure of goodness of fit (GoF), and, of course, upon the micro-simulation model applied. This calls for methodologies able to check the robustness of a calibration framework as well as further investigations of the problem, in order to identify possible “classes” of problems to be treated using the same approach. Therefore in the present work, first we describe a general method for verifying a traffic micor-simulation calibration procedure, based on a test with synthetic data. Then we investigate the influence that the choice of a particular GoF measure may have on the results of a calibration exercise. In all, 16 GoF measures are analyzed by using them to create and visualize the objective function of eight different calibration configurations. It was thus also possible i) to test the hypothesis presented in Punzo and Ciuffo (2009) to calibrate different parameters independently, on different time series of measurements and ii) to check the effect on the calibration itself of random errors in traffic measurement. Results show the importance of verifying the calibration procedure with synthetic data before using real measurements. In addition they highlight limitations of some GoF measures as well as give major insights into the topic.