Many researchers have indicated the energy performance gap (difference between actual and predicted energy used in buildings), not only on an individual building level, but also on a building stock level. For policy makers it is important that predictions are correct on an building stock level to make them a useful tool to predict the effect of their proposed energy saving policies. Often not all input parameters for building energy simulations are known (e.g. insulation rates are often only possible to determine with destructive inspection or extensive measurements), therefore assumptions are made (e.g. assumptions for insulation rates are often made based on construction year). It is expected that a large part of the energy performance gap on building stock level are caused by incorrect assumptions of the unknown parameters in the building simulations. Previous research has shown that automated calibration of the assumptions on building stock level seems a promising method to reduce the energy performance gap and therewith make building energy simulations on building stock level a more reliable tool for policy makers. The previous research about calibration on building stock level was a proof of concept and still needs some improvements before it can be applied in practice. One of the aspects to improve the method is to determine the most suitable objective function and the most suitable optimization algorithm. In this paper we compare different objective functions (e.g. Root Mean Square Error, Mean Absolute Error, Sum of Absolute Errors). Next to that we compare different optimization algorithms (e.g. Genetic Algorithm, Particle Swarm and simulated Annealing Algorithm). For the comparison of the objective functions and the algorithms the former Dutch calculation method to determine the energy label in dwellings is used, in combination with the SHAERE database and data from the Dutch Statistics. The SHAERE database contains all input information on individual dwelling level to calculate the energy label of a dwelling of almost 2 million dwellings. The Dutch Statistics database contains the individual annual energy use of all dwelling of the Netherlands and can be linked to the SHAERE database.