The IEC 61853 standard series "Photovoltaic (PV) module performance testing and energy rating" aims to provide a standardized measure for PV module performance, namely the Climate Specific Energy Rating (CSER). An algorithm to calculate CSER is specified in part 3 based on laboratory measurements defined in parts 1 and 2 as well as the climate data set given in part 4. To test the comparability and clarity of the algorithm in part 3, we share the same input data, obtained by measuring a standard photovoltaic module, among different research organizations. Each participant then uses their individual implementations of the algorithm to calculate the resulting CSER values. The initial blind comparison reveals differences of 0.133 (14.7%) in CSER between the ten different implementations of the algorithm. Despite the differences in CSER, an analysis of intermediate results revealed differences of less than 1% at each step of the calculation chain among at least three participants. Thereby, we identify the extrapolation of the power table, the handling of the differences in the wavelength bands between measurement and climate data set, and several coding errors as the three biggest sources for the differences. After discussing the results and comparing different approaches, all participants rework their implementations individually and compare the results two more times. In the third intercomparison, the differences are less than 0.029 (3.2%) in CSER. When excluding the remaining three outliers, the largest absolute difference between the other seven participants is 0.0037 (0.38%). Based on our findings we identified four recommendations for improvement of the standard series.