Despite progress in multiobjective evolutionary algorithms (MOEAs) research, their efficacy in real-world scenarios remains unclear. This article introduces a diagnostic benchmarking framework to evaluate MOEAs, comprising (1) flexible MOEA construction software, (2) performance evaluation metrics and (3) real-world applications for benchmarking, reflecting diverse mathematical challenges. Utilizing this framework, NSGA-II, NSGA-III, RVEA, MOEA/D and Borg MOEA were evaluated across four applications with three to ten objectives. Collectively, the four applications capture challenges such as stochastic objectives, severe constraints, nonlinearity and complex Pareto frontiers. The study demonstrates how MOEAs that have shown strong performance on standard test problems can struggle on real-world applications. The benchmarking framework and results have value for enhancing the design and use of MOEAs in real-world applications. Further, the results highlight the need to improve the adaptability and ease-of-use of MOEAs given the often ill-defined nature of real-world problem-solving.