Evolutionary algorithms have been successfully applied to attack Physically Unclonable Functions (PUFs). CMA-ES is recognized as the most powerful option for a type of attack called the reliability attack. In this paper, we take a step back and systematically evaluate several metaheuristics for the challenge-response pair-based attack on strong PUFs. Our results confirm that CMA-ES has the best performance, but we note several other algorithms with similar performance while having smaller computational costs.