Interactions in the real world are subject to mistakes and miscommunications. The presence of this noise in interactions challenges cooperation, as one party cannot determine whether the other party did not cooperate on purpose. The Prisoner's Dilemma has commonly been used to study mutual cooperation. Strategies like Tit for Tat that do well in the classic version of the game, perform badly once noise is present. Recent studies that have used the Prisoner's Dilemma to show that harsh environments promote cooperation, currently do not take noise into account. This article therefore uses the Prisoner's Dilemma and Agent Based Modeling and Simulation (ABMS) to describe the relation between the harshness of the environment and noise. From the simulations it follows that the adversity of the environment benefits cooperators and can make cooperation more robust against mistakes. Harsher environments also encourage greater generosity to cope with noise. Yet when uncertainty is high due to higher probability of mistakes or more potential defectors in the environment, contrite behaviours are most successful.