Mechanisms for large-scale vulnerability notifications have been confronted with disappointing remediation rates. It has proven difficult to reach the relevant party and, once reached, to incentivize them to act. We present the first empirical study of a potentially more effective mechanism: Quarantining the vulnerable resource until it is remediated. We have measured the remediation rates achieved by a medium-sized ISP for 1, 688 retail customers running open DNS resolvers or Multicast DNS services. These servers can be abused in UDP-based amplification attacks. We assess the effectiveness of quarantining by comparing remediation with two other groups: One group which was notified but not quarantined and another group where no action was taken. We find very high remediation rates for the quarantined users, 87%, even though they can self-release from the quarantine environment. Of those who received the email-only notification, 76% remediated. Surprisingly, over half of the customers who were not notified at all also remediated, though this is tied to the fact that many observations of vulnerable servers are transient. All in all, quarantining appears more effective than other notification and remediation mechanisms, but it is also clear that it can not be deployed as a general solution for Internet-wide notifications.