Airborne collision avoidance systems (ACASs) form a key safety barrier by providing last-moment resolution advisories (RAs) to pilots for avoiding midair collisions. Intrinsic uncertainties, such as noise in ACAS input signals and variability in pilot performance, imply that the generation of RAs and the effectuated aircraft trajectories are nondeterministic processes. Existing ACAS validation methods reflect the intrinsic uncertainties to a limited extent only. This paper develops an agent-based model, which systematically captures uncertainties in ACAS input and pilot performance for Monte Carlo (MC) simulation of encounter scenarios. The agent-based model has been integrated with industry-specific implementations of Traffic Alert and Collision Avoidance System II and ACAS Xa in a novel collision avoidance validation and evaluation tool. Through illustrative MC simulation results, it is demonstrated that the intrinsic uncertainties can have a significant effect on the variability in timing and types of RAs, and subsequently on the variability in miss distance. Even the MC simulation estimated mean miss distance can differ significantly from the deterministically simulated miss distance. Most important, the tails of miss distance probability distributions and probabilities of near-midair collisions are affected. This stipulates that addressing intrinsic uncertainties through MC simulation is essential in evaluating ACASs.