We study the “Immediate Exchange Model”, a wealth distribution model introduced in Heinsalu and Patriarca (Eur Phys J B 87:170, 2014). We prove that the model has a discrete dual, where the duality functions are natural polynomials associated to the Gamma distribution with shape parameter 2 and are exactly those connecting the Brownian Energy Process (with parameter 2) and the corresponding Symmetric Inclusion Process in Carinci et al. (J Stat Phys 152:657–697, 2013) and Giardinà et al. (J Stat Phys 135(1):25–55, 2009). As a consequence, we recover invariance of products of Gamma distributions with shape parameter 2, and obtain ergodicity results. Next we show similar properties for a more general model, where the exchange fraction is Beta(s, t) distributed, and product measures with Gamma (s+t) marginals are invariant. We also show that the discrete dual model itself is self-dual and has the original continuous model as its scaling limit. We show that the self-duality is linked with an underlying SU(1, 1) symmetry, reminiscent of the one found before for the Symmetric Inclusion Process and related processes.