Context. Direct imaging has paved the way for atmospheric characterization of young and self-luminous gas giants. Scattering in a horizontally-inhomogeneous atmosphere causes the disk-integrated polarization of the thermal radiation to be linearly polarized, possibly detectable with the newest generation of high-contrast imaging instruments. Aims. We aim to investigate the effect of latitudinal and longitudinal cloud variations, circumplanetary disks, atmospheric oblateness, and cloud particle properties on the integrated degree and direction of polarization in the near-infrared. We want to understand how 3D atmospheric asymmetries affect the polarization signal in order to assess the potential of infrared polarimetry for direct imaging observations of planetary-mass companions. Methods. We have developed a three-dimensional Monte Carlo radiative transfer code (ARTES) for scattered light simulations in (exo)planetary atmospheres. The code is applicable to calculations of reflected light and thermal radiation in a spherical grid with a parameterized distribution of gas, clouds, hazes, and circumplanetary material. A gray atmosphere approximation is used for the thermal structure. Results. The disk-integrated degree of polarization of a horizontally-inhomogeneous atmosphere is maximal when the planet is flattened, the optical thickness of the equatorial clouds is large compared to the polar clouds, and the clouds are located at high altitude. For a flattened planet, the integrated polarization can both increase or decrease with respect to a spherical planet which depends on the horizontal distribution and optical thickness of the clouds. The direction of polarization can be either parallel or perpendicular to the projected direction of the rotation axis when clouds are zonally distributed. Rayleigh scattering by submicron-sized cloud particles will maximize the polarimetric signal whereas the integrated degree of polarization is significantly reduced with micron-sized cloud particles as a result of forward scattering. The presence of a cold or hot circumplanetary disk may also produce a detectable degree of polarization (1%) even with a uniform cloud layer in the atmosphere.