We developed an impedance bridge that operates at cryogenic temperatures (down to 60 mK) and in perpendicular magnetic fields up to at least 12 T. This is achieved by mounting a GaAs HEMT amplifier perpendicular to a printed circuit board containing the device under test and thereby parallel to the magnetic field. The measured amplitude and phase of the output signal allows for the separation of the total impedance into an absolute capacitance and a resistance. Through a detailed noise characterization, we find that the best resolution is obtained when operating the HEMT amplifier at the highest gain. We obtained a resolution in the absolute capacitance of 6.4 aF/Hz at 77 K on a comb-drive actuator while maintaining a small excitation amplitude of 15 k_BT/e. We show the magnetic field functionality of our impedance bridge by measuring the quantum Hall plateaus of a top-gated hBN/graphene/hBN heterostructure at 60 mK with a probe signal of 12.8 k_BT/e.