The double helical nature of DNA links many cellular processes such as DNA replication, transcription, and repair to rotational motion and the accumulation of torsional strain. Magnetic tweezers (MTs) are a single-molecule technique that enables the application of precisely calibrated stretching forces to nucleic acid tethers and to control their rotational motion. However, conventional magnetic tweezers do not directly monitor rotation or measure torque. Here, we describe a method to directly measure rotational motion of particles in MT. The method relies on attaching small, non-magnetic beads to the magnetic beads to act as fiducial markers for rotational tracking. CCD images of the beads are analyzed with a tracking algorithm specifically designed to minimize crosstalk between translational and rotational motion: first, the in-plane center position of the magnetic bead is determined with a kernel-based tracker, while subsequently the height and rotation angle of the bead are determined via correlation-based algorithms. Evaluation of the tracking algorithm using both simulated images and recorded images of surface-immobilized beads demonstrates a rotational resolution of 0.1°, while maintaining a translational resolution of 1–2 nm. Example traces of the rotational fluctuations exhibited by DNA-tethered beads confined in magnetic potentials of varying stiffness demonstrate the robustness of the method and the potential for simultaneous tracking of multiple beads. Our rotation tracking algorithm enables the extension of MTs to magnetic torque tweezers (MTT) to directly measure the torque in single molecules. In addition, we envision uses of the algorithm in a range of biophysical measurements, including further extensions of MT, tethered particle motion, and optical trapping measurements.