A new approach is proposed to extract the slow-time feature of human motion in high-resolution radars. The approach is based on the self-similarity matrix (SSM) of the radar signals. The Mutual Information is used as a measure of similarity. The SSMs of different radar signals (high-resolution range profile, micro-Doppler, and range-Doppler video sequence) are compared, and the angel-invariant property of the SSMs is demonstrated. The SSM for different activities (i.e. walking and running) is extracted from range-Doppler video sequence and analyzed. Finally, simulation result is validated by experimental data.