Influence of coherence time drift on the secret key rate