A new temporal vortex tracking algorithm allows the first long-term temporal observation of the lives of the intense vorticity structures (IVS). The algorithm is applied to direct numerical simulations of statistically stationary isotropic turbulence, with Taylor-based Reynolds numbers in the range. In the highest Reynolds number case, the continuous time tracking of millions of 'worms' is achieved for more than seven integral time scales and close to 200 Kolmogorov time scales. Within an integral scale volume, more than 66 structures exist, and approximately 20 new structures are created per Kolmogorov time. More than of the structures live a solitary 'life' without any visible interaction with the other structures, while approximately break into new structures. Less than of the structures merge with others to form new vortices. A 'population model' is developed to estimate the numbers of existing vortices for very long simulated times, and it is observed that the birth rate density of these structures slowly increases with the Reynolds number. The survival functions of the IVS lives exhibit an exponential distribution, with some structures living for more than Kolmogorov time scales (more than four integral time scales). The mean lifetime of the IVS scales with the mean turnover time scale of the worms, defined by their radii and tangential velocity, attaining turnover time scales at high Reynolds numbers.