Ultrafast Time Dynamics of Plasmonic Fractional Orbital Angular Momentum