This paper presents a comprehensive investigation into novel Recurrent Neural Network (RNN) architectures for enhancing the efficiency and performance of Digital Predistortion (DPD) for Radio Frequency (RF) Power Amplifiers (PAs). The research first introduces Delta Just Another Network (DeltaJANET), a computationally efficient model that leverages a sparse delta update rule. By updating only a small fraction of the hidden state at each time step, DeltaJANET significantly reduces the complexity and computational cost associated with traditional RNNs, establishing a new baseline for efficient DPD solutions. Building upon this foundation of efficiency, the paper then proposes a second, more powerful architecture: the Temporal Convolutional Just Another Network (TC-JANET). This advanced hybrid model synergistically combines a Temporal Convolutional Network (TCN) for long-range feature extraction with a lightweight Just Another Network (JANET) unit. Key innovations, including a direct memory input module and a dynamic phase normalization scheme applied to the recurrent state, enable the TC-JANET to robustly model complex PA behaviors. A systematic multi-seed evaluation demonstrates the exceptional performance and scalability of this architecture, showing that it significantly surpasses existing benchmarks and establishing a new benchmark for high-performance DPD solutions.