The ever-increasing energy demands of traditional computing platforms (CPU, GPU) for large-scale deployment of Artificial Intelligence (AI) has spawned an exploration for better alternatives to existing von-Neumann compute architectures. Computation In-Memory (CIM) using emerging memory technologies such as Resistive Random Access Memory (RRAM) provide an energy-efficient and scalable alternative for Deep Neural Networks (DNN) applications. However, the benefits of CIM frameworks come at the cost of low DNN accuracy due to non-idealities in RRAM devices. In this thesis we address the conductance variation non-ideality in RRAM devices at an architectural level. We present two mapping schemes to improve the accuracy of CIM-based DNNs in the presence of RRAM conductance variation. Experimentation conducted with five datasets show that all proposed schemes provide up to 5.4x accuracy improvement over state-of-the-art implementations while inducing a 1.5% area cost and up to 10% energy overhead. Based on accuracy-energy trade-off, the thesis concludes the proposed Complementary Conductance Matrix (CCM) is the best candidate to improve inference accuracy of neural networks on CIM hardware using RRAM. It reports an accuracy improvement up to 5x with 1.52% area overhead and 9% energy overhead.