Registration between histology and ex-vivo Full Field Optical Coherence Tomography (FF-OCT) can help in many clinical applications including identifying surgical margin in a tumour tissue during intra-operative pathological diagnosis, quantifying features in FF-OCT for diagnosis and reduction of tissue processing time. In this work, we present a framework for non-rigid registration between Histology and ex-vivo FF-OCT images. The proposed framework consists of a two-stage registration process. The first step consists of large-scale misalignment correction while also establishing the match between FF-OCT and one of the several histology tissue samples based on iterative closest point of prominent edge points. The second step starts with an area-based affine registration algorithm and culminates with a Deep Flow based registration algorithm. To facilitate the use of optical-flow based algorithm for inter-modality registration, mutual transform based modality transformation has been used here. Traditionally used mutual transform does not preserve the edges well. Since edges are an important component of the objective function used for minimization, this affects the accuracy of the registration algorithm. To address this problem, an edge-preserving mutual transform has been introduced in this work. This newly proposed variant of mutual transform as well as the use of Deep Flow algorithm for registration is seen to improve the accuracy significantly.