The response of a sound system in a room primarily varies with the room itself, the position of the loudspeakers and the listening position. The room boundaries cause reflections of the sound that can lead to undesired effects such as echoes, resonances or reverberation. Knowledge of the location of these large reflecting surfaces can help to better estimate the sound field behavior inside the room. This work focuses on exploiting the inherent information present in echoes measured by microphones to infer the location of nearby reflecting surfaces. The investigated application uses a loudspeaker to emit a known signal and record the resulting sound field with a co-located microphone array. A signal model is proposed which provides a relationship between reflector locations and measured microphone signals. The locations of reflections are estimated by fitting a sparse set of modeled reflections with measurements. We present two novelties with respect to prior art. First, the method is end-to-end where from raw microphones measurements it outputs an estimate of the location of reflectors. For the case of a compact uniform circular microphone array, the symmetry can be exploited to reduce the computational complexity of the inference process. Secondly, the model is extended to include a loudspeaker model that is aware of the inherent directivity pattern of the loudspeaker. The performance of the proposed localization method is compared in simulation to the existing state-of-the-art localization methods. An experimental study with real world measurements was also conducted to investigate the performance of the model.