Conoscopic interferometry is a promising detection technique for ultrafast acoustics. By focusing a probe beam through a birefringent crystal before passing it through a polarizer, conoscopic interferences sculpt the spatial profile of the beam. The use of these patterns for acoustic wave detection revealed a higher detection sensitivity over existing techniques, such as reflectometry and beam distortion detection. However, the physical origin of the increased sensitivity is unknown. In this work, we present a model, describing the sensitivity behavior of conoscopic interferometry with respect to the quarter-wave plate orientation and the diaphragm aperture, which is validated experimentally. Using the model, we optimize the detection sensitivity of conoscopic interferometry. We obtain a maximal sensitivity of detection when placing the diaphragm edge on the dark fringes of the conoscopic interference patterns. In the configurations studied in this work, conoscopic interferometry can be 18 dB more sensitive to acoustic waves than beam distortion detection.