Heterostructures (HSs) based on two-dimensional transition metal dichalcogenides (TMDCs) are highly intriguing materials because of the layers’ pronounced excitonic properties and their nontrivial contributions to the HS. These HSs exhibit unique properties that are not observed in either of the constituent components in isolation. Interlayer excitons (IEs), which are electron-hole pairs separated across the HSs, play a central role in determining these HS properties and are of interest both fundamentally and for device applications. In recent years, a major focus has been on understanding and designing HSs composed of two or more TMDC materials. Less attention has been paid to HSs composed of one TMDC layer and a layer of perovskite material. A central challenge in the understanding of HS properties is that basic measurements such as optical spectroscopic analysis can be misinterpreted due to the complexity of the charge transfer dynamics. Addressing these aspects, this review presents an overview of the most common and insightful optical spectroscopic techniques used to study TMDC/TMDC and TMDC/halide perovskite HSs. Emphasis is placed on the interpretation of these measurements in terms of charge transfer and the formation of IEs. Recent advances have started to uncover highly interesting phenomena, and with improved understanding these HSs offer great potential for device applications such as photodetectors and miniaturized optics.