At the heart of digital signal processing (DSP) are the sampling and quantization processes, which convert analog signals into discrete samples and which are implemented in the form of analog to digital converters (ADCs). In some recent applications, there is an increased demand for DSP applications to process signals having a very wide bandwidth. For such signals, the minimum allowable sampling rate is also very high and this has put a very high demand on the ADCs in terms of power consumption. Recently, the emergence of compressive sampling (CS) has offered a solution that allows us to reconstruct the original signal from samples collected from a sampling device operating at sub-Nyquist rate. The application of CS usually involves applying an additional constraint such as a sparsity constraint on the original signal. However, there are also applications where the signal to deal with has a high bandwidth (and thus sub-Nyquist rate sampling is still important) but where only the second-order statistics (instead of the original signal) are required to be reconstructed. In the latter case, depending on the characteristics of the signals, it might be possible to reconstruct the second-order statistics of the received analog signal from its sub-Nyquist rate samples without applying any additional constraints on the original signals. This idea is the key starting point of this thesis. We first focus on time-domain wide-sense stationary (WSS) signals and introduce a method for reconstructing their power spectrum from their sub-Nyquist rate samples without requiring the signal or the power spectrum to be sparse. Our method is examined both in the time- and frequency-domain and the solution is computed using a simple least-squares (LS) approach, which produces a solution if the rank condition of the resulting system matrix is satisfied. To satisfy this rank condition, two options of sampling design are proposed, one of which is the so-called multi-coset sampling. It is show in this thesis that any of the so-called sparse ruler can produce a multi-coset sampling design that guarantees the full rank condition of the system matrix, and thus the optimal compression is achieved by a minimal sparse ruler. While the approach in the previous paragraph is related to time-domain signals, we could extend the discussion about the power spectrum reconstruction from sub-Nyquist rate samples in the context of the spatial-domain signal, which is defined as a sequence of outputs of the antennas in the antenna array at a particular time instant. Given the compressed spatial domain signals, which are obtained from the output of a uniform linear array (ULA) with some antennas turned off, of particular interest is to reconstruct the angular power spectrum, from which the direction of arrival (DOA) of the sources can generally be located. In this thesis, a method to estimate the angular power spectrum and the DOA of possibly fully correlated sources based on second-order statistics of the compressed spatial-domain signals is proposed by employing a so-called dynamic array which is built upon the so-called underlying ULA. In this method, we present the spatial correlation matrices of the output of the dynamic active antenna arrays at all time slots as a linear function of the spatial correlation matrix of the entire underlying uniform array and we solve for this last correlation matrix using LS. The required theoretical condition to ensure the full column rank condition of the system matrix is formulated and designs are proposed to satisfy this condition. Next, we consider both spatio-angular and time-frequency domains and propose a compressive periodogram reconstruction method as our next contribution. We introduce the multibin model, where the entire band is divided into equal-size bins such that the spectra at two frequencies or angles, whose distance is at least equal to the bin size, are uncorrelated. This model results in a circulant structure in the so-called coset correlation matrix, which enables us to introduce a strong compression. We propose the sampling patterns based on a circular sparse ruler to guarantee the full column rank condition of the system matrix and to allow the LS reconstruction of the periodogram. We also provide a method for the case when the bin size is reduced such that the spectra at two frequencies or angles, whose distance is larger than the bin size, can still be correlated. To combine frequency and DOA processing, we also introduce a compressive two-dimensional (2D) frequency- and angular-domain power spectrum reconstruction for multiple uncorrelated time-domain WSS signals received from different sources by a linear array of antennas. We perform spatial-domain compression by deactivating some antennas in an underlying ULA and time-domain compression by multi-coset sampling. Finally, we propose a compressive cyclic spectrum reconstruction approach for wide-sense cyclostationary (WSCS) signals, where we consider sub-Nyquist rate samples produced by non-uniform sampling. This method is proposed after first observing that the block Toeplitz structure emerges in theWSCS signal correlation matrix. This structure is exploited to solve the WSCS signal correlation matrix by LS. The condition for the system matrix to have full column rank is provided and some possible non-uniform sampling designs to satisfy this full column rank condition are presented. Based on all the works that have been done, we have found that focusing on reconstructing the statistical measure of the received signals has significantly relax the sampling requirements and the constraints on both the statistics and the signals themselves. Hence, we would like to conclude that, for given tasks of applications in hand, we should ask ourselves whether statistical measure reconstruction is sufficient since the answer for this question will likely to determine how we should collect the data from the observed phenomena. This underlines the importance of awareness on what kind of information is necessary and sufficient for the tasks in hand before conducting the sensing/sampling process.