Long-term time series forecasting has found many utilities in various domains. Nevertheless, it remains difficult to perform by many existing methods. One of the most well-known forecasting techniques, the ARIMA, does not suffice the long-term forecasting task due to the mean convergence problem. Therefore, this research empirically assesses the alternative solution based on the Gaussian process (GP) regression. This study presents two approaches of Gaussian process regression for our problem: the structure modelling and the autoregressive approach. These techniques are evaluated on two synthetic datasets and two real-world datasets, which are the wind speed and electricity consumption dataset. From the experiment, it can be concluded that the GP-based forecasting techniques show more favourable long-term forecasting performance than the ARIMA model, particularly in cases where the data contain apparent trend and season. The experiment also demonstrates that the structure modelling method slightly outperforms the autoregressive approach for long-term forecast and offers a benefit that the autoregressive model does not have, which is the interpretability of the model.