The production of cocoa beans contributes to 7.5 % of European Union (EU) driven deforestation. As a result, the recent European Union Deforestation-free Regulation (EUDR) mandates producers to track cocoa farm extents comprehensively. While Remote Sensing has enormous capacity in dynamic crop monitoring, cocoa crop detection shows challenges due to cocoa complex canopy structure, spectral similarity to forest, variable farming methods, and location in frequently cloudy regions. Previous research on cocoa crop detection has mainly focused on pixel-based classification, disregarding spatial context. In this research we have performed a semantic segmentation approach to incorporate spatial configuration and enhance cocoa crop detection. We have applied Convolutional Neural Network (CNN) for the to semantic segmentation of cocoa parcels, considering both spectral and spatial characteristics. Additionally, we have evaluated the impact of combining Synthetic Aperture RADAR (SAR) and MSI (Multi-Spectral Imagery) data in the training of a CNN to demonstrate the importance of texture, moisture, and canopy characteristics in identifying cocoa canopies. The impact of MSI dataset stack with different SAR polarizations, seasons and temporality has been evaluated. The methodology is tested on Sentinel 1 and 2 data over an area of 100 × 100 km in Ghana for which an extensive ground truth data set of almost 90,000 polygons was available for training and validation. The results show that the addition of single-day and temporal SAR to a single-day MSI image can improve the predictions, reaching an F1 score of 86.62 %. This research demonstrates the influence of SAR measurements, seasons, polarization, and ground truth classes on the semantic segmentation of cocoa.