Although the pixel-wise labelling approaches have been exploited in depth and achieve good results in segmentation tasks, the grouped pixels are not ideal output for many end-users. In this paper, we propose a vertex-voting-based approach that can directly extract the polygon representations of objects. In order to better solve overlapping scenarios, we also propose a novel method that distinguishes objects by learning a virtual depth axis. When compared with the state-of-the-art method, our experiments demonstrate that this voting-based method is more robust to occlusion and shows a potential research direction.