In order to achieve redundancy and improve the robustness of an autonomous driving system, radar is a suitable choice for road user detection task in severe working conditions (e.g. darkness, bad weather). However, the real-time multi-class radar based road user detection algorithm is less explored compared with camera and LiDAR solutions. To fill this gap, the current thesis proposes a pipeline for radar based road user detection task, which is able to detect pedestrians, cyclists and cars by a single radar sweep. The pipeline effectively utilizes the advantages of radar low-level data by using it as region descriptor and combining it with radar high-level data for region proposal. A novel convolutional neural network structure called LLTnet is designed, in combination with proper pre-processing and post-processing stages. Ensemble learning is used to further improve the inter-class detection accuracy. The LLTnet itself performs radar targets segmentation. If needed, its output can be fused into object-level detection. To better train the network, a real-life dataset containing different moving road users is created during the study by a moving test vehicle, which simulates the real-life urban driving scenarios. After the network is trained, it is firstly evaluated by target-level metrics, such as the classification accuracy and F1 score. Then object-level metrics are used for object-level evaluation, such as the precision, recall and intersection over union (IoU). Comprehensive experiments are performed which not only evaluate the performance of the proposed model but also test the importance of different stages and features, such as the importance of the ensemble learning and the validity of adding low-level data. The proposed pipeline improves the target-level F1 score from 0.59 of the baseline to 0.64 using LLTnet without ensemble learning. By adding the ensemble learning stage, the target-level F1 score is further improved from 0.64 to 0.70. The object-level recall of pedestrian class greatly improves from 0.37 to 0.68. The validity of adding low-level data to the algorithm is verified by bypassing the low-level data branch of the network. With only high-level branch, the target-level F1 score drops from 0.70 to 0.60. Furthermore, the trained model also shows good generalization ability on unseen data.