This study focuses on automated malaria diagnosis in low quality blood smear images, captured by a low-cost smartphone based microscope system. The aim is to localize and classify the healthy and infected erythrocytes (red blood cells) in order to evaluate the parasitaemia in an infected blood smear. Due to the lower quality of the smartphone microscope system compared to traditional high-end light microscopes, conventional algorithms fail to process these images. We propose a framework using a convolutional neural network as a pixel classifier to localize the erythrocytes. Afterwards we classify them accordingly, using a convolutional neural network as an object classifier. Such a system can offer in the-field malaria diagnosis without human intervention or can act as an aid for human experts to lower workload and increase diagnosis accuracy. The algorithm successfully localizes the erythrocytes with an average sensitivity of 97.31% and precision of 92.21%. Classification performed inadequate, in terms of low agreement with two human experts. This can be due to the low image quality or the small amount of training data available at the time.