Multi-inference on the Edge: Scheduling Networks with Limited Available Memory

Multi-inference on the Edge: Scheduling Networks with Limited Available Memory

Galjaard, Jeroen (TU Delft Electrical Engineering, Mathematics and Computer Science)

Chen, Lydia Y. (mentor)
Ghiassi, S. (graduation committee)
Cox, B.A. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2020-06-25

The execution of multi-inference tasks on low-powered edge devices has become increasingly popular in recent years for adding value to data on-device. The focus of the optimization of such jobs has been on hardware, neural network architectures, and frameworks to reduce execution speed. However, it is yet not known how different scheduling policies affect the execution speed of a multi-inference job. An empirical study has been performed to investigate the effects of scheduling policies on multi-inference. The execution performance information of multi-inference batch jobs under combinations of loading and scheduling policies were determined under varying levels of constrained memory. These results were obtained using EdgeCaffe: a framework developed to execute Caffe networks on edge oriented devices. Our research showed that a novel scheduling policy, MeMa, can significantly reduce execution speed under stringent memory availability. Overall, this study demonstrates that scheduling policies can significantly reduce the execution speed of multi-inference jobs.

Edge Computing
Scheduling
Constrained Memory
Memory Aware
Multi-inference
Convolutional Neural Networks

http://resolver.tudelft.nl/uuid:2fadc0f7-e381-419e-8593-92af74ead039

Student theses

bachelor thesis

© 2020 Jeroen Galjaard