Artifact

Masa: Responsive Multi-DNN Inference on the Edge

Conference paper (2021)

Authors

B.A. Cox Data-Intensive Systems - EEMCS
J.M. Galjaard Electrical Engineering, Mathematics and Computer Science
S. Ghiassi Data-Intensive Systems - EEMCS
Robert Birke ABB Switzerland Ltd.
Lydia Y. Chen Data-Intensive Systems - EEMCS
Research Group
Data-Intensive Systems (EEMCS) (TU Delft)
Multiple DNNs inference Edge devices Memory-aware scheduling Mean response time
More Info
expand_more

Published Date

2021

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Data-Intensive Systems

Abstract

This artifact is a guideline how the Edgecaffe framework, presented in [1], can be used. Edgecaffe is an open-source Deep Neural Network framework for efficient multi-network inference on edge devices. This framework enables the layer by layer execution and fine-grained control during inference of Deep Neural Networks. Edgecaffe is created to give more fine grained-control over the execution during inference than offered by the original code of Caffe [2]. Edgecaffe made it possible for Masa to outperform Deepeye [3] and normal bulk execution. Besides the core implementation of Edgecaffe, the repository holds additional tools, Queue Runner and ModelSplitter, that make more convenient to run experiments and prepare newly trained networks