Monitoring hardware utilization when training on GPUs in distributed machine learning

Large­scale machine learning frameworks can accelerate training of a neural network by per­ forming distributed training on a cluster using multiple GPUs per node and multiple nodes. Because distributed training on a cluster involves many nodes which need to communicate and load and exchange data, a machine learning framework may at certain...