Abstract: Increasingly large-scale models and rich data sets make communication overhead a key bottleneck for distributed Deep Neural Network (DNN) training, constantly attracting the attention of ...
Abstract: In this paper we introduce the novel routing scheme based on load balancing and shortest-path routing. First, we present the linear program for routing optimization. The nonblocking network ...