Poster: A novel shared memory framework for distributed deep learning in high-performance computing architecture

Abstract

This paper proposes a novel virtual shared memory framework, Soft Memory Box (SMB), which directly shares the memory of remote nodes among distributed processes to improve communication performance/speed via deep learning parameter sharing.