Strong demands for efficient deployment of Deep Learning (DL) applications prompt the rapid development of a rich DL ecosystem. To keep up with its fast advancement, it is crucial for DL frameworks to efficiently integrate a variety of optimized libraries and runtimes as their backends. However, current DL frameworks require significant manual effort to integrate diverse backends and often fail to deliver optimal performance. In this paper, we propose Collage, an automatic framework for deep learning backend integration. Collage provides a backend registration interface that allows accurate specification of various backend capabilities. With this specification, Collage searches for an optimized backend placement for a given workload and execution environment. Our evaluation shows that Collage automatically integrates multiple backends without manual intervention, and outperforms existing frameworks by 1.21x, 1.39x, 1.40x on two different NVIDIA GPUs and an Intel CPU respectively.
