Our research work addresses the issues in utilizing TVM on mobile devices. In our last year presentation at TVM conference, we presented our work in mapping TVM Relay to NNAPI. In this year work, we further extend the framework to Mediatek Neuron.
This work uses the BYOC infrastructure for mapping relay IR to Neuron IR. TVM can use APU to accelerate inference time by Neuron runtime on the android system with Mediatek SoC. It also enables the Neuron compiler to take advantage of the TVM ecosystem. Achieve a win-win situation.
Mediatek Neuron is software solutions to boost AI computing by APU (AI Processing Unit) at Mediatek’s platform. Neuron compiles AI models into a low-level binary for the APU on the host, and Neuron Runtime provides APIs to load low-level binary on the device side for inference. Compared with TVM Relay to NNAPI last year we presented, it reduces the compile-time at the device by offline compilation and the runtime is fully optimized for low latency.
