Event Details

Date: 12/16/2021 1:35 pm
Track:
Organization: VMWare
Speakers: Tiejun Chen
Download Slides

TVM aims to optimize deep models pretrained by any ML upstream frameworks. But as a popular compiler, users have many additional works to make one given model to work over TVM. For example:

• users have to know some model information like input layer {name, shape, type}, and output layer, in order to help TVM compiler understand the model to compile that.
• users have to call different TVM APIs to compile the model trained by different ML frameworks.
• users have to rebuild {pre, post} process to handle TVM “model” – TVM Relay IR.

So, we were thinking what if we can make TVM as a backend acceleration to those ML framework? When users still use consistent development and deployment with ML upstream frameworks but they can enjoy the best TVM performance. With this goal we are building an automated ML Infer Booster that just tricks the ML framework into thinking it still takes over everything, but essentially, the underlying TVM backend helps boost this ML inference automatically.