We present Tsunami, a work-in-progress training framework built on top of TVM using Relay. Tsunami lets us use TVM’s compilation flow to optimize and train BERT and DLRM. We also discuss some shortcomings that could be addressed by improving Relay.