# PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections

* Current system considers only full equivalent transformations&#x20;
  * Pro: preserve functionality&#x20;
  * Con: miss optimization opportunities&#x20;
* Partial Equivalent Transformation&#x20;
  * Pro: better performance&#x20;
    * Faster ML operators&#x20;
    * More efficient tensor layouts&#x20;
    * HW-specific optimizations&#x20;
  * Con: potential accuracy loss&#x20;
* Key: benefits while preserving equivalence?&#x20;
  * Correction preserves equivalence&#x20;

![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-Me_oGu5RA5axHm-VwUv%2F-Me_qz09wavPCMxBrYsP%2Fimage.png?alt=media\&token=e359a3d9-8c3d-40e4-9e97-f77833d78b51)

1. Super optimization
2. Multi-linearity of DNN computations&#x20;
   1. DNN computation = MLTP + non-linear activations&#x20;
   2. Reduce complexity from O(m\*n) to O(1)&#x20;

Corrector:

1. Re-compute the incorrect outputs using the original program&#x20;
2. opportunistically fuse correction kernels with other operators&#x20;
3. Eval: less than 1% overhead&#x20;

Program optimizer:

1. Search-based program optimizer&#x20;
   1. Beam search&#x20;
