# Are Machine Learning Cloud APIs Used Correctly?

### Presentation&#x20;

* Machine learning provides effective solutions&#x20;
* Software development: problems --> bugs&#x20;
* ML cloud API
  * Function as a service&#x20;
  * Help incorporating learning solutions into software systems&#x20;
    * Require less domain knowledge&#x20;
    * No need to design and train neural networks&#x20;
* ML APIs raise unique challenges&#x20;
  * Performing cognitive tasks: how people ask questions greatly affect the result &#x20;
  * Largely defined by training data: properties might not be known by API users &#x20;
  * Numeric vector output: high-dim, tricky to interpret
  * Complicated accuracy - performance tradeoffs&#x20;
* &#x20;Corpus&#x20;
  * Google / Amazon ML cloud API&#x20;
  * 3 ML domains: vision, language, speech&#x20;
  * 18 months, size of 2,200 lines&#x20;
* Anti-pattern identification methodology
  * Manual examine
  * Design test cases&#x20;
  * Report bugs&#x20;
* Result&#x20;
  * Most applications: misuses!&#x20;
  * Pattern
    * Calling the wrong API&#x20;
      * Subtle semantics difference among cognitive tasks&#x20;
      * e.g. image classification, object detection. Which one to use?&#x20;
      * e.g. text-detection, document-text-detection&#x20;
      * Escape the traditional testing&#x20;
    * Misinterpreting outputs&#x20;
      * Numeric vector outputs are difficult to interpret&#x20;
    * Misuse of async APIs&#x20;
      * complicated accuracy-performance tradeoffs&#x20;
    * Necessarily high-resolution inputs&#x20;
      * higher resolution - performance degrades&#x20;
    * Many other misuses --> what types of impact (reduce functionality, degraded performance, increased cost)&#x20;
* Design checkers&#x20;
  * Three static analysis tools for three misuses&#x20;
  * API wrappers for four misuses&#x20;
