Declarative Machine Learning Systems

https://arxiv.org/abs/2107.08148

Introduction

• Machine learning: central in the strategy of tech companies and has gathered more attention from academia than ever before

• Hastened by

  • Hardware improvements: enabled massive parallel processing

  • Data infrastructure improvements: enabled storage and consumption of massive datasets needed to train most ML models

  • Algorithmic improvements: better performance and scaling

• Future envision: larger amount of people without skills to perform the same tasks. More declarative and abstract interfaces to hide complexity.

• Describe

  • How current ML systems are structured

  • What factors are important for ML project success, which ones will determine wider ML adoption

  • What are the issues current ML systems are facing

  • How the systems being developed address them

  • What the next generation of ML systems look like

1. Software engineering meets ML

   1. Distill common patterns that abstract the most mechanical parts of the process of building ML projects in a set of tools, systems, and platforms

2. The coming wave of ML systems

   1. Substantial increases in adoption always come with separation of interests and hiding of complexity

   2. Next-generation of ML systems that will allow people without ML expertise to train models and obtain predictions through more abstract interfaces

Machine Learning Systems

Challenges & desiderata:

1. Exponential decision explosion

   1. Need: good defaults and automation

2. New model-itis

   1. Need: standardization and focus on quality

3. Organizational chasms

   1. Need: common interfaces

4. Scarcity of expertise

   1. Need: higher level abstractions

5. Process slowness

   1. Need: rapid iteration

6. Many diverse state

   1. Need: separation of interests

Declarative ML systems