> For the complete documentation index, see [llms.txt](https://sliu583.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sliu583.gitbook.io/blog/conference/index/sosp-21/smartnic/linefs-efficient-smartnic-offload-of-a-distributed-file-system-with-pipeline-parallelism.md).

# LineFS: Efficient SmartNIC offload of a distributed file system with pipeline parallelism

### Presentation 

* Growing DFS host resource consumption 
  * DFS is popular in cloud and HPC --> applications are consolidated 
    * ![](/files/5Q9vkgoTib8qeLCOQ7we)
  * Resource contention between DFS and co-running applications 
    * ![](/files/KgzpUfNjfv37zatnKERd)
* Problem: performance interference 
  * Host resource contention degrades the performance of both 
    * 1\) DFS application 2) co-running applications 
* Solution: offload DFS to SmartNIC 
  * To reduce interference 
  * ![](/files/CAd9yKMkEYK4jEZljGbL)
  * Challenges 
    * C1: high access latency from SmartNIC CPU to host PM 
    * Wimpy SmartNIC architecture 
    * ![](/files/IR2KjGL45EdQNq6MqpxJ)
* LineFS: SmartNIC offload of DFS with pipeline parallelism 
  * Goal: minimizing **host application interference** and **DFS slowdown** 
  * Design principles 
    * D1: persist-and-publish 
      * Persist data / metadata: latency critical task (fast host CPU)
      * Background publish: deferrable task (Wimpy SmartNIC CPU) 
    * D2: pipeline parallelism 
      * Increase throughput of offloaded tasks 
        * D2-1: publishing pipeline 
        * D2-2: replication pipeline 

#### D1: persist-and-publish 

![](/files/jmrF1Xs6fssUyDox45sM)

#### D2-1: pipeline parallelism - publishing 

![](/files/hfkge3JV60T2ymUi0sME)

#### D2-2: pipeline parallelism - replication 

* Replication? "persisting data to all the nodes on fsync"
* Persist data in advance: use SmartNIC resources 
* Reduce fsync latency: most of the data has already been persisted 

![](/files/KARt8J8Saawke7l9M2Lm)

Other design ideas in the paper:

* Linearizability & prefix crash consistency 
* Leveraging data-path processing 
* Availability during host OS failure 
* Shared file management with lease mechanism 

Evaluation:

* Does LineFS provide adequate offloaded DFS performance?
* Does LineFS alleviate interference? 

Conclusion

* Increasing DFS host resource use causes contention 
  * Degrading applications' performance
  * Unpredictable DFS operation latencies 
* Naive DFS offload to SmartNIC has high overhead
  * Long distance to host PM
  * Wimpy processing power 
* **LineFS: Efficient DFS SmartNIC offload**
  * Design principles: persist-and-publish, pipeline parallelism 
  * 90% lower LevelDB latency, 79% higher Filebench throughput versus Assise under host resource contention