> For the complete documentation index, see [llms.txt](https://sliu583.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sliu583.gitbook.io/blog/conference/index/sosp-21/smartnic/xenic-smartnic-accelerated-distributed-transacitions.md).

# Xenic: SmartNIC-accelerated distributed transacitions

### Presentation 

* Distributed transactions in the datacenter 
  * Our target: distributed ACID transactions are a replicated, in-memory database
  * Common approach: optimistic concurrency control + replication 
  * Viability depends on efficient remote operations --> hardware acceleration
*   Recent work applies RDMA 
  * One-sided read/write primitives are high-performance, but restrict design 
    * Impact data structure and protocol overheads 
    * ![](/files/EvXkdaBhm9ssKtoRyaHt)
    * ![](/files/qVvYiVc9Ebrtl3XvY02C)
  * Two-sided RPCs are costly 
    * Add latency overhead, processing costs 
  * FaRM: one-sided RDMA
  * FaSST: two-sided RPCs 
  * DrTM+H: uses both  
  * Ongoing debate of applying RDMA: trade-offs are necessary 
* On-path SmartNICs: another option for hardware acceleration 
  * Programmable remote operations, without host processing 
  * Cost-effective compute: \~30% of NIC die area, 25W line-rate processing 
* SmartNIC opportunities 
  * Flexible CPU-bypass remote operations
  * Latency savings via stateful NIC operations, efficient PCIe DMA 
  * Efficient NIC-to-NIC communication 
  * But 
    * Software packet pipeline --> latency overhead 
    * Limited NIC resources 
* Xenic 
  * Distributed transactions accelerated with on-path SmartNICs 
  * Key
    * Co-designed data store, spread across NIC + host DRAM 
      * Minimize lookup overhead, utilizing NIC's on-board memory 
    * SmartNIC function shipping
      * Offload transaction logic to avoid PCIe crossings 
    * Multi-hop OCC protocols 
      * Reduce communication with optimized message patterns
    *  Stateful, asynchronous SmartNIC operation framework 
      * Exploit the SmartNIC's hardware interfaces 
* Xenic: Robinhood Data Store 
  * Host DRAM contains all objects; SmartNIC caches objects and lookup hints
  * Critical path accesses: NIC memory hit or DMA read, DMA log write 
    * Lookup hints limit DMA cost for cache misses 
      * Cache miss: bounded DMA R
      * Cache hit: NIC DRAM 
    * OCC + pinning ensure NIC/host consistency 
    * ![](/files/Cf1igwTdhg36LVErE9Zd)
* Xenic: SmartNIC function shipping 
  * Provides SmartNIC cores as a function shipping target 
  * Shipping execution can reduce overhead, depending on application-level computation and state requirements 
  * Saves coordinator PCIe crossings 
* Xenic: multi-hop OCC protocols 
  * Ships execution to remote SmartNICs
  * Multi-hop NIC-to-NIC communication increases network efficiency 
    * ![](/files/hTAp0uy4eqyx9asUwrqB)
* Evaluation 
  * Robinhood + NIC lookup hints effectively reduce cost
  * SmartNIC increases DMA lookup efficiency, even for cache misses
  * For FaRM and DrTM+H, end-to-end bandwidth/latency cost 
  * Better latency & throughput than RPC, RDMA, hybrid designs 
  * Also measured in our paper 
    * Cumulative core savings 
    * Full TPC-C, Retwis, Smallbank 
* Summary: high-performance, CPU-efficient distributed transactions 
  * Leveraging on-path SmartNICs:
    * Avoids RDMA compromises 
    * Provides a new, remote access-optimized data store
    * Selectively offloads transaction logic 
    * Applies multi-hop communication patterns 
    * Delivers >2x throughput over 100Gbps RDMA, latency savings relative to RPCs