# Dagger: Efficient and Fast RPCs in Cloud Microservices in Near-Memory Reconfigurable NICs

### Presentation 

* Trends in cloud computing (monoliths) 
  * Tightly-coupled application logic in a single statically / dynamically linked library 
  * Shift towards microservices 
    * Loosely-coupled application logic split into many independent small applications
  * Shift towards serverless 
    * Fine application granularity
    * Fine lifetime granularity 
* Cloud applications today are interactive 
  * Frequent interaction with large sets of users 
  * Strict performance requirements as SLO
    * Low tail latency under high load
    * Performance predictability 
* Focus on: improve communication stack in microservices 
  * over RPCs 
  * RPC requests in microservices are small and vary by tiers 
  * Take
    * Per-request communication overheads are large
    * Cannot tune communication stacks for small messages only
    * Need an adaptive stack 
  * RPC stacks run on the same CPUs as highly concurrent applications 
    * Already high pressure on CPUs from applications
    * Intensive traffic of small messages 
* Dagger: a HW/SW co-designed end-host RPC stack 
  * Design principles 
    * **Hardware offload** 
      * Existing techniques to improve efficiency of cloud networking 
        * Kernel bypass: IX, eRPC, mTCP, and many others 
          * Removes per-packet kernel overheads, tightly couples networking stacks with applications, but still run everything in SW 
        * RDMA system 
          * Offloads networking stacks to hardware 
          * But: 
            * only provides low-level abstractions, the RPC part runs in SW
            * requires specialized adapters 
      * hardware NIC for end-host communication stacks, from the L1 (PHY) layer, and all the way up to the application (RPC) layer 
        * Completely free CPU from doing any work related to data exchange
    * **Reconfigurability** 
      * Networking protocols, load balances, threading, data representation, data manipulation. HW should also be 
      * Dagger is based on an FPGA! 
        * Configurable transport: UDP, TCP, mTCP, HOMA, TONIC 
        * Configurable load balancer / flow controller: static, round-robin, random, application-specific 
        * Configurable host-NIC interface: PCIe doorbells, PCIe MMIOs, coherency-based 
        * Configurable threading model: connection/thread/queue/flow mapping, number of NIC flows / queues 
    * **Tight coupling** 
      * Dagger is based on a cache-coherent FPGA tightly-coupled with the host CPU
        * Inspired by soNUMA, series of RDMA studies 
        * an FPGA acting as NUMA node
          * No DMAs are required to exchange data between NUMA nodes
          * No explicit MMIO requests
          * Minimal software overhead
          * NUMA interconnects have lower latency 
      * Existing SmartNICs are based on PICe! (introduce overheads) 
        * Doorbell scheme 
          * Multiple PCIe roundtrips
          * Expensive and CPU-inefficient rings based on MMIOs 
        * Existing optimizations: combined descriptors and packets, packet write with MMIOs, doorbell batching... (but fail to eliminate) 

![](/files/HU1t9IenXN43syLE0PoK)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://sliu583.gitbook.io/blog/networking/index/reading-list/dagger-efficient-and-fast-rpcs-in-cloud-microservices-in-near-memory-reconfigurable-nics.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.