Floem: A programming system for NIC-accelerated network applications
https://www.usenix.org/conference/osdi18/presentation/phothilimthana
Challenge of accelerating server applications by off-loading applications to a programmable network card
Solution: language compiler runtime that makes it easier for developers to explore different NIC offloading designs
Recently: CPU can no longer keep up with the network
Offload the computation to programmable network card that sits between the CPU and the network
Traditionally: cannot be programmed
Now: NIC
Wimpy multi-core processor
Cavium LiquidIO
Netronome Agilio
Mellanox BlueField
Field-programmable gate array (FPGA)
Microsoft catapult
NetFPGA
Reconfigurable match table (RMT)
NIC offload
Offload computation
Fast path processing: filtering, classifying, caching, etc.
Transformation: encryption / decryption, compression etc
Steering
Congestion control
Offloading computation to a NIC requires a large amount of effort
Why? have to deal with distributed heterogenous system
CPU and NIC have o cache coherence
NIC can access CPU memory via PCIe
Hard to manage and optimize for
Heterogeneity in the system
NIC: slower cores, less power; faster cores higher power, instruction, memory hierarchy
Space of offload designs
Example: key-value store
CPU core, each handle one distinct set
NIC to steer the packet to the right CPU core: key-based steering
Can increase throughput
Using NIC as cache
3x power efficiency
Require: enough memory on NIC
No one-size-fit-all offload. Non-trivial to predict which offload is best
System, and performance objectives
Challenge: packet marshaling (define what fields to send, copy those fields)
Tedious & error-prone, hinder exploration of different offload designs
Challenge: communication strategies
No steering, key-based steering, separate GET & SET
Exploring different offload designs requires huge amount of efforts!
Floem
DSL makes it easy to explore alternative offloads
Compiler minimizes communication and generates efficient code
Runtime manages data transfer over PCIe
Language overview
Data-flow programming model
Extend to support: heterogeneity, parallelism
Contributions
Goal: explore offload designs
Inferred data transfer
Logical-to-physical queue mapping
Caching construct
Goal: Integration with existing app
Interface to external programs
Compiler 7 Runtime
Example
Element class: allow users to embed C code to define a computation of that element
One thread processes one packet at a time
Data parallelism
Annotate the segment with multiple cores
Pipeline parallelism
Multiple segments, insert queues
Offload: device
First: inferred data transfer
Solution: infer fields to send
Per-packet state: a packet and its metadata can be accessed anywhere in the program
Compiler infers which fields of packet and metadata to send
Second: logical-to-physical queue mapping
Observation: different communication strategies can be expressed by mapping logical queues to physical queues
Degrees of resource sharing
Dynamic packet steering
Packet ordering
Solution: queue construct with explicit logical-to-physical queue mapping
I.e. Queue(channels=2, instances=3)
Logical queues, physical queues
Example
No steering: queue construct with one physical instance
Key-based steering: multiple physical queue instances, specify which CPU cores get the data from which queue, and specify how to steer by assigning queue id to the per-packet state
Third: caching construct
Difficult to implement a complete cache protocol
Maintain consistency of data on NIC and CPU
High performance
Runtime & Communication
Runtime: responsible for managing the communication over the PCIe
Hard thing: manage data synchronization between the NIC and CPU
For performance, need a lot of optimizations in order to achieve high-throughput data transfer
I/O batching
Overlapping DMA operations with useful computation
Decouple the queue logic (in queue library) from the data sync and the DMA optimizations
Less compact and more modular
Can swap up queue library but reuse the queue sync layer
Evaluation
Help the programmers explore different offload designs?
Server: with smart NIC, without smart NIC
Case study: key-value store
Code relevant to communication
Floem program: 15 lines --> C program: 240 lines
Add cache in Floem (write-through cache --> write-back cache)
Other: distributed real-time data analytics
First offload: worse than CPU-only
Second offload: 96% improvement with 23 lines of code
Take-away: high-level programming abstractions
Control implementation strategies
Avoid low-level details
Result: minimal changes to explore different designs
Questions
Storage limitation of the NIC
Cavium: pretty big memory (4GB memory), fitting nicely with it
Might not be the right strategy
Allow you to try different offloading technologies
Multiple communication? multiple times back and forth
Can we have multiple communications?
Language allows you to use how many queues as you want
Whether that is the best strategy to use?
Best to minimize the communication
Congestion control beyond single-packet
TCP stack offload
One thing: implement the entire stack in the NIC, but use a lot of resources, and only left with small number of resources to offload other applications
Right thing: offload some parts (fast-path processing), but handle the flow state carefully (share between NIC and CPU)
Last updated
