# Leveraging Service Meshes as a New Network Layer

* A recent shift from monolithic to microservice architecture 
  * For better scalability and functionality 
  * Talk to each other through network calls 
  * Network communication has become intrinsic to application's functioning 
* Visualization of a large scale microservice 
  * Usually is a multi-hop DAG with many stages 
  * Developers end up writing a of "communication code" 
* **Service Meshes have emerged to "factor out communication" in microservices** 
  * Take the existing microservices and convert it into an architecture where all the communication functionality has been factored out and placed within a separate process called a sidecar 
  * Sidecar performs all inbound and outbound network communication 
  * The sidecar routes traffic at the granularity of an application-level request (e.g., HTTP) 
  * End to end path involves 3 transport connections 
    * Microservice A --> sidecar A (source)
    * Sidecar A --> sidecar B
    * Sidecar B --> Microservice B (destination)
  * **Data plane** (i.e., sidecar) offers: 
    * Load balancing: between replicas of the service 
    * Name resolution: not keep track of the IP address  
    * Encryption 
    * Tracing 
    * Authorization 
    * ... 
* **Control plane** offers global view and policy 
  * Service discovery 
  * Certificate management 
  * Configuration management 
* Data plane executes local function 
* Service Meshes as a new network layer 
  * How is it a layer? 
    * Service mesh abstracts lower layer functionality for the application
    * E.g., app is unaware which exact replica of a service it is talking to 
    * All inbound/outbound app traffic goes through service mesh via the sidecars 
  * Why is that a useful abstraction?
    * Highlights service meshes as a convenient location to observe and enhance cross-layer communication 
    * Can be application-agnostic and expose standard APIs which make any optimizations available to all apps running on top of it 
  * Cloud-native network stack 
* New opportunities enabled by service meshes 
  * **Enhanced visibility**
    * So far: IP packet
    * Now: HTTP request --> internal app-level, API call tree  
    * Enhanced visibility is useful for 
      * Monitoring
      * Troubleshooting
      * Root cause analysis 
  * **Better knowledge of application needs** 
    * Dozens of proposals assume knowledge of application specifics 
      * Priority aware flow scheduling
      * App-aware traffic engineering 
      * ...
    * With service mesh APIs, apps can directly signal preferences 
    * These proposals could be made more practical through service meshes 
  *  **Easier evolvability** 
    * Service meshes provide convenient platform to deploy new functionality 
      * Run in user space on end-hosts
      * Service Mesh APIs are easily extensible 
      * E.g., WASM extensions can be used to extend sidecars directly 
      * E.g. transport protocols 
  * **Coordination with lower layers** 
    * SDN controller can provide explicit hints to the sidecar (e.g., via congestion notification)
    * Example: congestion increases, SDN controller notifies service mesh, service mesh reroutes traffic 
* Case study: 
  * ![](/files/p6jV57jONKQdalYRJvc9)
  * Goal: provide request-level prioritization for latency-sensitive requests in this microservice applications which serves a mix of workloads 
  * Aim: implement this prioritization, by leveraging the capability of the sidecar to do cross-layer coordination with many minimal coupling with the application 
  * Design components 
    * Classify application's performance objectives at the ingress 
    * Carry these performance objectives (i.e. priorities) through the entire system 
    * Implement cross-layer prioritization at the sidecars 
      * Prioritization at the request/message queue level (e.g., at the RabbitMQ queue)
      * Choice: transport protocol (BBR, PCC) as per QoS of the request
      * Kernel prioritization for latency sensitive packets 
    * What does this prototype demonstrate?
      * Use knowledge of app needs across the microservice via provenance tracing (in this case, we trace priority of requests through the microservice) 
      * Easily evolvable sidecar can deploy many optimization schemes without modifying the app 
      * Abstract the details from the app and make these optimizations available to any app running on top of the service mesh layer (i.e., no tight coupling) 
  * Challenges and future directions 
    * Added latency due to sidecars 
      * Explore kernel-bypass techniques to improve performance 
    * Different layers may conflict in terms of their roles and functionality 
      * For example, app layer LB may conflict with network layer LB
      * How do we define the scope of responsibilities 
    * Extending cross-layer prioritization 
      * Include compute and storage level prioritization 


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://sliu583.gitbook.io/blog/specific-work/seminar-and-talk/fall-21-reading-list/leveraging-service-meshes-as-a-new-network-layer.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.