# A Deep Reinforcement Learning Perspective on Internet Congestion Control

* Internet congestion control: one of the most fundamental and challenging problems in communication networks 
  * Determines what you get out of the internet 
    * At what rate data goes in 
  * Always running on every connection 
    * With no prior knowledge 
* Water pipe
  * How fast do we send the data? 
  * Too slow: harm performance 
  * Too fast: overflow the pipe 
* Internet CC 
  * Very limited information, massive agent churn, enormous, dynamic, complicated network 
  * CC revisited 
    * Default algorithm: transmission control algorithm (TCP) 
* Motivating deep RL 
  * CC protocol
    * Blackbox 
    * Locally-perceived history of feedback: loss rates, latency 
    * On the other end: next sending rate 
  * Hypothesis: this feedback contains information about patterns that can improve the choice of sending rates 
* RL formulation 
  * Goal: maximize the cumulative reward
  * Agent observe the state, new action in the environment, output a reward 
  * NN: agent, and simulator for the environment 
    * Observe state, based on locally-perceived feedback 
* Aurora 
  * Monitor interval: send ratio, latency ration, latency inflation 
    * Keep design choice: scale-free observations affect robustness 
  * 3-layer NN 
  * Rate change factor 
  * Training 
    * Very simple simulated network
    * Each episode chooses different link parameters
    * Entire training platform: PCC-RL 
  * Real-World testing setup 
    * Real packets in linux kernel network emulation 
    * Using inference only 
      * Allowing faster adaptation 
      * Still producing state-of-the-art results 


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://sliu583.gitbook.io/blog/specific-work/seminar-and-talk/reading-groups/network-reading-group/ml-and-networking/congestion-control/a-deep-reinforcement-learning-perspective-on-internet-congestion-control.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.