> For the complete documentation index, see [llms.txt](https://sliu583.gitbook.io/blog/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://sliu583.gitbook.io/blog/specific-work/seminar-and-talk/reading-groups/network-reading-group/ml-and-networking/other/snicket-query-driven-distributed-tracing.md).

# Snicket: Query-Driven Distributed Tracing

https\://dl.acm.org/doi/10.1145/3484266.3487393

* The rise of microservices 
  * Complexity and scaling --> microservices 
  * Advantages 
    * Flexibility with languages
    * Development velocity 
  * Challenges 
    * Distribution 
* Trace: a directed tree representing the calls made as a result of one user request 
  * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2F3F3TGXnG6XRtNIBeqhuM%2Fimage.png?alt=media\&token=1c122361-edf9-4cf6-a462-e83a9469a540)
* Head-Based Sampling 
  * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2Fo61RoHm2VJaBcfJ2vBMR%2Fimage.png?alt=media\&token=684466ee-9b66-42df-91b4-9ebad0a31966)
  * Unusual data is not collected 
* Tail-based sampling 
  * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2Fnwb1vyUVdozgnH8MPeVs%2Fimage.png?alt=media\&token=9fdea222-f52d-4345-aa91-199d6f72a79a)
  * May lose information of "how common"
* Problems with current approaches 
  * Persists information at granularity of traces 
    * Most queries of the data are interested only in the subset of the trace 
  * Queries run on incomplete data 
    * Uniform sampling misses unusual traces
    * Trace data is biased by the sampling strategy 
* Key idea: collect only the information necessary for the query 
* **What is Snicket?**
  * Query-driven
  * Allows for complex queries that include graph-based reasoning 
  * Tightly ties collection and computation: collect only what you need 
* **Why is Snicket possible now?**
  * The emergence of service proxies, analogous to application-level switches (e.x., envoy, linkerd)
  * Rise of extensions with proxies that allow computation close to the source 
    * Allows network programmability higher up the stack 
* **Snicket Design Overview** 
  * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2FoQHWEUJXApOoiYlFMEGT%2Fimage.png?alt=media\&token=760d1906-355f-4b12-8d03-f3f97537f11e)
  * The Snicket Query Language 
    * Structural filter: match on isomorphic subtree (Match)
    * Attribute filter: match on attributes of nodes or traces (Where) 
    * Map: create new developer-defined attributes (latency)
    * Return: return attribute value 
    * Aggregate: aggregate return values (avg)
* Query example
  *  ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2FNxwNwkDwO6WB1wPiQUz9%2Fimage.png?alt=media\&token=272153da-b6b8-4244-8cfb-aca47c8e906d)
* Evaluation: expressiveness, interactivity, and latency 
  * Online Boutique: 10 microservices 
  * 7 nodes of e2-highmem-4
  * Horizontal and vertical autoscaling enabled: 11->12 nodes
  * Locust as load generator 
  * \~15ms difference