# Data Management

[DMon: Efficient Detection and Correction of Data Locality Problems Using Selective Profiling](https://www.usenix.org/conference/osdi21/presentation/khan) 

[CLP: Efficient and Scalable Search on Compressed Text Logs](https://www.usenix.org/conference/osdi21/presentation/rodrigues) 

[Polyjuice: High-Performance Transactions via Learned Concurrency Control](https://www.usenix.org/conference/osdi21/presentation/wang-jiachen) 

[Retrofitting High Availability Mechanism to Tame Hybrid Transaction/Analytical Processing](https://www.usenix.org/conference/osdi21/presentation/shen) 

### DMon 

Motivation 

* Single search query (\~700ms): 8 processor cores 
  * Millions of cores 
  * Cost in management and energy cost 
* CPU performance 
  * 32% doing useful work 
  * Other: fetching, decoding, bad speculation, execution units unavailable, memory stalls (poor data locality) 

Existing techniques

* Compiler Optimizations 
  * Improve data locality via program transformation 
  * Con: rely on static heuristic, can sometimes hurt performance 
* Dynamic profilers 
  * Accurate execution information, identify and resolve poor locality 
  * Con: mostly manual repair, high profiling overhead 

Contribution 

* Selective profiling 
* Apply specific compiler optimizations based on profiling results

 

![](/files/-MeaTnEfYbpaoWon_K01)

Evaluation

* Efficiency 
* Effectiveness 
* Real-world case studies 

Data locality problem that is not solved by optimization? 

* Tail latency problem 
  * Can't identify and repair all 

### CLP

Workload / problems: 

* Logs --> search tools --> compress (i.e. gzip) 
  * Problem 
    * Perabytes of logs 
    * Consume a lot of resources for uncompressed logs 
  * Current solution 
    * Build indexes: adds storage overhead 
      * Total usage is the same magnitude 
    * Can only retain indexed logs for a few weeks 
    * Compress (2.27% of the original size) 
      * Unsearchable once compressed 
      * Requires decompression 

Method key: 

* Lossless log compression: better than general-purpose compressors 
* Can search compressed logs 
  * Without decompression 
    * Only decompress matching results 
  * With good performance 

Method:

* Domain-specific algorithm to extract the overlapping parts 
  * De-duplicate 
  * Log Type Dictionary 
  * Variable Dictionary 
  * Encoded Message 

Comparison:

* Search: ripgrep, ElasticSearch 
* Archieve tools: lzma, ... 

How to change the structure? And prevent them from blowing down?

* Compress logs to multiple archives 
  * Different set of archives 
* Do the search? Affecting the performance? 
  * Time is neglectable compared to the time to read 
* Accurate schema? 
  * Yes 
  * Based on the variable, tend to be quite general (generic schema) 
    * Aren't the best? Search performance 

### Polyjuice 

Motivation: 

* CC: control how concurrent executions interleave 
  * Maximizing interleaving --> better performance 
* Currently, no single CC algorithm is the best depending on the workload (high-, low- contention)

Current solutions:

* Coarse-grained approach to combine few known CC algorithms 
* Con 
  * Cumbersome: requires manual partition of the workload 
  * Suboptimal: uses a single CC within each partition 

Approach:

* RL
  * State 
    * Differentiate state that require different CC actions 
    * Two part 
      * Type of transaction 
      * Data accessed 
  * Action
    * Should be able to encode most existing CC algorithms 
    * Exert control on the interleaving 
    * Solution 
      * Whether to wait, and how long?
        * OCC: no wait 
        * 2PL: wait until dependent transactions commit 
        * IC3, RP, DRP: wait until ... finish execution up to some point 
      * Which versions of data to read?
        * Latest committed version 
        * Latest published dirty version 
      * Whether to make a dirty write visible?
        * Keep itself in the private buffer 
        * Publish the dirty write 
      * Whether to validate now, prior to commit? 
        * Whether or not to validate after this access 
        * Based on Silo's protocol 

![](/files/-MeacE2AHKFx0XuYYmGE)

### VEGITO  

Workload: 

* Typical workloads 
  * OLTP: short-term 
  * OLAP: long-term
* New type of workload: HTAP 

Requirement:

* Performance 
  * Minimizes performance degradation 
  * Millions of transactions per second 
* Freshness 
  * Delay.. 

Data analysis: TP + ETL + AP

* HTAP alternatives 
* Alternative 1: Dual system 
  * Good performance 
  * Large time delay 
* Alter 2: single-layout 
  * Short time delay 
  * Huge performance degradation 
* Alter 3: dual-layout 
  * Lightweight sync. 

Idea: VEGITO 

* High availability for fast in-memory OLTP 
  * Replication-based 
  * Synchronous log shipping 
* Challenge 
  * Log cleaning 
    * Need to be consistent and log-parallel 
  * Epoch-based design 
    * Gossip-style 
      * Guarantee epoch assignment is consistent 
  * MVCS 
    * Multi-version column store (MVCS) 
    * Different locality for read & write 
      * column-wise v.s row-wise 
    * Use block-based MVCS 
      * Row-split 
      * Col-merge 
  * Tree-based index 
    * Write-optimized tree index
    *  Read-optimized index 
    * Tree insert = in-place insert + balance (costly)
      * Balance in batch 


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://sliu583.gitbook.io/blog/conference/index/osdi-21/data-management.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.