π£
π£
π£
π£
Reading List
Searchβ¦
π£
π£
π£
π£
Reading List
Starting point
Reference list
PhD application guidelines
Big Data System
Index
Architecture
Execution Engine, Resource Negotiator, Schedulers
Machine Learning
SQL Framework
Stream Processing
Graph Processing
Potpourri: Hardware, Serverless and Approximation
Operating System
Index
Networking
Index
Persistence
Index
Database
Index
Group
WISR Group
Shivaram's Group
Seminar & Talk
Conference
Index
Cloud Computing
Index
Random
Reading List
Grad School
Index
Distributed System
Distributed Systems Lecture Series
Powered By
GitBook
Index
Reference: CS744 (UW-Madison) and CS 494 (UIC)
Architecture
Compute + Overall
β
The Datacenter as a Computer
: An Introduction to the Design of Warehouse-Scale Machines , L.A. Barroso, U. Holzle, Synthesis Lectures on Computer Architecture, 2009. Chapter 1 and 2.
Networks
β
VL2: A Scalable and Flexible Data Center Network
, Greenberg et al., SIGCOMM 2009.
β
Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Googleβs Datacenter Network
, Singh et al., SIGCOMM 2015.
Storage (in a bit detailed fashion)
β
The Hadoop Distributed File System
, Schvachko et al, MSST, 2010
β
The Google File System
, Ghemawat et al, SOSP, 2003.
β
NFS: Sun's Network File System
β
β
Flat Datacenter Storage
. Nightingale et. al, OSDI, 2012.
β
EC-Cache: Load-balanced, Low-latency Cluster Caching with Online Erasure Coding.
Rashmi et. al, OSDI, 2016
β
f4: Facebookβs Warm BLOB Storage System.
Muralidhar et. al, OSDI, 2014.
β
Bigtable: A Distributed Storage System for Structured Data.
Chang et. al, OSDI, 2006.
β
Dynamo: Amazonβs Highly Available Key-value Store.
DeCandia et. al, SOSP, 2007.
β
Spanner: Googleβs Globally-Distributed Database.
Corbett et. al, OSDI, 2012.
β
An Analysis of Facebook Photo Caching.
Huang et. al, SOSP, 2013.
β
Scaling Memcache at Facebook.
Nishtala et. al, NSDI, 2013.
β
The Chubby lock service for loosely-coupled distributed systems.
Mike Burrows, OSDI, 2006.
Execution Engines, Resource Negotiators, Schedulers
Execution Engines
β
MapReduce: Simplified Data Processing on Large Clusters, Dean and Ghemawat
, OSDI, 2004
β
Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks.
Isard et. al, EuroSys, 2007.
β
CIEL: a universal execution engine for distributed data-flow computing
. Murray et. al, NSDI, 2011.
β
Reining in the Outliers in Map-Reduce Clusters using Mantri,
Ananthanarayanan et al, OSDI, 2010.
β
DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language.
Yu et. al, OSDI, 2008.
β
Encapsulation of parallelism in the Volcano query processing system.
Goetz Graefe, SIGMOD, 1990.
β
PACMan: Coordinated Memory Caching for Parallel Jobs,
Ananthanarayanan et. al, NSDI, 2012.
β
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
, Zaharia et al, NSDI, 2012.
β
Apache Tez: A Unifying Framework for Modeling and Building Data Processing Applications
, Saha et al, SIGMOD, 2015.
β
Flare: Optimizing Apache Spark with Native Compilation for Scale-Up Architectures and Medium-Size Data.
Essertel et. al, OSDI, 2018.
Transaction:
Obladi: Oblivious Serializable Transactions in the Cloud.
Crooks et. al, OSDI, 2018.
Load balancing
β
Ananta: Cloud Scale Load Balancing
. Patel et. al, SIGCOMM, 2013.
β
Duet: Cloud Scale Load Balancing with Hardware and Software.
Gandhi et. al, SIGCOMM, 2014.
β
SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs
, Miao et. al, SIGCOMM, 2017.
Resource Negotiator
β
Apache Hadoop YARN: Yet Another Resource Negotiator
, Vavilapalli et al, SOCC, 2013.
β
Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center,
Hindman et al, NSDI, 2011.
β
Dominant Resource Fairness: Fair Allocation of Multiple Resource Types,
Ghodsi et al, NSDI, 2011.
β
Borg: Large-scale cluster management at Google with Borg.
Verma et. al, EuroSys, 2015.
Scheduling
Packing
β
Altruistic Scheduling in Multi-Resource Clusters.
Grandl et. al, OSDI, 2016.
β
Multi-Resource Packing for Cluster Schedulers,
Grandl et. al, SIGCOMM, 2014.
β
Quincy: Fair Scheduling for Distributed Computing Clusters.
Isard et. al, SOSP, 2009.
Re-Planning
β
Dynamic Query Re-Planning using QOOP.
Mahajan et. al, OSDI, 2018.
Threads
β
Arachne: Core-Aware Thread Management.
Qin et. al, OSDI, 2018.
Cache
β
RobinHood: Tail Latency Aware Caching β Dynamic Reallocation from Cache-Rich to Cache-Poor.
Berger et. al, OSDI, 2018.
Applications: Machine Learning
β
Scaling Distributed Machine Learning with the Parameter Server
, Li et al, OSDI, 2014.
β
STRADS: A Distributed Framework for Scheduled Model Parallel Machine Learning
, Kim et al, EuroSys, 2016.
β
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning
, Zhang et al, SoCC, 2017.
β
TensorFlow: A System for Large-Scale Machine Learning
, Abadi et al, OSDI, 2016.
β
Pytorch Distributed: Experiences on Accelerating Data Parallel Training
, Shen et al, VLDB, 2020
β
Gandiva: Introspective Cluster Scheduling for Deep Learning,
Xiao et al, OSDI, 2018.
β
Clipper: A Low-Latency Online Prediction Serving System
, Crankshaw et al, NSDI, 2017.
β
PipeDream: Generalized Pipeline Parallelism for DNN Training.
Narayanan et al, SOSP 2019.
β
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
, Chen et al, OSDI, 2018
β
Ray: A Distributed Framework for Emerging AI Applicationss
, Moritz et al, OSDI, 2018.
β
Towards a Unified Architecture for in-RDBMS Analytics
, Feng et al, SIGMOD, 2012
β
DeepCPU: Serving RNN-based Deep Learning Models 10x Faster.
Zhang et. al, USENIX ATC, 2018.
β
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
. Lee et. al, OSDI, 2018.
β
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
, Hazelwood et. al, HPCA, 2018.
β
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems.
Chen et. al, Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015.
β
Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud
, Low et al, VLDB, 2012.
β
Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters
. Peng et. al, EuroSys, 2018.
β
Tiresias: A GPU Cluster Manager for Distributed Deep Learning.
Gu et. al, NSDI, 2019.
β
Janus: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs
. Jeong et. al, 2018.
β
KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analystics
, Sparks et al, ICDE, 2017.
β
Project Adam: Building an Efficient and Scalable Deep Learning Training System
, Chilimbi et al, OSDI, 2014.
β
DimmWitted: A Study of Main-Memory Statistical Analytics.
Zhang and Re, VLDB, 2014.
Applications: Batch Analytics and SQL Frameworks
β
Spark SQL: Relational Data Processing in Spark
, Armburst et al, SIGMOD, 2015.
β
Major technical advancements in Apache Hive
, Huai et al, SIGMOD, 2014.
β
Clarinet: WAN-Aware Optimization for Analytics Queries
, Viswanathan et al, OSDI, 2016.
β
Global Analytics in the Face of Bandwidth and Regulatory Constraints,
Vulimiri et al, NSDI, 2015.
β
SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets
. Chaiken et al, VLDB
β
The Snowflake Elastic Data Warehouse.
Dageville et al, SIGMOD 2016.
β
Building an Elastic Query Engine on Disaggregated Storage
. Vuppalapati et al, NSDI 2020.
β
Impala: A Modern, Open-Source SQL Engine for Hadoop.
Kornacker et. al, CIDR, 2015.
β
Dremel: Interactive Analysis of Web-Scale Datasets.
Melnik et. al, VLDB, 2010.
β
Trill: A High-Performance Incremental Query Processor for Diverse Analytics.
Chandramouli et. al, VLDB, 2014.
β
Rethinking SIMD Vectorization for In-Memory Databases.
Polychroniou et. al, SIGMOD, 2015.
β
Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited.
Balkesen et. al, VLDB, 2013.
β
TAG: a Tiny AGgregation Service for Ad-Hoc Sensor Networks
. Madden et. al, OSDI, 2002.
Applications: Stream Processing
β
Storm @Twitter
, Toshniwal et al, SIGMOD, 2014.
β
Twitter Heron: Stream Processing at Scale
, Kulkarni et al, SIGMOD, 2015.
β
Realtime Data Processing at Facebook
. Chen et. al, SIGMOD, 2016.
β
Discretized Streams: Fault-Tolerant Streaming Computation at Scale
, Zaharia et al, SOSP, 2013.
Reading:
Spark Structured Streaming
β
β
Apache Flink: Stream and Batch Processing in a Single Engine,
Carbone et al, Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015.
β
Kafka Distributed Messaging System for Log Processing,
Kreps et al, NetDB Workshop, 2011.
Also this document of comparison of widely used
Queuing Messaging Processing Systems
.
β
StreamScope: Continuous Reliable Distributed Processing of Big Data Streams
, Lin et al, NSDI, 2016.
β
Drizzle: Fast and Adaptable Stream Processing at Scale
. Venkataraman et. al, SOSP, 2017.
β
Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems
. Mai et. al, PVLDB, 2018.
β
Gloss: Seamless Live Reconfiguration and Reoptimization of Stream Programs.
Rajadurai et. al, ASPLOS, 2018.
β
Aurora: a new model and architecture for data stream management.
Abadi et. al, VLDB, 2003.
β
Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows.
Kalavri et. al, OSDI, 2018.
β
Naiad: A Timely Dataflow System
, Murray et al, SOSP, 2013.
β
The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing.
Akidau et. al, VLDB, 2015.
Applications: Graph Processing
β
Pregel: A System for Large-Scale Graph Processing,
Malewicz et al, SIGMOD, 2010.
β
TAO: Facebookβs Distributed Data Store for the Social Graph
. Bronson et. al, USENIX ATC, 2013.
β
PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs
, Gonzalez et al, OSDI, 2012.
β
GraphX: Graph Processing in a Distributed Dataflow Framework,
Gonzalez et al, OSDI, 2014.
β
PyTorch-BigGraph: A Large-Scale Graph Embedding System.
Lerer et al, Proceedings of the 2nd SysML Conference, 2019.
β
Scalability! But at what COST?
McSherry et al, HOTOS 2015.
β
Arabesque: A System for Distributed Graph Mining.
Teixeira et. al, SOSP, 2015.
β
Fast and Concurrent RDF Queries with RDMA-based Distributed Graph Exploration.
Shi et. al, OSDI, 2016.
β
ASAP: Fast, Approximate Pattern Mining at Scale.
Iyer et. al, OSDI, 2018.
β
Grappa: A Latency-Tolerant Runtime for Large-Scale Irregular Applications.
Nelson et. al, USENIX ATC, 2015.
β
One Trillion Edges: Graph Processing at Facebook-Scale.
Ching et. al, VLDB, 2015.
Potpourri: Runtime, New Hardware Models, Serverless, and Approximation
Runtime
β
Weld: A Commom Runtime for High Performance Data Analytics
, Palkar et al, CIDR, 2017.
Hardware
β
In-Datacenter Performance Analysis of a Tensor Processing Unit
, Jouppi et al, CIDR, 2017.
β
A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services.
Putnam et. al, ISCA, 2014.
β
Strata: A Cross Media File System
. Kwon et. al, SOSP, 2017.
Serverless
β
Occupy the Cloud: Distributed Computing for the 99%
, Jonas et al, SoCC, 2017.
β
Serverless Computation with OpenLambda.
Hendrickson et. al, HotCloud, 2016.
β
Pocket: Elastic Ephemeral Storage for Serverless Analytics.
Klimovic et. al, OSDI, 2018.
β
Peeking Behind the Curtains of Serverless Platforms,
Wang et. al, USENIX ATC, 2018.
β
SOCK: Rapid Task Provisioning with Serverless-Optimized Containers
, Oakes et. al, USENIX ATC, 2018.
Approximation
β
BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data,
Agarwal et al, Eurosys, 2013.
β
BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees
. Park et. al, SIGMOD, 2019.
β
Quickr: Lazily Approximating Complex AdHoc Queries in BigData Clusters
. Kandula et. al, SIGMOD, 2016.
Other: RDMA
β
FaRM: Fast Remote Memory.
Dragojevic et. al, NSDI, 2014.
β
No compromises: distributed transactions with consistency, availability, and performance.
Dragojevic et. al, SOSP, 2015.
β
FaSST: Fast, Scalable and Simple Distributed Transactions with Two-Sided (RDMA) Datagram RPCs.
Kalia et. al, OSDI, 2016.
β
Datacenter RPCs can be General and Fast.
Kalia et. al, NSDI, 2019.
β
Distributed Lock Management with RDMA: Decentralization without Starvation.
Yoon et. al, SIGMOD, 2018.
β
Efficient Memory Disaggregation with Infiniswap.
Gu et. al, NSDI, 2017.
β
Accelerating Relational Databases by Leveraging Remote Memory and RDMA.
Li et. al, SIGMOD, 2016.
β
Remote Memory in the Age of Fast Networks.
Aguilera et. al, SoCC, 2017.
Other: Offload
β
Floem: A Programming System for NIC-Accelerated Network Applications.
Phothilimthana et. al, OSDI, 2018.
β
iPipe: A Framework for Building Datacenter Applications Using In-networking Processors.
Liu et. al, 2018.
β
Direct Universal Access: Making Data Center Resources Available to FPGA.
Shu et. al, NSDI, 2019.
β
β
β
Previous
PhD application guidelines
Next
Architecture
Last modified
1yr ago
Copy link
Outline
Architecture
Execution Engines, Resource Negotiators, Schedulers
Applications: Machine Learning
Applications: Batch Analytics and SQL Frameworks
Applications: Stream Processing
Applications: Graph Processing
Potpourri: Runtime, New Hardware Models, Serverless, and Approximation