🐣
Reading List
  • Starting point
  • Reference list
  • PhD application guidelines
  • Big Data System
    • Index
      • Architecture
        • Storage
          • Sun's Network File System (NFS)
      • Execution Engine, Resource Negotiator, Schedulers
        • Execution Engines
        • Resource Negotiator
        • Schedulers
      • Machine Learning
      • SQL Framework
      • Stream Processing
      • Graph Processing
      • Potpourri: Hardware, Serverless and Approximation
  • Operating System
    • Index
      • OSTEP
        • Virtualization
          • CPU Abstraction: the Process
          • Interlude: Process API
          • Mechanism: Limited Direct Execution
        • Intro
  • Networking
    • Index
      • CS 294 (Distributed System)
        • Week 1 - Global State and Clocks
          • Distributed Snapshots: Determining Global States of Distributed Systems
          • Time, Clocks, and the Ordering of Events in a Distributed System
        • Weak 5 - Weak Consistency
          • Dynamo: Amazon's Highly Available Key-value Store
          • Replicating Data Consistency Explained Through Baseball
          • Managing update conflicts in Bayou, a weakly connected replicated storage system
      • CS 268 (Adv Network)
        • Intro
        • Internet Architecture
          • Towards an Active Network Architecture
          • The Design Philosophy of the DARPA Internet Protocols
        • Beyond best-effort/Unicast
          • Core Based Trees (CBT)
          • Multicast Routing in Internetworks and Extended LANs
        • Congestion Control
        • SDN
          • ONIX: A Distributed Control Platform for Large-scale Production Networks
          • B4: Experience with a Globally-Deployed Software Defined WAN
          • How SDN will shape networking
          • The Future of Networking, and the Past of Protocols
        • Datacenter Networking
          • Fat tree
          • Jellyfish
        • BGP
          • The Case for Separating Routing from Routers
        • Programmable Network
          • NetCache
          • RMT
        • Datacenter Congestion Control
          • Swift
          • pFabric
        • WAN CC
          • Starvation (Sigcomm 22)
        • P2P
          • Design and Evaluation of IPFS: A Storage Layer for the Decentralized Web
          • The Impact of DHT Routing Geometry on Resilience and Proximity
        • Net SW
          • mTCP
          • The Click modular router
        • NFV
          • Performance Interfaces for Network Functions
          • Making Middleboxes Someone Else's Problem: Network Processing as a Cloud Service
        • Ethics
          • On the morals of network research and beyond
          • The collateral damage of internet censorship by DNS injection
          • Encore: Lightweight Measurement of Web Censorship with Cross-Origin Requests
        • Low Latency
          • Aquila: A unified, low-latency fabric for datacenter networks
          • cISP: A Speed-of-Light Internet Service Provider
        • Disaggregation
          • Network Requirements for Resource Disaggregation
        • Tenant Networking
          • Invisinets
          • NetHint: While-Box Networking for Multi-Tenant Data Centers
        • Verification
          • A General Approach to Network Configuration Verification
          • Header Space Analysis: Static Checking for Networks
        • ML
          • SwitchML
          • Fast Distributed Deep Learning over RDMA
      • Computer Networking: A Top-Down Approach
        • Chapter 1. Computer Network and the Internet
          • 1.1 What Is the Internet?
          • 1.2 The Network Edge
          • 1.3 The Network Core
        • Stanford CS144
          • Chapter 1
            • 1.1 A Day in the Life of an Application
            • 1.2 The 4-Layer Internet Model
            • 1.3 The IP Service Model
            • 1.4 A Day in the Life of a Packet
            • 1.6 Layering Principle
            • 1.7 Encapsulation Principle
            • 1.8 Memory layout and Endianness
            • 1.9 IPv4 Addresses
            • 1.10 Longest Prefix Match
            • 1.11 Address Resolution Protocol (ARP)
            • 1.12 The Internet and IP Recap
      • Reading list
        • Elastic hyperparameter tuning on the cloud
        • Rethinking Networking Abstractions for Cloud Tenants
        • Democratizing Cellular Access with AnyCell
        • Dagger: Efficient and Fast RPCs in Cloud Microservices in Near-Memory Reconfigurable NICs
        • Sage: Practical & Scalable ML-Driven Performance Debugging in Microservices
        • Faster and Cheaper Serverless Computing on Harvested Resources
        • Network-accelerated Distributed Machine Learning for Multi-Tenant Settings
        • User-Defined Cloud
        • LegoOS: A Disseminated Distributed OS for Hardware Resource Disaggregation
        • Beyond Jain's Fairness Index: Setting the Bar For The Deployment of Congestion Control Algorithms
        • IncBricks: Toward In-Network Computation with an In-Network Cache
  • Persistence
    • Index
      • Hardware
        • Enhancing Lifetime and Security of PCM-Based Main Memory with Start-Gap Wear Leveling
        • An Empirical Guide to the Behavior and Use of Scalable Persistent Memory
  • Database
    • Index
  • Group
    • WISR Group
      • Group
        • Offloading distributed applications onto smartNICs using iPipe
        • Semeru: A memory-disaggregated managed runtime
      • Cache
        • Index
          • TACK: Improving Wireless Transport Performance by Taming Acknowledgements
          • LHD: Improving Cache Hit Rate by Maximizing Hit Density
          • AdaptSize: Orchestrating the Hot Object Memory Cache in a Content Delivery Network
          • Clustered Bandits
          • Important Sampling
          • Contexual Bandits and Reinforcement Learning
          • Reinforcement Learning for Caching with Space-Time Popularity Dynamics
          • Hyperbolic Caching: Flexible Caching for Web Applications
          • Learning Cache Replacement with CACHEUS
          • Footprint Descriptors: Theory and Practice of Cache Provisioning in a Global CDN
      • Hyperparam Exploration
        • Bayesian optimization in cloud machine learning engine
    • Shivaram's Group
      • Tools
      • Group papers
        • PushdownDB: Accelerating a DBMS using S3 Computation
        • Declarative Machine Learning Systems
        • P3: Distributed Deep Graph Learning at Scale
        • Accelerating Graph Sampling for Graph Machine Learning using GPUs
        • Unicorn: A System for Searching the Social Graph
        • Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless
        • Garaph: Efficient GPU-accelerated GraphProcessing on a Single Machine with Balanced Replication
        • MOSAIC: Processing a Trillion-Edge Graph on a Single Machine
        • Fluid: Resource-aware Hyperparameter Tuning Engine
        • Lists
          • Wavelet: Efficient DNN Training with Tick-Tock Scheduling
          • GPU Lifetimes on Titan Supercomputer: Survival Analysis and Reliability
          • ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training
          • ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
          • KungFu: Making Training inDistributed Machine Learning Adaptive
        • Disk ANN
      • Queries Processing
        • Building An Elastic Query Engine on Disaggregated Storage
        • GRIP: Multi-Store Capacity-Optimized High-Performance NN Search
        • Milvus: A Purpose-Built Vector Data Management System
        • Query2box: Reasoning over Knowledge Graphs in Vector Space using Box Embeddings
        • Billion-scale Approximate Nearest Neighbor Search
        • DiskANN: Fast accurate billion-point nearest neighbor search on a single node
        • KGvec2go - Knowledge Graph Embeddings as a Service
    • Seminar & Talk
      • Berkeley System Seminar
        • RR: Engineering Record and Replay for Deployability
        • Immortal Threads: Multithreaded Event-driven Intermittent Computing on Ultra-Low-Power Microcontroll
      • Berkeley DB Seminar
        • TAOBench: An End-to-End Benchmark for Social Network Workloads
      • PS2
      • Sky Seminar Series
        • Spring 23
          • Next-Generation Optical Networks for Emerging ML Workloads
      • Reading List
        • Confluo: Distributed Monitoring and Diagnosis Stack for High-speed Networks
        • Rearchitecting Linux Storage Stack for µs Latency and High Throughput
        • eBPF: rethinking the linux kernel
        • BPF for Storage: An Exokernel-Inspired Approach
        • High Velocity Kernel File Systems with Bento
        • Incremental Path Towards a Safe OS Kernel
        • Toward Reconfigurable Kernel Datapaths with Learned Optimizations
        • A Vision for Runtime Programmable Networks
        • The Demikernel and the future of kernal-bypass systems
        • Floem: A programming system for NIC-accelerated network applications
        • High Performance Data Center Operating Systems
        • Leveraging Service Meshes as a New Network Layer
        • Automatically Discovering Machine Learning Optimizations
        • Beyond Data and Model Parallelism for Deep Neural Networks
        • IOS: Inter-Operator Scheduler for CNN Acceleration
        • Building An Elastic Query Engine on Disaggregated Storage
        • Sundial: Fault-tolerant Clock Synchronization for Datacenters
        • MIND: In-Network Memory Management for Disaggregated Data Centers
        • Understanding host network stack overheads
        • From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers
        • Redesigning Storage Systems for Future Workloads Hardware and Performance Requirements
        • Are Machine Learning Cloud APIs Used Correctly?
        • Fault-tolerant and transactional stateful serverless workflows
      • Reading Groups
        • Network reading group
          • Recap
          • ML & Networking
            • Video Streaming
              • Overview
              • Reducto: On-Camera Filtering for Resource Efficient Real-Time Video Analytics
              • Learning in situ: a randomized experiment in video streaming
              • SENSEI: Aligning Video Streaming Quality with Dynamic User Sensitivity
              • Neural Adaptive Video Streaming with Pensieve
              • Server-Driven Video Streaming for Deep Learning Inference
            • Congestion Control
              • ABC: A Simple Explicit Congestion Controller for Wireless Networks
              • TCP Congestion Control: A Systems Approach
                • Chapter 1: Introduction
              • A Deep Reinforcement Learning Perspective on Internet Congestion Control
              • Pantheon: the training ground for Internet congestion-control research
            • Other
              • On the Use of ML for Blackbox System Performance Prediction
              • Marauder: Synergized Caching and Prefetching for Low-Risk Mobile App Acceleration
              • Horcrux: Automatic JavaScript Parallelism for Resource-Efficient Web Computation
              • Snicket: Query-Driven Distributed Tracing
            • Workshop
          • Homa: A Receiver-Driven Low-Latency Transport Protocol Using Network Priorities
        • DB reading group
          • CliqueMap: Productionizing an RMA-Based Distributed Caching System
          • Hash maps overview
          • Dark Silicon and the End of Multicore Scaling
        • WISR
          • pFabric: Minimal Near-Optimal Datacenter Transport
          • Scaling Distributed Machine Learning within-Network Aggregation
          • WCMP: Weighted Cost Multipathing for Improved Fairness in Data Centers
          • Data center TCP (DCTCP)
      • Wisconsin Seminar
        • Enabling Hyperscale Web Services
        • The Lottery Ticket Hypothesis
        • External Merge Sort for Top-K Queries: Eager input filtering guided by histograms
      • Stanford MLSys Seminar
        • Episode 17
        • Episode 18
  • Cloud Computing
    • Index
      • Cloud Reading Group
        • Owl: Scale and Flexibility in Distribution of Hot Contents
        • RubberBand: cloud-based hyperparameter tuning
  • Distributed System
    • Distributed Systems Lecture Series
      • 1.1 Introduction
  • Conference
    • Index
      • Stanford Graph Learning Workshop
        • Overview of Graph Representation Learning
      • NSDI 2022
      • OSDI 21
        • Graph Embeddings and Neural Networks
        • Data Management
        • Storage
        • Preview
        • Optimizations and Scheduling for ML
          • Oort: Efficient Federated Learning via Guided Participant Selection
          • PET: Optimizing Tensor Programs with Partially Equivalent Transformations and Automated Corrections
      • HotOS 21
        • FlexOS: Making OS Isolation Flexible
      • NSDI 21
        • Distributed System
          • Fault-Tolerant Replication with Pull-Based Consensus in MongoDB
          • Ownership: A Distributed Futures System for Fine-Grained Tasks
          • Caerus: NIMBLE Task Scheduling for Serverless Analytics
          • Ship Computer or Data? Why not both?
          • EPaxos Revisited
          • MilliSort and MilliQuery: Large-Scale Data-Intensive Computing in Milliseconds
        • TEGRA: Efficient Ad-Hoc Analytics on Evolving Graphs
        • GAIA: A System for Interactive Analysis on Distributed Graphs Using a High-Level Language
      • CIDR 21
        • Cerebro: A Layered Data Platform for Scalable Deep Learning
        • Magpie: Python at Speed and Scale using Cloud Backends
        • Lightweight Inspection of Data Preprocessingin Native Machine Learning Pipelines
        • Lakehouse: A New Generation of Open Platforms that UnifyData Warehousing and Advanced Analytics
      • MLSys 21
        • Chips and Compilers Symposium
        • Support sparse computations in ML
      • SOSP 21
        • SmartNic
          • LineFS: Efficient SmartNIC offload of a distributed file system with pipeline parallelism
          • Xenic: SmartNIC-accelerated distributed transacitions
        • Graphs
          • Mycelium: Large-Scale Distributed Graph Queries with Differential Privacy
          • dSpace: Composable Abstractions for Smart Spaces
        • Consistency
          • Efficient and Scalable Thread-Safety Violation Detection
          • Understanding and Detecting Software Upgrade Failures in Distributed Systems
        • NVM
          • HeMem: Scalable Tiered Memory Management for Big Data Applications and Real NVM
        • Learning
          • Bladerunner: Stream Processing at Scale for a Live View of Backend Data Mutations at the Edge
          • Faster and Cheaper Serverless Computing on Harvested Resources
  • Random
    • Reading List
      • Random Thoughts
      • Hesse
      • Anxiety
  • Grad School
    • Index
      • Resources for undergraduate students
Powered by GitBook
On this page
  • Architecture
  • Execution Engines, Resource Negotiators, Schedulers
  • Applications: Machine Learning
  • Applications: Batch Analytics and SQL Frameworks
  • Applications: Stream Processing
  • Applications: Graph Processing
  • Potpourri: Runtime, New Hardware Models, Serverless, and Approximation

Was this helpful?

  1. Big Data System

Index

Reference: CS744 (UW-Madison) and CS 494 (UIC)

PreviousPhD application guidelinesNextArchitecture

Last updated 4 years ago

Was this helpful?

Architecture

Compute + Overall

  • : An Introduction to the Design of Warehouse-Scale Machines , L.A. Barroso, U. Holzle, Synthesis Lectures on Computer Architecture, 2009. Chapter 1 and 2.

Networks

  • , Greenberg et al., SIGCOMM 2009.

  • , Singh et al., SIGCOMM 2015.

Storage (in a bit detailed fashion)

  • , Schvachko et al, MSST, 2010

  • , Ghemawat et al, SOSP, 2003.

  • . Nightingale et. al, OSDI, 2012.

  • Rashmi et. al, OSDI, 2016

  • Muralidhar et. al, OSDI, 2014.

  • Chang et. al, OSDI, 2006.

  • DeCandia et. al, SOSP, 2007.

  • Corbett et. al, OSDI, 2012.

  • Huang et. al, SOSP, 2013.

  • Nishtala et. al, NSDI, 2013.

  • Mike Burrows, OSDI, 2006.

Execution Engines, Resource Negotiators, Schedulers

Execution Engines

  • Load balancing

Resource Negotiator

Scheduling

  • Packing

  • Re-Planning

  • Threads

  • Cache

Applications: Machine Learning

Applications: Batch Analytics and SQL Frameworks

Applications: Stream Processing

Applications: Graph Processing

Potpourri: Runtime, New Hardware Models, Serverless, and Approximation

  • Runtime

  • Hardware

  • Serverless

  • Approximation

  • Other: RDMA

  • Other: Offload

, OSDI, 2004

Isard et. al, EuroSys, 2007.

. Murray et. al, NSDI, 2011.

Ananthanarayanan et al, OSDI, 2010.

Yu et. al, OSDI, 2008.

Goetz Graefe, SIGMOD, 1990.

Ananthanarayanan et. al, NSDI, 2012.

, Zaharia et al, NSDI, 2012.

, Saha et al, SIGMOD, 2015.

Essertel et. al, OSDI, 2018.

Transaction: Crooks et. al, OSDI, 2018.

. Patel et. al, SIGCOMM, 2013.

Gandhi et. al, SIGCOMM, 2014.

, Miao et. al, SIGCOMM, 2017.

, Vavilapalli et al, SOCC, 2013.

Hindman et al, NSDI, 2011.

Ghodsi et al, NSDI, 2011.

Verma et. al, EuroSys, 2015.

Grandl et. al, OSDI, 2016.

Grandl et. al, SIGCOMM, 2014.

Isard et. al, SOSP, 2009.

Mahajan et. al, OSDI, 2018.

Qin et. al, OSDI, 2018.

Berger et. al, OSDI, 2018.

, Li et al, OSDI, 2014.

, Kim et al, EuroSys, 2016.

, Zhang et al, SoCC, 2017.

, Abadi et al, OSDI, 2016.

, Shen et al, VLDB, 2020

Xiao et al, OSDI, 2018.

, Crankshaw et al, NSDI, 2017.

Narayanan et al, SOSP 2019.

, Chen et al, OSDI, 2018

, Moritz et al, OSDI, 2018.

, Feng et al, SIGMOD, 2012

Zhang et. al, USENIX ATC, 2018.

. Lee et. al, OSDI, 2018.

, Hazelwood et. al, HPCA, 2018.

Chen et. al, Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015.

, Low et al, VLDB, 2012.

. Peng et. al, EuroSys, 2018.

Gu et. al, NSDI, 2019.

. Jeong et. al, 2018.

, Sparks et al, ICDE, 2017.

, Chilimbi et al, OSDI, 2014.

Zhang and Re, VLDB, 2014.

, Armburst et al, SIGMOD, 2015.

, Huai et al, SIGMOD, 2014.

, Viswanathan et al, OSDI, 2016.

Vulimiri et al, NSDI, 2015.

. Chaiken et al, VLDB

Dageville et al, SIGMOD 2016.

. Vuppalapati et al, NSDI 2020.

Kornacker et. al, CIDR, 2015.

Melnik et. al, VLDB, 2010.

Chandramouli et. al, VLDB, 2014.

Polychroniou et. al, SIGMOD, 2015.

Balkesen et. al, VLDB, 2013.

. Madden et. al, OSDI, 2002.

, Toshniwal et al, SIGMOD, 2014.

, Kulkarni et al, SIGMOD, 2015.

. Chen et. al, SIGMOD, 2016.

, Zaharia et al, SOSP, 2013.

Reading:

Carbone et al, Bulletin of the IEEE Computer Society Technical Committee on Data Engineering, 2015.

Kreps et al, NetDB Workshop, 2011.

Also this document of comparison of widely used .

, Lin et al, NSDI, 2016.

. Venkataraman et. al, SOSP, 2017.

. Mai et. al, PVLDB, 2018.

Rajadurai et. al, ASPLOS, 2018.

Abadi et. al, VLDB, 2003.

Kalavri et. al, OSDI, 2018.

, Murray et al, SOSP, 2013.

Akidau et. al, VLDB, 2015.

Malewicz et al, SIGMOD, 2010.

. Bronson et. al, USENIX ATC, 2013.

, Gonzalez et al, OSDI, 2012.

Gonzalez et al, OSDI, 2014.

Lerer et al, Proceedings of the 2nd SysML Conference, 2019.

McSherry et al, HOTOS 2015.

Teixeira et. al, SOSP, 2015.

Shi et. al, OSDI, 2016.

Iyer et. al, OSDI, 2018.

Nelson et. al, USENIX ATC, 2015.

Ching et. al, VLDB, 2015.

, Palkar et al, CIDR, 2017.

, Jouppi et al, CIDR, 2017.

Putnam et. al, ISCA, 2014.

. Kwon et. al, SOSP, 2017.

, Jonas et al, SoCC, 2017.

Hendrickson et. al, HotCloud, 2016.

Klimovic et. al, OSDI, 2018.

Wang et. al, USENIX ATC, 2018.

, Oakes et. al, USENIX ATC, 2018.

Agarwal et al, Eurosys, 2013.

. Park et. al, SIGMOD, 2019.

. Kandula et. al, SIGMOD, 2016.

Dragojevic et. al, NSDI, 2014.

Dragojevic et. al, SOSP, 2015.

Kalia et. al, OSDI, 2016.

Kalia et. al, NSDI, 2019.

Yoon et. al, SIGMOD, 2018.

Gu et. al, NSDI, 2017.

Li et. al, SIGMOD, 2016.

Aguilera et. al, SoCC, 2017.

Phothilimthana et. al, OSDI, 2018.

Liu et. al, 2018.

Shu et. al, NSDI, 2019.

The Datacenter as a Computer
VL2: A Scalable and Flexible Data Center Network
Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google’s Datacenter Network
The Hadoop Distributed File System
The Google File System
NFS: Sun's Network File System
Flat Datacenter Storage
EC-Cache: Load-balanced, Low-latency Cluster Caching with Online Erasure Coding.
f4: Facebook’s Warm BLOB Storage System.
Bigtable: A Distributed Storage System for Structured Data.
Dynamo: Amazon’s Highly Available Key-value Store.
Spanner: Google’s Globally-Distributed Database.
An Analysis of Facebook Photo Caching.
Scaling Memcache at Facebook.
The Chubby lock service for loosely-coupled distributed systems.
MapReduce: Simplified Data Processing on Large Clusters, Dean and Ghemawat
Dryad: Distributed Data-Parallel Programs from Sequential Building Blocks.
CIEL: a universal execution engine for distributed data-flow computing
Reining in the Outliers in Map-Reduce Clusters using Mantri,
DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language.
Encapsulation of parallelism in the Volcano query processing system.
PACMan: Coordinated Memory Caching for Parallel Jobs,
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
Apache Tez: A Unifying Framework for Modeling and Building Data Processing Applications
Flare: Optimizing Apache Spark with Native Compilation for Scale-Up Architectures and Medium-Size Data.
Obladi: Oblivious Serializable Transactions in the Cloud.
Ananta: Cloud Scale Load Balancing
Duet: Cloud Scale Load Balancing with Hardware and Software.
SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs
Apache Hadoop YARN: Yet Another Resource Negotiator
Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center,
Dominant Resource Fairness: Fair Allocation of Multiple Resource Types,
Borg: Large-scale cluster management at Google with Borg.
Altruistic Scheduling in Multi-Resource Clusters.
Multi-Resource Packing for Cluster Schedulers,
Quincy: Fair Scheduling for Distributed Computing Clusters.
Dynamic Query Re-Planning using QOOP.
Arachne: Core-Aware Thread Management.
RobinHood: Tail Latency Aware Caching – Dynamic Reallocation from Cache-Rich to Cache-Poor.
Scaling Distributed Machine Learning with the Parameter Server
STRADS: A Distributed Framework for Scheduled Model Parallel Machine Learning
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning
TensorFlow: A System for Large-Scale Machine Learning
Pytorch Distributed: Experiences on Accelerating Data Parallel Training
Gandiva: Introspective Cluster Scheduling for Deep Learning,
Clipper: A Low-Latency Online Prediction Serving System
PipeDream: Generalized Pipeline Parallelism for DNN Training.
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
Ray: A Distributed Framework for Emerging AI Applicationss
Towards a Unified Architecture for in-RDBMS Analytics
DeepCPU: Serving RNN-based Deep Learning Models 10x Faster.
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems.
Distributed GraphLab: A Framework for Machine Learning and Data Mining in the Cloud
Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters
Tiresias: A GPU Cluster Manager for Distributed Deep Learning.
Janus: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs
KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analystics
Project Adam: Building an Efficient and Scalable Deep Learning Training System
DimmWitted: A Study of Main-Memory Statistical Analytics.
Spark SQL: Relational Data Processing in Spark
Major technical advancements in Apache Hive
Clarinet: WAN-Aware Optimization for Analytics Queries
Global Analytics in the Face of Bandwidth and Regulatory Constraints,
SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets
The Snowflake Elastic Data Warehouse.
Building an Elastic Query Engine on Disaggregated Storage
Impala: A Modern, Open-Source SQL Engine for Hadoop.
Dremel: Interactive Analysis of Web-Scale Datasets.
Trill: A High-Performance Incremental Query Processor for Diverse Analytics.
Rethinking SIMD Vectorization for In-Memory Databases.
Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited.
TAG: a Tiny AGgregation Service for Ad-Hoc Sensor Networks
Storm @Twitter
Twitter Heron: Stream Processing at Scale
Realtime Data Processing at Facebook
Discretized Streams: Fault-Tolerant Streaming Computation at Scale
Spark Structured Streaming
Apache Flink: Stream and Batch Processing in a Single Engine,
Kafka Distributed Messaging System for Log Processing,
Queuing Messaging Processing Systems
StreamScope: Continuous Reliable Distributed Processing of Big Data Streams
Drizzle: Fast and Adaptable Stream Processing at Scale
Chi: A Scalable and Programmable Control Plane for Distributed Stream Processing Systems
Gloss: Seamless Live Reconfiguration and Reoptimization of Stream Programs.
Aurora: a new model and architecture for data stream management.
Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows.
Naiad: A Timely Dataflow System
The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing.
Pregel: A System for Large-Scale Graph Processing,
TAO: Facebook’s Distributed Data Store for the Social Graph
PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs
GraphX: Graph Processing in a Distributed Dataflow Framework,
PyTorch-BigGraph: A Large-Scale Graph Embedding System.
Scalability! But at what COST?
Arabesque: A System for Distributed Graph Mining.
Fast and Concurrent RDF Queries with RDMA-based Distributed Graph Exploration.
ASAP: Fast, Approximate Pattern Mining at Scale.
Grappa: A Latency-Tolerant Runtime for Large-Scale Irregular Applications.
One Trillion Edges: Graph Processing at Facebook-Scale.
Weld: A Commom Runtime for High Performance Data Analytics
In-Datacenter Performance Analysis of a Tensor Processing Unit
A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services.
Strata: A Cross Media File System
Occupy the Cloud: Distributed Computing for the 99%
Serverless Computation with OpenLambda.
Pocket: Elastic Ephemeral Storage for Serverless Analytics.
Peeking Behind the Curtains of Serverless Platforms,
SOCK: Rapid Task Provisioning with Serverless-Optimized Containers
BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data,
BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees
Quickr: Lazily Approximating Complex AdHoc Queries in BigData Clusters
FaRM: Fast Remote Memory.
No compromises: distributed transactions with consistency, availability, and performance.
FaSST: Fast, Scalable and Simple Distributed Transactions with Two-Sided (RDMA) Datagram RPCs.
Datacenter RPCs can be General and Fast.
Distributed Lock Management with RDMA: Decentralization without Starvation.
Efficient Memory Disaggregation with Infiniswap.
Accelerating Relational Databases by Leveraging Remote Memory and RDMA.
Remote Memory in the Age of Fast Networks.
Floem: A Programming System for NIC-Accelerated Network Applications.
iPipe: A Framework for Building Datacenter Applications Using In-networking Processors.
Direct Universal Access: Making Data Center Resources Available to FPGA.