# Faster and Cheaper Serverless Computing on Harvested Resources ### Presentation * Serverless computing * User's angle * Pay per use * No worry about VMs * Most popular offering - function as a service (FaaS) * Bounded completion time with no persistent states between invocations * FaaS Provider's angle * Provision, manage, and pay laaS providers for hosting VMs * Harvested Resources in Datacenters * IaaS providers offer surplus resources as VMs * Harvest VM * Allocated with a minimum size * Grows and shrinks to harvest all unallocated resources * Only evicted if minimum size is needed for a regular VM * Grace period before eviction (e.g., 30 seconds in Azure) * Characterization - Harvest VMs * Eviction: * \>90% harvest VMs live longer than 1 day * \>60% Harvest VMs live longer than 1 month * Harvest VM evictions are generally infrequent * Resource variation: * \>70% intervals longer than 10 minutes * \~35% intervals longer 1 hour * Resource variation much more frequent than evictions * Characterization - FaaS workloads * Invocation duration & VM grace period * Long invocation - longer than 30s (VM eviction grace period) * Long application - at least 1 invocation longer than 30 seconds * Short invocations can always terminate in VM grace period * 96% invocations shorter than 30 seconds * Long invocations take over 82% of the total execution time * Application's angle * 48.7% applications are long applications * Long applications take over 99.7% of the total execution time * Nearly half of the applications are long, and long applications take over almost all the execution time * Take-aways * FaaS workloads are a good fit for Harvest VMs * Short invocation durations * Relatively long harvest VM lifetime * Harvest VM resource variation much more common than evictions * Long applications take the vast majority of the total execution time * Handling Evictions * How to eliminate or minimize invocation failure caused by Harvest VM evictions? * Use a mix of Harvest VMs and regular VMs * Strategies with different tradeoff between reliability and efficiency * No failure: guaranteed no failure * Guarantee no invocation failure caused by Harvest VM evictions * Assign all long applications to regular VMs * Efficiency? * Fraction of computation hosted by cheap Harvest VMs * 12% of computation capacity hosted by Harvest VMs * Bounded failure * Upper bound (100 - x)% per application eviction failure rate * Assign application to regular VM if xth duration percentile longer than 30s * Worst case: all long invocations on Harvest VM fails --> (100-x)% failure rate * How about efficiency? * Failure rate < 1% --> 45.7% computation hosted by Harvest VMs * Failure rate < 0.1% --> 28% computation hosted by Harvest VMs * Live and Let Die * No guarantee on failure rate * All applications on Harvest VMs * How about reliability? * Invocation failure rate * Worst: 99.99% success rate * 7 nines of reliability * Failure require two low-probability events to happen simultaneously * A harvest VM gets evicted while it is running a long invocation * Best efficiency and actual low failure rate * Handling Resource Variability * Join-the-shortest-queue (JSQ) leads to high cold start rate * Invocations of the same application distributed across all VMs * Inter-arrival time longer than container keep-alive * Min-worker-set (MWS) * Consolidates each application to a minimal set of backend * Shorter inter-arrival time --> warm starts * Consistent hashing to minimize reshuffling of home VMs * Implementation on OpenWhisk * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2FnbUn8YQuFSAZX2Trs8vO%2Fimage.png?alt=media\&token=7be68160-6b6b-4a43-bf91-cebcad9d44b1) * Harvest Monitor * Collects resource information & eviction signal * Controller * Maintains data from Harvest Monitor * Implement MWS * Resource monitor * Tracks resource variation in the system * Spins up new VMs to maintain available resources Evaluation ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2F33y39rz0jMPKNWCnCVza%2Fimage.png?alt=media\&token=052e8aec-edb4-4419-a0fe-56fb3acb74af) Conclusion * To host serverless platforms on harvested resources * Quantify the challenges of using harvested resources for serverless invocations, including Harvest VM evictions and resource variation * Demonstrate the reliability of hosting serverless workloads on harvested resources * Demonstrate the performance and economic benefits of hosting serverless platforms on harvested resources --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://sliu583.gitbook.io/blog/conference/index/sosp-21/learning/faster-and-cheaper-serverless-computing-on-harvested-resources.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.