# Faster and Cheaper Serverless Computing on Harvested Resources ### Presentation * Serverless computing * User's angle * Pay per use * No worry about VMs * Most popular offering - function as a service (FaaS) * Bounded completion time with no persistent states between invocations * FaaS Provider's angle * Provision, manage, and pay laaS providers for hosting VMs * Harvested Resources in Datacenters * IaaS providers offer surplus resources as VMs * Harvest VM * Allocated with a minimum size * Grows and shrinks to harvest all unallocated resources * Only evicted if minimum size is needed for a regular VM * Grace period before eviction (e.g., 30 seconds in Azure) * Characterization - Harvest VMs * Eviction: * \>90% harvest VMs live longer than 1 day * \>60% Harvest VMs live longer than 1 month * Harvest VM evictions are generally infrequent * Resource variation: * \>70% intervals longer than 10 minutes * \~35% intervals longer 1 hour * Resource variation much more frequent than evictions * Characterization - FaaS workloads * Invocation duration & VM grace period * Long invocation - longer than 30s (VM eviction grace period) * Long application - at least 1 invocation longer than 30 seconds * Short invocations can always terminate in VM grace period * 96% invocations shorter than 30 seconds * Long invocations take over 82% of the total execution time * Application's angle * 48.7% applications are long applications * Long applications take over 99.7% of the total execution time * Nearly half of the applications are long, and long applications take over almost all the execution time * Take-aways * FaaS workloads are a good fit for Harvest VMs * Short invocation durations * Relatively long harvest VM lifetime * Harvest VM resource variation much more common than evictions * Long applications take the vast majority of the total execution time * Handling Evictions * How to eliminate or minimize invocation failure caused by Harvest VM evictions? * Use a mix of Harvest VMs and regular VMs * Strategies with different tradeoff between reliability and efficiency * No failure: guaranteed no failure * Guarantee no invocation failure caused by Harvest VM evictions * Assign all long applications to regular VMs * Efficiency? * Fraction of computation hosted by cheap Harvest VMs * 12% of computation capacity hosted by Harvest VMs * Bounded failure * Upper bound (100 - x)% per application eviction failure rate * Assign application to regular VM if xth duration percentile longer than 30s * Worst case: all long invocations on Harvest VM fails --> (100-x)% failure rate * How about efficiency? * Failure rate < 1% --> 45.7% computation hosted by Harvest VMs * Failure rate < 0.1% --> 28% computation hosted by Harvest VMs * Live and Let Die * No guarantee on failure rate * All applications on Harvest VMs * How about reliability? * Invocation failure rate * Worst: 99.99% success rate * 7 nines of reliability * Failure require two low-probability events to happen simultaneously * A harvest VM gets evicted while it is running a long invocation * Best efficiency and actual low failure rate * Handling Resource Variability * Join-the-shortest-queue (JSQ) leads to high cold start rate * Invocations of the same application distributed across all VMs * Inter-arrival time longer than container keep-alive * Min-worker-set (MWS) * Consolidates each application to a minimal set of backend * Shorter inter-arrival time --> warm starts * Consistent hashing to minimize reshuffling of home VMs * Implementation on OpenWhisk * ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2FnbUn8YQuFSAZX2Trs8vO%2Fimage.png?alt=media\&token=7be68160-6b6b-4a43-bf91-cebcad9d44b1) * Harvest Monitor * Collects resource information & eviction signal * Controller * Maintains data from Harvest Monitor * Implement MWS * Resource monitor * Tracks resource variation in the system * Spins up new VMs to maintain available resources Evaluation ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MVORxAomcgtzVVUqmws%2Fuploads%2F33y39rz0jMPKNWCnCVza%2Fimage.png?alt=media\&token=052e8aec-edb4-4419-a0fe-56fb3acb74af) Conclusion * To host serverless platforms on harvested resources * Quantify the challenges of using harvested resources for serverless invocations, including Harvest VM evictions and resource variation * Demonstrate the reliability of hosting serverless workloads on harvested resources * Demonstrate the performance and economic benefits of hosting serverless platforms on harvested resources