# Semeru: A memory-disaggregated managed runtime ### Motivation ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdai4VKvv74igZ-txD%2Fimage.png?alt=media\&token=c43d4d36-3759-4d7f-92e0-9d9cf2afdaa2) * All resources sitting in different resource pools * CPU servers: small memory * Memory servers: not high processing powers, handling memory accesses * Today usage? * Storage servers: weak core * Network: InfiniBand, high bandwidth ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdbaLbzXyAuDnLkGQw%2Fimage.png?alt=media\&token=a14d9e6d-df14-4c7a-b85c-fc8c97fbcf74) * Resource is not fully utilized * This can help all the workloads to share resource better * Multiplex different people's workload * Moving towards * But most cloud services are still pre-configured ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdcjXNETwKhQ4xvqSO%2Fimage.png?alt=media\&token=7a4e7619-a49a-4ccf-a7a5-d9212c4c3567) * Don't understand the actual workload * Managed languages: * Program relies on language runtime (sit between user runtime and OS) to manage memory * Java (objects), how these are allocated (memory), the allocated memory is not contiguous (?) digging into JVM ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWddWVE8mzD6okpW0K4%2Fimage.png?alt=media\&token=ebb09b06-1710-4e8f-b89d-de44adde354d) * Write something in C, manage memory allocation * JVM takes care of tracing in the memory ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWddnJ6ZjZWAOizCfLL%2Fimage.png?alt=media\&token=baadc274-839d-4e95-98b8-deb1b0ffc83a) * Takes over CPU and bandwidth ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWde2ee8Ijf1QYj6Uxi%2Fimage.png?alt=media\&token=4d556ca0-e9d9-4901-8e93-568ca009eefb) * Remote memory, two Spark applications (graph like applications) * Cache Ratio * 50%/75% goes remote * No swap: best case (baseline), all accesses go locally ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfMdb5IkA-32z4Sdp%2Fimage.png?alt=media\&token=38fad748-9b08-461b-9c66-c79e906bb316) * GC work * Not much compute (suitable for memory server compute power) * Data loaded locally to the memory server, cheaper * Run concurrently with the program itself (not compete for resources) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfw5-TCZ-0VTNHH48%2Fimage.png?alt=media\&token=786026f0-e07d-476b-927e-061fe9508ee6) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfzuDqwsQrNMY1rt3%2Fimage.png?alt=media\&token=2b532ded-44e6-4808-a6f7-c700c9cc668f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgC42iwVVPBKQ4GdU%2Fimage.png?alt=media\&token=2cbeb563-9e45-4240-b9ed-1b708da59aba) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgFkoB1gCjk_TmRKD%2Fimage.png?alt=media\&token=26ca787f-2047-4581-b119-fd0495bbb972) * Accessing the dirty page again? ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgct4ax0bT-7ZAE6j%2Fimage.png?alt=media\&token=e2b4315c-b9eb-4c69-a7fa-aca941c9e5c1) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdglOeWKLNcWSb-LMF%2Fimage.png?alt=media\&token=c56d7629-66ca-49e9-99f9-767e660f52a7) * Java * GC: find all objects that will never be used again * E.x. variable within the loop, can be cleaned after this loop * Tracing is not computationally heavy * Cheaper to store all tracing on local memory in memory server ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdhvmCJ-MhlNwfc3YP%2Fimage.png?alt=media\&token=8814d827-7012-42cd-9566-0304dc62b59f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdiDL7poirQchhypqh%2Fimage.png?alt=media\&token=25a82bd0-cf54-4182-9f20-9e54f485599a) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdiHhp57s413euLhNB%2Fimage.png?alt=media\&token=1d3ae15e-c389-4a41-9ff2-a0ed2189c991) * Several rounds of GC: * If these objects are still there, likely to remain for longer period of time * Java does less cleaning to older objects, and spend more times on newly-created objects ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdirfLqKVbeLjvAOO9%2Fimage.png?alt=media\&token=a8ae29eb-bc65-4277-b804-8f08262c1b6e) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdj2MsdJZKpuKFqciQ%2Fimage.png?alt=media\&token=4e40fc73-0d3b-49b8-b1d7-0e1b96182274) * Memory servers: recycling the regions of themselves * Data layout: reduce memory segmentation #### #3: how to efficiently swap data ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdjPRqhjr-sO0ujZZ6%2Fimage.png?alt=media\&token=bd15430b-098f-48f7-b04a-5defc30c89c9) * Control path onto different memory servers * Data Path * Not interfere with each other * Paging: * Managed by the OS * Runtime access the page? * Bypass the OS? * JVM has the mapping between the two * Map the virtual pages in runtime to physical allocation on the memory servers * LegoOS: * this kind of disaggregated architecture * Not support JVM * Existing OS, then for OS, not be able to distinguish (which to handle locally, which to handle on remote). Implement this, having more flexibility. * Language runtime knows the program much better instead of OS ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdjb_zcV_tEw-wZ5Eb%2Fimage.png?alt=media\&token=96905434-4e22-4eea-a1cc-347e1d5df71f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdmUjkA0xQJ9o3YjU9%2F-MWdnUPMqG2e0bK2grBZ%2Fimage.png?alt=media\&token=b1c5f344-9503-428e-a63e-c1bf633c809a) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdmUjkA0xQJ9o3YjU9%2F-MWdnsm_SnBaaw8y4F9S%2Fimage.png?alt=media\&token=0a6998da-bf54-4517-b797-0d99c92a83eb) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdnu-4R8othAeFvFnm%2F-MWdo9titZBvLeZd5Bz0%2Fimage.png?alt=media\&token=85dfd5f8-4ee0-44c5-a999-a5264de64d60) * JVM on disaggregated * GC implemented inside of JVM rather than doing this directly from the OS