# Semeru: A memory-disaggregated managed runtime ### Motivation ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdai4VKvv74igZ-txD%2Fimage.png?alt=media\&token=c43d4d36-3759-4d7f-92e0-9d9cf2afdaa2) * All resources sitting in different resource pools * CPU servers: small memory * Memory servers: not high processing powers, handling memory accesses * Today usage? * Storage servers: weak core * Network: InfiniBand, high bandwidth ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdbaLbzXyAuDnLkGQw%2Fimage.png?alt=media\&token=a14d9e6d-df14-4c7a-b85c-fc8c97fbcf74) * Resource is not fully utilized * This can help all the workloads to share resource better * Multiplex different people's workload * Moving towards * But most cloud services are still pre-configured ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdcjXNETwKhQ4xvqSO%2Fimage.png?alt=media\&token=7a4e7619-a49a-4ccf-a7a5-d9212c4c3567) * Don't understand the actual workload * Managed languages: * Program relies on language runtime (sit between user runtime and OS) to manage memory * Java (objects), how these are allocated (memory), the allocated memory is not contiguous (?) digging into JVM ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWddWVE8mzD6okpW0K4%2Fimage.png?alt=media\&token=ebb09b06-1710-4e8f-b89d-de44adde354d) * Write something in C, manage memory allocation * JVM takes care of tracing in the memory ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWddnJ6ZjZWAOizCfLL%2Fimage.png?alt=media\&token=baadc274-839d-4e95-98b8-deb1b0ffc83a) * Takes over CPU and bandwidth ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWde2ee8Ijf1QYj6Uxi%2Fimage.png?alt=media\&token=4d556ca0-e9d9-4901-8e93-568ca009eefb) * Remote memory, two Spark applications (graph like applications) * Cache Ratio * 50%/75% goes remote * No swap: best case (baseline), all accesses go locally ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfMdb5IkA-32z4Sdp%2Fimage.png?alt=media\&token=38fad748-9b08-461b-9c66-c79e906bb316) * GC work * Not much compute (suitable for memory server compute power) * Data loaded locally to the memory server, cheaper * Run concurrently with the program itself (not compete for resources) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfw5-TCZ-0VTNHH48%2Fimage.png?alt=media\&token=786026f0-e07d-476b-927e-061fe9508ee6) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdfzuDqwsQrNMY1rt3%2Fimage.png?alt=media\&token=2b532ded-44e6-4808-a6f7-c700c9cc668f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgC42iwVVPBKQ4GdU%2Fimage.png?alt=media\&token=2cbeb563-9e45-4240-b9ed-1b708da59aba) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgFkoB1gCjk_TmRKD%2Fimage.png?alt=media\&token=26ca787f-2047-4581-b119-fd0495bbb972) * Accessing the dirty page again? ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdgct4ax0bT-7ZAE6j%2Fimage.png?alt=media\&token=e2b4315c-b9eb-4c69-a7fa-aca941c9e5c1) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdglOeWKLNcWSb-LMF%2Fimage.png?alt=media\&token=c56d7629-66ca-49e9-99f9-767e660f52a7) * Java * GC: find all objects that will never be used again * E.x. variable within the loop, can be cleaned after this loop * Tracing is not computationally heavy * Cheaper to store all tracing on local memory in memory server ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdhvmCJ-MhlNwfc3YP%2Fimage.png?alt=media\&token=8814d827-7012-42cd-9566-0304dc62b59f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdiDL7poirQchhypqh%2Fimage.png?alt=media\&token=25a82bd0-cf54-4182-9f20-9e54f485599a) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdiHhp57s413euLhNB%2Fimage.png?alt=media\&token=1d3ae15e-c389-4a41-9ff2-a0ed2189c991) * Several rounds of GC: * If these objects are still there, likely to remain for longer period of time * Java does less cleaning to older objects, and spend more times on newly-created objects ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdirfLqKVbeLjvAOO9%2Fimage.png?alt=media\&token=a8ae29eb-bc65-4277-b804-8f08262c1b6e) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdj2MsdJZKpuKFqciQ%2Fimage.png?alt=media\&token=4e40fc73-0d3b-49b8-b1d7-0e1b96182274) * Memory servers: recycling the regions of themselves * Data layout: reduce memory segmentation #### #3: how to efficiently swap data ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdjPRqhjr-sO0ujZZ6%2Fimage.png?alt=media\&token=bd15430b-098f-48f7-b04a-5defc30c89c9) * Control path onto different memory servers * Data Path * Not interfere with each other * Paging: * Managed by the OS * Runtime access the page? * Bypass the OS? * JVM has the mapping between the two * Map the virtual pages in runtime to physical allocation on the memory servers * LegoOS: * this kind of disaggregated architecture * Not support JVM * Existing OS, then for OS, not be able to distinguish (which to handle locally, which to handle on remote). Implement this, having more flexibility. * Language runtime knows the program much better instead of OS ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdaKX5nt20A5KBrApk%2F-MWdjb_zcV_tEw-wZ5Eb%2Fimage.png?alt=media\&token=96905434-4e22-4eea-a1cc-347e1d5df71f) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdmUjkA0xQJ9o3YjU9%2F-MWdnUPMqG2e0bK2grBZ%2Fimage.png?alt=media\&token=b1c5f344-9503-428e-a63e-c1bf633c809a) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdmUjkA0xQJ9o3YjU9%2F-MWdnsm_SnBaaw8y4F9S%2Fimage.png?alt=media\&token=0a6998da-bf54-4517-b797-0d99c92a83eb) ![](https://2097630930-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MVORxAomcgtzVVUqmws%2F-MWdnu-4R8othAeFvFnm%2F-MWdo9titZBvLeZd5Bz0%2Fimage.png?alt=media\&token=85dfd5f8-4ee0-44c5-a999-a5264de64d60) * JVM on disaggregated * GC implemented inside of JVM rather than doing this directly from the OS --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://sliu583.gitbook.io/blog/specific-work/wisr-group/group/semeru-a-memory-disaggregated-managed-runtime.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.