KV-Cache and Storage Interaction

This project would investigate key-value(KV)-cache offloading in large language model (LLM) inference engines, with particular emphasis on data placement and storage efficiency. First, the project would use an emulator such as FEMU to characterize workload access patterns, including hot-key distributions, write intensity, and write amplification factor (WAF). Then, we would evaluate the potential of Flexible Data Placement (FDP) for improving data placement decisions on SSDs for KV-cache accesses.

The project will use open source state-of-the-art KV-cache types such as LMCache or the ones implemented by the inference frameworks such as vLLM and SGLang. Furthermore, all experimental analysis will be conducted using open-source workload traces, including datasets released by Mooncake, Alibaba, Swiss AI Initiative, etc.

This project would be the most suitable as a research project with an MSc thesis followup, but we can also talk about other possibilities. If you are interested in LLM serving efficiency, storage devices, benchmarking, and performance analysis, this project would be a great fit for you. Furthermore, there is a possibility of collaborating with Samsung Research Denmark.

Some reading material:

[1] Liu et al. “LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference.”.

[2] Song et al. “Characterizing and Emulating FDP SSDs with WARP.”, FAST 2026.