# MinIO MemKV RELEASE.2026-08-04T22-19-45Z Released: 2026-08-05 Four changes: a new LMCache multi-process adapter, a deadline on vLLM tier lookups that could otherwise stall an engine, and two silent read-path fixes — false misses after a scale-out, and a B+tree page leak under overwrites. --- ## Downloads ### Server Binary | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [memkv](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv) | | Linux | arm64 | [memkv](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv) | ### NIXL Plugin (for Dynamo / KVBM integrations) | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [libplugin_MEMKV.so](https://dl.min.io/aistor/memkv/release/linux-amd64/libplugin_MEMKV.so) | | Linux | arm64 | [libplugin_MEMKV.so](https://dl.min.io/aistor/memkv/release/linux-arm64/libplugin_MEMKV.so) | ### LD_PRELOAD Shim (for MLPerf-Storage kvcache workloads) | Platform | Architecture | Download | | -------- | ------------ | -------- | | Linux | amd64 | [libmemkv_preload.so](https://dl.min.io/aistor/memkv/release/linux-amd64/libmemkv_preload.so) | | Linux | arm64 | [libmemkv_preload.so](https://dl.min.io/aistor/memkv/release/linux-arm64/libmemkv_preload.so) | ### Packages `.deb`, `.rpm`, and `.apk` packages bundle the server + both `.so` sidecars + the LMCache, sglang, and vLLM-tier Python wheels into a single per-arch install. | Format | Architecture | Download | | ------ | ------------ | -------- | | DEB | amd64 | [memkv\_20260804221945.0.0_amd64.deb](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv_20260804221945.0.0_amd64.deb) | | DEB | arm64 | [memkv\_20260804221945.0.0_arm64.deb](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv_20260804221945.0.0_arm64.deb) | | RPM | amd64 | [memkv-20260804221945.0.0-1.x86_64.rpm](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv-20260804221945.0.0-1.x86_64.rpm) | | RPM | arm64 | [memkv-20260804221945.0.0-1.aarch64.rpm](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv-20260804221945.0.0-1.aarch64.rpm) | | APK | amd64 | [memkv\_20260804221945.0.0_x86_64.apk](https://dl.min.io/aistor/memkv/release/linux-amd64/memkv_20260804221945.0.0_x86_64.apk) | | APK | arm64 | [memkv\_20260804221945.0.0_aarch64.apk](https://dl.min.io/aistor/memkv/release/linux-arm64/memkv_20260804221945.0.0_aarch64.apk) | The NIXL plugin is auto-symlinked to `/opt/nvidia/nvda_nixl/lib/plugins/` when that directory exists (postinstall hook). ### Python Plugins Install from PyPI, for amd64 and arm64: ```bash pip install -U memkv-lmcache memkv-sglang # Python 3.8+ pip install -U memkv-vllm # Python 3.10+, needs vLLM main ``` The wheels `dlopen` the RDMA libraries at runtime — the transport needs host `rdma-core`, otherwise it falls back to TCP. Without network access, the deb/rpm installs the same wheels at `/usr/share/memkv/wheels/`: ```bash cd /usr/share/memkv/wheels pip install --force-reinstall memkv_lmcache-*.whl memkv_sglang-*.whl pip install --force-reinstall memkv_vllm-*.whl ``` ### Container Image ```bash docker pull quay.io/minio/memkv:RELEASE.2026-08-04T22-19-45Z docker pull quay.io/minio/memkv:latest ``` The container ships the server and the NIXL plugin (under `/usr/local/lib/plugins/`). For the LD_PRELOAD shim and the Python wheels, use the deb/rpm. ### Verification Each binary is signed with both minisign (preferred) and GPG; sha256sums are published alongside. ```bash # minisign minisign -Vm memkv -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav # sha256 sha256sum -c memkv.sha256sum ``` --- ## Changes since RELEASE.2026-07-15T20-45-38Z ### New Features - **LMCache multi-process L2 adapter.** MemKV now works as an L2 backend behind LMCache's MP server (`lmcache server` + `LMCacheMPConnector`) — currently the only LMCache mode that attaches to hybrid Mamba/attention models. One MemKV value holds one L1 page, full-length-or-miss, reusable across engine instances and restarts. Enable it with LMCache's `plugin` adapter type pointing at `memkv_lmcache.l2_adapter`. Validated on Nemotron-3 Ultra over DC-RDMA to four MemKV instances, including serving a 100k-token context byte-identically to a fresh engine with an empty L1. ### Bug Fixes - **vLLM tier lookups now time out instead of stalling the engine.** Probes had no deadline, and vLLM defers a request for as long as its lookup retries, so one slow probe starved everything behind it: 62 of 64 requests deferred, no completions for 12 minutes, probes averaging 188 s. A lookup that cannot answer promptly now returns MISS and the block recomputes. Tune with `lookup_timeout_s` (default 5 s; any value at or below zero disables it), and watch `vllm:kv_offload_memkv_lookups{result="timeout"}`. Needs a patched vLLM — see Known Limitations. - **Batch reads no longer miss live data after a scale-out.** Batch reads — the path every plugin uses — asked only the key's primary owner and never retried, so adding a server made roughly `1/(N+1)` of them report false misses until those keys were rewritten. They now retry at the previous owner, as single-key reads always have. No wrong bytes were ever returned; the cost was lost cache hits. The retry shows up in `memkv_client_step_count`. - **Overwrites no longer fail healthy drives.** An overwrite landing on a full B+tree leaf consumed a page from the reserved metadata region without using it, one per write, until the region tripped `B+Tree region full` and failed the device — reproduced at 1494 live keys after 2685 inserts. The same path also skipped marking the leaf dirty, a durability gap for full-leaf overwrites. Both are fixed. ### Known Limitations - **The lookup deadline needs a patched vLLM.** The deadline lives in vLLM's async lookup manager, not in the plugin, and no vLLM release carries the patch — contact MinIO for it. On an unpatched engine the plugin warns, skips the setting, and lookups stay unbounded. - **The vLLM tier needs an unreleased vLLM.** The `SecondaryTierManager` API the plugin implements first appears in the v0.25.0 release candidates, so v0.25.0 should be the first stable release that runs it. Until then, use vLLM's per-commit main builds and re-qualify with the conformance suite in the serving image. The plugin never raises on engines without the tiering framework, so bundling the wheel is safe. --- ## Documentation - Hosted docs: - Embedded docs (in the binary): `memkv doc` serves the same site locally. ## Support - Security disclosures: security@min.io