Skip to content
Crimson, Ceph's own Seastar rewrite of the OSD with SeaStore, against croit FlashStore, a Rust rewrite with its own store

FlashStore Against Crimson: Two Rewrites Of The Same OSD

Crimson is Ceph’s own rewrite of the OSD, in C++ on the Seastar framework, with a new native store called SeaStore; FlashStore is croit’s rewrite, in Rust, with a store of its own. Both are in preview and both are aimed at NVMe and fast networks. This compares them as Ceph 21.2.0 Umbrella ships Crimson, from Ceph’s source and documentation and croit’s ceph-rust snapshot: where each comes from and under what licence, why Rust meets the Five Eyes memory-safety guidance and C++ does not, why Crimson needs its own pools and an experimental flag while FlashStore drops into existing pools, why Umbrella has no way to turn an existing pool into a Crimson pool so the data has to be copied, what each tells the monitor at boot, Seastar reactors against pinned polling cores, the fact that a default Crimson OSD still runs BlueStore, SeaStore’s onode, OMAP and LBA trees against FlashStore’s one tree per shard, the order each commits a write in, SeaStore’s segment cleaner, and what hardware and pools each will take.

11th November 2026 · 20 min · 4679 words · Damien Dye
Umbrella BlueStore, RocksDB on BlueFS on any drive, against croit FlashStore, a copy-on-write B+tree and WAL per core on NVMe with power-loss protection

FlashStore Against Umbrella BlueStore: What croit Changed Under The OSD

croit’s ceph-rust is a clean-room Rust rewrite of the Ceph OSD with its own object store, FlashStore, in public preview since 9th November 2026. This puts it side by side with BlueStore exactly as Ceph 21.2.0 Umbrella ships it, read from both projects’ source and documentation: what is in the repository and under which licences, where each store keeps its metadata, how a 4 KiB write and a write under 4 KiB reach the drive in each, Umbrella’s thread pools against FlashStore’s one thread per core, why FlashStore makes power-loss protection a hard requirement, what it will not run on, why it tells your monitors it is BlueStore, which BlueStore options it honours and which it ignores, and why the only way between the two is backfill.

11th November 2026 · 20 min · 4665 words · Damien Dye
A Proxmox VE node running Proxmox's own Ceph monitor and manager beside croit FlashStore OSDs on eight Optane drives

FlashStore OSD on Proxmox VE: What It Took on One Node

croit’s FlashStore OSD is a Rust replacement for ceph-osd and BlueStore, in public preview since 9th November 2026. This deploys it hyperconverged on a single Proxmox VE 9.2 node: Ceph Umbrella 21.2.0 from the PVE wizard for the monitor and manager, FlashStore OSDs on Intel Optane P4800X drives. It covers why Umbrella is the only Proxmox release that matches, formatting the drives to 4 KiB and the controller reset that aborts a format, what croit-flashstore-osd –check reports, why the shipped unit cannot read Proxmox’s ceph.conf and the drop-in that fixes it, the NVMe passthrough Proxmox’s kernel refuses and the direct backend that works, preparing OSDs by PCI address, what the PVE OSD panel makes of them, a pool, RBD storage and a Fisher-Price OS (Windows) VM installed onto it, CrystalDiskMark from inside that VM read against the drives’ own counters, Ceph’s memory with all eight OSDs active, three VMs at once on a 1 GiB memory target, and a setup script that does all of it, tested by rebuilding an OSD from nothing.

11th November 2026 · 35 min · 7721 words · Damien Dye
Two Proxmox defaults: Hyper-V enlightenments switched on by the VM's OS Type, and CPU microcode switched on by how the host was installed

Two Proxmox VE 9 Defaults You Might Not Have: Hyper-V Enlightenments and CPU Microcode

Proxmox VE 9 turns on Hyper-V enlightenments for Fisher-Price OS (Windows) guests and installs CPU microcode on new hosts, but only if the VM was given a Microsoft Windows OS type and the host came from the 9.x installer. This covers what the enlightenments do, the exact rules qemu-server uses to pick them (measured on a live 9.2 host), why machine version pinning keeps older VMs off the July 2026 hv-evmcs fix, the pve-qemu-kvm build that fix needs and what each QEMU release since 10.0 fixed for those guests, why they belong on q35 and never i440fx (which the wizard still hands most of them), the two flags you can set by hand, how to prove what a VM is running, where microcode comes from (board firmware, Debian and Proxmox’s own repository), why an upgraded host has none (with an Intel node four Intel releases behind as the example), the initramfs rebuild that installing it triggers and the older kernels that rebuild misses, the Zen 5 RDSEED failure that follows, enabling it, a rolling reboot across a Ceph cluster, and rolling back a bad update.

27th October 2026 · 25 min · 6016 words · Damien Dye

Who Actually Writes & Uses Ceph

I counted every commit to Ceph’s main branch for the last ten years — 69,613 of them from 1,718 people across 478 organisations — traced all 38 Steering Committee members to their employers, and gathered the publicly stated deployments. The result is a project that absorbed the loss of its founder and its second-largest contributor without dropping a beat, and is still shipping at the same rate today.

27th August 2026 · 36 min · 7504 words · Damien Dye
An NVMe drive entering a tri-mode adapter and leaving as a SCSI disk with a queue depth of 128

Tri-Mode Adapters Buy Flexibility With Your NVMe Queues

A tri-mode adapter lets any bay take SAS, SATA or NVMe — real flexibility, and the reason U.3 backplanes exist. What it does not advertise is that your NVMe drives stop being NVMe drives: they arrive in Linux as SCSI disks on mpt3sas, queue depth 128, sharing one tag pool and one x8 uplink. Two Gen4 drives saturate the card. A single Gen5 drive is already past it. Whether that trade still makes sense in 2026, and why the universal bays belong to the backplane rather than the controller.

13th August 2026 · 25 min · 5202 words · Damien Dye
Three storage tiers: Optane absorbing writes, NVMe holding Ceph metadata, and HDDs providing bulk capacity

Making HDD-Backed Proxmox Ceph Clusters Fast — NVMe Metadata, One Optane Per Spindle, and What It Costs You

Flash pricing has made all-NVMe hard to justify, and enterprise HDDs are worth another look. Getting acceptable latency out of them on hyper-converged Proxmox Ceph means moving RocksDB off the spindle, pairing each disk with its own Optane through bcache — Optane specifically, because NAND endurance is wrong for that job — and being honest about the failure modes it buys you.

7th August 2026 · 35 min · 7359 words · Damien Dye
Three Proxmox nodes wired in a triangle mesh with direct cables and no switch

A Switchless Proxmox Mesh with SDN OpenFabric — Ceph and Client Networks Without a 100G Switch

Three Proxmox nodes wired directly to each other in a triangle, with OpenFabric routing over the mesh and Ceph plus VXLAN client networks riding on top. Built entirely in the web interface, and honest about where the design stops scaling.

7th August 2026 · 14 min · 2783 words · Damien Dye
A guest clock drifting away from the host clock, and a hypercall pulling it back

A VM's Clock Cannot Be Trusted — and the ptp_kvm Fix for QEMU/KVM

A virtual machine’s clock is built on an assumption virtualisation breaks: that the CPU keeps counting. Why guest time drifts on every hypervisor, what the skew actually breaks — Kerberos, TLS, Ceph, Windows — and how ptp_kvm fixes it properly on QEMU/KVM.

7th August 2026 · 14 min · 2897 words · Damien Dye
Eight 512-byte logical blocks mapped onto one 4K physical sector, beside a one-to-one 4K mapping

4Kn, 512e and 512n — Why Native 4K Wins, and What the Emulation Costs

512e drives present 512-byte sectors they do not have, and the firmware makes up the difference on every unaligned write. What that costs on the medium, in the host and in write amplification — why direct synchronous writes are the worst case — and how to convert a fleet to 4Kn.

7th August 2026 · 26 min · 5489 words · Damien Dye