Flash pricing has made all-NVMe hard to justify, and enterprise HDDs are worth another look. Getting acceptable latency out of them on hyper-converged Proxmox Ceph means moving RocksDB off the spindle, pairing each disk with its own Optane through bcache — Optane specifically, because NAND endurance is wrong for that job — and being honest about the failure modes it buys you.
512e drives present 512-byte sectors they do not have, and the firmware makes up the difference on every unaligned write. What that costs on the medium, in the host and in write amplification — why direct synchronous writes are the worst case — and how to convert a fleet to 4Kn.
Why PCIe devices lose throughput when passed through to a VM via VFIO, and the practical tuning steps that claw most of it back.
On multi-socket systems, a VM with its vCPUs on one NUMA node and its passed-through device on another loses 20–30% throughput before you’ve even looked at anything else.
Active State Power Management saves a few watts on idle PCIe links. Under VFIO passthrough, it adds latency jitter that’s hard to diagnose and easy to fix.
QEMU’s virtual root complex defaults to 128-byte TLP payloads. Most devices support 256 or 512. One kernel parameter fixes it.