Do I really need 10GbE for Ceph in a homelab?

For a cluster you intend to actually run VMs from, yes — 2.5GbE inter-node replication quickly becomes the bottleneck that makes Ceph feel broken: 50-80MB/s replicated writes and long recovery windows after maintenance. A used MikroTik CRS310-class SFP+ switch or a pair of direct-attach DAC connections gets real 10G for the price of one premium NVMe drive. See our 10GbE switch guide for tested picks.

Can Ceph run on consumer SSDs like the Samsung 990 Pro?

Yes, with tuning: consumer NVMe lacks power-loss protection and has limited endurance headroom, so keep replication factor 3, avoid enterprise write bursts on WAL/DB-only placement, and monitor wear. Blue/green splitting of WAL+DB onto a small dedicated NVMe (or an Optane-class leftover) markedly improves write latency on hybrid nodes. It is a homelab-acceptable trade: lose a drive and Ceph recovers; lose a house power event and you rest on backups.

How many OSDs should a first Ceph cluster have?

Three OSDs is the absolute minimum (one per node) but six is the practical sweet spot for a homelab — two NVMe or three HDD OSDs per 3 nodes — because one-drive failure no longer shifts 33% of capacity into recovery weight. Keep it symmetrical: same number of OSDs per node, same sizes per OSD family, and the cluster self-balances predictably.

Proxmox Ceph Homelab Guide 2026: 3-Node Cluster That Actually Survives Node Loss

Q: Is Ceph worth it for a home lab, or should I just use ZFS?

Ceph earns its complexity only when you have 3+ nodes and want VMs to keep running when a node dies, with storage that stays consistent across the cluster. If you run one node (or two with no shared-storage requirement), ZFS mirrors on each node are simpler, faster on light hardware, and far easier to debug. A useful rule: Ceph when the answer to 'what happens when this node dies?' is 'the lab keeps serving'; ZFS when you accept downtime in exchange for simplicity.

Q: How much RAM does a small Ceph cluster need?

Plan roughly 1GB of RAM per 1TB of OSD capacity as the practical floor, plus 4-6GB per node for the monitors/managers, plus whatever Proxmox and your VMs already need. For a 3x12TB-node cluster, 32GB per node is the comfortable minimum and 64GB removes the temptation to tune too aggressively. Undersized Ceph clusters degrade long before they fail outright — slow writes, deep recovery times — so skimp on cores before you skimp on RAM.

Q: What happens when a node dies in a 3-node Ceph cluster?

With size=3/min_size=2, the cluster stays writable when one node is down and the remaining two hold all data (in degraded state with one replica missing). VMs keep running with elevated latency on the surviving nodes. This is exactly why three nodes — not two — is the Ceph minimum: two nodes cannot lose one and keep quorum. Recovery starts the moment the failed rejoins or the OSD is replaced.

Affiliate disclosure: Some links in this guide earn us a commission at no extra cost to you. We only recommend hardware we'd run ourselves. Read our full disclosure.

Is Ceph worth it? (the honest test)

Answer one question: when a node dies, does the lab keep serving, or do you accept downtime? If downtime is acceptable, stop here — a ZFS mirror per node (our Proxmox build guide covers that topology) is simpler and faster on light hardware. Ceph is for the 3+ node design where HA is the point, and where the VMware exit already committed you to a cluster mindset.

The payoffs that make it worth it in 2026: live migration between any two nodes without shared NAS vendor lock, self-balancing storage that absorbs a new node overnight, and recovery that starts the second hardware returns. The costs: RAM, network, and a debugging rabbit-hole you enter whenever you deviate from defaults.

The 3-node hardware recipe

Symmetry is the design principle — same OSD count, same sizes, same NICs per node:

Component	Budget node (~$700)	Comfort node (~$1,100)
CPU	Ryzen 5 8600G / i5-12400 (6c, IOMMU)	Ryzen 9 7900 / i5-13500 (12-14c)
RAM	32GB DDR5	64GB DDR5 ECC-optional
OS/boot	500GB SATA/NVMe (Proxmox ZFS mirror optional)	2x 500GB NVMe ZFS mirror
Ceph OSDs	2x 2TB NVMe (990 Pro / WD SN850X class)	2x 4TB NVMe + small WAL/DB NVMe
Network	2x 10G SFP+ (Intel X520/X710 or Mellanox ConnectX-3/4)	2x 10/25G SFP+ (dedicated Ceph fabric nic)

RAM rule of thumb: ~1GB per 1TB of OSD plus 4-6GB monitor/manager overhead per node. Cores before RAM is wrong for Ceph — RAM before cores.

Network fabric: where Ceph dreams go to die

Ceph replicates every write between OSDs. On 1GbE, replicated NVMe writes collapse to spinning-disk speeds and recovery after maintenance takes hours. The 2026 fix is cheap:

Minimum viable fabric: one 10G SFP+ switch — the MikroTik CRS310-8G+2S+IN (~$240) plus X520-class NICs per node (~$30-60 used).
Better: separate public (VM) and cluster (replication) networks — either physically (two switches) or virtually (VLANs; keep the cluster VLAN jumbo-frame-clean and untagged-noise-free).
Two-node temptation: direct-attach DAC between two nodes works for testing, but a two-node Ceph cluster has no quorum story — add a third node or do not run Ceph.

Budget check: the whole fabric (switch + 3 DAC/sfp modules + NICs) is roughly one premium consumer NVMe's price — and it is the difference between Ceph feeling enterprise and feeling broken.

Install on Proxmox VE 9, step by step

Proxmox VE 9.1 ships Ceph Squid; the wizard does the YAML-free heavy lifting:

Install VE 9 on all three nodes from the ISO — pick ZFS raid0 on the boot drive if single, mirror if paired.
Configure the network first in /etc/network/interfaces: cluster VLAN or dedicated NIC (ceph0/ceph1 bridges) before anything touches Ceph.
Datacenter → Ceph: install the packages on all nodes (one wizard run per node), then Create Monitor + Manager on node A; add monitors to B and C so all three run MON+MGR. Odd monitors (3) — quorum lives here.
Create OSDs on each node from the wizard (bluestore default; tick WAL/DB split only if you have the dedicated small NVMe). Two OSDs per node = six total for the sweet-spot lab.
Create the pool: replication x3 (size=3/min_size=2), PG autoscaler on, name it rbd-vm or similar. Ignore EC pools for a first build — erasure coding is the classic homelab regret.
Validate: ceph -s shows HEALTH_OK, 6 OSDs, 3 monitors quorate. Then and only then create a test VM on the Ceph RBD and live-migrate it node-to-node.

Full command-level detail including the virtio and import-wizard steps for migrating real VMs lives in the Proxmox Migration Playbook (shop).

Tuning for consumer SSDs

Most of Ceph's "it's slow" reputation in homelabs is consumer-hardware defaults. The three that matter:

WAL/DB placement: on all-NVMe nodes keep WAL+DB colocated (defaults). On HDD nodes, always split WAL+DB onto a small NVMe — HDD Ceph without NVMe metadata is a patience simulator.
Recovery aggressiveness: for a lab, throttle recovery so a failing disk doesn't tank the whole cluster: ceph tell 'osd.*' injectargs '--osd-recovery-max-active 4 --osd-max-backfills 2'. Enterprise configs trade this for speed; homelabs trade it for stability.
Monitor wear: ceph osd df weekly plus smartctl — consumer NVMe at replication 3 writes 3x. A 600TBW drive at homelab write rates lasts years, but know your trend.

Skip: kernel tweaks, deep crush-map edits, custom bluestore compression. Defaults on good hardware beat tuned jank on bad hardware in the homelab weight class.

Pools, replication & the VM storage class

size=3, min_size=2 — the only homelab replication setting. size=2 loses quorum on one failure; EC saves space at CPU/latency cost and breaks simple live-migration patterns.
PG autoscaler on (default in VE 9) — do not hand-calculate PGs. If you must: ~100 PGs per pair of OSDs, power-of-two.
Separate pool per workload class: rbd-vm (fast NVMe replicas) for VM disks, cephfs-slow (HDD nodes, if any) for ISOs/backups/files. Mixing a backup storm into the VM pool is self-inflicted latency.
RBD vs CephFS for VMs: RBD (block) for VM disks, always. CephFS for shared files/ISO library.

Three failure drills to run before you trust it

Trust is a test result, not a feeling. On the new cluster, in this order:

Kill one node (hard power-off): VMs should keep running (min_size=2 holds), cluster degrades to HEALTH_WARN, writes slow but complete. Reboot the node; wait for recovery to HEALTH_OK; watch ceph -s do its thing. Time it — that number is your real maintenance window.
Kill one OSD on one node only (pull the NVMe): with 6 OSDs the cluster should stay HEALTH_OK-ish and rebalance without drama. Replace the drive, watch it re-weight. This is why 6 OSDs beats 3.
Reboot the whole cluster one node at a time: the quarterly Windows Update of the cluster world. Do it once now so the first "reboot everything" isn't during a real incident.

Backup still applies: Ceph replication is not backup (rm -RF propagates, as does ransomware). vzdump from the Ceph pool to a separate ZFS box/NAS remains mandatory — our migration guide's 3-2-1 section covers the tiers.

What it costs

Budget build (3 nodes): ~$2,100 in nodes + ~$400 fabric (switch/NICs/DACs) ≈ $2,500 all-in for a hyperconverged 3-node cluster with 12TB raw NVMe and real HA.
Comfort build: ~$3,300 nodes + fabric ≈ $3,700, with 24TB raw, 64GB/node and a separate fabric network.
Alternative for context: a 2-node ZFS + NAS design hits similar reliability for less, if node-loss downtime is acceptable — see the decision tree in our hypervisor comparison.

FAQ

Is Ceph worth it for a home lab, or should I just use ZFS?

ZFS for 1-3 nodes with downtime-tolerance; Ceph for 3+ nodes where live HA is the point. The full breakdown is in the FAQ schema above.

How much RAM does a small Ceph cluster need?

~1GB per 1TB of OSD plus 4-6GB per node for monitors — 32GB/node floor, 64GB/node comfortable.

Do I really need 10GbE for Ceph?

For a VM-serving cluster, yes. 2.5GbE replication makes Ceph feel broken; the fabric fix costs about one premium NVMe.

Can Ceph run on consumer SSDs?

Yes — replication 3, wear monitoring, and optional WAL/DB splitting make 990 Pro-class drives a valid homelab OSD at these write rates.

How many OSDs should a first cluster have?

Six (two per node in a 3-node): one drive lost no longer shifts a third of capacity into recovery.

What happens when a node dies in a 3-node Ceph cluster?

Cluster stays writable (min_size=2), VMs keep running, one replica is missing until the node returns — measured in your drill #1.