ZEDEDA WiKi

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== EVE-OS Volume Provisioning: Pool Layer vs. Per-App Layer ======

Companion page to [[eve-app-volumes|EVE-OS / ZEDEDA Application Volumes]]. That page covers the **controller/app-instance config layer** (''maxsize'', ''preserve'', ''drvtype''). This page covers the **storage backend layer** underneath it — what actually happens on the edge node's disks, and why those two layers must not be conflated.

===== The core distinction =====

There are two completely different things that can both reasonably be called "a zvol," configured at two different times, by two different actors:

^ ^ Pool layer ^ Per-app volume layer ^
| **What it is** | The ZFS pool's RAID topology across physical disks (mirror / raidz1 / raidz2) | One zvol (or qcow2 file) per individual app volume, sized from that volume's ''maxsize'' |
| **Set via** | ''grub.cfg'' on the CONFIG partition, ''eve_install_zfs_with_raid_level=...'' | Controller app-instance config (''maxsize''), read by ''volumemgr'' on the node |
| **When** | Once, at install or reinstall | Dynamically, every time an app instance with a volume is deployed, purged, or resized |
| **Who acts on it** | The EVE installer | The ''volumemgr'' pillar agent |
| **Covered on** | EVE-OS Partitioning / ZFS wiki page | This page, and [[eve-app-volumes|EVE-OS / ZEDEDA Application Volumes]] |

The install-time grub.cfg RAID setting decides **only** the physical pool topology. It does not create, size, or reserve space for any individual app's disk. Every app volume's actual backing object is created **later**, per app, by ''volumemgr'', driven entirely by that app's ''maxsize'' — the same way qcow2 files are created per app on an ext4-persist node. Setting ''raid1'' at install time tells EVE "mirror these two physical disks"; it says nothing about how big any given app's disk will be or whether it's a zvol or a file.

===== What volumemgr actually creates per app volume =====

Source: EVE's ''pillar/docs/volumemgr.md'', ''pillar/types'' (''ZVolDevicePrefix = "/dev/zvol"'', ''ZFSBinary'', ''ZPoolBinary''), and EVE release notes referencing zvol support for ZFS persist.

  * **ext4 persist** → ''volumemgr'' creates a **qcow2 file** per volume, under ''/persist/img'' (or the encrypted/clear volume dirs under ''/persist/vault/volumes'' or ''/persist/clear/volumes''). Sized to ''maxsize'' as the qcow2 virtual size.
  * **ZFS persist** → ''volumemgr'' creates a **separate zvol per app volume**, inside the pool, sized to ''maxsize'' as the zvol's ''volsize''. Confirmed from EVE release notes: a past PR description reads "refactoring of volumemgr which makes zvol usage configurable for zfs persist type" — meaning per-app zvol creation is a real, distinct code path in volumemgr, and (at least at some point) whether it's used at all on ZFS persist was itself a build-time toggle. Treat "ZFS persist always means per-app zvols" as **not fully confirmed for every EVE version** — it may instead store a qcow2 file inside a ZFS dataset in some configurations. Worth confirming against the specific EVE-OS version in use before assuming either way.

Either way, the app-instance config layer doesn't know or care which of these it gets — ''maxsize'' is the same field regardless of backend, and ''volumemgr'' is what translates it.

===== Does maxsize fully provision the disk immediately? =====

**qcow2 (ext4 persist):** No — thin-provisioned by default. A ''maxsize'' of 100 GB on an 8 GB image creates a qcow2 file with a **virtual size** of 100 GB; actual bytes on ''/persist'' start near the image size and grow only as the guest writes new data. ''qemu-img info'' would show ''virtual size: 100 GiB'' but a much smaller ''disk size:'' until filled in. This changes only if preallocation is explicitly forced to ''metadata''/''falloc''/''full'', which none of the GUI/TF/zcli fields covered in the companion page do.

**zvol (ZFS persist), if per-app zvols are in use:** Depends on whether ''volumemgr'' creates the zvol sparse or thick:
  * A zvol created with ''zfs create -V 100G ...'' (no ''-s'' flag) is **thick-provisioned by default** — ZFS immediately reserves (''refreservation'') the full 100 GB against the pool, whether or not the guest has written anything.
  * A zvol created with the ''-s'' (sparse) flag behaves like qcow2 — virtual size only, grows into actual usage.
  * Which of these ''volumemgr'' uses for app volumes is **not confirmed** from available docs/source at the time of writing. This is the single most important open item for capacity planning on any ZFS-persist node — a 100 GB thick zvol immediately eats 100 GB of pool space regardless of what the app actually writes; a sparse one doesn't.
  * **Action item:** on a test ZFS-persist node, deploy an app volume with real headroom (e.g. 8 GB image, 100 GB ''maxsize''), then run ''zfs get volsize,refreservation,used <pool>/<volume-dataset>'' and compare ''refreservation'' to ''used''. If ''refreservation'' ≈ 100 GB immediately, it's thick. If it tracks ''used'' instead, it's sparse.

===== Can a per-app zvol be resized after the fact? =====

Yes, mechanically — this is a normal, well-supported ZFS operation. Growing a zvol's ''volsize'' (''zfs set volsize=<new size> pool/vol'') is safe and routine; shrinking is the risky/restricted direction, not growing. So a ''maxsize'' increase on a ZFS-persist app volume, combined with ''allow_storage_resize=true'' on the companion page, should translate to ''volumemgr'' issuing the equivalent of a ''zfs set volsize'' call on that specific app's zvol.

**Not yet confirmed:** the exact ''volumehandlers'' code path that performs this translation on the ZFS side. Treat this as mechanically plausible and consistent with how ZFS works, but not verified against EVE source line-by-line. Same Purge & Update lifecycle from the companion page still applies regardless — this is not a live resize while the app instance is running.

===== Why this matters practically =====

  * **Admission control:** before a deploy is allowed, EVE/the controller checks whether the requested ''maxsize'' could fit on the node's persist capacity. On ext4/thin-qcow2, this check is against a number larger than what will actually be consumed. On a thick-provisioned ZFS zvol, the check and the real consumption are the same number — capacity gets used up much faster in practice.
  * **Fleet capacity planning (Quvia-relevant):** nodes running ZFS persist with raid1/5/6 for HA already lose usable capacity to parity/mirroring overhead (see the Partitioning/ZFS page). If per-app zvols also turn out to be thick-provisioned, the effective usable margin for headroom-heavy ''maxsize'' settings is smaller than an ext4-persist node with the same raw disk capacity. This is worth confirming before sizing ''maxsize'' generously "just in case" on any ZFS-persist Quvia deployment.

===== Related =====

  * [[eve-app-volumes|EVE-OS / ZEDEDA Application Volumes]] — controller/app-instance layer: drive types, maxsize/preserve/allow_storage_resize, deploy and grow steps.
  * EVE-OS Partitioning / ZFS page — pool layer: RAID levels, install-time grub.cfg options, ZFS ARC/memory overhead, minimum specs.