User Tools

Site Tools


zededa:sharing-persist-volume-by-two-apps

This is an old revision of the document!


Sharing a Volume Between Multiple Edge Applications (ZEDEDA / EVE-OS)

Can a volume be shared by two apps?

Yes — but the rules differ for read-only vs read-write:

  • Read-only sharing — any volume type can be shared read-only by multiple apps.
  • Read-write sharing (multiple apps read and write the same data) — the volume must be backed by a CONTAINER content-tree. A bare block-storage / QCOW2 / RAW / ISO (file-based) volume cannot be shared for writing across VMs, because multiple VMs cannot write to a single file.

This is a documented ZEDEDA feature (“Attach Volume Instances to Multiple Edge Applications”, EVE-OS >= 12.7.0). When done correctly, EVE shares the volume to the guests over plan 9 (9P), which arbitrates concurrent access — so all attached apps can read AND write the same data.

The common error (and its real cause)

If you attach a writable, file-based volume (plain block-storage/QCOW2) to two apps, EVE rejects the second one with:

//Multiple app instances (N) are trying to use the same file-based volume <name>//

This is a deliberate safeguard, verified in EVE source (pkg/pillar/cmd/volumemgr/handlevolumeref.go). The guard fires when a volume is shared by ≥2 apps and it is not container-based and it is not read-only. It is not related to the node's storage backend, ZFS, zvols, disk format, or the multiattach flag — none of those bypass it. The fix is to use a container content-tree volume (below), or make the shared volume read-only.

How to set it up (writable shared volume)

  1. Content-tree volume — create a volume instance of type Content Tree, based on a container image. The image is never executed; its rootfs simply seeds the shared volume's initial contents, so use a small/empty image (e.g. busybox, or a FROM scratch image if you want the share to start empty). Do not use something like nginx — you'd just get its filesystem as clutter.
  2. Block-storage volume — create a Block Storage volume instance connected to that content-tree (content_tree_id), with an access mode and a label. Size it larger than the container.
  3. Attach to each app — on every edge app (container or VM), add a drive that references the block-storage volume by its label, and set a mount path.
  4. Deploy all the app instances on the same edge node.

Access modes

The accessmode field (each value prefixed with VOLUME_INSTANCE_ACCESS_MODE_):

Value (suffix) Meaning
READ / READONLY Read-only — any volume type can be shared this way
READWRITE Read + write
MULTIREAD_SINGLEWRITE Treated the same as Read,Write by EVE

For a writable shared content-tree volume, READWRITE (or MULTIREAD_SINGLEWRITE) plus multiattach = true.

Terraform (zedcloud provider)

# A tiny CONTAINER image — only used to seed the content-tree (never executed)
resource "zedcloud_image" "shared_ct_image" {
  datastore_id  = zedcloud_datastore.docker_hub.id
  image_type    = "IMAGE_TYPE_APPLICATION"
  image_arch    = "AMD64"
  image_format  = "CONTAINER"
  image_size_bytes = 0
  name          = "SHARED-CT-BUSYBOX"
  title         = "SHARED-CT-BUSYBOX"
  image_rel_url = "busybox:latest"
}
 
# 1) Content-tree volume seeded from that container image
resource "zedcloud_volume_instance" "shared_ct" {
  device_id = zedcloud_edgenode.my_node.id
  type      = "VOLUME_INSTANCE_TYPE_CONTENT_TREE"
  image     = zedcloud_image.shared_ct_image.name
  label     = "shared-ct"
  name      = "shared-ct"
  title     = "shared-ct"
}
 
# 2) Block-storage volume connected to the content-tree (this is what apps attach)
resource "zedcloud_volume_instance" "shared_blk" {
  device_id       = zedcloud_edgenode.my_node.id
  type            = "VOLUME_INSTANCE_TYPE_BLOCKSTORAGE"
  content_tree_id = zedcloud_volume_instance.shared_ct.id
  accessmode      = "VOLUME_INSTANCE_ACCESS_MODE_READWRITE"
  multiattach     = true
  size_bytes      = 2147483648
  label           = "shared-blk"
  name            = "shared-blk"
  title           = "shared-blk"
}

Then, on each app instance, add a drive referencing the block-storage volume by label:

  drives {
    imagename   = ""
    volumelabel = zedcloud_volume_instance.shared_blk.label
    mountpath   = "/data"
    target      = "Disk"
    drvtype     = "HDD"
    preserve    = true
    readonly    = false   # both apps may read AND write (9P arbitrates)
  }

Mounting inside the guest

  • Container apps — EVE auto-mounts the share at the drive's mountpath. Nothing to do in the guest.
  • VMs — you must mount the 9P share manually. EVE exposes it as a virtio-9P device with mount tag share_dir (from EVE hypervisor/kvm.go):
modprobe 9pnet_virtio 2>/dev/null || true
mkdir -p /data
# fstab (nofail so a missing share never blocks boot):
echo "share_dir  /data  9p  trans=virtio,version=9p2000.L,rw,nofail  0  0" >> /etc/fstab
mount /data

Do this via cloud-init so it happens on every boot. No mkfs, no block device — it is a 9P filesystem, not a raw disk.

Requirements & gotchas

  • EVE-OS >= 12.7.0.
  • All sharing apps on the same edge node.
  • Writable share ⇒ container content-tree base. A plain block-storage/QCOW2 volume shared writable across VMs is the blocked case (see the error above).
  • The content-tree image is a seed, not a workload — it is never run; its rootfs is the volume's starting content. Use a tiny/empty base.
  • VMs mount manually (9P, tag share_dir); containers auto-mount at the mount path.
  • 9P allows concurrent read+write for all apps — no single-writer restriction, unlike a raw shared block device.

Doing it via zcli / REST / WebUI

Same objects as Terraform:

  • WebUI: Library → Volume Instances → create a Content Tree (from a container image), then a Block Storage connected to it; add a drive on each edge app referencing the block-storage label + mount path.
  • REST: POST/PUT /api/v1/volumes/instances for both volumes; app-instance drives via PUT /api/v1/apps/instances/id/{id}.
  • zcli: zcli volume-instance … and zcli edge-app-instance ….

Summary

  • Read-only sharing: any volume type.
  • Read-write sharing across VMs: container content-tree volume + block-storage connected to it, attached to each app; shared via 9P (all apps read+write).
  • Not a ZFS/zvol/disk-format/multiattach-flag matter — the gate is container-vs-file and read-only-vs-writable.
  • VMs mount the 9P share manually (mount -t 9p … share_dir); containers auto-mount.
  • Requires EVE-OS >= 12.7.0, all apps on the same node.

Sources

  • ZEDEDA Help Center — “Attach Volume Instances to Multiple Edge Applications”
  • EVE source: pkg/pillar/cmd/volumemgr/handlevolumeref.go (the share guard), pkg/pillar/hypervisor/kvm.go (9P mount tag share_dir)
zededa/sharing-persist-volume-by-two-apps.1784896534.txt.gz · Last modified: by mc