====== App Instances ====== An App Instance is the deployment of an App Bundle to a specific edge node or EVE-k cluster. The bundle defines **what** to run; the instance defines **where** to run it, how to wire its network interfaces to actual network instances, what cloud-init config to pass, and whether to activate immediately. Every app instance has its own lifecycle, logs, and run state independent of other instances of the same bundle. ===== App Instance vs App Bundle ===== ^ | App Bundle (zedcloud_application) ^ App Instance (zedcloud_application_instance) ^ | Defines | Image, resources, interface names, ACL rules | Where to deploy, network wiring, cloud-init config | | Scope | Enterprise-wide, reusable | Node or cluster-specific | | Created | Once | Once per target node/cluster | | Modified | Rarely -- triggers purge on instances | Per-instance config (cloud-init, MAC, IP) | | Lifecycle | Persists until deleted | Tied to deployment target | ===== Key Fields ===== ^ Field ^ Description ^ | ''app_id'' | ID of the App Bundle this instance deploys | | ''device_id'' | Target standalone edge node. Mutually exclusive with ''edge_node_cluster''. | | ''edge_node_cluster.id'' | Target EVE-k cluster. Mutually exclusive with ''device_id''. | | ''project_id'' | Project this instance belongs to | | ''activate'' | ''true'' = deploy immediately on apply. ''false'' = stage config without starting the VM. | | ''drives'' | Override drive settings from the bundle (image, size, encryption, purge behavior) | | ''interfaces'' | Bind each interface name from the bundle to a network instance on the target node | | ''custom_config'' | Cloud-init configuration delivered to the VM on first boot | | ''logs.access'' | ''true'' = enable log collection for this instance | | ''start_delay_in_seconds'' | Delay VM start after node boot. Useful for sequencing (e.g. start firewall before VMs behind it) | ===== interfaces block (Instance Level) ===== The instance-level ''interfaces'' block binds the logical interface names defined in the bundle to real network instances on the target node. This is where the rubber meets the road for networking. ^ Field ^ Description ^ | ''intfname'' | Must match the interface name in the App Bundle manifest (e.g. ''eth0'', ''enp1s0'') | | ''intforder'' | Ordering of NICs as presented to the guest OS. 1 = first NIC. | | ''netinstname'' | Name of the network instance on the target node to connect this interface to | | ''directattach'' | ''true'' = SR-IOV VF passthrough. ''false'' = virtual NIC via EVE switch | | ''io'' | For directattach VFs: specifies the VF name and type | | ''macaddr'' | Optional static MAC address for this interface inside the VM. Leave empty for auto-assigned. | | ''access_vlan_id'' | VLAN tag for the interface on the network instance. ''0'' = untagged. | | ''ipaddr'' | Static IP inside the VM. Usually empty -- let the guest handle DHCP or cloud-init sets it. | | ''privateip'' | ''true'' = assign a private IP from the network instance pool | | ''default_net_instance'' | Marks this interface as the default route interface for the VM | ===== custom_config block ===== custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(file("./c-init/fw-cloud-init.txt")) } ^ Field ^ Description ^ | ''add = true'' | Enable custom configuration for this instance | | ''allow_storage_resize'' | Allow EVE-OS to resize the root filesystem to fill the allocated drive size on first boot | | ''field_delimiter'' | Delimiter for template variable substitution. ''@@@'' means variables in the cloud-init file are wrapped as ''@@@VAR_NAME@@@'' | | ''override = true'' | This instance config overrides any template defined at the bundle level | | ''template'' | Base64-encoded cloud-init content. Use ''base64encode(file(...))'' for static files or ''base64encode(templatefile(..., {...}))'' for variable substitution | ===== Terraform Examples ===== ==== Firewall VM Instance -- EVE-k Cluster (3 NICs) ==== Deploys the FortiGate firewall bundle to an EVE-k cluster. Three interfaces wired to WAN, LAN, and DMZ network instances. Static MAC on eth0 (WAN) ensures consistent IP assignment from upstream DHCP. resource "zedcloud_application_instance" "tf_atl_cl_fw_1" { name = "TF-CL-ATL-FW-1" title = "TF-CL-ATL-FW-1" activate = true project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_atl_fw_app_1.id custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(file("./c-init/fw-cloud-init.txt")) } logs { access = true } edge_node_cluster { id = zedcloud_edgenode_cluster.edgenode_cluster_1.id } drives { imagename = zedcloud_image.tf_demo_fw_atl_image.name cleartext = false ignorepurge = true maxsize = 40971520 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { #### WAN -- eth0 intfname = "eth0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false ipaddr = "" macaddr = "02:16:8A:12:34:56" netinstname = zedcloud_network_instance.tf_cl_atl_wan_1.name privateip = false } interfaces { #### LAN -- eth1 intfname = "eth1" intforder = 2 directattach = false access_vlan_id = 0 default_net_instance = false ipaddr = "" macaddr = "" netinstname = zedcloud_network_instance.tf_cl_atl_ni_1.name privateip = false } interfaces { #### DMZ -- eth2 intfname = "eth2" intforder = 3 directattach = false access_vlan_id = 0 default_net_instance = false ipaddr = "" macaddr = "" netinstname = zedcloud_network_instance.tf_cl_atl_ni_2.name privateip = false } } Notes: * ''ignorepurge = true'' on the drives block -- preserves FortiGate's config partition across image updates. Critical for appliances that store their config inside the image. * ''macaddr = "02:16:8A:12:34:56"'' on the WAN interface -- static MAC ensures the upstream network always assigns the same IP to this firewall, regardless of node or redeploy. * ''macaddr = ""'' on LAN/DMZ -- EVE-OS auto-assigns a MAC. Consistent within the instance's lifecycle. * ''logs.access = true'' -- enables log streaming from this instance to ZEDEDA Cloud (viewable in the instance detail view). * ''allow_storage_resize = true'' -- EVE-OS expands the qcow2 sparse image to fill the full ''maxsize'' allocation on first boot. ==== Ubuntu VM Instance -- EVE-k Cluster with templatefile ==== Ubuntu VM on a cluster. Uses ''templatefile()'' to inject a per-instance hostname into the cloud-init at deploy time. resource "zedcloud_application_instance" "tf_atl_cl_vm_1" { name = "VM-1" title = "VM-1" project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_atl_ub_app_1.id activate = true custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(templatefile("./c-init/vm-1-cl.txt", { hostname = var.vm1_hostname })) } edge_node_cluster { id = zedcloud_edgenode_cluster.edgenode_cluster_1.id } drives { imagename = zedcloud_image.demo_atl_ub_image_1.name cleartext = false ignorepurge = false maxsize = 20971520 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { intfname = "enp1s0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false ipaddr = "" macaddr = "" netinstname = zedcloud_network_instance.tf_cl_atl_ni_1.name privateip = false } logs { access = true } } The ''templatefile()'' approach lets you reuse a single cloud-init template across many instances, with only the variable values changing per instance. In the template file, reference variables as ''${hostname}'': ./c-init/vm-1-cl.txt: #cloud-config hostname: ${hostname} users: - name: ubuntu ssh_authorized_keys: - ssh-rsa AAAA... The ''field_delimiter = "@@@"'' is ZEDEDA's own variable substitution -- separate from Terraform's ''templatefile()''. ZEDEDA replaces ''@@@VAR@@@'' tokens at instance creation time using system variables like ''$zri.system.edge-node.serial''. Both mechanisms can coexist in the same cloud-init file. ==== Firewall VM Instance -- Standalone Node ==== Same firewall bundle, deployed to a standalone EVE-KVM node instead of a cluster. The only structural change is ''device_id'' replacing ''edge_node_cluster''. resource "zedcloud_application_instance" "tf_stnd_sjc_fw_1" { name = "TF-SJC-FW-1" title = "TF-SJC-FW-1" activate = true project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_sjc_fw_app_1.id device_id = zedcloud_edgenode.demo_en_stnd_sjc_1.id custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(file("./c-init/sjc-fw-cloud-init.txt")) } logs { access = true } drives { imagename = zedcloud_image.tf_demo_fw_sjc_image.name cleartext = false ignorepurge = true maxsize = 40971520 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { intfname = "eth0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "02:16:8A:12:34:56" netinstname = zedcloud_network_instance.tf_stnd_sjc_wan_1.name privateip = false } interfaces { intfname = "eth1" intforder = 2 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = zedcloud_network_instance.tf_stnd_sjc_ni_1.name privateip = false } interfaces { intfname = "eth2" intforder = 3 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = zedcloud_network_instance.tf_stnd_sjc_ni_2.name privateip = false } } ==== Ubuntu VM Instance with start_delay_in_seconds ==== When deploying a topology where a firewall must be up before downstream VMs can reach the network, use ''start_delay_in_seconds'' to sequence startup: resource "zedcloud_application_instance" "tf_stnd_sjc_vm_1" { name = "TF-STND-VM-1" title = "TF-STND-VM-1" project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_sjc_ub_app_1.id activate = true device_id = zedcloud_edgenode.demo_en_stnd_sjc_1.id start_delay_in_seconds = 90 custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(templatefile("./c-init/vm-1.txt", { hostname = var.vm1_hostname })) } drives { imagename = zedcloud_image.demo_sjc_ub_image_1.name cleartext = false ignorepurge = false maxsize = 20971520 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { intfname = "enp3s0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = var.vm1_macaddr netinstname = zedcloud_network_instance.tf_stnd_sjc_ni_1.name privateip = false } logs { access = true } } ''start_delay_in_seconds = 90'' tells EVE-OS to wait 90 seconds after the node boots before starting this VM. The firewall instance has no delay and starts immediately, so it is routing by the time the VM comes up. ==== SR-IOV VM Instance (2 VFs + 1 virtual NIC) ==== Ubuntu VM with one virtual management NIC and two SR-IOV VFs passed directly to the guest. This VM **exclusively owns** both VFs -- direct attach means the VM has raw hardware access to those VFs via VFIO/IOMMU. No other VM can use the same VF simultaneously. resource "zedcloud_application_instance" "tf_demo_ub_atl_sriov_1" { name = "VM-1-SRIOV" title = "VM-1-SRIOV" project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_demo_atl_srio_ub_local_ni_1.id activate = true device_id = zedcloud_edgenode.demo_en_advantech_1.id custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(templatefile("./c-init/vm-1-sriov.txt", { hostname = var.vm1_hostname_sriov })) } drives { imagename = zedcloud_image.demo_atl_ub_image_1.name cleartext = false ignorepurge = true maxsize = 20971520 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { #### Management -- virtual NIC via EVE switch intfname = "enp1s0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = zedcloud_network_instance.tf_demo_sriov_ni_1.name privateip = false } interfaces { #### Data NIC 1 -- SR-IOV VF0 from eth2 intfname = "enp6s0" intforder = 2 directattach = true io { name = "eth2vf0" type = "IO_TYPE_ETH_VF" } access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = "" privateip = false } interfaces { #### Data NIC 2 -- SR-IOV VF0 from eth3 intfname = "enp7s0" intforder = 3 directattach = true io { name = "eth3vf0" type = "IO_TYPE_ETH_VF" } access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = "" privateip = false } } SR-IOV VF naming convention on EVE-OS: * ''eth2vf0'' = physical NIC labeled ''eth2'', Virtual Function index 0 * ''eth3vf0'' = physical NIC labeled ''eth3'', Virtual Function index 0 * A VF is **dedicated exclusively to one VM** -- it cannot be shared. Once bound via VFIO to a KVM domain, no other VM or the EVE-OS host can use it. * The physical NIC (PF) is split into multiple VFs at the EVE-OS level. Each VF is then assigned to a different VM. In this example: VM-1 gets ''eth2vf0'' and ''eth3vf0''; a second VM would get ''eth2vf1'' and ''eth3vf1'' -- different VF indexes, different VMs, never the same VF to two VMs. * The number of VFs available per PF depends on the NIC hardware (Intel X710 supports up to 128 VFs per port, E810 up to 256) and how many are configured on the edge node. * ''netinstname = ""'' on SR-IOV interfaces -- direct-attach VFs bypass the EVE network switch entirely. There is no network instance to reference. * EVE-OS must have SR-IOV enabled and the VFs pre-created on the node before an app instance can reference them. The VF must appear in the node's IO adapter list. ==== MikroTik Router Instance ==== resource "zedcloud_application_instance" "tf_mikrotik_vm_1" { name = "MIK-1" title = "MIK-1" project_id = zedcloud_project.demo_zededa_project_1.id app_id = zedcloud_application.tf_demo_atl_mik_1.id activate = true device_id = zedcloud_edgenode.demo_en_advantech_1.id custom_config { add = true allow_storage_resize = true field_delimiter = "@@@" name = "cloud-config" override = true template = base64encode(file("./c-init/vm-1.txt")) } drives { imagename = zedcloud_image.demo_mikrotick_1.name cleartext = false ignorepurge = true maxsize = 20000000 preserve = false target = "Disk" drvtype = "HDD" readonly = false } interfaces { intfname = "eth0" intforder = 1 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = zedcloud_network_instance.tf_demo_sriov_ni_1.name privateip = false } interfaces { intfname = "eth1" intforder = 2 directattach = false access_vlan_id = 0 default_net_instance = false macaddr = "" netinstname = zedcloud_network_instance.tf_demo_sriov_ni_2.name privateip = false } } ''ignorepurge = true'' -- MikroTik CHR stores its full config inside the QCOW2 image (license, interfaces, routing, firewall rules). Purging the drive wipes all of that. Always set ''ignorepurge = true'' for appliances. ===== Cloud-Init Template Patterns ===== Cloud-init is delivered to VMs as a NoCloud datasource. EVE-OS presents it via a virtual CD-ROM on first boot. The ''#cloud-config'' header is required. ==== Static Config (base64encode(file(...))) ==== Use when every instance of this bundle gets the same config. No variables. template = base64encode(file("./c-init/fw-cloud-init.txt")) ==== Per-Instance Variables (base64encode(templatefile(...))) ==== Use when each instance needs a different hostname, IP, or key. Terraform substitutes variables at plan time. template = base64encode(templatefile("./c-init/vm-1.txt", { hostname = var.vm1_hostname })) In the template file: #cloud-config hostname: ${hostname} ==== ZEDEDA System Variables (field_delimiter) ==== Use ''@@@VAR@@@'' tokens for values ZEDEDA substitutes at deploy time from node context. These are resolved by ZEDEDA Cloud, not Terraform: #cloud-config runcmd: - echo "@@@zri.system.edge-node.serial@@@" > /etc/node-serial Common ZEDEDA system variables: * ''@@@zri.system.edge-node.serial@@@'' -- physical serial number of the node * ''@@@zri.system.edge-instance.name@@@'' -- name of this app instance Both Terraform ''${var}'' and ZEDEDA ''@@@var@@@'' substitution can coexist in the same file. Terraform replaces its variables at plan time; ZEDEDA replaces its tokens at instance creation time. ===== Instance Run States ===== ^ State ^ Meaning ^ | INIT | Instance config received, preparing to start | | DOWNLOAD | EVE-OS downloading the image from the datastore | | INSTALL | Creating KVM domain, attaching volumes and network interfaces | | RUNNING (green) | VM is up and running | | STOPPED (red) | VM is halted | | CONFIG (blue) | Configuration received, processing | | SUSPECT (amber) | No heartbeat from the app for 3+ minutes | | INVALID (purple) | Unknown or inconsistent state | ===== Manage via ZCLI ===== # Deploy a new instance zcli edge-app-instance create MY-FW-1 \ --edge-app=TF-FW-ATL-7.4.3 \ --edge-node=my-edge-node \ --network-instance=eth0:tf-cl-atl-wan-1 \ --network-instance=eth1:tf-cl-atl-ni-1 \ --custom-configuration=./c-init/fw-cloud-init.json # Show all instances zcli edge-app-instance show # Show instances on a specific node zcli edge-app-instance show --edge-node=my-edge-node # Stop an instance zcli edge-app-instance stop MY-FW-1 # Start a stopped instance zcli edge-app-instance start MY-FW-1 # Export manifest from an existing app (to clone or modify) zcli edge-app export-manifest TF-FW-ATL-7.4.3 ===== Related Resources ===== * [[07_edge_apps|Edge Apps (Bundles)]] * [[06_persistent_volumes_and_content_trees|Persistent Volumes and Content Trees]] * [[05_patch_envelopes|Patch Envelopes]]