User Tools

Site Tools


eve-k:network-instances:start

VM deployment -- confirmed live trace

Device: c86eddc2-9a2f-4a0e-a761-25e70e7cbb89
App Instance UUID: 01859d42-b9c5-48f4-a7c3-d40be41b5877
Network Instance: 60321c32-0825-478c-905f-d9a8f698ff20 (Switch type, eth0)
Image volume: cbd8bf0c-3683-4f99-912c-92a81442cf40 (20Gi Longhorn)

Phase timeline (confirmed live)

15:06:52 → Pending    VMI created by zedkube in k3s
15:06:53 → Scheduling KubeVirt scheduler running
15:07:11 → Scheduled  placed on node tf-demo-adv-en-1
15:07:15 → Running    VM booted

Total time: 23 seconds from VMI creation to running.

EVE host side (confirmed)

Pubsub object Path State
AppInstanceConfig /run/zedagent/AppInstanceConfig/01859d42…json Present
AppNetworkStatus /run/zedrouter/AppNetworkStatus/01859d42…json Activated: true
VolumeStatus /run/volumemgr/VolumeStatus/cbd8bf0c…#0.json Ready
DomainStatus /run/domainmgr/DomainStatus/01859d42…json Present
CloudInit disk /run/domainmgr/cloudinit/01859d42…cidata 374KB

k3s objects created (confirmed)

Object Name Namespace Detail
VirtualMachineInstanceReplicaSet tf-stnd-atl-vm-1-01859-0 eve-kube-app Owner of the VMI – zedkube creates VMIRS not VM
VirtualMachineInstance tf-stnd-atl-vm-1-01859-0kpr7l eve-kube-app Running · node: tf-demo-adv-en-1
PersistentVolumeClaim cbd8bf0c…-pvc-0 eve-kube-app 20Gi · RWO · Block mode · Longhorn
NetworkAttachmentDefinition network-instance-attachment eve-kube-app Bootstrap NAD – eve-bridge CNI

Key finding: VMIRS not VM

zedkube creates a VirtualMachineInstanceReplicaSet (VMIRS), not a VirtualMachine or bare VirtualMachineInstance. The VMI is owned by the VMIRS. This is why kubectl get vms -n eve-kube-app returns nothing while kubectl get vmis -n eve-kube-app shows the running instance.

kubectl get vmirs -n eve-kube-app   # the actual object zedkube creates
kubectl get vmis -n eve-kube-app    # the running instance owned by VMIRS
kubectl get vms -n eve-kube-app     # empty -- EVE does not use VirtualMachine objects

Key finding: single bootstrap NAD

EVE does NOT create a separate NetworkAttachmentDefinition per Network Instance. It uses the single bootstrap NAD network-instance-attachment created at cluster initialisation. The eve-bridge CNI plugin handles NI routing at runtime using context passed by EVE – not via separate NAD objects per NI.

This corrects the earlier assumption that a NAD would be created per NI on app deploy.

VM spec (confirmed from kubectl)

CPU and memory

Field Value
CPU cores 2
CPU model host-model
Machine type pc-q35-rhel9.6.0 (KubeVirt Q35)
Memory guest 2Gi
Memory maxGuest 8Gi (hotplug capable)
QoS class Burstable

Disk layout

Disk Target Type Source Size
disk1 vda Longhorn PVC · virtio bus cbd8bf0c…-pvc-0 20Gi Block
disk2 sda hostDisk · SATA CDROM · read-only /run/domainmgr/cloudinit/01859d42…cidata 374KB

disk1 is the VM boot disk – a Longhorn PVC in Block mode (not filesystem).
disk2 is the cloud-init config disk – written by domainmgr on the host and mounted directly into the VMI via KubeVirt hostDisk. This means cloud-init config is generated by the EVE pillar layer, not by Kubernetes.

Network

networks:
- multus:
    networkName: network-instance-attachment   # bootstrap NAD
  name: net1

interfaces:
- bridge: {}                                   # bridge mode = Switch NI (L2)
  macAddress: "02:11:22:33:44:55"              # EVE-assigned MAC
  name: net1
  podInterfaceName: pod6c270ef2f25             # Multus VIF on the pod
  linkState: up                                # confirmed UP
  infoSource: domain, multus-status            # confirmed by both KubeVirt AND Multus

infoSource: domain, multus-status means both the QEMU domain and Multus independently confirm the interface is up and connected.

VifList: null in AppNetworkStatus confirms Switch NI behaviour – EVE does not assign or track the VM's IP address. The VM gets its IP from the upstream network on eth0.

Node affinity

The VMI has a strong node affinity preference (weight 100) for tf-demo-adv-en-1:

affinity:
  nodeAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - preference:
        matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - tf-demo-adv-en-1
      weight: 100

EVE pins VMs to specific nodes via node affinity. This is a preference not a requirement – KubeVirt can schedule elsewhere if the preferred node is unavailable.

EVE domain name label

labels:
  App-Domain-Name: 01859d42-b9c5-48f4-a7c3-d40be41b5877.1.1

Format: <app-uuid>.<generation>.<instance> – EVE's internal domain name scheme carried into k8s as a label on the VMI.

Migration status

Type Status Reason
Live migration Not possible PVC is ReadWriteOnce – not shared across nodes
Storage live migration Possible StorageLiveMigratable: true
Migration method BlockMigration Copies block device across nodes
Migration transport Unix Local Unix socket

Live migration requires ReadWriteMany PVCs. EVE uses ReadWriteOnce Longhorn volumes by default which means VMs cannot be live migrated without reconfiguration.

How to inspect a running VM

# k3s side
kubectl get vmirs -n eve-kube-app
kubectl get vmis -n eve-kube-app
kubectl get vmis -n eve-kube-app -o yaml
kubectl get pvc -n eve-kube-app
kubectl describe vmi <name> -n eve-kube-app
 
# EVE host side
cat /run/domainmgr/DomainStatus/<app-uuid>.json | jq '{State, VifList, DiskList}'
cat /run/zedrouter/AppNetworkStatus/<app-uuid>.json | jq
ls -lh /run/domainmgr/cloudinit/

See also: NAD - NetworkAttachmentDefinition explained

eve-k/network-instances/start.txt · Last modified: by 127.0.0.1