ZEDEDA WiKi

User Tools

Site Tools

Trace: • edge-view
eve-kvm:core-isolation

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
eve-kvm:core-isolation [2026/09/18 16:47] – mceve-kvm:core-isolation [2026/09/19 17:35] (current) – mc
Line 1: Line 1:
-====== EVE CPU Core Pinning and Isolation ======+====== EVE CPU Core Isolation — Step-by-Step Runbook ======
  
-How to enable kernel CPU isolation on an EVE node, prove it took effect, and demo it.+Follow the steps in order. Every step has a command, the output you should see, and what to do if you do not see it. Do not skip **Step 5** (the BEFORE baseline) — it is what makes the AFTER checks meaningful.
  
-**Scope.** Applies to the ''core-pinning'' feature branch (eve [[https://github.com/lf-edge/eve/pull/6335|PR #6335]], eve-api [[https://github.com/lf-edge/eve-api/pull/155|#155]], adam [[https://github.com/lf-edge/adam/pull/158|#158]], NFR-172). Verified on EVE ''0.0.0-core-pinning-gmwtus-551edfcb-kvm-amd64''.+**Scope.** The ''core-pinning'' feature branch (eve [[https://github.com/lf-edge/eve/pull/6335|PR #6335]], eve-api [[https://github.com/lf-edge/eve-api/pull/155|#155]], adam [[https://github.com/lf-edge/adam/pull/158|#158]], NFR-172). Verified end to end on EVE ''0.0.0-core-pinning-gmwtus-551edfcb-kvm-amd64'', node = Intel Core i3-13100TE, on 2026-09-18. Every transcript below is real output from that run except Step 17, which is marked.
  
-**Provenance.** The transcripts in §3, §4.1 and §4.4 were captured on a live node on 2026-09-18. The outputs in §4.2 and §4.3 are expected values, not yet captured.+**Time required.** About 20 minutes, including one reboot. 
 + 
 +**Just here to run the demo?** Use the four-act script immediately below. The numbered steps in Parts 1-4 are the full procedure behind it.
  
 ---- ----
  
-===== 1. Requirements =====+===== Quick demo script — 1, 2, 3, 4 =====
  
-==== 1.1 CPU ====+Four commands-and-a-sentence. Run them in order, with the pinned VM **not yet deployed**.
  
-Pinning on this branch means **whole-core SMT**: each vCPU gets one hardware thread of a physical core, and both siblings go to the same workload. The allocator will only use a core that has **exactly two hardware threads**.+Set this up first so every command is a single keystroke away:
  
-  * **SMT / Hyper-Threading is mandatory and must be enabled in the BIOS.** A node with no sibling threads fails closed with ''cpu.topology.unsupported'' — there is no fallback to thread-granular placement. +<code bash> 
-  * At least **3 physical cores** to be useful: one for EVE housekeeping, one for the dedicated pool, one or more to isolate. +alias iso='echo "isolated: [$(cat /sys/devices/system/cpu/isolated)]"' 
-  * Even vCPU counts only. An odd count fails closed with ''cpu.policy.odd_vcpu''. +alias pools='cat /run/domainmgr/CPUPoolStatus/*.json' 
-  * Homogeneous cores are strongly preferred. On Intel hybrid parts the E-cores have no SMT and are silently skipped for pinning.+alias plan='cat /run/domainmgr/cpuplan.json' 
 +alias ticks='awk "/^cpu[0-9] /{printf \"%s=%s  \", \$1, \$2+\$4}" /proc/stat; echo' 
 +</code> 
 + 
 +==== 1. BEFORE — a plain node: no isolation, no assignment ==== 
 + 
 +<code bash> 
 +iso 
 +pools 
 +ticks 
 +</code> 
 + 
 +<code> 
 +isolated: [] 
 +{"Pools":[{"Kind":1,"CPUs":[0,1,2,3,4,5,6,7],"FreeCPUs":[1,2,3,4,5,6,7],"TotalCores":4,"FreeWholeCores":3}, 
 +          {"Kind":2,"CPUs":null},{"Kind":3,"CPUs":null}]} 
 +cpu0=73851  cpu1=586  cpu2=496  cpu3=390  cpu4=1120  cpu5=980  cpu6=842  cpu7=771 
 +</code> 
 + 
 +**Say:** "The kernel isolates nothing. There is one pool — all eight threads, shared. Every CPU is doing work. Any workload can be scheduled anywhere, and so can the kernel's own housekeeping." 
 + 
 +//This is the state before any configuration. If your node is already configured and you want to rehearse the whole arc, revert it with:// ''eve config mount /tmp/cfg && rm /tmp/cfg/grub.cfg && eve config unmount && reboot'' //— or just show a saved capture of this output.// 
 + 
 +==== 2. AFTER the config — the isolated pool exists, and nobody may use it ==== 
 + 
 +Enable per Part 2 (property, ''grub.cfg'', reboot), then: 
 + 
 +<code bash> 
 +iso 
 +pools 
 +ticks 
 +</code> 
 + 
 +<code> 
 +isolated: [4-7] 
 +{"Pools":[{"Kind":1,"CPUs":[0,1,2,3,4,5,6,7],"FreeCPUs":[2,3],"AllocatedThreads":6,"FreeWholeCores":1}, 
 +          {"Kind":2,"CPUs":null}, 
 +          {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[4,5,6,7],"TotalCores":2,"FreeWholeCores":2}]} 
 +cpu0=9525  cpu1=376  cpu2=496  cpu3=390  cpu4=0  cpu5=0  cpu6=0  cpu7=0 
 +</code> 
 + 
 +**Say:** "Now there are three pools. Four CPUs are isolated. Look at the housekeeping pool — of eight threads, six are gone: two reserved for EVE, four withheld for isolation. Only one whole core is left for ordinary work. And cores 4 through 7 have executed **zero** cycles since boot — not the scheduler, not a workload, nothing." 
 + 
 +**Point at:** ''FreeWholeCores'' dropping 3 → 1, and the four zeros. 
 + 
 +==== 3. DEPLOY — a pinned VM takes an isolated core ==== 
 + 
 +Deploy a **2-vCPU** app with CPU pinning enabled, then: 
 + 
 +<code bash> 
 +plan 
 +pools 
 +</code> 
 + 
 +<code> 
 +{ "display_name": "TF-STND-VM-1", "mode": "whole-core-smt", "vcpus": 2, 
 +  "status": "success", "host_cpus": [4, 5] } 
 + 
 +{"Pools":[{"Kind":1,"CPUs":[0,1,2,3,6,7],"FreeCPUs":[2,3],"FreeWholeCores":1}, 
 +          {"Kind":2,"CPUs":[4,5],"AllocatedThreads":2}, 
 +          {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[6,7],"FreeWholeCores":1}]} 
 +</code> 
 + 
 +**Say:** "Two vCPUs, one whole physical core, taken from the isolated set — cores 4 and 5. The isolated pool drops from two free cores to one. The app asked for nothing but 'CPU pinning'; the node-level switch did the rest." 
 + 
 +==== 4. PROVE IT — the kernel agrees, down to the thread ==== 
 + 
 +<code bash> 
 +PID=$(pgrep -f qemu-system | head -1) 
 +for t in /proc/$PID/task/*; do 
 +  printf '%-18s %s 
 +' "$(cat $t/comm)" "$(awk '/Cpus_allowed_list/{print $2}' $t/status)" 
 +done | sort -u 
 +cat /sys/fs/cgroup/cpuset/eve-user-apps/*.1/cpuset.cpus 
 +ticks 
 +</code> 
 + 
 +<code> 
 +qemu-system-x86    4         <-- vCPU 0 
 +qemu-system-x86    5         <-- vCPU 1 
 +qemu-system-x86    4-5       <-- emulator / IO threads 
 +vhost-7981         4-5 
 + 
 +4-5 
 + 
 +cpu0=9525  cpu1=376  cpu2=496  cpu3=390  cpu4=1990  cpu5=813  cpu6=0  cpu7=0 
 +</code> 
 + 
 +**Say:** "One vCPU thread per hardware thread, pinned 1:1. Everything else the VM needs is confined to the same core. The cgroup agrees. And the second isolated core is **still at zero** — it is reserved and untouchable, waiting for the next workload that asks." 
 + 
 +**The closing line:** cores 6 and 7 are idle and cannot be used by anything that did not ask for isolation. On this node, a workload that needs guaranteed CPU gets it — not by priority, not by best effort, but because nothing else is allowed to run there. 
 + 
 +---- 
 + 
 +===== Part 0 — Understand what you are doing ===== 
 + 
 +There are **two independent switches**. Neither works without the other. 
 + 
 +^ # ^ Switch ^ Where it is set ^ What it does ^ Needs reboot? ^ 
 +| 1 | ''isolcpus'' kernel argument | ''/config/grub.cfg'' on the node | creates the isolated CPU set | **yes** — kernel command line | 
 +| 2 | ''cpu.pinning.use.isolated'' | node-level property in the controller — **Terraform or REST only, not in the UI** | lets pinned workloads use that set | no, but see Step 9 | 
 + 
 +A third thing is often confused with these: 
 + 
 +  * **"CPU pinning" on the application** (''pin_cpu'') is a //per-app// setting. It gives the workload whole physical cores out of the **dedicated** pool. It is **not** the isolation switch. If you only set this, your VM gets pinned — just not onto the isolated cores. 
 + 
 +Target layout on a 4-core / 8-thread node: 
 + 
 +^ Role ^ Physical core ^ CPUs ^ 
 +| EVE housekeeping | core 0 | 0, 1 | 
 +| Dedicated pool (ordinary pinned workloads) | core 1 | 2, 3 | 
 +| **Isolated pool** | cores 2-3 | **4, 5, 6, 7** | 
 + 
 +---- 
 + 
 +===== Part 1 — BEFORE: prerequisites and baseline ===== 
 + 
 +Run every command in this part **on the node**, over SSH or the console, before changing anything. 
 + 
 +==== Step 1 — Confirm the CPU can do this ==== 
 + 
 +<code bash> 
 +lscpu | grep -E 'Model name|Thread\(s\) per core|Core\(s\) per socket|NUMA node\(s\)' 
 +</code> 
 + 
 +Expected: 
 + 
 +<code> 
 +Model name:            13th Gen Intel(R) Core(TM) i3-13100TE 
 +Thread(s) per core:    2 
 +Core(s) per socket:    4 
 +NUMA node(s):          1 
 +</code> 
 + 
 +**PASS if ''Thread(s) per core'' is 2.** If it is 1, stop: either SMT is disabled in the BIOS (enable it and reboot) or the CPU has no SMT and cannot run this feature at all. There is no fallback — pinning fails closed with ''cpu.topology.unsupported''. 
 + 
 +You also need **at least 3 physical cores**: one for EVE, one for the dedicated pool, one to isolate.
  
 ^ CPU ^ Topology ^ Usable ^ ^ CPU ^ Topology ^ Usable ^
Line 28: Line 166:
 | Core Ultra 200S / Meteor Lake / Lunar Lake | SMT removed | **no** | | Core Ultra 200S / Meteor Lake / Lunar Lake | SMT removed | **no** |
  
-Reference node for this page: **Intel Core i3-13100TE**, 4 P-cores / 8 threads, no E-cores, 1 NUMA node, 12 MiB shared L3, VT-x.+Avoid Intel hybrid parts with E-cores (i5/i7 12th gen and up): the E-cores have no SMT and are silently skipped for pinning, so they add nothing and confuse the results.
  
-==== 1.2 Check the node before you start ====+==== Step 2 — Find out which CPUs are SMT siblings ==== 
 + 
 +This decides the numbers you will type in Step 8. **Do not assume them.**
  
 <code bash> <code bash>
-lscpu | grep -E 'Model name|Thread\(s\) per core|Core\(s\) per socket|NUMA node\(s\)'+cat /sys/devices/system/cpu/cpu[0-9]*/topology/thread_siblings_list | sort -u
 </code> </code>
  
-''Thread(s) per core: 2'' is the requirement. A ''1'' here means either the CPU has no SMT or it is disabled in the BIOS; both fail the same way.+Expected on the reference node:
  
-----+<code> 
 +0-1 
 +2-3 
 +4-5 
 +6-7 
 +</code>
  
-===== 2. Enable core isolation =====+^ If you see ^ Enumeration ^ Use the values in ^ 
 +| ''0-1  2-3  4-5  6-7'' | adjacent (Intel 12th gen and newer) | **Step 8a** | 
 +| ''0,4  1,5  2,6  3,7'' | split (older Intel) | **Step 8b** |
  
-==== 2.1 Determine the SMT sibling enumeration ====+==== Step 3 — Confirm nothing is isolated yet ====
  
-This decides your CPU indexes. Do not assume it.+<code bash> 
 +cat /sys/devices/system/cpu/isolated 
 +</code> 
 + 
 +Expected: **empty output.** That is the starting state. If it already lists CPUs, someone has been here before — read the existing ''/config/grub.cfg'' before you overwrite it. 
 + 
 +==== Step 4 — Confirm the property is currently off ====
  
 <code bash> <code bash>
-cat /sys/devices/system/cpu/cpu[0-9]*/topology/thread_siblings_list | sort -u+grep -o 'cpu.pinning.use.isolated[^}]*}' /run/zedagent/ConfigItemValueMap/global.json
 </code> </code>
  
-Two possible answers on a 4C/8T part:+Expected:
  
-^ Output ^ Meaning ^ Core map ^ +<code> 
-| ''0-1  2-3  4-5  6-7'' | adjacent siblings (Intel 12th gen and newer) | core0={0,1} core1={2,3} core2={4,5} core3={6,7} | +cpu.pinning.use.isolated":{"Key":"cpu.pinning.use.isolated","ItemType":2,"IntValue":0,"StrValue":"","BoolValue":false,"TriStateValue":0} 
-| ''0,4  1,5  2,6  3,7'' | split siblings (older Intel) | core0={0,4} core1={1,5} core2={2,6} core3={3,7} |+</code>
  
-==== 2.2 Decide the split ====+''BoolValue'' is ''false''. If the key is missing entirely, this EVE build does not have the feature — check ''eve version''.
  
-Three roles. The isolated set **must be sibling-complete** — a core with one thread isolated and the other in housekeeping serves neither ordinary requests nor the isolated pool, and simply disappears from usable capacity.+==== Step 5 — Capture the BEFORE baseline ====
  
-^ Role ^ Cores ^ CPUs (adjacent enumeration) ^ +Paste this whole block. Save the output somewhere; you will compare against it in Step 12.
-| EVE housekeeping | core 0 | 0,1 | +
-| Dedicated pool (ordinary pinned workloads) | core 1 | 2,3 | +
-| Isolated pool | cores 2-3 | 4,5,6,7 |+
  
-==== 2.3 Set the controller property FIRST ====+<code bash> 
 +echo "=== cmdline ===" 
 +tr ' ' '\n' < /proc/cmdline | grep -E 'isolcpus|nohz_full|rcu_nocbs|irqaffinity|eve_max_vcpus' 
 +echo "=== kernel isolated set ===" 
 +echo "[$(cat /sys/devices/system/cpu/isolated)]" 
 +echo "=== CPU pools ===" 
 +cat /run/domainmgr/CPUPoolStatus/*.json; echo 
 +echo "=== placement plan ===" 
 +cat /run/domainmgr/cpuplan.json 2>/dev/null 
 +echo "=== eve cpuset ===" 
 +cat /sys/fs/cgroup/cpuset/eve/cpuset.cpus 
 +echo "=== per-CPU busy ticks ===" 
 +awk '/^cpu[0-9] /{print $1, $2+$4}' /proc/stat 
 +</code> 
 + 
 +Expected BEFORE (no isolation, nothing pinned): 
 + 
 +<code> 
 +=== cmdline === 
 +eve_max_vcpus=1 
 +=== kernel isolated set === 
 +[] 
 +=== CPU pools === 
 +{"Pools":[ 
 + {"Kind":1,"CPUs":[0,1,2,3,4,5,6,7],"FreeCPUs":[1,2,3,4,5,6,7],"TotalThreads":8,"AllocatedThreads":1,"FreeThreads":7,"TotalCores":4,"FreeWholeCores":3}, 
 + {"Kind":2,"CPUs":null,"TotalThreads":0,"FreeWholeCores":0}, 
 + {"Kind":3,"CPUs":null,"TotalThreads":0,"FreeWholeCores":0} 
 +]} 
 +=== eve cpuset === 
 +0 
 +</code> 
 + 
 +Pool ''Kind'' values, needed for every later step: **1 = housekeeping, 2 = dedicated, 3 = isolated.** 
 + 
 +What this baseline says: all 8 threads are in housekeeping, only CPU 0 is taken (by EVE), and the dedicated and isolated pools do not exist. 
 + 
 +---- 
 + 
 +===== Part 2 — ENABLE ===== 
 + 
 +Do these three steps **in this order**. Doing the property last costs you a second reboot. 
 + 
 +==== Step 6 — Set the controller property ==== 
 + 
 +**This property is not exposed in the ZedControl UI.** Do not go looking for it there — the UI only offers properties it knows about, and this one is new in the feature branch. It can only be set via **Terraform** or the **REST API**. 
 + 
 +**Terraform** — a ''config_item'' block on the **''zedcloud_edgenode''** resource. Not on the application, not on the app instance. 
 + 
 +<code> 
 +resource "zedcloud_edgenode" "my_node" { 
 +  # ... existing fields ... 
 + 
 +  config_item { 
 +    key          = "cpu.pinning.use.isolated" 
 +    string_value = "true" 
 +  } 
 +} 
 +</code> 
 + 
 +Then ''terraform apply''. 
 + 
 +**Use ''string_value = "true"''.** This is the single most common way to get "I set it and nothing happened", and the reason is that two different schemas are involved: 
 + 
 +^ Hop ^ Schema ^ Fields ^ 
 +| you → controller | ''EDConfigItem'' | ''key'', ''valueType'', ''stringValue'', ''boolValue'', ''floatValue'', ''uint32Value'', ''uint64Value'' | 
 +| controller → device | eve-api ''ConfigItem'' | ''key'', ''value'' — **both strings** | 
 + 
 +The controller has to collapse the typed fields into a single string ''value'' for the device. An entry carrying only ''boolValue: true'' with no ''valueType'' can arrive at the device as an empty ''value'', which pillar cannot parse, so the item keeps its default — ''false''. ''stringValue'' passes straight through. The provider also exposes an optional ''value_type'' attribute; it is not needed when you use ''string_value''. 
 + 
 +**REST** — ''PUT /api/v1/devices/id/{id}'' (operation ''EdgeNodeConfiguration_UpdateEdgeNode''). The property goes in the device object's ''configItem'' array, whose elements are ''EDConfigItem'': 
 + 
 +<code javascript> 
 +{ "key": "cpu.pinning.use.isolated", "stringValue": "true" } 
 +</code> 
 + 
 +**This is a full-object PUT, not a patch.** The body carries the whole edge node — ''name'', ''title'', ''modelId'', ''projectId'', ''interfaces'', ''adminState'', every existing ''configItem'' and about forty more fields. Send a partial body and you erase what you left out. It also requires ''revision.curr'', the current database version of the record; a stale value is rejected with //409 Version mismatch//. 
 + 
 +So the REST procedure is read-modify-write: 
 + 
 +  - ''GET /api/v1/devices/id/{id}'' 
 +  - append ''{ "key": "cpu.pinning.use.isolated", "stringValue": "true" }'' to the returned ''configItem'' array, keeping every existing entry 
 +  - ''PUT'' the entire modified object back, including ''revision'' 
 + 
 +This is why Terraform is the easier route: the provider does that read-modify-write for you. Schema verified against ''swagger/zedge_node_service.swagger.json'' (ZEDEDA Edge Node Service v1.0) in the provider source; confirm against your own controller's ''/api/v1/docs/'' if it runs a different version. 
 + 
 +**Do not expect to find the key itself in the API documentation.** In the spec ''key'' is a plain ''string'' with no enum and no validation — ''configItem'' is a free-form key/value list that the controller stores and forwards to the device verbatim. The only component that knows ''cpu.pinning.use.isolated'' is a real property is EVE itself (pillar's ''types/global.go''). That is why the property works through a controller that has never heard of it, and equally why the UI does not offer it: the UI renders a curated list of known properties, the API accepts any string. 
 + 
 +Whichever route you use, **Step 7 is what confirms it** — do not assume it landed. 
 + 
 +==== Step 7 — Confirm the property reached the node ==== 
 + 
 +Do not go further until this passes. Allow a minute for the config poll. 
 + 
 +<code bash> 
 +grep -o 'cpu.pinning.use.isolated[^}]*}' /run/zedagent/ConfigItemValueMap/global.json 
 +logread | grep 'use.isolated' 
 +</code>
  
-In the controller, set the edge-node configuration property:+Expected:
  
 <code> <code>
-cpu.pinning.use.isolated = true+"BoolValue":true 
 +domainmgr: CPU placement: cpu.pinning.use.isolated is now true; workloads placed from now on may use the kernel-isolated CPUs
 </code> </code>
  
-**Order matters.** Set this //before// the reboot that applies ''isolcpus''. The switch only affects workloads placed after it is set, so setting it after the reboot silently does nothing until the next placement. Setting it on a node whose kernel isolates nothing warns rather than errors.+**If ''BoolValue'' is still ''false'':** you used ''bool_value'' instead of ''string_value'' (go back to Step 6), or the controller rejected the key — see Troubleshooting.
  
-Without this property, kernel-isolated CPUs are reported in the pool report and usable by **nobody**: they are withheld from every workload that did not explicitly ask for isolation, and only ''isolation_tier=hard'' can claim them — which current controllers cannot express.+==== Step 8 — Write /config/grub.cfg ====
  
-==== 2.4 Write /config/grub.cfg ====+''/config/grub.cfg'' does **not** exist on a fresh install. The image ships only ''grub.cfg.tmpl'', and GRUB reads ''grub.cfg'' exactly. **Create the file. Do not rename the template.**
  
-''/config/grub.cfg'' does **not** exist on a fresh install. The image ships only ''grub.cfg.tmpl'', and GRUB reads ''grub.cfg'' exactly — so create the file, do not rename the template.+=== Step 8a — adjacent siblings (0-1, 2-3, 4-5, 6-7) ===
  
 <code bash> <code bash>
Line 93: Line 340:
 </code> </code>
  
-For the **split** enumeration use ''isolcpus=managed_irq,domain,2,3,6,7 rcu_nocbs=2,3,6,7 nohz_full=2,3,6,7 irqaffinity=0,4'' and **omit** the ''eve_max_vcpus'' line: reservation is by lowest CPU id and a core is dropped if any sibling is reserved, so ''eve_max_vcpus=2'' there would reserve CPUs 0 and 1, which belong to two different cores, and cost a second core for nothing.+=== Step 8b — split siblings (0,4  1,5  2,6  3,7) ===
  
-Notes:+<code bash> 
 +eve config mount /tmp/cfg 
 +cat > /tmp/cfg/grub.cfg <<'EOF' 
 +set_getty 
 +set_global dom0_extra_args "$dom0_extra_args isolcpus=managed_irq,domain,2,3,6,7 rcu_nocbs=2,3,6,7 nohz_full=2,3,6,7 irqaffinity=0,4" 
 +EOF 
 +cat /tmp/cfg/grub.cfg 
 +sync 
 +eve config unmount 
 +</code>
  
-  * The quoted heredoc (''<<'EOF''') is required. Unquoted, the shell expands ''$dom0_extra_args'' to nothing and GRUB loses everything it had accumulated. +There is **no ''eve_max_vcpus'' line in 8b** on purpose. Reservation is by lowest CPU id and a core is dropped if any sibling is reserved, so ''eve_max_vcpus=2'' with split enumeration would reserve CPUs 0 and 1 — two halves of two different cores — and cost you a second core for nothing.
-  * ''irqaffinity'' names the **housekeeping** CPUs, not the isolated ones. Pointing it at the isolated set steers interrupts onto the cores you are trying to shield. +
-  * ''set_getty'' keeps a console shell available; it is the only content of the shipped template, so without it you lose that. +
-  * Do **not** use GRUB's ''set_isolcpus'' helper (the interactive menu entry //isolate CPU0 (only for PREEMPT_RT)//). It emits ''isolcpus=inverse,0'', which with the default ''eve_max_vcpus=1'' isolates CPU 0's own SMT sibling and produces a sibling-incomplete core. +
-  * ''/config/grub.cfg'' is sourced after EVE's defaults and before the boot entry is built, so ''set_global'' overrides the shipped ''eve_max_vcpus=1''.+
  
-==== 2.5 Reboot ====+**Check the ''cat'' output before moving on.** The third line must still contain the literal text ''$dom0_extra_args''. If that text is missing, your shell expanded it and GRUB will lose everything it had accumulated — rewrite it, making sure the heredoc marker is quoted as ''<<'EOF''' and not ''<<EOF''. 
 + 
 +Rules that matter here: 
 + 
 +  * ''irqaffinity'' lists the **housekeeping** CPUs, never the isolated ones. Pointing it at the isolated set steers interrupts onto the cores you are shielding. 
 +  * ''set_getty'' keeps a console shell. It is the only thing in the shipped template, so if you do not include it you lose it. 
 +  * Do **not** use GRUB's ''set_isolcpus'' helper (menu entry //isolate CPU0 (only for PREEMPT_RT)//). It emits ''isolcpus=inverse,0'', which isolates CPU 0's own sibling and produces a core that serves nobody. 
 + 
 +==== Step 9 — Reboot ====
  
 <code bash> <code bash>
Line 109: Line 369:
 </code> </code>
  
-Kernel isolation is a command-line decision, so it is reboot-gated. Expect roughly 3 minutes before SSH answers again.+Allow about 3 minutes. The reboot is required for two reasons: ''isolcpus'' is a kernel command-line argument, and a reboot is also what lets **already-running** workloads be re-placed (see Troubleshooting — a restart is not enough). 
 + 
 +Expect the SSH host key to change: EVE regenerates ''/etc/ssh/ssh_host_*'' on every boot. Clear the old entry with ''ssh-keygen -R <node-ip>''.
  
 ---- ----
  
-===== 3. Validate =====+===== Part 3 — AFTER: validate =====
  
-Run these in order. Stop at the first one that disagrees.+Run these in order. Stop at the first failure.
  
-==== 3.1 The kernel accepted the arguments ====+==== Step 10 — The kernel accepted the arguments ====
  
 <code bash> <code bash>
Line 131: Line 393:
 </code> </code>
  
-==== 3.2 The kernel is actually isolating ====+**If nothing appears:** GRUB never read your file. Confirm it is named ''grub.cfg'' (not ''grub.cfg.tmpl'') on the CONFIG partition. 
 + 
 +==== Step 11 — The kernel is actually isolating ====
  
 <code bash> <code bash>
Line 141: Line 405:
 </code> </code>
  
-**An empty result means there is no isolated pool and nothing downstream will work.** Stop and fix the command line.+**This is the make-or-break check.** An empty result means there is no isolated pool and nothing after this point can work. Do not continue — fix the command line first.
  
-==== 3.3 domainmgr saw the topology and the isolated set ====+==== Step 12 — domainmgr saw the topology and the isolated set ====
  
 <code bash> <code bash>
Line 154: Line 418:
 </code> </code>
  
-''CPU topology: N physical cores'' must appear. If topology discovery failed, the model is synthetic and **whole-core placement is refused outright** rather than performed against a fabricated topology.+''CPU topology: N physical cores'' **must** appear. If topology discovery failed, the model is synthetic and whole-core placement is refused outright rather than performed against a fabricated topology.
  
-Note: the second line is printed whenever the isolated set is non-empty, regardless of ''cpu.pinning.use.isolated''. It does not prove the promotion switch is on — check the property itself (§3.5).+Ignore the wording of the second line — it is printed whenever the isolated set is non-empty, whether or not the switch is on. Step 7 is what tells you the switch is on.
  
-==== 3.4 The pool report ====+==== Step 13 — The pools changed ====
  
 <code bash> <code bash>
 cat /run/domainmgr/CPUPoolStatus/*.json cat /run/domainmgr/CPUPoolStatus/*.json
 </code> </code>
- 
-Pool ''Kind'' values: **1 = housekeeping, 2 = dedicated, 3 = isolated**. 
  
 <code javascript> <code javascript>
 {"Pools":[ {"Pools":[
  {"Kind":1,"CPUs":[0,1,2,3,4,5,6,7],"FreeCPUs":[2,3],"TotalThreads":8,"AllocatedThreads":6,"FreeThreads":2,"TotalCores":4,"FreeWholeCores":1},  {"Kind":1,"CPUs":[0,1,2,3,4,5,6,7],"FreeCPUs":[2,3],"TotalThreads":8,"AllocatedThreads":6,"FreeThreads":2,"TotalCores":4,"FreeWholeCores":1},
- {"Kind":2,"CPUs":null,"FreeCPUs":null,"TotalThreads":0,"AllocatedThreads":0,"FreeThreads":0,"TotalCores":0,"FreeWholeCores":0},+ {"Kind":2,"CPUs":null,"TotalThreads":0,"FreeWholeCores":0},
  {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[4,5,6,7],"TotalThreads":4,"AllocatedThreads":0,"FreeThreads":4,"TotalCores":2,"FreeWholeCores":2}  {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[4,5,6,7],"TotalThreads":4,"AllocatedThreads":0,"FreeThreads":4,"TotalCores":2,"FreeWholeCores":2}
 ]} ]}
 </code> </code>
  
-What to read from it:+==== Step 14 — BEFORE vs AFTER at a glance ====
  
-  * **Kind 3 is non-empty** — the isolated set is now allocatable capacity, not just a number in the device info. +^ Check ^ BEFORE ^ AFTER ^ 
-  * **Kind 1 ''FreeCPUs'' is ''[2,3]'' only** — of 8 threads, 6 are allocated: CPUs 0-1 reserved for EVE and CPUs 4-7 withheld for isolation. This is the point of the feature: housekeeping freeness answers //"will an ordinary workload fit?"//, which isolated CPUs cannot serve. +| ''/sys/devices/system/cpu/isolated'' | empty | ''4-7'' | 
-  * **Kind 1 ''FreeWholeCores'' is 1** — exactly core 1.+| ''eve_max_vcpus'' | 1 | 2 | 
 +| Isolated pool (Kind 3) | does not exist | ''[4,5,6,7]'', 2 whole cores | 
 +| Housekeeping free CPUs (Kind 1) | ''[1,2,3,4,5,6,7]'' | ''[2,3]'' | 
 +| Housekeeping free whole cores | 3 | 1 |
  
-==== 3.5 The promotion switch ====+The housekeeping line is the one to point at. Of 8 threads, 6 are now allocated: CPUs 0-1 reserved for EVE and CPUs 4-7 withheld for isolation. Only core 1 is left for an ordinary workload. That is the feature working: housekeeping freeness answers //"will an ordinary workload fit?"//, and isolated CPUs cannot serve that.
  
-<code bash> +==== Step 15 — Nothing is running on the isolated cores ====
-grep -o 'cpu.pinning.use.isolated[^}]*}' /run/zedagent/ConfigItemValueMap/global.json +
-</code> +
- +
-<code> +
-cpu.pinning.use.isolated":{"Key":"cpu.pinning.use.isolated","ItemType":2,"IntValue":0,"StrValue":"","BoolValue":true,"TriStateValue":0} +
-</code> +
- +
-''BoolValue'' must be ''true''. This value comes from the controller only. A value written locally into ''/config/GlobalConfig/global.json'' survives just until the first config poll, because zedagent rebuilds the whole map from defaults plus the controller's items. +
- +
-==== 3.6 Derived placement plan ====+
  
 <code bash> <code bash>
-cat /run/domainmgr/cpuplan.json +awk '/^cpu[4-7] /{print $1, $2+$4}' /proc/stat
-</code> +
- +
-<code javascript> +
-{ +
-  "workloads": [ +
-    { "uuid": "...", "display_name": "TF-STND-VM-1", "mode": "whole-core-smt", +
-      "vcpus": 2, "status": "success", "host_cpus": [4, 5] } +
-  ] +
-} +
-</code> +
- +
-Diagnostic only — nothing reads it back — but it is the quickest way to see what the allocator decided, including for workloads that are configured but not running. +
- +
-==== 3.7 Known limitations on the shipped kernel ==== +
- +
-^ Argument ^ Effect ^ Why ^ +
-| ''isolcpus'' | **works** | ''CONFIG_CPU_ISOLATION=y'' | +
-| ''irqaffinity'' | **works** | accepted by the kernel | +
-| ''nohz_full'' | **no effect** | ''# CONFIG_NO_HZ_FULL is not set'' | +
-| ''rcu_nocbs'' | **no effect** | ''CONFIG_RCU_NOCB_CPU'' not set | +
- +
-<code bash> +
-dmesg | grep -iE 'nohz|Unknown kernel command line' +
-(zcat /proc/config.gz || cat /proc/config) | grep -E 'CONFIG_NO_HZ_FULL|CONFIG_RCU_NOCB_CPU|CONFIG_CPU_ISOLATION'+
 </code> </code>
  
 <code> <code>
-Housekeeping: nohz unsupported. Build with CONFIG_NO_HZ_FULL +cpu4 0 
-Unknown kernel command line parameters "... rcu_nocbs=4,5,6,7 nohz_full=4,5,6,7", will be passed to user space +cpu5 0 
-# CONFIG_NO_HZ_FULL is not set +cpu6 0 
-CONFIG_CPU_ISOLATION=y+cpu7 0
 </code> </code>
  
-So today you get the ''isolcpus'' half of hard isolation — the scheduler's load balancer is kept off the isolated cores — but **not** tick shedding or RCU callback offload. Getting those requires ''CONFIG_NO_HZ_FULL'' and ''CONFIG_RCU_NOCB_CPU'' in eve-kernel. Leave the arguments in place; they are harmless and become effective as soon as the kernel supports them. +Zero busy ticks since boot. Keep this number — it is the strongest single line in the demo.
- +
-Two more expected observations: +
- +
-  * **''/sys/fs/cgroup/cpuset/eve/cpuset.cpus'' stays ''0''** even with ''eve_max_vcpus=2''. That cpuset is driven by ''dom0_max_vcpus'' (still 1); ''eve_max_vcpus'' only sets the ''eve/services'' child, which cannot exceed its parent. No workload capacity is lost — core 0 is dropped from placement either way — but CPU 1 sits reserved and unused. To give EVE its whole core, also set ''set_global hv_dom0_cpu_settings "dom0_max_vcpus=2 dom0_vcpus_pin"''. +
-  * **The vault reports a PCR mismatch.** Changing the kernel command line changes PCRs 8, 9 and 14, so ''eve diag'' shows ''vault: ENABLED unlock:controller-key mismatchPCRs:[8 9 14]''. The vault unlocks with the controller key instead of the TPM seal. Expected after any ''grub.cfg'' change.+
  
 ---- ----
  
-===== 4. Demo ===== +===== Part 4 — Demo =====
- +
-Four steps, each one showing a distinct guarantee. Run §3 first so you are demoing from a known-good base. +
- +
-==== 4.1 An ordinary pinned workload avoids the isolated cores ====+
  
-With ''cpu.pinning.use.isolated = false'', deploy a **2-vCPU** app with CPU pinning enabled.+==== Step 16 — A pinned workload lands on the isolated cores ====
  
-Expected: it lands on the **dedicated** pool — CPUs 2,3 — and never on 4-7.+Deploy a **2-vCPU** app with CPU pinning enabled (even vCPU counts only — an odd count fails closed with ''cpu.policy.odd_vcpu'').
  
 <code bash> <code bash>
Line 254: Line 476:
  
 <code javascript> <code javascript>
-{ "uuid": "faa814f0-23d0-4f67-a0d0-ae5b39b3a354", "display_name": "TF-STND-VM-1", +{ "display_name": "TF-STND-VM-1", "mode": "whole-core-smt", "vcpus": 2, 
-  "mode": "whole-core-smt", "vcpus": 2, "status": "success", "host_cpus": [2, 3] }+  "status": "success", "host_cpus": [4, 5] }
  
 {"Pools":[ {"Pools":[
- {"Kind":1,"CPUs":[0,1,4,5,6,7],"FreeCPUs":null,"TotalThreads":6,"AllocatedThreads":6,"FreeWholeCores":0}, + {"Kind":1,"CPUs":[0,1,2,3,6,7],"FreeCPUs":[2,3],"AllocatedThreads":4,"FreeWholeCores":1}, 
- {"Kind":2,"CPUs":[2,3],"FreeCPUs":null,"TotalThreads":2,"AllocatedThreads":2,"FreeWholeCores":0}, + {"Kind":2,"CPUs":[4,5],"FreeCPUs":null,"AllocatedThreads":2,"FreeWholeCores":0}, 
- {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[4,5,6,7],"TotalThreads":4,"AllocatedThreads":0,"FreeWholeCores":2}+ {"Kind":3,"CPUs":[4,5,6,7],"FreeCPUs":[6,7],"AllocatedThreads":2,"FreeWholeCores":1}
 ]} ]}
 </code> </code>
  
-Kind 2 (dedicated) becomes ''[2,3]''; Kind 3 (isolated) stays **fully free**. **This is the guarantee:** an operator's isolated cores are not spent on a workload that never asked for them.+Read it as: the isolated pool went from 2 free whole cores to 1, the dedicated pool //is// ''[4,5]'' now (the isolated pool overlaps the other two rather than partitioning with them), and core 1 has been handed back to housekeeping.
  
-Note that a legacy ''pin_cpu'' app arrives with ''FullPCPUsOnly:false'' and ''ThreadsPerCore:0'' — the new API fields unset — and is still placed as ''whole-core-smt''. That reinterpretation is what makes the feature demonstrable from a controller that cannot send the new placement fields.+Worth saying out loud during a demo: the app asked only for ''pin_cpu''. It arrives with ''FullPCPUsOnly:false'' and ''ThreadsPerCore:0'' — the new API fields unset — and still gets whole-core placement on kernel-isolated cores. That is the point of the node-level switch: a controller that cannot express the new policy still gets the behaviour.
  
-==== 4.2 Capacity fails closed rather than spilling ====+==== Step 17 — Capacity fails closed instead of spilling ====
  
-Deploy a **second** 2-vCPU pinned app on the same node.+//Not yet captured on hardware; this is the expected behaviour.//
  
-Expected: it **fails** rather than taking an isolated core, and the error states how many cores were withheld for isolation — so the operator is not left comparing //"insufficient"// against CPUs that look idle.+With the switch **on**, a second 2-vCPU pinned app is promoted onto the remaining isolated core (''[6,7]''), and the **third** fails — while core 1 (''[2,3]'') sits completely free in housekeeping. A node with a visibly idle physical core refusing the workload is the most persuasive part of the demo, and it is by design: a promoted workload may only be served from the isolated set.
  
-This is the demo that makes the point: the node has 4 free threads at that moment and still refuses, //by design//.+With the switch **off**, it is the //second// app that fails, for the same reason.
  
-==== 4.3 The node-wide switch promotes a pinned workload ====+The error names how many cores were withheld for isolation, so the operator is not left comparing "insufficient" against CPUs that look idle.
  
-Set ''cpu.pinning.use.isolated = true'' in the controller, then reboot the node.+==== Step 18 — Prove it at the thread level ====
  
-Expected: the same 2-vCPU pinned app is now placed on **4,5** (or 6,7). A controller that can only send ''pin_cpu'' has reached the isolated cores without expressing anything new.+The pool report is EVE's own bookkeeping. This shows the kernel agrees.
  
 <code bash> <code bash>
-cat /run/domainmgr/cpuplan.json     # host_cpus should now be [4,5]+ls /run/hypervisor/kvm/                        # gives <app-uuid>.<gen>.<inst> 
 +PID=$(pgrep -f qemu-system | head -1) 
 +for t in /proc/$PID/task/*; do 
 +  printf '%-18s %s\n' "$(cat $t/comm)" "$(awk '/Cpus_allowed_list/{print $2}' $t/status)" 
 +done | sort -u 
 +find /sys/fs/cgroup/cpuset/eve-user-apps -maxdepth 2 -name cpuset.cpus | while read f; do echo "$f = $(cat $f)"; done 
 +awk '/^cpu[0-9] /{print $1, $2+$4}' /proc/stat
 </code> </code>
  
-Only whole-core workloads are promoted: kernel isolation means nothing on a core whose sibling the kernel still schedules freely.+<code> 
 +qemu-system-x86    4          <-- vCPU 0 
 +qemu-system-x86    5          <-- vCPU 1 
 +qemu-system-x86    4-5        <-- emulator / IO threads 
 +vhost-7981         4-5 
 +iou-wrk-7981       4-5
  
-==== 4.4 Prove it at the thread level ==== +/sys/fs/cgroup/cpuset/eve-user-apps/faa814f0-….2.1/cpuset.cpus = 4-5
- +
-The pool report is EVE's own bookkeeping. This shows the kernel agrees. +
- +
-<code bash> +
-# find the domain +
-ls /run/hypervisor/kvm/                       # <app-uuid>.<gen>.<inst> +
-PID=$(pgrep -f '<app-uuid>')+
  
-# per-thread affinity of the VM's vCPU threads +cpu4 1990   cpu5 813   cpu6 0   cpu7 0
-for t in /proc/$PID/task/*; do +
-  printf '%s  %s\n' "$(cat $t/comm)" "$(grep Cpus_allowed_list $t/status)" +
-done+
 </code> </code>
  
-Captured on the dedicated-pool run of §4.1 (2 vCPUs on core 1):+The two vCPU threads are pinned **1:1** to CPUs 4 and 5; everything else is confined to ''4-5''; the other isolated core is still at zero. Inside the guest, ''lscpu'' shows ''Thread(s) per core: 2'' — EVE truthfully advertising one physical core rather than two sockets.
  
-<code> +==== Step 19 — One-shot cross-check ====
-qemu pid=8752 +
-qemu-system-x86    2-3 +
-qemu-system-x86    2 +
-qemu-system-x86    3 +
-trace-thread       2-3 +
-vhost-8752         2-3 +
-iou-wrk-8752       2-3 +
-</code>+
  
-The two vCPU threads are pinned **1:1** to host CPUs 2 and 3; every other thread is confined to ''2-3''. Nothing touches 4-7. The app's cgroup agrees:+The branch ships a script that cross-checks the allocator's record against real per-thread affinity masks, host SMT siblings and the guest's view, and exits non-zero on any mismatch. It also catches an ''isolcpus'' range that is not sibling-complete.
  
 <code bash> <code bash>
-cat /sys/fs/cgroup/cpuset/eve-user-apps/<app-uuid>.1.1/cpuset.cpus+GUEST_PASS=<password> ./cpu-assignment-report.sh <node-ip> <app-ip>
 </code> </code>
  
-<code> +Override its lab defaults (''NODE_IP'', ''SSH_KEY=~/.ssh/ztest_key'', ''GUEST_USER=pocuser'') and make sure ''debug.enable.ssh'' is set on the node.
-2-3 +
-</code>+
  
-Expected on the isolated run of §4.3: the same shape with 4 and 5 (or 6 and 7) in place of 2 and 3, and the emulator threads confined to the housekeeping CPUs — **excluding** the isolated ones. That exclusion matters: ''isolcpus'' keeps the load balancer off a core but honours an explicit affinity, so the cpuset confining non-pinned workloads has to exclude the isolated CPUs or the kernel will happily run them there.+----
  
-Inside the guest, the topology EVE advertised should match:+===== Part 5 — Known limitations ===== 
 + 
 +==== 5.1 nohz_full and rcu_nocbs do nothing on the shipped kernel ====
  
 <code bash> <code bash>
-lscpu | grep -E 'CPU\(s\)|Thread\(s\) per core'+dmesg | grep -iE 'nohz|Unknown kernel command line' 
 +(zcat /proc/config.gz || cat /proc/config) | grep -E 'CONFIG_NO_HZ_FULL|CONFIG_RCU_NOCB_CPU|CONFIG_CPU_ISOLATION'
 </code> </code>
  
-A 2-vCPU whole-core-SMT workload sees ''Thread(s) per core: 2'' — a truthful topology, not a flat list of 2 sockets.+<code> 
 +Housekeeping: nohz unsupported. Build with CONFIG_NO_HZ_FULL 
 +Unknown kernel command line parameters "... rcu_nocbs=4,5,6,7 nohz_full=4,5,6,7", will be passed to user space 
 +# CONFIG_NO_HZ_FULL is not set 
 +CONFIG_CPU_ISOLATION=y 
 +</code>
  
-==== 4.5 One-shot cross-check ====+^ Argument ^ Effect ^ 
 +| ''isolcpus'' | **works** | 
 +| ''irqaffinity'' | **works** | 
 +| ''nohz_full'' | **no effect** — kernel not built with it | 
 +| ''rcu_nocbs'' | **no effect** — kernel not built with it |
  
-The branch ships a script that cross-checks the allocator's record against real per-thread affinity masks, host SMT siblings and the guest's view, and exits non-zero on any mismatch. It also catches an ''isolcpus'' range that is not sibling-complete.+So what you can demonstrate today is **scheduler isolation**: the load balancer is kept off the isolated cores and nothing else runs there. You cannot yet claim timer-tick shedding or RCU callback offload. That needs ''CONFIG_NO_HZ_FULL'' and ''CONFIG_RCU_NOCB_CPU'' in eve-kernel. Leave the arguments in place — they are harmless and become effective as soon as the kernel supports them.
  
-<code bash> +==== 5.2 EVE's own cpuset stays on CPU 0 ==== 
-GUEST_PASS=<password> ./cpu-assignment-report.sh <node-ip> <app-ip> + 
-</code>+''/sys/fs/cgroup/cpuset/eve/cpuset.cpus'' remains ''0'' even with ''eve_max_vcpus=2'', because that cpuset is driven by ''dom0_max_vcpus'' (still 1); ''eve_max_vcpus'' only sets the ''eve/services'' child, which cannot exceed its parent. No workload capacity is lost — core 0 is dropped from placement either way — but CPU 1 sits reserved and unused. To give EVE its whole core, add ''set_global hv_dom0_cpu_settings "dom0_max_vcpus=2 dom0_vcpus_pin"''. 
 + 
 +==== 5.3 The vault reports a PCR mismatch on the first boot ====
  
-Override its lab defaults (''NODE_IP'', ''SSH_KEY=~/.ssh/ztest_key'', ''GUEST_USER=pocuser'') and set ''debug.enable.ssh'' on the node.+Changing the kernel command line changes PCRs 8, 9 and 14, so ''eve diag'' shows ''vault: ENABLED unlock:controller-key mismatchPCRs:[8 9 14]'' on the boot right after the change. It re-seals itself and returns to ''unlock:tpm-local-sealed'' on the following boot. Expected, not a fault.
  
 ---- ----
  
-===== 5. Troubleshooting =====+===== Part 6 — Troubleshooting =====
  
 ^ Symptom ^ Cause ^ Fix ^ ^ Symptom ^ Cause ^ Fix ^
-| ''/sys/devices/system/cpu/isolated'' empty after reboot | kernel rejected the argument, or ''grub.cfg'' not read | check ''/proc/cmdline''; confirm the file is ''/config/grub.cfg'', not ''grub.cfg.tmpl'' | +| Cannot find the property in the controller UI | it is not exposed there | set it via Terraform or REST (Step 6) | 
-| ''cpu.topology.unsupported'' | no core has 2 hardware threads | enable SMT in the BIOS; the CPU may have none |+| Property set in Terraform, node still shows ''BoolValue:false'' | ''bool_value'' serializes empty | use ''string_value = "true"'' in the ''config_item'' block | 
 +| Property set on the application instead of the node | ''cpu.pinning.use.isolated'' is node-level only | move it to a ''config_item'' on ''zedcloud_edgenode'' | 
 +| ''/sys/devices/system/cpu/isolated'' empty after reboot | GRUB did not read the file, or the kernel rejected the argument | check ''/proc/cmdline''; confirm the file is ''grub.cfg'', not ''grub.cfg.tmpl'' | 
 +| ''$dom0_extra_args'' missing from ''/proc/cmdline'' | unquoted heredoc expanded it away | rewrite ''grub.cfg'' using ''<<'EOF''' | 
 +| **Pinned VM stays on the dedicated cores after enabling the switch** | a running workload holds its allocation; ''doActivate'' returns early on ''len(status.VmConfig.CPUs) > 0'' before the promotion is applied, and ''releaseCPUs'' only runs on the delete/deactivate path | **reboot the node** (''DomainStatus'' lives in ''/run'') or **deactivate and reactivate** the app instance. An app restart or a boot retry is **not** enough | 
 +| VM halted with ''BootFailed: true'', retries onto the same CPUs | someone killed the qemu process instead of stopping the workload properly | reboot, or deactivate/reactivate from the controller | 
 +| ''cpu.topology.unsupported'' | no core has two hardware threads | enable SMT in the BIOS; some CPUs have none |
 | ''cpu.policy.odd_vcpu'' | odd vCPU count on a whole-core request | use an even count | | ''cpu.policy.odd_vcpu'' | odd vCPU count on a whole-core request | use an even count |
-| Pinned workload placed on dedicated cores, never isolated | ''cpu.pinning.use.isolated'' is false, or it was set after the reboot | set the property, **then** reboot | +| Placement refused citing a synthetic topology | sysfs topology discovery failed | check ''/sys/devices/system/cpu/cpu*/topology/'' is populated | 
-| Isolated cores show as free but nothing can use them | expected without the property — they are withheld from every workload that did not ask for isolation | set the property, or use ''isolation_tier=hard'' | +| Isolated cores free but nothing can use them | expected with the switch off — they are withheld from every workload that did not ask for isolation | set the property (Step 6), or use ''isolation_tier=hard'' | 
-| Placement refused with a synthetic-topology error | sysfs topology discovery failed | check ''/sys/devices/system/cpu/cpu*/topology/'' is populated | +| SSH host key changed after reboot | EVE regenerates ''/etc/ssh/ssh_host_*'' every boot | ''ssh-keygen -R <node-ip>'' | 
-| Boot loop, ''fatal: agent zedbox[N]: zboot curpart: err exit status 64'' | two disks carry EVE's static PARTUUIDs — install media left in, or a second EVE install | remove the media or wipe the second disk's partition table | +| Boot loop, ''fatal: agent zedbox[N]: zboot curpart: err exit status 64'' | two disks carry EVE's static PARTUUIDs — install media left in, or a second EVE install | remove the media, or wipe the second disk's partition table |
-| ''$dom0_extra_args'' missing from ''/proc/cmdline'' | unquoted heredoc expanded it away when writing ''grub.cfg'' | rewrite with ''<<'EOF''' |+
  
 ---- ----
  
-===== 6. Reference =====+===== Part 7 — Reference =====
  
-  * ''docs/cpu-affinity-design.md'' in the feature branch — isolation tiers, placement vocabulary, the EVE↔controller interface. Large parts are marked //not yet implemented//; it describes the intended end state.+  * ''docs/cpu-affinity-design.md'' in the feature branch — isolation tiers, placement vocabulary, the EVE↔controller interface. Large parts are marked //not yet implemented//; it describes the intended end state, not this build.
   * ''docs/CONFIG-PROPERTIES.md'' — all configuration properties.   * ''docs/CONFIG-PROPERTIES.md'' — all configuration properties.
   * Pool kinds and error codes: ''pkg/pillar/types/cpuplacement.go''.   * Pool kinds and error codes: ''pkg/pillar/types/cpuplacement.go''.
   * Placement logic: ''pkg/pillar/cpuallocator/placement.go''.   * Placement logic: ''pkg/pillar/cpuallocator/placement.go''.
 +  * The allocation-reuse shortcut discussed in Troubleshooting: ''pkg/pillar/cmd/domainmgr/domainmgr.go'' (''doActivate''), and ''releaseCPUs'' in the same file.
  
eve-kvm/core-isolation.1789750033.txt.gz · Last modified: by mc