NodeRings Docs

Dedicated compute nodes

Sell Dedicated Compute plans with exclusive physical CPU cores that are never shared with other VMs

Dedicated compute nodes are how you sell Dedicated Compute (DC-*) plans on NodeRings. Each customer VM gets exclusive physical CPU cores pinned to that guest, so no other VM shares those cores. NodeRings tracks every logical CPU on the node and places dedicated VMs on whole physical cores with NUMA awareness.

Use this guide to prepare a Proxmox host as a dedicated node, register its CPU layout, and list Dedicated Compute plans in your sell regions. Node basics (platform, region, storage, provisioning settings) are unchanged. See Platforms and Nodes.

Affinity and pinning. On create, NodeRings allocates whole physical cores, stores the host cpuset on the VM, and the operator posts Proxmox affinity= plus numa=1. Placement prefers one NUMA node, then one socket, then cross-socket. NodeRings does not guarantee every dedicated VM stays on a single NUMA node. See Proxmox: allow the agent to set CPU affinity.


Shared or dedicated: one type per node

Every node has a compute type: shared (default) or dedicated. A node is exactly one type, never both.

Shared nodeDedicated node
SellsShared Compute (SC-*) plans onlyDedicated Compute (DC-*) plans only
CPU overallocationConfigurableFixed at 100% (no oversubscription)
Memory overallocationConfigurableFixed at 100% (no oversubscription)
Storage overallocationConfigurableStill configurable
CPU topology pasteNot neededRequired before plans can pass preflight

Mixing both products on one host would force shared VMs to compete with pinned cores. That is the noisy-neighbor problem dedicated plans exist to prevent, so NodeRings does not allow it.

Compute type and CPU topology are set at create time and cannot be changed afterward (UI or API). To switch shared vs dedicated or supply a different topology, delete the node and create it again.


SMT required: threads are your sellable vCPUs

NodeRings counts every hardware thread as one sellable vCPU. Your dedicated capacity is physical cores × threads per core with SMT (AMD) / Hyper-Threading (Intel):

HostCapacity with SMT
AMD EPYC 9654 (96 physical cores)192 threads → 192 vCPUs
DC-*-8C plan8 vCPUs (4 physical cores × 2 threads)

Enable SMT in the BIOS before collecting topology. Dedicated node create rejects pastes where SMT is off. With SMT disabled you cannot register the node as dedicated.

Allocation is still whole physical cores. Both threads of a core always go to the same VM, so no other customer ever runs on a sibling thread (see How plans consume cores). The topology board shows vCPU cells (one per hardware thread), not physical-core cards.

Toggling SMT changes the host's logical CPU numbering. If you change it later you must drain and recreate the node with a fresh topology paste (see Re-paste after BIOS changes).


Proxmox: allow the agent to set CPU affinity

NodeRings pins dedicated VMs by posting affinity= (and numa=) to the Proxmox API:

POST /api2/json/nodes/{node}/qemu/{vmid}/config

For every dedicated VM the backend has allocated a cpuset for, the operator always posts both affinity=<cpuset> and numa=1 together. Shared VMs have no affinity and leave the template's NUMA setting alone.

Upstream Proxmox treats affinity as root-only. Admin API tokens also fail when calling this API.

Without a one-line host patch, the agent cannot pin dedicated VMs. Apply this on every Proxmox node that will sell Dedicated Compute.

Edit the permission whitelist

As root on the Proxmox node, open:

nano /usr/share/perl5/PVE/API2/Qemu.pm

Search for my $cpuoptions. You should see a hash like:

my $cpuoptions = {
    'cores' => 1,
    'cpu' => 1,
    'cpulimit' => 1,
    'cpuunits' => 1,
    'numa' => 1,
    'smp' => 1,
    'sockets' => 1,
    'vcpus' => 1,
};

Add 'affinity' => 1, inside that hash:

my $cpuoptions = {
    'affinity' => 1,
    'cores' => 1,
    'cpu' => 1,
    'cpulimit' => 1,
    'cpuunits' => 1,
    'numa' => 1,
    'smp' => 1,
    'sockets' => 1,
    'vcpus' => 1,
};

Save and exit

Restart the API services

systemctl restart pvedaemon pveproxy

If affinity posts still fail after the restart, reboot the node once.

Verify

NodeRings live validation for Dedicated Compute plans also checks this after the pilot VM is created: it polls pve_node_cpu_pinned until every affinity bit from the allocation appears in metrics (or fails if Proxmox never accepted the pin).

This edit is unsupported by Proxmox and will be overwritten when the qemu-server package is upgraded. Re-apply the 'affinity' => 1 line (and restart pvedaemon / pveproxy) after every Proxmox upgrade that refreshes /usr/share/perl5/PVE/API2/Qemu.pm. Bake the change into your node bootstrap image if you provision hosts often.


Provide the CPU topology

Core pinning needs the node's exact CPU layout: which logical CPU belongs to which physical core, socket, and NUMA node. This mapping is per host. Identical machines can number their CPUs differently depending on BIOS settings, firmware, and kernel, so NodeRings never assumes it and instead asks you to paste it once.

When you create a dedicated node, the dialog shows this command with a copy button (copy pastes it as a single line). Run it on the node itself as root and paste the single JSON line into the topology field:

jq -nc \
  --argjson topology "$(lscpu -J --extended=CPU,CORE,SOCKET,NODE,ONLINE)" \
  --argjson summary "$(lscpu -J)" \
  --argjson siblings "$(
    for f in /sys/devices/system/cpu/cpu*/topology/thread_siblings_list; do
      [[ $f =~ /cpu([0-9]+)/ ]] || continue
      jq -n --arg k "${BASH_REMATCH[1]}" --arg v "$(<"$f")" '{($k):$v}'
    done | jq -cs add
  )" \
  --argjson numa_mem_kb "$(
    for f in /sys/devices/system/node/node*/meminfo; do
      [[ $f =~ /node([0-9]+)/ ]] || continue
      jq -n --arg k "${BASH_REMATCH[1]}" --argjson v "$(awk '/MemTotal:/{print $4}' "$f")" '{($k):$v}'
    done | jq -cs add
  )" \
  --argjson numa_distance "$(
    for f in /sys/devices/system/node/node*/distance; do
      [[ $f =~ /node([0-9]+)/ ]] || continue
      jq -n --arg k "${BASH_REMATCH[1]}" --arg line "$(<"$f")" '{($k):($line|split(" ")|map(tonumber))}'
    done | jq -cs add
  )" \
  '{noderings_topology:1,topology:$topology,summary:$summary,siblings:$siblings,numa_mem_kb:$numa_mem_kb,numa_distance:$numa_distance}'

Requires bash, util-linux lscpu -J, and jq (apt install jq if missing). The JSON envelope includes:

FieldSourceWhat it provides
topologylscpu -J -ePer logical CPU: core, socket, NUMA node, and online status (the canonical map)
summarylscpu -JCPU model, sockets, and threads per core (used for validation and display)
siblingssysfs thread_siblings_listThe kernel's explicit SMT sibling sets, cross-checked against the topology
numa_mem_kbsysfs meminfo per NUMA nodeMemory physically local to each NUMA node
numa_distancesysfs distance per NUMA nodeNUMA distance matrix, used to pick the closest pair when a large VM must span

Re-paste after BIOS changes

BIOS changes that affect the CPU layout (enabling or disabling SMT/hyper-threading, changing NPS (NUMA-per-socket) modes, firmware updates) can renumber logical CPUs. Topology cannot be updated on an existing node: drain and delete the node, recreate it as dedicated with a fresh paste, then migrate VMs back. Otherwise pinned VMs and the core map will no longer match reality.


The node topology board

Dedicated nodes get a topology board on /infrastructure/platforms-nodes: click a dedicated node row to expand it and see a visual map of sellable vCPUs.

  • Machine header: node name, CPU model, socket count, total vCPUs, and allocated/free vCPU counts.
  • Socket panels: one panel per physical CPU package. Hosts running NPS2/NPS4 BIOS modes show multiple NUMA panels inside one socket panel.
  • NUMA node panels: inside each socket panel, with the NUMA-local RAM total and a usage bar splitting it into host reserve / allocated to NodeRings VMs / free, plus free/total vCPU counts. This makes the physical memory split visible: a 512 GB dual-socket machine is typically 256 GB local to each NUMA node. Placement prefers one NUMA node, then one socket, then cross-socket, and memory stays co-located with the cores on the NUMA nodes that contribute them.
  • vCPU cells: one cell per hardware thread, labeled with its affinity bit (the OS logical CPU ID). Non-contiguous numbering (common on EPYC, for example NUMA 0 = 0-31,64-95) displays as-is.

Each cell has a status, with an IPAM-style legend:

StatusMeaning
AvailableFree and ready to allocate to a new dedicated VM
AllocatedPinned to a NodeRings VM; the tooltip names the VM
Foreign VMPinned by a VM created on the hypervisor outside NodeRings; treated as unavailable

Per-NUMA preflight messages

For DC-* plans, preflight replaces the shared-plan Sell capacity math with a real dry-run allocation: it checks that some NUMA node has both enough free physical cores and enough free NUMA-local memory for the plan. When a plan doesn't fit, the failure tells you exactly which NUMA node and which dimension blocks it, for example:

not enough free dedicated CPU cores (requested 4 physical cores and 8 GB RAM): NUMA 0: needs 4 free cores, has 2/16 free; RAM OK (58 GB free) | NUMA 1: cores OK (9 free); needs 8 GB NUMA-local RAM, has 3 GB free

How to read and fix these:

Message saysMeaningTypical fix
needs N free cores, has M/T freeNot enough whole physical cores free on that NUMA nodeSell smaller plans, wait for churn, or add a node (cores free up when VMs are deleted)
needs X GB NUMA-local RAM, has YCores are free but that NUMA node's local memory is exhaustedMemory-heavy plans on that NUMA node are the cause; balance your plan mix or add a node
requested N physical cores and X GB RAMWhat the plan needs (physical cores + VM RAM), not host reserveCompare against the per-NUMA free counts below
Spanning not possibleThe plan doesn't fit one NUMA node and can't be split cleanly eitherThe plan is too large for this node's current occupancy
run the topology command and paste its outputThe node is dedicated but has no topology storedComplete the topology paste
N VM(s) you run on the hypervisor … not pinnedVMs you created on the hypervisor have no pinning and can use any host corePin or migrate them

A small reserve of each NUMA node's memory is held back for the hypervisor itself, so a node never sells 100% of its physical RAM to pinned VMs.


Existing VMs must be pinned too

A node you already ran VMs on can still become a dedicated node, but every guest on the hypervisor must have a CPU affinity config. A VM without affinity is scheduled by the kernel across all host cores, including cores sold as dedicated, which silently breaks the isolation your customers pay for.

Preflight fails the node when any unpinned guest exists:

This node has 3 VM(s) you run on the hypervisor that are not pinned to specific CPU cores. Unpinned VMs can use any core on the host, so dedicated-core plans cannot be sold here.

To fix it, either pin each guest VM to explicit cores on the hypervisor:

qm set <vmid> --affinity 0-3   # pick cores you keep off the sellable pool

or migrate the VM to another (shared) node. Cores reserved for your own VMs show as Foreign VM on the topology board and are never sold. Stopped VMs count too, because they can be started at any time.

The same guard runs when a customer VM is actually placed. An unpinned VM created between preflight runs cannot cause a double allocation, because the node is skipped until it is clean again.


Sell Dedicated Compute plans

Creating DC-* plans works exactly like shared plans on /infrastructure/plans: pick a Dedicated Compute template, set the region, assign one or more catalog-eligible nodes, then set price and bandwidth terms. Fixed and usage-based billing, bandwidth egress, and addons all behave identically. See Plans & Addons for details.

Templates cover General Purpose (DC-GP-*), CPU Optimized (DC-CPU-*), Memory Optimized (DC-MEM-*), Storage Optimized (DC-STO-*), High Frequency (DC-HF-*), and Very High Frequency (DC-VHF-*). Which host CPUs can sell which category (and why) is documented in CPU catalog tags & plan eligibility.

Placement rules to keep in mind:

  • Each plan must have nodes explicitly assigned (only catalog-eligible nodes for that category appear in the picker); preflight and placement never fall back to every node in the region.
  • DC-* plans only sell on dedicated nodes; SC-* plans only on shared nodes. A region needs at least one dedicated node before any dedicated plan can pass preflight there.
  • A plan's vCPU count is logical threads, but allocation is whole physical cores: on an SMT-2 host an 8 vCPU plan consumes 4 physical cores, and a node's sellable dedicated capacity is physical cores × threads per core. For example, an EPYC 9654 node with SMT enabled sells 192 vCPUs (see SMT required).

Next