NUMA Makes Memory Location Part of Access Cost
A large multiprocessor server can expose one physical address space while giving different processors different paths to that memory. A load from a page attached to the processor running a thread can take a shorter route than a load from memory attached to another processor package or NUMA node.
This arrangement is called non-uniform memory access, or NUMA. It lets systems scale memory capacity and bandwidth across multiple processor sockets or chiplet groups without forcing every memory request through one centralized controller.
The practical result is simple: physical placement becomes part of memory performance. Two pointers can refer to equally valid RAM yet carry different access costs for the same CPU.
Local and remote memory use different paths
A NUMA node commonly groups processor cores with one or more memory controllers and a portion of system RAM. Access to memory associated with that node is local. Access to memory associated with another node is remote and travels through a processor interconnect.
The remote path can add latency and consume interconnect bandwidth. Exact penalties vary by platform, topology, memory generation, processor load, and traffic from other nodes.
Remote memory is not a slower class of RAM in isolation. The distinction comes from the path between the requesting core and the memory controller serving the target address. A page that is remote to one node can be local to another.
This directional property separates NUMA from a simple hierarchy such as L1, L2, L3, and DRAM. Cache level describes where a copy currently resides. NUMA locality describes which memory controller owns the physical page and how the requester reaches it.
One address space does not imply one access cost
Operating systems normally present ordinary processes with a virtual address space. Virtual pages map to physical pages, and those physical pages can reside on different NUMA nodes.
Application code can therefore allocate a large buffer without encoding a node number in each pointer. The operating system and runtime policies decide where backing pages are placed unless the application requests a specific policy.
That abstraction preserves normal pointer semantics. It does not erase hardware topology.
A thread can migrate from one CPU to another after its pages have already been placed. The virtual addresses remain valid, but accesses that were local can become remote. Likewise, moving data without moving the threads that use it can reverse the locality pattern.
NUMA performance is therefore a relationship between computation placement and data placement, not a property of either one alone.
First-touch placement ties initialization to page location
Many operating-system NUMA policies use a first-touch strategy for anonymous memory. Reserving a virtual range does not necessarily assign every physical page immediately. A physical page is commonly committed when code first writes to or otherwise faults in that page, and placement can favor the node where the faulting thread runs.
This makes parallel initialization important.
Suppose one thread allocates a large array and initializes every page while running on node 0. Worker threads later split the array across cores on nodes 0 and 1. Workers on node 1 may perform much of their traffic remotely because the initialization concentrated pages on node 0.
If workers initialize the portions they will later process while pinned to their intended nodes, first-touch placement can distribute pages closer to the consumers.
The exact policy is operating-system dependent, so first touch is a common behavior rather than a universal rule for every allocation type.
CPU affinity can stabilize locality
Schedulers move runnable threads among CPUs to balance load and satisfy scheduling policy. Migration is often beneficial, but frequent movement can work against deliberate NUMA placement.
CPU affinity constrains where a thread or process may run. Combined with a matching memory policy, affinity can keep a workload near the pages it accesses most heavily.
Pinning is not automatically faster. Tight affinity can leave cores idle, increase contention, or prevent the scheduler from reacting well to changing load. It is most useful when a workload has stable ownership of large memory regions or when measurements show substantial remote traffic.
The goal is not to eliminate migration at any cost. The goal is to keep the dominant compute-to-data relationship efficient.
Interleaving trades locality for balanced capacity and bandwidth
A NUMA memory policy can spread pages across nodes rather than placing them mainly near one thread. Interleaving can be useful for workloads whose threads access a shared region uniformly across several nodes.
This policy sacrifices some local accesses because each node will encounter pages owned elsewhere. In return, it can distribute memory-controller traffic and reduce the risk that one node becomes a capacity or bandwidth hotspot.
The best policy depends on the access pattern. A partitioned workload with strong per-thread data ownership often benefits from local placement. A broadly shared streaming workload can benefit from distributing pages across memory controllers.
A single rule such as “always keep memory local” misses that bandwidth balance can matter as much as latency.
Caches reduce traffic but do not erase NUMA effects
Processor caches can satisfy repeated accesses without returning to DRAM, so a cache hit can hide the location of the backing physical page for that access.
NUMA still matters when cache lines must be fetched, evicted, or exchanged among processors. Large working sets, streaming scans, database buffers, scientific arrays, and in-memory analytics can generate enough memory traffic for topology to become visible in throughput and tail latency.
Shared writable data adds another cost. A cache line modified by cores on different nodes can move ownership across the interconnect even when the underlying page stays fixed. This is a coherence effect layered on top of NUMA placement.
Placing a page near a thread cannot fix a design that continuously transfers ownership of the same cache lines among distant cores.
Capacity pressure can force remote allocation
Local placement is constrained by available memory. If one node lacks sufficient free pages, the operating system may place new pages elsewhere, depending on policy and platform configuration.
A process can therefore begin with favorable locality and later accumulate remote pages as memory pressure changes. Long-running services are especially exposed because thread placement, allocation patterns, and free capacity evolve over time.
Strict node-binding policies can reject allocations rather than fall back remotely. That behavior can be appropriate for specialized workloads but creates a different failure mode: available RAM may exist elsewhere in the machine while the requested node cannot satisfy the allocation.
NUMA tuning must account for capacity as well as speed.
Measurement should separate locality from total memory pressure
A slow memory-heavy workload is not automatically a NUMA problem. Cache misses, insufficient memory bandwidth, page faults, swapping, false sharing, lock contention, and poor data layout can produce similar symptoms.
Useful diagnosis combines topology information with counters or operating-system statistics that distinguish local and remote allocation or traffic. Comparing runs with controlled CPU affinity and memory placement can show whether topology materially changes the result.
Benchmarks also need enough duration and working-set size to represent the real application. A tiny test that fits in cache may show almost no NUMA penalty even when the production workload streams through hundreds of gigabytes.
NUMA adds another coordinate to memory performance: where the code runs relative to where its physical pages reside. Good placement keeps heavily used data close when latency dominates, distributes traffic when bandwidth dominates, and avoids treating a unified address space as a uniform physical machine.