numa_zonelist_order

This sysctl is only for NUMA. 'Where the memory is allocated from' is controlled by zonelists.

In non-NUMA case, a zonelist for GFP_KERNEL is ordered as following: ZONE_NORMAL -> ZONE_DMA. This means that a memory allocation request for GFP_KERNEL will get memory from ZONE_DMA only when ZONE_NORMAL is not available.

In NUMA case, you can think of following 2 types of order. Assume 2 node NUMA and below is zonelist of Node(0)'s GFP_KERNEL:

(A) Node(0) ZONE_NORMAL -> Node(0) ZONE_DMA -> Node(1) ZONE_NORMAL
(B) Node(0) ZONE_NORMAL -> Node(1) ZONE_NORMAL -> Node(0) ZONE_DMA.

Type(A) offers the best locality for processes on Node(0), but ZONE_DMA will be used before ZONE_NORMAL exhaustion. This increases possibility of out-of-memory (OOM) of ZONE_DMA because ZONE_DMA is tend to be small.

Type(B) cannot offer the best locality but is more robust against OOM of the DMA zone.

Type(A) is called as "Node" order. Type (B) is "Zone" order.

"Node order" orders the zonelists by node, then by zone within each node. Specify "[Nn]ode" for node order.

"Zone Order" orders the zonelists by zone type, then by node within each zone. Specify "[Zz]one" for zone order.

Specify "[Dd]efault" to request automatic configuration. Autoconfiguration will select "node" order in following case:

(1) if the DMA zone does not exist or
(2) if the DMA zone comprises greater than 50% of the available memory or
(3) if any node's DMA zone comprises greater than 60% of its local memory and the amount of local memory is big enough.

Otherwise, "zone" order will be selected. Default order is recommended unless this is causing problems for your system/application.