maximum_memory
SingleStoreDB will not allocate more than maximum_memory
megabytes of memory. If a SELECT
query’s memory allocations put the memory usage by SingleStoreDB over maximum_memory
, query execution stops and an error message is generated.
By default, maximum_memory
is set to 90% of the physical memory on the host, or physical memory minus 10GB, whichever is larger.
As Kubernetes does not support swap, maximum_memory
for the Operator is set to 80% of physical memory, or physical memory minus 10 GB, whichever is larger
Query compilations whose allocations exceed this limit will also terminate the server. See Code Generation for more information on query compilations.
SingleStoreDB supports disk spilling. Instead of query failure, disk spilling allows query execution to continue by moving certain query operators onto disk and out of memory.
When a node's memory load is above 75% and in-memory data structures (for the current thread) are at least 100MB, the intermediate result of the operation spills to disk in the data/spill directory. The node memory load ratio and threshold are configurable variables: spilling_node_memory_threshold_ratio
and spilling_query_operator_memory_threshold
, respectively. Disk spilling will usually reduce memory usage (can spill at most 64x physical memory), but may result in longer execution time. It is recommended to use an SSD for the spilling directory since it can result in up to a tenfold increase in performance.
Currently, the following query operator(s) can be spilled onto disk:
GROUP BY
SORT
orORDER BY
Note
Disk spilling must be enabled by setting the engine variable, enable_spilling
to ON
. Starting with version 7.8, ON is the default value for enable_spilling
. See the Sync Variables Lists page for more information.