Sign inTry Free

Managing Plancache Memory and Disk Usage

On this page

Compiled query plan images, sometimes referred to as 'modules' or 'code modules', are stored in a plancache for later use. The plancache consists of both an in-memory plancache and an on-disk plancache, also known as the persistent plancache or PPC. Thus, the plancache incrementally consumes both memory and disk storage space. This topic addresses managing that memory and storage space. Refer to Code Generation for more information on the plancache.

Compiled query plan images may be removed from the in-memory plancache through expiration or eviction.

A plan expires from the in-memory plancache (called expiration) after the time interval specified by the plan_expiration_minutes engine variable has elapsed. When a plan expires from the in-memory plan cache, the plan is removed fully from the in-memory plancache so when that plan is used again it must be fully reloaded from disk.

Plans are evicted from the in-memory plan cache (called eviction) when the plancache memory usage goes over the thresholds specified by compiled_images_eviction_memory_limit_mb and compiled_images_eviction_memory_limit_percent, as described below. Once the plancache goes over the threshold, the engine evicts the least recently used query plan images until the memory use of the plancache falls below the threshold.

Whether a plan expires or is evicted from the in-memory plancache, that plan remains in the on-disk plancache and is loaded back into memory the next time the query is run. Evicted plans will reload more quickly than expired plans, but in neither case will the plan be recompiled. Further, when a node restarts, the in-memory plancache starts off empty and plans are loaded back in from the on-disk plancache as queries are run.

Query plans expire from the on-disk plancache as described in Set Plan Expiration Limits. When a plan expires from the on-disk plancache, it is completely removed from the plancache, and the query must be recompiled when it is run again.

Query plans can be manually removed from the in-memory plancache as described in Drop Plans from the Plancache. All plans can be removed from the in-memory and on-disk plancache as described in Drop All Plans from the Plancache.

Set Plan Expiration Limits

Plancache expiration policy can be set through the plan_expiration_minutes and disk_plan_expiration_minutes engine variables.

Expiration for the in-memory and on-disk plancaches is always enabled.

Plans will expire from the on-disk plancache if the following three conditions are met: 

  • The time limit specified by disk_plan_expiration_minutes is reached (assuming the plan has not been read from disk during that time), and

  • The plan is not in the in-memory plancache.

This process also applies to temporary (temp) tables. SingleStore recommends reusing the same name for temporary tables to the degree possible.

The following example shows how to set these variables to change the in-memory expiration limit to eight hours and set the on-disk expiration to one week.

SHOW VARIABLES LIKE '%plan_expiration%';
+------------------------------+-------+
| Variable_name                | Value |
+------------------------------+-------+
| disk_plan_expiration_minutes | 20160 |
| enable_disk_plan_expiration  | ON    |
| plan_expiration_minutes      | 720   |
+------------------------------+-------+
3 rows in set (0.08 sec)
SET GLOBAL plan_expiration_minutes = 480;
Query OK, 0 rows affected (0.07 sec)
SET GLOBAL disk_plan_expiration_minutes = 10080;
Query OK, 0 rows affected (0.09 sec)
SHOW VARIABLES LIKE '%plan_expiration%';
+------------------------------+-------+
| Variable_name                | Value |
+------------------------------+-------+
| disk_plan_expiration_minutes | 10080 |
| enable_disk_plan_expiration  | ON    |
| plan_expiration_minutes      | 480   |
+------------------------------+-------+
3 rows in set (0.07 sec)

Manage Eviction of Compiled Query Plans

Note

Plan eviction and the information in this section is applicable when interpreter_mode is set to interpret_first or compile.

Compiled query plan images (sometimes referred to as 'modules' or 'code modules') are evicted using LRU (least-recently used) when the in-memory plancache reaches a set memory limit. The compiled_images_eviction_memory_limit_percent and compiled_images_eviction_memory_limit_mb variables are used to specify the memory limit available for caching query plan images. Query plan images are evicted when the enable_compiled_images_eviction engine variable is set to ON, which is the default setting.

If the compiled_images_eviction_memory_limit_percent and compiled_images_eviction_memory_limit_mb variables are not set or are set to 0, compiled_images_eviction_memory_limit_percent will be set as a percentage of maximum_memory as shown in the following table.

maximum_memory (GB)

compiled_images_eviction_memory_limit_percent Value

In-memory Plancache Memory Limit

0-1

Prohibited (maximum_memory minimum is 1GB)

1-64

10

10% of maximum_memory

64+

5

5% of maximum_memory

If the compiled_images_eviction_memory_limit_mb or compiled_images_eviction_memory_limit_percent variables are set, the memory limit for the in-memory plancache is derived at runtime as shown in the following table.

The notation _mb denotes compiled_images_eviction_memory_limit_mb and _percent denotes compiled_images_eviction_memory_limit_percent.

_percent

_mb

In-memory Plancache Memory Limit

0.0

0

The value is derived in accordance with the default allocation rule specified in the table above.

0.0

> 0

If _mb exceeds the maximum_memory , a value derived from the above table is used. Otherwise, the value of _mb is used as is.

> 0.0

Any value

The value of _percent is used.

If the compiled_images_eviction_memory_limit_percent variable is set to a value other than 0.0, the value of compiled_images_eviction_memory_limit_mb is ignored.

Note

Changes to maximum_memory may result in changes in the module cache limit, in accordance with the rules specified above.

Drop Plans from the Plancache

In addition to setting a retention policy for in-memory and on-disk plans, you may also choose to drop plans from the plancache to either compile a new query plan or reduce memory usage. For running nodes, the following three commands can drop a plan from the plancache: DROP … FROM PLANCACHE , ALTER TABLE, and ANALYZE.

Successful completion of a new ALTER TABLE will invalidate the plancache for all queries that operate on the affected table, requiring their recompilation. The invalidation of the plancaches happens asynchronously. That is, new queries will compile and use new query plans, while currently running queries will use the old query plans. Once no queries are using the old query plans, they are marked as ready to be purged by the garbage collector.

ANALYZE invalidates stale plans from both the in-memory and the on-disk plancache, including plans which are on-disk but not in-memory. This invalidation ensures that all stale plans are discarded and that they are recompiled the next time the query is run.

Note

The ANALYZE command invalidates plans by marking them as invalid so they won’t be used in the future, but it does not actually delete all the files in the on-disk plancache. The files will still be on disk, so it is not possible to reduce disk usage of the on-disk plancache with this command.

Drop All Plans from the Plancache

To remove all plans from the in-memory and on-disk plancaches, SingleStore recommends the practice of temporarily setting plan_expiration_minutes and disk_plan_expiration_minutes to 0. Setting these engine variables to 0 will cause the plans in the plancaches to be marked as invalid and subsequently deleted by the garbage collector.

To forcibly drop all plans in the in-memory and on-disk plancaches:

Run SHOW VARIABLES and save the current values for the plan expiration minutes engine variables.

SHOW VARIABLES LIKE '%plan_expiration_minutes'; /* be sure to save these values */

Set the plan expiration minutes engine variables to 0 and invoke SLEEP to give the garbage collector time to delete the plans. A larger value for SLEEP may be needed depending on the size of the plancache and the speed of your disks.

SET GLOBAL plan_expiration_minutes=0;
SET GLOBAL disk_plan_expiration_minutes=0;
SELECT SLEEP(300);

Reset the plan expiration minutes engine variables to their previous values.

SET GLOBAL plan_expiration_minutes=<oldvalue>;


SET GLOBAL disk_plan_expiration_minutes=<oldvalue>;

Related Topics

Last modified: April 7, 2025

Was this article helpful?

On this page

Was this article helpful?

Verification instructions

Note: You must install cosign to verify the authenticity of the SingleStore file.

Use the following steps to verify the authenticity of singlestoredb-server, singlestoredb-toolbox, singlestoredb-studio, and singlestore-client SingleStore files that have been downloaded.

You may perform the following steps on any computer that can run cosign, such as the main deployment host of the cluster.

  1. (Optional) Run the following command to view the associated signature files.

    curl undefined

  2. Download the signature file from the SingleStore release server.

    • Option 1: Click the Download Signature button next to the SingleStore file.

    • Option 2: Copy and paste the following URL into the address bar of your browser and save the signature file.

    • Option 3: Run the following command to download the signature file.

      curl -O undefined

  3. After the signature file has been downloaded, run the following command to verify the authenticity of the SingleStore file.

    echo -n undefined |
        cosign verify-blob --certificate-oidc-issuer https://oidc.eks.us-east-1.amazonaws.com/id/CCDCDBA1379A5596AB5B2E46DCA385BC \
          --certificate-identity https://kubernetes.io/namespaces/freya-production/serviceaccounts/job-worker \
          --bundle undefined \
          --new-bundle-format -
    Verified OK