SingleStore DB

Columnstore Table Considerations

Columnstore tables (in MemSQL 4.1 and above) have a hidden rowstore table that buffers small inserts into the columnstore table in memory. When enough rows have accumulated in memory, the rows are removed from memory and converted to columnstore format in batch and pushed to disk. The memory use for this hidden rowstore table will show up the same way as a regular rowstore table.

Columnstore tables also store metadata about the files stored on disk in memory. Each file has a row stored in memory with some information about this file (max and min value of the column in this file, bitmap of rows that are deleted, etc.). The memory use for columnstore metadata is currently not broken out into a separate component, but it is included as rowstore memory use along with all other rowstore tables because metadata tables are implemented as hidden rowstore tables. The memory use for columnstore metadata should be small unless the columns are storing large values (the max and min value will be large in this case).