Leaf Error (HOST:PORT): Unable to connect to leaf @HOST:PORT with user USER, using password YES/NO: [1040] Too many connections.

Issue

As memory is maxed out, queries may hit the maximum_memory limit on the node and be killed automatically. And this error message is recorded in the logs.

Solution

As the CPU maxes out, queries will start taking longer to complete as the kernel queues them (load average will increase above 1 per cpu).Tuning max_connection_threads will allow you to find the correct balance between the count of concurrently running queries, and resource utilization. As you increase the max threads per aggregator, monitor the resource usage (CPU, load average, memory) during a typical workload.