PostgreSQL Connection Limit Error

Problem Description

Gitlab component logs (for example sidekiq, webservice, praefect) show database errors such as: pq: sorry, too many clients already

Root Cause

Gitlab opened more concurrent PostgreSQL connections than the database allows (exceeded PostgreSQL max_connections).

Troubleshooting

Check the logs of the Gitlab Deployments. Confirm if the pq: sorry, too many clients already error message is present.

kubectl -n <NAMESPACE> logs <RELEASE>-praefect-xxxxx
kubectl -n <NAMESPACE> logs <RELEASE>-sidekiq-all-in-1-v2-xxxxx
kubectl -n <NAMESPACE> logs <RELEASE>-webservice-default-xxxxx

Solution

If PostgreSQL's max_connections is too low for your Gitlab scale, increase it on your PostgreSQL side. The exact steps depend on how you manage PG and are out of scope here.

You can use the following command to check PostgreSQL capacity and current usage:

-- Show the global upper limit on concurrent connections allowed by this PostgreSQL instance.
-- Note: the effective limit available to non-superusers ≈ max_connections - superuser_reserved_connections.
SHOW max_connections;

-- Show how many connections are reserved for superusers,
-- so administrators can still log in when all regular slots are taken.
SHOW superuser_reserved_connections;

-- Count the total number of currently open backend connections across all databases and states
-- (e.g., active, idle, idle in transaction, autovacuum, replication).
SELECT count(*) AS current_connections FROM pg_stat_activity;