[#1010900] Current Connection pooling scheduling results in an out of memory in high loaded environment

View Trackers | Bugs | Download .csv | Monitor

Date:
2010-09-22 08:54 Priority:
3

State:
Open

Submitted by:
Adriaan (adriaanvk) Assigned to:
Nobody (None)

Npgsql Version:
2.0.10 Category:

Group: Resolution:
None

Summary:
Current Connection pooling scheduling results in an out of memory in high loaded environment

Detailed description
hi, I have an postgresql database to which I connect using Npgsql. I saw that there is connection pooling included in the application. ConnectionLifeTime cleans up idle connections to free up resources. I am firing a heavy bulk of queries to the database using one thread. Those queries are most in sense of 'create or replace function'. I run this for some hours, which slows down more and more, resulting in an out of memory after an hour. After analyzing what happens, I saw one postgres process on the database server claiming more and more memory. After running a Npgsql.NpgsqlConnection.ClearAllPools(); frequently during my queries, the memory claim is released and the right performance is back! My conclusion on this is that the ConnectionLifeTime is fine for a regular, not heavy loaded database, but if you are using one connection heavily without giving the timeout time to release the connection, this results into memory problems on the database server! All heavy loaded application can run into this! A possible solution is to add a timeframe in the release of connections as well. For example, if the idle time has been expired OR the connection is open for 15 minutes, the connection will be recycled. If possible a check on the amount of memory used in the connection would be more great as when to recycle the connection. btw, my caching parameters in postgresql are set to a low value, so that might not be the problem. Hopefully this can be solved soon, so Npgsql also supports heavily loaded applications. regards, Adriaan

Followup

Message
Date: 2010-10-20 02:20 Sender: Josh Cooley Is the out of memory in the client process or in the db server? I can profile a test client if that's the case. But it sounds like problem you are reporting is in the server that can be worked around by clearing the pool regularly.

Attached Files:

Changes:

No Changes Have Been Made to This Item