SCM

[#1011284] Socket Error when connecting via ThreadPool

View Trackers | Bugs | Download .csv | Monitor

Date:
2012-11-26 09:17
Priority:
3
State:
Open
Submitted by:
Matei Craciun (mematei)
Assigned to:
Nobody (None)
Npgsql Version:
2.0.12beta
Category:
None
Group:
None
Resolution:
None
Summary:
Socket Error when connecting via ThreadPool

Detailed description
I am trying to implement a waiting mechanism to go with Connection Pooling. If all connections are in use, i make all other threads wait, until one becomes available.

In implementing this i stumbled across a problem.
I am attaching a (commented) class file that reproduces the problem.

In short, using ThreadPool i enqueue a thread that opens a connection to the database, and after that create other threads(via the same ThreadPool) that just wait, practically doing nothing.

The more threads i create, the longer it takes for the connection to establish(via the Flush method in NpgsqlStartupPacket.WriteToStream_Ver_3), and if i use let's say 70 connections, it throws a "An established connection was aborted by the software in your host machine" (code: 10053), after a minute or so(regardless of the timeout set to Send).

Any help is much appreciated.

Followup

Message
Date: 2012-11-27 13:28
Sender: Matei Craciun

I managed to fix it in the end. I set the ThreadPool.SetMinThreads(70,70), it is initially 4,4( the number of cores) so even if we set it to 50 it's still OK, and it only took about a few seconds to connect. Also added NpgsqlCommand and NpgsqlConnection in each thread, same outcome, works beautifully.
Problem solved.

Thank you for the help.
Date: 2012-11-27 03:32
Sender: Josh Cooley

I see the same thing. It appears the EndConnect call completes without an exception, but the connection isn't established. This appears to be a bug in the framework to me since my reading of the documentation says that it should throw if it doesn't connect.
Date: 2012-11-26 15:02
Sender: Matei Craciun

Thank you for the answer.

We have a server side application that can receive up to 70 requests at a time, and each request interrogates the database for as long as even 2 minutes. So the situation i was simulating does actually happen.

I have investigated the issue more thoroughly and i see that by specifying the Timeout parameter in the ConnectionString the calling threads are blocked for max of Timeout, waiting for a connection to be freed. I thought this behavior was not implemented(we never specified a large Timeout value, so it actually timed-out and threw a Unable to get Connection from Pool, i assumed this was the normal behavior), that is why i wanted to implement our own blocking mechanism. I now removed our thread blocking code.

I just want to outline the fact that inside the Open method from NpgsqlClosedState, the call to BeginConnect does finish, (specifically the call to result.AsyncWaitHandle.WaitOne and EndConnect() on the socket). And that the hang occurs when flushing the NpgsqlStartupPacket using output_stream.Flush().

I just want to confirm that even though the BeginConnect and EndConnect calls do finish, does the Socket Exception appear because internally it still makes use of the threadpool?


Date: 2012-11-26 13:28
Sender: Josh Cooley

This sample will consume all the threads in the pool and starve anything else that needs the pool. The connection timeout is implemented with an asynchronous BeginConnect call so that we can timeout waiting for the connection to succeed. BeginConnect also depends on free threads in the thread pool to complete the connection asynchronously.
You could get out of this bind if you consolidated your waits with ThreadPool.RegisterWaitForSingleObject. Perhaps there's an enhancement you are looking for. Why are you making other threads wait when all connections are in use? Do you want something better than the waiting that the pool does?

Attached Files:

Attachments:
ExceptionTest.cs

Changes:

Field Old Value Date By
File Added783: ExceptionTest.cs2012-11-26 09:17mematei
Powered By FusionForge