[PgPy] [ python-Bugs-1000420 ] Implement message streaming

[noreply at pgfoundry.org]

Bugs item #1000420, was opened at 2005-11-16 13:56
You can respond by visiting: 
http://pgfoundry.org/tracker/?func=detail&atid=442&aid=1000420&group_id=1000094

Category: PQ/PQueue
Group: To do
Status: Open
Resolution: None
Priority: 3
Submitted By: James Pye (jwp)
Assigned to: James Pye (jwp)
Summary: Implement message streaming

Initial Comment:
Currently, PQueue's buffer implementations require all of the message data to be received before returning the message. The problem shows it's ugly face when Very Large Messages(tm) are received by the client. This is quite likely to occur if the database stores large bytea columns.

My current vision is to alter the passive buffer's semantics(or perhaps extend them in a subclass thereof) to take the reader upon instantiation so that it creates raw message interfaces that takes data directly from the reader, if the message size were to justify streaming.
This idea would require some alterations to the message class parsers so that references to past accessed data is not done, and string operations are only done after a Python string is resolved from the raw message interface(The current parsing methods, more or less, assume a Python string interface).

This solution will only improve the performance of the transformations from raw messages to structured messages.

Further improvements to extend the streaming queue may be provided by the structured message parsing methods.

----------------------------------------------------------------------

You can respond by visiting: 
http://pgfoundry.org/tracker/?func=detail&atid=442&aid=1000420&group_id=1000094