[Pgcluster-general] pgcluster, TOAST and large objects

[Koen Martens]

Hi,

On Wed, Jul 11, 2007 at 05:59:39PM +0200, Benoit Mathieu wrote:
> I want to use postgresql to store large binary files (from 10Mo to 
> 200Mo). PostgreSQL offers 2 way to handle that : large objects, or a 
> bytea column with TOAST.
> 
> I plan to use pgcluster in the future, and I'd like to know if there's 
> any limitations in pgcluster with large objects or TOAST ? Has someone 
> already used pgcluster in that way ?

I tried pgcluster with large objects, and that did not work at all. I did some digging around in the source code, and it seems it ought to work, or at least it is partially implemented. Unfortunately, repeated posts on this list never resulted in a reply so i gave up on it alltogether.

I never tried bytea columns, because i was warry of the performance issues that might give. The project i was working on when this issue came up, was using the JDBC driver that supports the special fastpath calls that allow high-throughput/stream access to large objects. Now i recall that it is possible to do something like that with bytea's, but not with the jdbc driver.

Be aware that bytea comes with its own troubles, one of them being a high memory usage (at least 2 or 3 times the size of the bytea array iirc).

Note that replication of large binary objects is a problem not only for pgcluster. Other clustering products like sequoia have trouble with it too. In fact, large objects are problematic in other DBMS's too.

Gr,

Koen

-- 
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, hosting, embedded systems, unix, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://pgfoundry.org/pipermail/pgcluster-general/attachments/20070711/d9f25319/attachment.bin