pgFoundry Project: PostgreSQL Loader - News

pgloader 2.3.1 with fixed format support

dim@users.pgfoundry.org (Dimitri Fontaine) — Tue, 03 Jun 2008 19:03:31 GMT

This new version has seen some testing of max_parallel_sections setting, a user using 10 parallel sections to load a 1000 sections file. So this version saw some bugfixes too...

pgloader 2.3.0

dim@users.pgfoundry.org (Dimitri Fontaine) — Sat, 15 Mar 2008 16:30:30 GMT

pgloader 2.3.0 implements parallel loading of data at several level, and comes with some minor improvements and fixes, such as the long awaited support for columns = * configuration!

Parallel loading means pgloader is now multi-threaded and able to load several sections at the same time (see the max_parallel_sections global configuration option), and has support for spreading a single file loading to several active workers threads.

See the matching documentation section for details: http://pgloader.projects.postgresql.org/#_parallel_loading

pgloader 2.2.6

dim@users.pgfoundry.org (Dimitri Fontaine) — Tue, 05 Feb 2008 04:49:37 GMT

This new version adds -D (--disable-triggers) option to pgloader, which disablestriggers before loading the data, then re-enables them afterward.

-V (--vacuum) now issues a VACUUM ANALYZE for each table instead of a vacuumdb at the end of the load.

pgloader 2.2.5: templates & logging

dim@users.pgfoundry.org (Dimitri Fontaine) — Sun, 09 Dec 2007 18:08:19 GMT

This version of pgloader supports template configuration section and an improved logging mecanism.

You can now share whatever settings you need to from many configuration sections by using template section (template = True) from your sections (use_template = TemplateName).

pgloader now uses python's logging facilities (per user gripe), and supports log_min_messages and client_min_messages options, which you're certainly used to using.

Oh, and its debian packaging has been reviewed, which allows it to enter official repositories soon! Stay tuned for further news.

pgloader 2.2.4: reformating input data on-the-fly

dim@users.pgfoundry.org (Dimitri Fontaine) — Fri, 23 Nov 2007 17:27:13 GMT

New pgloader version allows user provided modules to reformat data input, and provides a mysql timestamp reformating function.

See for yourself how this first pluggable reformating function behaves:
>>> from reformat import mysql
>>> mysql.timestamp(None, '20041120175542')
'2004-11-20 17:11:42'

Now you can use mysqldump -T and pgloader to simply load data into PostgreSQL. Including timestamps!

pgloader 2.2.3 : User-defined Columns

dim@users.pgfoundry.org (Dimitri Fontaine) — Thu, 15 Nov 2007 10:03:05 GMT

pgloader now supports user-defined columns.

This allows for loading into the same database table of data coming from more than one file, keeping track of the data origin.

Simply add a column (say "origin") to the table definition and define the udc_origin option for pgloader to add a constant value per section (thus per input data file) in the database.

pgloader 2.2.2 now available

dim@users.pgfoundry.org (Dimitri Fontaine) — Sun, 11 Nov 2007 20:15:06 GMT

The new pgloader version supports a --quiet option, and the summary is now only printed when the --summary option is given. Some users have already reported this as a big improvement ;)

Please also note the debian package now properly contains the documentation, which is the main reason of this 2.2.2 release.

A user reported a bug promptly corrected, where a temporary file was created by pgloader with too long a name (containing tablename and columns instead of just tablename).

Please enjoy this new version!

Version 2.2.1 and new public website

dim@users.pgfoundry.org (Dimitri Fontaine) — Thu, 23 Aug 2007 23:30:27 GMT

pgloader 2.2.1 released, along with a news documentation format, allowing for a better-than-ever public site!
Do check out http://pgloader.projects.postgresql.org/

Release include supports for datestyle settings and automatic column numbering.

pgloader rewrite : version 2.0.2

dim@users.pgfoundry.org (Dimitri Fontaine) — Wed, 29 Nov 2006 20:31:17 GMT

While preparing an Informix to PostgreSQL migration, Dalibo ( http://www.dalibo.com/ ) had to load the data. They created pgLoader 2.

pgloader v.1.1

jpargudo@users.pgfoundry.org (Jean-Paul Argudo) — Mon, 03 Apr 2006 16:17:53 GMT

This new version of pgloader is a bugfix:
- allows use of separator character and
- allows use of quoting
..mainly for CVS files

Plus now allows importing data from files in other encodings than utf-8 to be imported in their original encoding.

I took time to test some different configurations, but I would be glad if some testers could help me enhance and debug this tool.

Please note it wont work yet with versions <8.0, mainly in CVS mode. Next step is to adapt the tool the the PostgreSQL version you use, wheter its 7.x or 8.x.

Thanks a lot for any comment, test, debug...

--
Jean-Paul