[Lazarus] ETL tool - make or use?

[Michael Van Canneyt]

On Fri, 23 Mar 2012, Marc Santhoff wrote:

> Der list members,
>
> does anyone of you know a tool for ETL[1] tasks?
>
> At best it would be written in object pascal and be open source.
>
> I'm currently charged with doing some simple data transfer actions like
> loading CSV (aka SDF) data and pushing those records to a relational
> database and the like. I have the strong feeling that someone has
> already done something like that. ;)
>
> Options to solve this request are
>
> 1. whipping together minimal code that does it
>
> 2. make a more generic tool that can be used for all tasks, having to be
> extended when needed
>
> 3. use some tool already existing
>
> Some tools I tried or looked at are mostly written in Java and do suck
> ressources, or they are not able to handle some requirements (like
> databases in use) or are closed source. Last tests were done using
> OpenAdaptor[2] which is really cool but has problems with posgresql and
> demands being familiar with spring. There is still Pentaho Data
> Integration[3] n the list of tools to do some testing with.
>
> What do you use for ETL tasks?

Always option 1.

Because it always turns out you need to check for doubles and do updates;
making self written code the most optimal tool. You always know your own 
data best of all.

Michael.