[Lazarus] Unicode and DBAware

[Ivan Gan]

Hi
Be warned that this will fail on some databases
Example, ZEOS components, Lazarus 0.9.25 Unicode UTF-8, database 
encoding UTF-8
Tables contain ansi & extended including Chinese & Hebrew
If you fail to set the driver to UTF-8, the transliteration is done 
before the table object receives it
The result is ????? instead of Chinese & Hebrew

This email is utf8 encoded & the following lines are supplied as a test 
reference
The Polish line may look like ansi but the Z has a punctuation mark 
above it which will be lost if you do conversion

Hebrew : ??? ?? ????? ?????????? NLSO <http://dual-core/nlso/index.php>
Chinese : ???????????????? <http://dual-core/nlso/index.php>
English : This site uses NLSO technology <http://dual-core/nlso/index.php>
Polish : Ta strona uz.ywa technologi NLSO <http://dual-core/nlso/index.php>

These will only show correctly if you have international support on your 
system,
The database will only work correctly if there is NO conversion or 
compatible conversion eg: utf8 to ucs2 or ucs2 to ucs4
The objective for smooth operation should be to have the table fields 
utf8 encoded, all communication between your application & the database 
also utf8 encoded
To achieve this with firebird, I have no clue
To achieve it with Zeos using MySql is easy when you know how
Drop a ZConnection on your form
set the ZConnection properties to

character_set_client=utf8
character_set_connection=utf8
character_set_database=utf8
character_set_results=utf8
character_set_server=utf8
character_set_system=utf8
collation_connection=utf8_general_ci
collation_database=utf8_general_ci
collation_server=utf8_general_ci
Codepage=utf8

This will force the Mysql driver to speak utf8 in all communication
Failure to do this with MySql driver will make all attempts to 
transliterate data absolutely useless on the above language samples as 
the data is mangled by the driver anyhow
I suggest you look first at the table encoding, then the communication 
with your server
If these are right then you should have no need for further 
transliteration, use pure utf8

Have fun

JoshyFun wrote:
> Hello Joost,
>
> Wednesday, July 23, 2008, 10:46:44 PM, you wrote:
>
> JvdS> You could override the TDataset.DataConvert procedure or the
> JvdS> TDataset.Translate procedure and set the Transliterate property to true
> JvdS> of the TStringFields of which you want to convert the data.
>
> JvdS> If you e.g. use a TSQLQuery, you can add your own
> JvdS> TTransSQLQuery=class(TSQLQuery) and override one of the above
> JvdS> procedures.
>
> Hmmm... it sounds interesting and "quite" simple to transform in full
> unicode when the DB be upgraded to unicode too.
>
> This means that I must create my own visual object derived from i.e.
> TSQLQuery and add it to the lazarus components, but at least looks
> like a good option if I'm not able to find a "automagical" way to do
> it without special components.
>
> A lot of thanks.
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20080724/3f061089/attachment-0007.html>