[Lazarus] Adding codepage-support to the RTL (making LConvEncoding obsolete)

[Guy Fink]

Hello

I have opened issue #0018144 in the bugtracker and uploaded a new version of my codepages unit.

My description on this :

In September we had a discussion on the Lazarus-mailing list to rewrite LConvEncoding and move the functionality to the RTL (Thread: rewriting of LConvEncoding).

Since there I did a lot of coding to implement an effective algorithm, both for Singlebyte- as for Doublebyte-Codepages. A first release was on the mailing list mid-October, mainly as a base for further discussions. But there were no comments or suggestions on this.

So here is a nearly final release with many changes to the first version.

Major points:
 - The unit supports Single- and Double-bytecodepages trough the same functions
 - Widestringsupport (configurable)
 - UTF8 and UTF16 support (UTF16 needs widestrings)
 - Direct conversion from CP to CP without intermediate string
 - Uppercase and Lowercase support
 - Underlying Unicodes as of V 6.0.0 (October 11, 2010)
 - A converter-application to convert Unicodedefinitions to a complete
   pascal unit. The cp_* units are entirely generated by this app.
 - Conversion up to 80% faster for SBCS. For DBCS up to 100 times



As for now there are only units for ISO-8859-1, ISO-8859-2 and CP932 (SHIFT_JIS). More to be added for the final release. The converter-subdir has all the definition files that I could find. I will add them all.

The units:
  codepages.pp : the main unit (highly configurable trough codepagesdef.inc)
  unicodemappings.pas : Some definitions from unicode.org, especially the tables
                        for uppercase, lowercase and the unicodeblocks.
  utf8.pas : mainly the UTF8 functions from LCLProc + some new
  utf16.pas: same for UTF16
  acpinfo.pas: info for codepages supported by Windows, as published on MSDN

Some first test results  as attachment.


Best regards



______________________________________________________
powered by GLOBER.LU
Luxembourg Internet Service Provider
Hosting. Domain Registration, Webshops, Webdesign, FreeMail ...

Our professional Web Hosting plans include all the features you are looking for at the best possible price.
www.globe.lu
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: TestResults1.txt
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20101203/8a41c476/attachment-0002.txt>