[Lazarus] Testing Unicode and Windows system codepage in Lazarus 1.6

[Juha Manninen]

On Sun, Jan 17, 2016 at 2:54 PM, Jürgen Hestermann
<juergen.hestermann at gmx.de> wrote:
> You may have a false impression of how Free Pascal/Lazarus is used by most
> people.
> At least I did not wait until the current version to use Unicode (and long
> paths)
> for file names and others. I think many did so.
> Therefore we have build our own infrastructure/functions to work with the
> Windows unicode API functions. In this case using "DisableUTF8RTL" would
> be no option as it bombs us back to ANSI/short paths and still requires to
> do code changes.
> That's not good.

I don't understand how DisableUTF8RTL bombs your system because it is
the backwards compatible thing.
If your system worked with FPC 2.6.4, it should work with FPC 3.0 +
DisableUTF8RTL.
If it does not work then there are bugs which should be fixed.
Do you mean the A- / W- versions of WinAPI calls by the short/long path thing?
Enabling or disabling the UTF8RTL does not affect WinAPI calls. It is
a separate issue.


> So we need to live with the changes and understand the logic behind it
> (which IMO is much more complicated than before). The amount of time we have to invest
> is unforseeable for us and therefore many get frustrated (especially,
> because the documentation is poor).
>
> I have done the changes to (some) of my programs now and it works quite okay
> but it was a hard time until this was finished (and I still don't know
> whether bugs lurk here and there).
>
>
>> I don't know why you always find excuses to complain. It is not nice.
>> Please stop it.
>
> Please stop ignoring and gibing those who have problems with the new unicode
> types.
> I know how they feel and such statements are not helpful.
> If you don't want to help then ignore these mails.

Excuse me! Are you really saying I have not helped with the Unicode issues?
I am amazed the system works already as well as it does.

> I know that those who have already worked on this for a long time
> cannot understand why not everybody instantly knows how the new
> codepage aware ansistrings work but be assured that for many this is
> completely new. So bare with our frustration and better ignore such
> mails than revile the writers.

In my mail I explicitly asked for feedback so we can still improve this system.
Yes, this is a very complicated issue. For example I am at my limits
of understanding / not-understanding the details, but still I tried to
improve things instead of complaining about how poorly the voluntary
developers have done the job.

I have worked on the DisableUTF8RTL feature although I don't use it
myself. Fortunately there were helpful people like "michl" (forum
name) who fixed bugs.
I also wrote some wiki pages. They are not perfect but they are better
than nothing.

Now we support 3 ways with Lazarus 1.6 :
1. The improved UTF-8 system with FPC 3.0. The default codepage of
AnsiString = UTF-8.
2. Backwards compatible system with FPC 3.0 by defining
DisableUTF8RTL. AnsiString uses system codepage.
3. Continue using FPC 2.6.4. All string types and encodings are 100%
compatible with earlier versions.

The 4. way will be a Delphi compatible UTF-16 sometime in future.

How to improve the 3 supported ways? The issues I know are that FPC
libs still use the old WinAPI calls in some places, and the
TFormatSettings separators issue. Are there other bugs? How to solve
them?

Juha