[Lazarus] Lazarus (UTF8) and Windows: SysToUTF8, UTF8ToSys... Is there a better solution?

[Marcos Douglas]

On Sat, Dec 21, 2013 at 5:56 AM, Juha Manninen
<juha.manninen62 at gmail.com> wrote:
> On Sat, Dec 21, 2013 at 3:08 AM, Marcos Douglas <md at delfire.net> wrote:
>> I didn't understand. If I have a TStringList instance, on Windows, I
>> need to convert Text property to ANSI. But some components, e.g.
>> TMemo, do these conversions automatically, but this is different.
>
> TMemo is a GUI component.

I know, of course... :)

> Then the string encoding matters and it must
> be converted to the native widgetset encoding. Still, the conversion
> is automatic. You don't need to care about it if you work with LCL
> components only and not with WinAPI directly.

Yes and so I wrote "TMemo, do these conversions automatically, but
this is different".

> TStringList is not a GUI component. It can be used for example in an
> embedded Linux program with no GUI.

Yes again. I use a lot TStrings as a transfer of information in many
cases... no GUI envolved.

> It does not need to know the encoding (except for sorting maybe). With
> FPC no automatic conversions happen.

And that is one of these problems because I need convert the Text
property to the right encode.

> In Delphi things are different. The auto-conversion happens ALWAYS
> when assigning between eg. UTF-8 and UTF-16. It has nothing to do with
> WinAPI, or any other widgetset API.
> Native string is UTF-16. If you have
>   var MyUTF8Str: UTF8String;
>   ...
>   StringList.Add(MyUTF8Str);   <- triggers conversion
>   MyUTF8Str := StringList[0];   <- triggers conversion again
>
> The amazing thing is that such code works. Delphi does a good job in
> converting the strings.

That's it!
I think you talking about of new versions of Delphi, right? So I
always read that "new Unicode implementation" in new versions of
Delphi is wrong, broke things, etc. but you is writing other vision.
These conversions, IMHO, could be automatic -- as Delphi does -- when
I use the correct type of string, in that case UT8String. So, I can
write my packages and opt to use only UTF8String or UTF16String in all
arguments and the compiler convert for me. What is wrong in that
approach?

> It is also reasonably fast, but still not acceptable in a speed
> critical code. This was the problem in my employer's code base. We are
> thinking how to use UTF-8 for the core program without triggering many
> auto-conversions. One choice is to dump Delphi and use only FPC. Now
> the code still works with both.

If you do not want automatic conversions, use the RawByteString type.
Delphi does not do conversions in that case, right?
Thank you, I'm learning.

Marcos Douglas

> P.S.
>   I am still wondering why you are so fond of WinAPI while you have a
> nice cross-platform API available.

Fond? Of course not! I use WinAPI when I need or when I don't know
another way to do the same using cross-plataform. I'm a "classic
Delphi programmer". I still use Delphi (stoped in 7 version) today but
all new projects I use Lazarus -- MSEgui a little.
For example, I use a lot PostMessage, SendMessage, PeekMessage... Are
these cross-plataform? If not, how can I do the same?