[Lazarus] Lazarus (UTF8) and Windows: SysToUTF8, UTF8ToSys... Is there a better solution?

[Marco van de Voort]

On Tue, Dec 24, 2013 at 06:18:41AM +0100, Hans-Peter Diettrich wrote:
> > 
> > Not necessarily. Supporting both on both platforms is a sane reason too.
> > 
> > One can't ditch utf16 because of Delphi compatibility. It will be hard to
> > ditch utf8 because of old Lazarus compatibility.
> 
> In the meantime we have 2 Delphi compiler/RTL versions:
> - Ansi (Win32)
> - Unicode (UTF-16, multi target)
> and 4 GUI versions
> - VCL Win32
> - CLX
> - VCL.NET
> - FireMonkey
> summing up to 8 versions in theory, and 3 versions in practice.

The older delphi compilers are unsupported. We never supported anything but
VCL 32/64, so this list seems artificially inflated to me.

> So what does "Delphi compatible" mean *really*?

The same as it always has. VCL, and language level at a distance. The rest
is irrelevant.
 
> The FPC compiler supports multiple targets, and most probably can be 
> managed to support both string types using the *same code base* 
> (maintenance issue!). 

Yes.

> IMO this  does *not* apply to the libraries (RTL 
> and LCL) 

RTL is less of a problem than one might think. The problem mostly only comes
in at the classes level.

>and existing applications, where Lazarus counts as the most 
> important and prominent application. 

Existing Lazarus applications are toast anyway, without changes.

> We can be happy to have one single LCL and IDE version, which is already
> incompatible due to the use of UTF-8 strings instead of Ansi.  Multiple
> versions, for compatibility with the other Delphi combinations, are beyond
> *development capacities*.

Then drop the old stuff, and simply go for full compatibility. Anything else
will only cause the loss of all OSS Delphi projects (and even the commercial
ones that support Lazarus).

And people like me that are torn between both systems.

> This sheds a very different light on Delphi compatibility, meaning that 
> a Unicode LCL and IDE can *not* be supported in parallel to the existing 
> UTF-8 implementation.

There is no existing UTF8 implementation that can be continued as is anyway.

> Dumping UTF-8 would discontinue support for the entire* range of
> **existing* LCL applications, i.e.  loose all the
> current Lazarus users :-(
> 
> So what should be the intended *audience* for a future Lazarus version?

> IMO the biggest group are old fashioned Delphi (D7) users, which want 
> their existing Ansi/VCL code base supported *without* complications and 
> incompatibilities introduced by the newer Delphi versions. The subject 
> of this thread clearly indicates that UTF-8 is *not* a solution for this 
> group of users.

It was like that two years ago. But I see more and more people migrate to
the unicode versions, and updating packages. The D7 base is eroding, and
worse, many of its users are mostly hedging bets to keep their codebases
running. Not to make new code. (and we need people that DO things)

It's like with turbo pascal in the (1.0.x) past. Yes, the numbers are huge,
but all they say is they want something 100% compatible to effortless keep
their codebases running.  But when the times come to actually _invest_ in
the code again, they pick something that is at least halfwhat modern.  And
all you are stuck with is oldtimers and l33t tinkerers.

That is the curse of supporting legacy targets, you can't do that forever
without making yourself irrelevant.

Keep in mind that any Lazarus solution in production use based on 2.8.x is
years away. The current activity levels in that group will be even less. Our
decisions must be aimed not at the situation now, but good for at least 5
years.

> Another important user group is targeting mobiles, where time will tell 
> whether FM will ever succeed, or shares the fate of Kylix or VCL.NET. 

Everywhere I see FM (Mobile plugin) buyers, I see existing Delphi users
hoping for an easy conversion to mobile and a quick buck to tide them over
the crisis.  Not real go-getters that really go for mobile.

That makes me think this is not sustainable.

But Embarcadero is said to use it heavily internally, so they won't quickly
kill it off, and I assume a certain kind of customers will adapt it.

But IMHO for us it is irrelevant

> IMO these should be happy already with fpGUI or mseGUI, no need to raise 
> another competitor in this area.

I don't really see any adaptation there. Those teams and offerings are again
a magnitude smaller than Lazarus, and for most of those users switching from
Embacadero to Lazarus is already the biggest step they are willing to make.

> > But if I have to chose to kill one, it is utf8. It is the lesser used choice
> > for unicode strings INSIDE APPLICATIONS.  Yes, UTF8 is dominant in documents, but
> > not in APIs.
> 
> That's my conclusion as well. But is that new audience worth to abandon 
> the entire existing Lazarus audience?

I myself hope for the two tracks way. It satisfies multiple demands, and the
extra work is offset by less rewriting from current Delphi sources and less
discussion.

But the prime point is that IMHO an utf8 Windows is insane, and it should be
possible to port modern Delphi VCL apps at least to Windows. Preferably to
all.