[Lazarus] [fpc-pascal] Tests results of several pascal based JSON parsers

[Michael Van Canneyt]

On Sat, 31 Aug 2019, Anthony Walter via lazarus wrote:

> Michael, regarding this unicode problem, all the code has already been
> posted in this thread.


>
> program Test;
>
> uses
>  FPJson, JsonParser, JsonTools;

There you are. You're missing the cwstring unit and the codepage directive.

Change the above code to

{$mode objfpc}
{$h+}
{$codepage utf8}
uses
   {$IFDEF UNIX}cwstring, {$ENDIF} FPJson, JsonParser, JsonTools;

and it will work correctly. 
(The objpas and $h+ are probably in your fpc.cfg or lazarus setup)

Your program will only work correctly in a utf8-only environment. 
(see also below)

But fpJSON relies on the fpc infrastructure to handle all codepages, as a
consequence this infrastructure also must be set up properly.

Now you can see why I insisted on using my program, it was known to work
correctly: it sets up things properly. If you look at my initial mail,
you'll also see that I explicitly mentioned including cwstring. 
You probably failed to pick up on that important piece of info.

So, mystery solved.

That said :

Unfortunately JSONTools is also not without problems.

I copied the program to a windows VM.

Attached screenshot of the output.

As you can see, jsontools also does not work correctly.

It's no mystery why not. I had to add

   DefaultSystemCodePage:=CP_UTF8;

as the first line in the program, then it does show TRUE for both tests.
Now, if you work in lazarus, it does this for you, so you don't notice/know it.

Codepages & strings require careful setup. Contrary to popular belief, it does not 'just work'.

All this is documented:

https://www.freepascal.org/docs-html/current/ref/refsu9.html#x32-390003.2.4

Many people tend to ignore this, because Lazarus does a lot behind the scenes 
(which is a good thing).

But if people use your JSONTools in a 'mixed' environment, you might 
get strange results, if you ignore the correct and careful setup.

You control your environment, and jsontools will function correctly in your
environment. But it's a big world out there, where things might be happening
that you didn't foresee but which do influence jsontools.

I hope with my explanations, you are now well equipped/informed to strengthen
jsontools and help people should problems nevertheless pop up.

Now that we've hopefully established that fpjson does work correctly, 
I would appreciate it if you could correct the JSON test comparison page you created.

Michael.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jsontools.png
Type: image/png
Size: 8535 bytes
Desc: 
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20190831/9e88b4fa/attachment-0001.png>