From sysrpl at gmail.com Wed Dec 1 01:03:36 2021 From: sysrpl at gmail.com (Anthony Walter) Date: Tue, 30 Nov 2021 19:03:36 -0500 Subject: [Lazarus] New Synthwave Demo In-Reply-To: <20211130231847.537099f5@limapholos.matflo.wg> References: <20211130231847.537099f5@limapholos.matflo.wg> Message-ID: Did someone steal access to Juha's email? Not only are his ramblings off-topic, but he sounds like a deluded conspiracy theorist bent on disenfranchising rational people from using these mailing lists. -------------- next part -------------- An HTML attachment was scrubbed... URL: From juha.manninen62 at gmail.com Wed Dec 1 09:26:58 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Wed, 1 Dec 2021 10:26:58 +0200 Subject: [Lazarus] New Synthwave Demo In-Reply-To: References: <20211130231847.537099f5@limapholos.matflo.wg> Message-ID: On Wed, Dec 1, 2021 at 2:04 AM Anthony Walter via lazarus < lazarus at lists.lazarus-ide.org> wrote: > Did someone steal access to Juha's email? Not only are his ramblings > off-topic, but he sounds like a deluded conspiracy theorist bent on > disenfranchising rational people from using these mailing lists. > It is me myself. I am ready to explain both my theories and pure facts to anyone. Yes, cognitive dissonance is inevitable. Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartjunk64 at gmail.com Wed Dec 1 10:26:09 2021 From: bartjunk64 at gmail.com (Bart) Date: Wed, 1 Dec 2021 10:26:09 +0100 Subject: [Lazarus] New Synthwave Demo In-Reply-To: References: <20211130231847.537099f5@limapholos.matflo.wg> Message-ID: On Wed, Dec 1, 2021 at 9:27 AM Juha Manninen via lazarus wrote: I'm breaking my promiss to not reply anymore. @All: please stop discussing this on the Lazarus mailinglist. Please, please, please, please, pretty please! -- Bart From etienne.leblois at free.fr Wed Dec 1 12:13:34 2021 From: etienne.leblois at free.fr (Etienne Leblois) Date: Wed, 1 Dec 2021 12:13:34 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: We love Lazarus and Pascal programming and anyone's best effort on this is deeply acknowledged. Let us stick to the topic that unites us. Etienne From serbod at gmail.com Wed Dec 1 23:04:41 2021 From: serbod at gmail.com (Sergey Bodrov) Date: Thu, 2 Dec 2021 01:04:41 +0300 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: Pascal walks into a bar There are 100 000 Pascals in the bar! Sorry, i am drunk. Keep bugs out of Lazarus! On Wed, Dec 1, 2021, 14:13 Etienne Leblois via lazarus < lazarus at lists.lazarus-ide.org> wrote: > We love Lazarus and Pascal programming and anyone's best effort on this > is deeply acknowledged. > > Let us stick to the topic that unites us. > > Etienne > > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From b_lists at patandbrian.org Thu Dec 2 00:13:42 2021 From: b_lists at patandbrian.org (Brian) Date: Wed, 1 Dec 2021 18:13:42 -0500 Subject: [Lazarus] New Synthwave Demo In-Reply-To: References: Message-ID: <45d07099-0585-c527-b249-734456367f83@patandbrian.org> On 11/30/21 14:24, Gordon Findlay via lazarus wrote: > > Your mails containing so many lies, and so dangerous to the community, > means that I will cease being a member of the mailing list. > To Gordon, if it's not already too late, and anybody else considering leaving the list due to recent postings :- Your leaving is not the answer. That alone will not stop the posts. Use an e-mail client with good filtering capabilities, and simply filter out those abusing the list. It's hard to do in some cases, particularly when the people you are filtering have helped you in the past, but some things have to be done. Brian. From ryansmithhe at gmail.com Thu Dec 2 00:49:04 2021 From: ryansmithhe at gmail.com (R.Smith) Date: Thu, 2 Dec 2021 01:49:04 +0200 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: <3eb501f8-cffe-8e88-dfeb-95998a902bf5@gmail.com> Sounds like you were pressured into making a joke... On 2021/12/02 00:04, Sergey Bodrov via lazarus wrote: > Pascal walks into a bar > There are 100 000 Pascals in the bar! > > Sorry, i am drunk. Keep bugs out of Lazarus! > > On Wed, Dec 1, 2021, 14:13 Etienne Leblois via lazarus > wrote: > > We love Lazarus and Pascal programming and anyone's best effort on > this > is deeply acknowledged. > > Let us stick to the topic that unites us. > > Etienne > > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpc at pascalprogramming.org Thu Dec 2 10:19:36 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 2 Dec 2021 10:19:36 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: Op 1-12-2021 om 23:04 schreef Sergey Bodrov via lazarus: > Pascal walks into a bar > There are 100 000 Pascals in the bar! > Nope, 100001, 100000 in the bar, and the one that just walked in. From dhkblaszyk at gmail.com Thu Dec 2 10:54:37 2021 From: dhkblaszyk at gmail.com (Darius Blaszyk) Date: Thu, 2 Dec 2021 10:54:37 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: That should be 33 in total! On Thu, Dec 2, 2021 at 10:20 AM Marco van de Voort via lazarus < lazarus at lists.lazarus-ide.org> wrote: > > Op 1-12-2021 om 23:04 schreef Sergey Bodrov via lazarus: > > Pascal walks into a bar > > There are 100 000 Pascals in the bar! > > > Nope, 100001, 100000 in the bar, and the one that just walked in. > > > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpc at pascalprogramming.org Thu Dec 2 10:56:57 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 2 Dec 2021 10:56:57 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: Message-ID: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> Op 2-12-2021 om 10:19 schreef Marco van de Voort via lazarus: > > Op 1-12-2021 om 23:04 schreef Sergey Bodrov via lazarus: >> Pascal walks into a bar >> There are 100 000 Pascals in the bar! >> > ... and then he left because there was no Atmosphere. From m.e.sanliturk at gmail.com Thu Dec 2 11:45:40 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Thu, 2 Dec 2021 13:45:40 +0300 Subject: [Lazarus] up with Lazarus ! In-Reply-To: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> Message-ID: Since the subject is "up with Lazarus" can I ask one question as follows : I have a Pascal program having ( around ) 12000 procedures and a small number of units and it is compiled with Lazarus / Free Pascal only in Unix-like operating systems such as Linux and FreeBSD . (1) After converting all of the variant records to non_variant records ( by removing "case " related part of its tag line , by keeping the tag variable ) , (2) There is no any "Initialization" , "Finalization" parts in units (3) Nearly all of the constants , types , and records are marked with their unit names such as Unit_A . constant_name , etc. . Can I convert that program to JavaScript by using Lazarus ? Later on I want to convert this program to C to link C sources from other permissively licensed repositories directly and then use only C ( not Pascal any more ) , and use JavaScript programs derived from that program where they are more suitable . Your suggestions are appreciated very much . With my best regards , Mehmet Erol Sanliturk On Thu, Dec 2, 2021 at 12:57 PM Marco van de Voort via lazarus < lazarus at lists.lazarus-ide.org> wrote: > > Op 2-12-2021 om 10:19 schreef Marco van de Voort via lazarus: > > > > Op 1-12-2021 om 23:04 schreef Sergey Bodrov via lazarus: > >> Pascal walks into a bar > >> There are 100 000 Pascals in the bar! > >> > > > ... and then he left because there was no Atmosphere. > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpc at pascalprogramming.org Thu Dec 2 11:48:43 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 2 Dec 2021 11:48:43 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> Message-ID: <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> Op 2-12-2021 om 11:45 schreef Mehmet Erol Sanliturk: > >  Can I convert that program to JavaScript by using Lazarus ? Afaik the pas2js transpiler compiles to an low level javascript that might not be suitable for maintenance after. It is not intended for pascal to javascript source conversion, but to compile Pascal codebases to run on javascript only systems. So this will probably all be manual converts. From michael at freepascal.org Thu Dec 2 11:48:53 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Thu, 2 Dec 2021 11:48:53 +0100 (CET) Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> Message-ID: On Thu, 2 Dec 2021, Mehmet Erol Sanliturk via lazarus wrote: > Since the subject is > "up with Lazarus" > can I ask one question as follows : > > I have a Pascal program having ( around ) 12000 procedures and a small > number of > units and it is compiled with Lazarus / Free Pascal only in Unix-like > operating systems > such as Linux and FreeBSD . > > (1) > After converting all of the variant records to non_variant records > ( by removing "case " related part of its tag line , by keeping the tag > variable ) , > > (2) There is no any "Initialization" , "Finalization" parts in units > > (3) Nearly all of the constants , types , and records are marked with their > unit names > such as Unit_A . constant_name , etc. . > > > Can I convert that program to JavaScript by using Lazarus ? Yes. You can use pas2js to convert it to javascript; provided your program does not do any file IO or GUI, this should be relatively painlesss. Michael. From michael at freepascal.org Thu Dec 2 11:52:17 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Thu, 2 Dec 2021 11:52:17 +0100 (CET) Subject: [Lazarus] up with Lazarus ! In-Reply-To: <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> Message-ID: On Thu, 2 Dec 2021, Marco van de Voort via lazarus wrote: > > Op 2-12-2021 om 11:45 schreef Mehmet Erol Sanliturk: >> >>  Can I convert that program to JavaScript by using Lazarus ? > > Afaik the pas2js transpiler compiles to an low level javascript that > might not be suitable for maintenance after. This is not correct. The code is perfectly readable and maintainable. Some of the constructs used are of course specific to the pascal object language. For instance I do all the debugging in Javascript, and this is perfectly doable. The code is very readable, and you should not have any problem stripping out what you think is too pascal-ish. What you cannot expect to be able to do is recompile from pascal and have the compiler incorporate changes you made to the generated javascript. Michael. From m.e.sanliturk at gmail.com Thu Dec 2 14:37:29 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Thu, 2 Dec 2021 16:37:29 +0300 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> Message-ID: On Thu, Dec 2, 2021 at 1:52 PM Michael Van Canneyt wrote: > > > On Thu, 2 Dec 2021, Marco van de Voort via lazarus wrote: > > > > > Op 2-12-2021 om 11:45 schreef Mehmet Erol Sanliturk: > >> > >> Can I convert that program to JavaScript by using Lazarus ? > > > > Afaik the pas2js transpiler compiles to an low level javascript that > > might not be suitable for maintenance after. > > This is not correct. The code is perfectly readable and maintainable. > Some of the constructs used are of course specific to the pascal object > language. > > For instance I do all the debugging in Javascript, and this is perfectly > doable. The code is very readable, and you should not have any problem > stripping out what you think is too pascal-ish. > > What you cannot expect to be able to do is recompile from pascal and have > the compiler incorporate changes you made to the generated javascript. > > Michael. Thank you very much for your prompt replies . After converting to C and JavaScript , Pascal will not be used for continuation of development . I have another program set in Fortran about Statistical and Mathematical analyses . These are convertible to C and working very well when they are executed . I will continue to develop some parts for improvement in Fortran and then convert them to C and push these programs as permissive licensed open sources into a suitable website maintaining such repositories ( as archived , enabling the people to use them freely without wasting them ) . The Pascal program is an implementation of my PhD thesis about a multimedia knowledge base management system ( definition and management ) based on expert system rules . The Fortran programs after conversion to C will be incorporated into this system and will become unusable independently . Reason of leaving Pascal is to enable me to pursue the development by using a SINGLE language . Otherwise , I need to use Pascal , C and Fortran . Another most important problem is the limitations of current operating systems . The Lazarus / Free Pascal is able to generate excellently high quality machine code , but the operating systems , both FreeBSD and Linux are tested , are not able to execute such a large system , because the executable program is hitting the internal limit(s) of the operating systems and they are erasing the executable from the screen and terminating the run without any single message about why that action is applied . The fault is not in the program . If it were in the program I am sure that it would produce a proper run time error . There is no such an error . The run is terminated from different parts of the program by the OSes after a ( large number of thousand ) recursive entries into almost all of the involved procedures covering a very large portion of ( around ) 12000 procedures during transaction processing which involves more recursions with respect to query . When only the query part is used there is no problem . During small transaction usage , again there is no problem . Increasing system defined run time stack size is not changing anything . This fact has blocked my development . Now I have decided to develop a NEW permissively licensed operating system from SCRATCH by using mostly FreeBSD parts in C able to manage ( not "Very" but ) "Large scale software stacks" . This requires changes on ( design , implement ,test , maintain ) operations of software stacks by using NEW compilation , linking ( actually not using it ) , and executing and testing , debugging of the programs distributed on not only NFS server networks but also using the Internet . Process migration over distinct and different computing systems is a vital requirement because a knowledge base management system can not stop some computations over decades . The problems are large and difficult to solve . At present , the choice is "To be or not to be . That's the only question ." With my best wishes for all of you . Mehmet Erol Sanliturk -------------- next part -------------- An HTML attachment was scrubbed... URL: From fpc at pascalprogramming.org Thu Dec 2 14:43:21 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 2 Dec 2021 14:43:21 +0100 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> Message-ID: Op 2-12-2021 om 14:37 schreef Mehmet Erol Sanliturk: > The fault is not in the program . If it were in the program I am sure > that it would produce a proper run time error . There is no such an > error . The run is terminated from different parts of the program by > the OSes  after a ( large number of thousand ) recursive entries into > almost > all of the involved procedures covering a very large portion of ( > around ) 12000 procedures during transaction processing which involves > more recursions with respect to query . > When only the query part is used there is no problem . During small > transaction usage , again > there is no problem . Increasing system defined run time stack size is > not changing anything . > ulimit limits? Does it also happen running as root? Debian seems to have a 8k ulimit by default.  (type "ulimit -h") From m.e.sanliturk at gmail.com Thu Dec 2 16:55:49 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Thu, 2 Dec 2021 18:55:49 +0300 Subject: [Lazarus] up with Lazarus ! In-Reply-To: References: <066b2e03-e17a-4a2b-ef7d-ac5b12f54235@pascalprogramming.org> <992cdb8a-4481-cf10-f289-b0a9b559ae73@pascalprogramming.org> Message-ID: On Thu, Dec 2, 2021 at 4:43 PM Marco van de Voort wrote: > > Op 2-12-2021 om 14:37 schreef Mehmet Erol Sanliturk: > > The fault is not in the program . If it were in the program I am sure > > that it would produce a proper run time error . There is no such an > > error . The run is terminated from different parts of the program by > > the OSes after a ( large number of thousand ) recursive entries into > > almost > > all of the involved procedures covering a very large portion of ( > > around ) 12000 procedures during transaction processing which involves > > more recursions with respect to query . > > When only the query part is used there is no problem . During small > > transaction usage , again > > there is no problem . Increasing system defined run time stack size is > > not changing anything . > > > > ulimit limits? Does it also happen running as root? > > Debian seems to have a 8k ulimit by default. (type "ulimit -h") > <-------- This limit is defined for shell . > > At present my NFS server has not been running for more than a few years . If I start it I need to determine how to restart my work . This will take some time . I did not try Debian . Many years ago I attempted to use Debian , but I found that Mandriva was more suitable for me . When Mandriva closed down , I switched to Fedora . At present I am using Fedora 25 : [s at localhost ~]$ uname -a Linux localhost.localdomain 4.13.16-100.fc25.x86_64 #1 SMP Mon Nov 27 19:52:46 UTC 20 17 x86_64 x86_64 x86_64 GNU/Linux [s at localhost ~]$ [s at localhost ~]$ ulimit -h bash: ulimit: -h: invalid option ulimit: usage: ulimit [-SHabcdefilmnpqrstuvxT] [limit] [s at localhost ~]$ ulimit -H unlimited [s at localhost ~]$ [s at localhost ~]$ ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 30781 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 30781 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited [s at localhost ~]$ uname -a Linux localhost.localdomain 4.13.16-100.fc25.x86_64 #1 SMP Mon Nov 27 19:52:46 UTC 20 17 x86_64 x86_64 x86_64 GNU/Linux [s at localhost ~]$ Now I do not want to attempt to "improve" the existing operating systems because such improvements will not solve the inherent problems in developing and using large scale software stacks . Reasons are shortly defined below . My opinion about developing a new ( distributed and able to manage large software stacks ) operating system is really a very important goal , because the existent methods about software development and maintenance is causing large amount of losses ( time , effort , money , ... ) . This structure based on ( let's say around ) 1970 technology levels are not suitable for today's requirements . Many years ago when there was a need to a "secure programming language" for large military system supporting software , the Ada is selected , but failed , ( with respect to my memory ( which I may be wrong )) because (1) Ada was not able to compile itself due to its design , (2) The Ada compiler written in a "not-secure language" such as C could not produce a "secure compiler" for Ada . In those days , ( I do not remember name of the computing scientist who expressed the idea ) the expressed idea was " ... the present day software systems can not support development of required secure software for such a large military system ... " . Software development ( with all its components ) is not different from construction of buildings conceptually : It is possible to construct a 1-storey building by using earth-bags , but you can not stack three of them to construct a 3-storey building . You need to use brick at least . You can not construct a 9-storey building buy stacking three 3-storey buildings constructed from brick : You need to use concrete . . . . Such technology and material changes are required during construction of taller and taller buildings . During development of my program I have encountered a similar problem . When the size of the program increased , it became necessary to change the program development system . When the size of the program increased to a larger level , the used method again collapsed , it became necessary to change the program development method once more . These changes became necessary during increasing size of the program . All of the ideas presented in "Software Engineering" literature usable in small programs ( because they derived from small program projects ) are becoming useless during increasing size of the program . Instead of attempting "patching" of existing systems ( OS , compilers , etc. ) which they are developed for small systems and have significant problems for management of them , it is necessary to start a new set of software systems with a suitable design and implementation . Mehmet Erol Sanliturk -------------- next part -------------- An HTML attachment was scrubbed... URL: From sysrpl at gmail.com Fri Dec 3 00:22:41 2021 From: sysrpl at gmail.com (Anthony Walter) Date: Thu, 2 Dec 2021 18:22:41 -0500 Subject: [Lazarus] All Demos Are Done Message-ID: I've completed writing all the demos and most of the documentation for my soon to be released vector graphics and physics simulation library. The library is named Tiny Sim and the first version will be published soon. I have lots of planned features and improvements to come. If you are interested I posted a very brief overview page and a video containing all nine demos for my library on this web page: https://www.getlazarus.org/videos/physics/demos/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.e.sanliturk at gmail.com Fri Dec 3 00:52:09 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Fri, 3 Dec 2021 02:52:09 +0300 Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: On Fri, Dec 3, 2021 at 2:23 AM Anthony Walter via lazarus < lazarus at lists.lazarus-ide.org> wrote: > I've completed writing all the demos and most of the documentation for my > soon to be released vector graphics and physics simulation library. The > library is named Tiny Sim and the first version will be published soon. I > have lots of planned features and improvements to come. > > If you are interested I posted a very brief overview page and a video > containing all nine demos for my library on this web page: > > https://www.getlazarus.org/videos/physics/demos/ > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus At the top of the your supplied link page , the following message is displayed ( which is not copyable , I wrote it below myself ) : (i) To play video , you may need to install the required video codecs. When "Play" is clicked , it is displaying the following message ( which is not copyable , I wrote it below myself ) : No video with supported format and MIME type found. A display occurred below the video play area when I clicked something with its partial output under "Console" tab is the following : Specified “type” attribute of “video/mp4” is not supported. Load of media resource //cache.getlazarus.org/videos/tiny-sim-demos.mp4 failed. demos All candidate resources failed to load. Media load paused. demos SyntaxHighlighter.config home.js:120:10 The video on this page can’t be played. Your system may not have the required video codecs for: video/mp4 demos I am playing Youtube videos without any problem . [s at localhost ~]$ uname -a Linux localhost.localdomain 4.13.16-100.fc25.x86_64 #1 SMP Mon Nov 27 19:52:46 UTC 20 17 x86_64 x86_64 x86_64 GNU/Linux [s at localhost ~]$ With my best regards Mehmet Erol Sanliturk -------------- next part -------------- An HTML attachment was scrubbed... URL: From samps at unplugd.com Fri Dec 3 04:22:27 2021 From: samps at unplugd.com (Samps) Date: Fri, 3 Dec 2021 13:52:27 +1030 Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: Works fine from where I am (in front of a MacBook) and it looks absolutely awesome!! Samps On 3/12/21 10:22 am, Mehmet Erol Sanliturk via lazarus wrote: > > > On Fri, Dec 3, 2021 at 2:23 AM Anthony Walter via lazarus > wrote: > > I've completed writing all the demos and most of the documentation > for my soon to be released vector graphics and physics simulation > library. The library is named Tiny Sim and the first version will > be published soon. I have lots of planned features and > improvements to come. > > If you are interested I posted a very brief overview page and a > video containing all nine demos for my library on this web page: > > https://www.getlazarus.org/videos/physics/demos/ > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > > > > > At the top of the your supplied link page , the following message is > displayed > ( which is not copyable , I wrote it below myself ) : > > (i) To play video , you may need to install the required video codecs. > > > > When "Play" is clicked , it is displaying the following message > ( which is not copyable , I wrote it below myself ) : > > > No video with supported format and MIME type found. > > > A display occurred below the video play area when I clicked something > with > its partial output under "Console" tab is the following : > > > Specified “type” attribute of “video/mp4” is not supported. Load of > media resource //cache.getlazarus.org/videos/tiny-sim-demos.mp4 > failed. demos > All candidate resources failed to load. Media load paused. demos > SyntaxHighlighter.config home.js:120:10 > The video on this page can’t be played. Your system may not have the > required video codecs for: video/mp4 demos > > > I am playing  Youtube videos without any problem . > > [s at localhost ~]$ uname -a > Linux localhost.localdomain 4.13.16-100.fc25.x86_64 #1 SMP Mon Nov 27 > 19:52:46 UTC 20 > 17 x86_64 x86_64 x86_64 GNU/Linux > [s at localhost ~]$ > > > > > With my best regards > > > Mehmet Erol Sanliturk > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sam_herzog at yahoo.com Fri Dec 3 06:33:04 2021 From: sam_herzog at yahoo.com (Samuel Herzog) Date: Fri, 3 Dec 2021 05:33:04 +0000 (UTC) Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: <1993949350.1778566.1638509584770@mail.yahoo.com> Wow, so cool! Great  work. Congratulations. Am Freitag, 3. Dezember 2021, 00:23:34 MEZ hat Anthony Walter via lazarus Folgendes geschrieben: I've completed writing all the demos and most of the documentation for my soon to be released vector graphics and physics simulation library. The library is named Tiny Sim and the first version will be published soon. I have lots of planned features and improvements to come. If you are interested I posted a very brief overview page and a video containing all nine demos for my library on this web page: https://www.getlazarus.org/videos/physics/demos/ -- _______________________________________________ lazarus mailing list lazarus at lists.lazarus-ide.org https://lists.lazarus-ide.org/listinfo/lazarus -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkitty42 at windstream.net Fri Dec 3 11:13:09 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Fri, 3 Dec 2021 05:13:09 -0500 Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: On 12/2/21 6:52 PM, Mehmet Erol Sanliturk via lazarus wrote: > At the top of the your supplied link page , the following message is displayed > ( which is not copyable , I wrote it below myself ) : > > (i) To play video , you may need to install the required video codecs. if your system cannot play the video, then this is the correct message... apparently you are missing some video codecs that you need to install if you expect to view this video and others like it... at a guess, i'd say that you do not have the AVC (Advanced Video Codec) installed... the below info pulled from a kubuntu 14.04 system that easily viewed the video both online and after downloading it... $ ffprobe -report "cv137g.mp4" Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'cv137g.mp4': Metadata: major_brand : isom minor_version : 512 compatible_brands: isomiso2avc1mp41 encoder : Lavf58.76.100 Duration: 00:01:42.53, start: 0.000000, bitrate: 6094 kb/s Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 5955 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc (default) Metadata: handler_name : VideoHandler Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : SoundHandler -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From m.e.sanliturk at gmail.com Fri Dec 3 11:25:53 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Fri, 3 Dec 2021 13:25:53 +0300 Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: On Fri, Dec 3, 2021 at 1:13 PM wkitty42--- via lazarus < lazarus at lists.lazarus-ide.org> wrote: > On 12/2/21 6:52 PM, Mehmet Erol Sanliturk via lazarus wrote: > > At the top of the your supplied link page , the following message is > displayed > > ( which is not copyable , I wrote it below myself ) : > > > > (i) To play video , you may need to install the required video codecs. > > if your system cannot play the video, then this is the correct message... > apparently you are missing some video codecs that you need to install if > you > expect to view this video and others like it... at a guess, i'd say that > you do > not have the AVC (Advanced Video Codec) installed... the below info pulled > from > a kubuntu 14.04 system that easily viewed the video both online and after > downloading it... > > $ ffprobe -report "cv137g.mp4" > > Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'cv137g.mp4': > Metadata: > major_brand : isom > minor_version : 512 > compatible_brands: isomiso2avc1mp41 > encoder : Lavf58.76.100 > Duration: 00:01:42.53, start: 0.000000, bitrate: 6094 kb/s > Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, > bt709), 1920x1080 [SAR 1:1 DAR 16:9], 5955 kb/s, 60 fps, 60 tbr, 15360 > tbn, 120 > tbc (default) > Metadata: > handler_name : VideoHandler > Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, > fltp, > 128 kb/s (default) > Metadata: > handler_name : SoundHandler > > > -- > NOTE: No off-list assistance is given without prior approval. > *Please keep mailing list traffic on the list where it belongs!* > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus You are right : The Firefox is using different resources other than Fedora supplied ones during Youtube video playing . It is necessary to install Fedora supplied packages for other HTML pages . >From responses to my message I understand that Fedora KDE 25 supplied , related packages are missing from my computer , because my version of Fedora KDE 25 was installed a few days ago by replacing a broken HDD containing Fedora KDE 28 , and required codecs are not installed yet . Thank you very much . Mehmet Erol Sanliturk -------------- next part -------------- An HTML attachment was scrubbed... URL: From mguerra13 at gmail.com Fri Dec 3 13:27:42 2021 From: mguerra13 at gmail.com (Mario Guerra) Date: Fri, 3 Dec 2021 09:27:42 -0300 Subject: [Lazarus] All Demos Are Done In-Reply-To: References: Message-ID: <9b02101f-7333-ca31-71ff-b10eda32f3da@gmail.com> Thanks for sharing your great work !! El 2/12/21 a las 20:22, Anthony Walter via lazarus escribió: > I've completed writing all the demos and most of the documentation for > my soon to be released vector graphics and physics simulation library. > The library is named Tiny Sim and the first version will be published > soon. I have lots of planned features and improvements to come. > > If you are interested I posted a very brief overview page and a video > containing all nine demos for my library on this web page: > > https://www.getlazarus.org/videos/physics/demos/ > > From steveg at nevets.com.au Mon Dec 6 00:36:28 2021 From: steveg at nevets.com.au (Steve Gatenby) Date: Mon, 6 Dec 2021 09:36:28 +1000 Subject: [Lazarus] Assigning Class to object Message-ID: Please dont judge on simplicity of question (or questioner) -  :) if I call the following procedure with either a TButton or Panel ctrl (for example), how can I use the class info contained within the MyObj to choose the correct Class procedure SetThis( MyObj :TObject ); begin   // this works -   if pos('Button', MyObj.ClassName) > 0   then TButton(MyObj).OnMouseDown := tmethod (not important here)   else TButton(MyObj).OnMouseDown := tmethod; (not important here)   // what I would like is the single assignment as could cover multiple controls / events   TClass(MyObj).OnMouseDown := tmethod (not important here) end; Regards SteveG From ryansmithhe at gmail.com Mon Dec 6 01:15:33 2021 From: ryansmithhe at gmail.com (R.Smith) Date: Mon, 6 Dec 2021 02:15:33 +0200 Subject: [Lazarus] Assigning Class to object In-Reply-To: References: Message-ID: <188ffbdf-4c89-4cb4-76d4-58a7790b1c30@gmail.com> > if I call the following procedure with either a TButton or Panel ctrl (for example), how can I use the class info contained within the MyObj to choose the correct Class Two things, Firstly the correct way of determinig class (as with your current solution) is like this:   if (MyObj is TButton) then TButton(MyObj).OnMouseDown := tmethod (not important here)  else   if (MyObj is TPanel)  then TPanel(MyObj).OnMouseDown := tmethod (not important here)  else   //  etc... If you want to change a property/method of many descendants of class which all contain the same property, that is quite easy and what object orientation is great at, but you have to figure out what is the most recent ancestor class that is the ancestor of all the classes that you wish to treat similarly, AND if the property(ies) are defined in the ancestor or higher.  This is trivial to do just looking through the FPC help/documentation pages for the classes in question. In the case you describe, from memory I believe TControl is the recent-most ancestor of both those classes that already has the OnMouseDown event defined, so this should work: procedure SetThis( MyObj :TObject ); begin   TControl(MyObj).OnMouseDown := @someMouseDownMethod; end; HTH. From steveg at nevets.com.au Mon Dec 6 01:48:56 2021 From: steveg at nevets.com.au (Steve Gatenby) Date: Mon, 6 Dec 2021 10:48:56 +1000 Subject: [Lazarus] Assigning Class to object In-Reply-To: <188ffbdf-4c89-4cb4-76d4-58a7790b1c30@gmail.com> References: <188ffbdf-4c89-4cb4-76d4-58a7790b1c30@gmail.com> Message-ID: <6f1fe790-da27-4dd9-e1ba-98bc12abc717@nevets.com.au> Thank you, that should work for me :) On 6/12/21 10:15 am, R.Smith via lazarus wrote: > > if I call the following procedure with either a TButton or Panel > ctrl (for example), how can I use the class info contained within the > MyObj to choose the correct Class > > Two things, > > Firstly the correct way of determinig class (as with your current > solution) is like this: > >   if (MyObj is TButton) then TButton(MyObj).OnMouseDown := tmethod > (not important here)  else >   if (MyObj is TPanel)  then TPanel(MyObj).OnMouseDown := tmethod (not > important here)  else >   //  etc... > > If you want to change a property/method of many descendants of class > which all contain the same property, that is quite easy and what > object orientation is great at, but you have to figure out what is the > most recent ancestor class that is the ancestor of all the classes > that you wish to treat similarly, AND if the property(ies) are defined > in the ancestor or higher.  This is trivial to do just looking through > the FPC help/documentation pages for the classes in question. > > In the case you describe, from memory I believe TControl is the > recent-most ancestor of both those classes that already has the > OnMouseDown event defined, so this should work: > > procedure SetThis( MyObj :TObject ); > begin >   TControl(MyObj).OnMouseDown := @someMouseDownMethod; > end; > > > HTH. > > > From badsectoracula at gmail.com Mon Dec 6 10:49:38 2021 From: badsectoracula at gmail.com (Kostas Michalopoulos) Date: Mon, 6 Dec 2021 11:49:38 +0200 Subject: [Lazarus] Assigning Class to object In-Reply-To: <188ffbdf-4c89-4cb4-76d4-58a7790b1c30@gmail.com> References: <188ffbdf-4c89-4cb4-76d4-58a7790b1c30@gmail.com> Message-ID: On 12/6/21 02:15, R.Smith via lazarus wrote: > Firstly the correct way of determinig class (as with your current > solution) is like this: > >   if (MyObj is TButton) then TButton(MyObj).OnMouseDown := tmethod (not > important here)  else >   if (MyObj is TPanel)  then TPanel(MyObj).OnMouseDown := tmethod (not > important here)  else >   //  etc... > One small detail is that the "is Thing" operator checks if the object belongs to the class hierarchy introduced by the Thing class, but not necessarily if it *is* of Thing type - it can be a subclass too. Sometimes this distinction is important, so in those cases instead of "is" you can use the ClassType property of objects (e.g. if MyObj.ClassType=TButton then ...). (yeah the name of the operator is kinda optimistic in assuming Liskov substitution principle always applies, but in practice this is often not the case :-P) In general assuming two f: TFoo and b: TBar objects with TBar being a subclass of TFoo, this code: Writeln(f is TFoo, ' ', b is TFoo, ' ', f.ClassType=TFoo, ' ', b.ClassType=TFoo); Will write TRUE TRUE TRUE FALSE (ie, both f and b belong to the class hierarchy introduced by TFoo but only f is really TFoo and b is not). Kostas From mlnglsts at bgss.hu Mon Dec 6 17:17:00 2021 From: mlnglsts at bgss.hu (Gabor Boros) Date: Mon, 6 Dec 2021 17:17:00 +0100 Subject: [Lazarus] FpDebug with aarch64-linux Message-ID: <4604f31c-cbea-2483-1a52-c6820f368e1b@bgss.hu> Hi All, FPC 3.3.1, Lazarus fixes_2_2, LazDebuggerFp installed and configured in Tools/Options. If start the project got "Error starting process in debugger". With gdb works as expected. Is FpDebug not supported on aarch64? Gabor From lazarus at mfriebe.de Mon Dec 6 17:47:39 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Mon, 6 Dec 2021 17:47:39 +0100 Subject: [Lazarus] FpDebug with aarch64-linux In-Reply-To: <4604f31c-cbea-2483-1a52-c6820f368e1b@bgss.hu> References: <4604f31c-cbea-2483-1a52-c6820f368e1b@bgss.hu> Message-ID: <8181b9bb-06e7-8afc-6fac-63f8ab0f695b@mfriebe.de> On 06/12/2021 17:17, Gabor Boros via lazarus wrote: > Hi All, > > FPC 3.3.1, Lazarus fixes_2_2, LazDebuggerFp installed and configured > in Tools/Options. > If start the project got "Error starting process in debugger". > With gdb works as expected. Is FpDebug not supported on aarch64? > FpDebug only supports Intel/AMD 32/64 bit. There is an remote "avr" extension (included, but must be installed), contributed (and currently worked on) by C. Crause. From md at delfire.net Wed Dec 8 17:08:00 2021 From: md at delfire.net (Marcos Douglas B. Santos) Date: Wed, 8 Dec 2021 13:08:00 -0300 Subject: [Lazarus] TComboBox using csSimple, but with a dropdown list Message-ID: Hi, Is it possible to have a TComboBox, Style=csSimple, but with a dropdown list? I would like to have a combo with no arrow, as csSimple style, but with the dropdown (auto) list. Even though TComboBox has the option to hide the arrow, the dropdown list doesn't work. best regards, Marcos Douglas -------------- next part -------------- An HTML attachment was scrubbed... URL: From bo.berglund at gmail.com Wed Dec 8 21:44:59 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Wed, 08 Dec 2021 21:44:59 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? Message-ID: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> I need to port an old Delphi program to FPC/Lazarus and it contains two TRichEdit components. When I try to open the form in Lazarus 2.0.12 on Windows 10 with Fpc 3.2.0 I get an error because the TRichEdit component does not exist.... Is there some non-standard package I could install (from OLPM or similar) in order to get it working again? I have a new laptop now and I have no new Delphi version installed (except for Delphi 7 with the no-nonsense license). I am only using Fpc + Lazarus nowadays. Another question while I am here: --------------------------------- The form I have problems with is a special calculation sheet in one single form, which is part of the source program. But I only need to port this form into the new Lazarus program so I tried to create a new program from scratch and intended to replace the default form as the main form. But I don't know really how to do this... So How can I: 1) Replace the main form of a Lazarus program with another form? or 2) Create a new program in Lazarus where I can specify an existing form as the main program form. Either of these will work, but I don't really know how to do any of them. -- Bo Berglund Developer in Sweden From ryansmithhe at gmail.com Wed Dec 8 22:28:23 2021 From: ryansmithhe at gmail.com (R.Smith) Date: Wed, 8 Dec 2021 23:28:23 +0200 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: On 2021/12/08 22:44, Bo Berglund via lazarus wrote: > I need to port an old Delphi program to FPC/Lazarus and it contains two > TRichEdit components. > > When I try to open the form in Lazarus 2.0.12 on Windows 10 with Fpc 3.2.0 I get > an error because the TRichEdit component does not exist.... > > Is there some non-standard package I could install (from OLPM or similar) in > order to get it working again? Not really. The problem is TRichEdit was just a wrapper for the MS Windows RichEdit component, which was born before the days of HTML componenets, so far inferior to anything new. It does not exist anywhere else in the world and so not useful on a cross-platform development system. Not sure if anybody reverse-engineered the RichEdit functionality, but if Google doesn't know then I am relatively sure there isn't. > > I have a new laptop now and I have no new Delphi version installed (except for > Delphi 7 with the no-nonsense license). I am only using Fpc + Lazarus nowadays. > > Another question while I am here: > --------------------------------- > > The form I have problems with is a special calculation sheet in one single form, > which is part of the source program. > But I only need to port this form into the new Lazarus program so I tried to > create a new program from scratch and intended to replace the default form as > the main form. > But I don't know really how to do this... > > So How can I: > > 1) Replace the main form of a Lazarus program with another form? > or > 2) Create a new program in Lazarus where I can specify an existing form as the > main program form. > > Either of these will work, but I don't really know how to do any of them. This is quick to do - via the GUI: In the Project Options, simply go to "Forms" and make sure the topmost one is the one you want to load as your main form. via Code: Open the program file (Project --> View Source) and make sure in the code section the very first "Application.CreateForm(...)" that appears is the one for your desired main form. Basically TApplication regards the first form it opens as the "main" form, which simply means that that form is shown automatically first (unless specified not to) and that it will exit when that form closes (and a few more small things). From skalogryz.lists at gmail.com Wed Dec 8 22:39:55 2021 From: skalogryz.lists at gmail.com (Dmitry Boyarintsev) Date: Wed, 8 Dec 2021 16:39:55 -0500 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: On Wednesday, December 8, 2021, Bo Berglund via lazarus < lazarus at lists.lazarus-ide.org> wrote: > > Is there some non-standard package I could install (from OLPM or similar) > in > order to get it working again? > you might want to try TRichMemo it provides some sort of compatibility with TRichEdit https://wiki.freepascal.org/RichMemo#TRichEditForMemo thanks, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From bo.berglund at gmail.com Wed Dec 8 23:52:07 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Wed, 08 Dec 2021 23:52:07 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: <4ad2rgpavu2bopil1qj50r7h2u0rm8c7rd@4ax.com> On Wed, 8 Dec 2021 23:28:23 +0200, "R.Smith via lazarus" wrote: >> So How can I: >> >> 1) Replace the main form of a Lazarus program with another form? >> or >> 2) Create a new program in Lazarus where I can specify an existing form as the >> main program form. >> >> Either of these will work, but I don't really know how to do any of them. > >This is quick to do - >via the GUI: > >In the Project Options, simply go to "Forms" and make sure the topmost >one is the one you want to load as your main form. Wel, my form is not shown here either as active or available. I added the unit via Project Inspector using the Add button... >via Code: > >Open the program file (Project --> View Source) and make sure in the >code section the very first "Application.CreateForm(...)" that appears >is the one for your desired main form. In the source I had to: - Add the form unit to the uses clause and remove Unit1 - Change the Application.CreateForm call to use the specific form - Erase Form1 from the lpr file - Remove Unit1 in Project Inspector. >Basically TApplication regards the first form it opens as the "main" >form, which simply means that that form is shown automatically first >(unless specified not to) and that it will exit when that form closes >(and a few more small things). OK, so now I have to solve the dependencies that popped up regarding used units in the now main form... -- Bo Berglund Developer in Sweden From jmlandmesser at gmx.de Thu Dec 9 11:22:43 2021 From: jmlandmesser at gmx.de (John Landmesser) Date: Thu, 9 Dec 2021 11:22:43 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> Am 08.12.21 um 22:39 schrieb Dmitry Boyarintsev via lazarus: > On Wednesday, December 8, 2021, Bo Berglund via lazarus > wrote: > > Is there some non-standard package I could install (from  OLPM or > similar) in > order to get it working again? > >  you might want to try TRichMemo > > it provides some sort of compatibility with TRichEdit > https://wiki.freepascal.org/RichMemo#TRichEditForMemo > > thanks, > Dmitry > > If i remember correct  TRichMemo won't be usable on Linux. -------------- next part -------------- An HTML attachment was scrubbed... URL: From skalogryz.lists at gmail.com Thu Dec 9 11:53:18 2021 From: skalogryz.lists at gmail.com (Dmitry Boyarintsev) Date: Thu, 9 Dec 2021 05:53:18 -0500 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> Message-ID: On Thu, Dec 9, 2021 at 5:23 AM John Landmesser via lazarus < lazarus at lists.lazarus-ide.org> wrote: > If i remember correct TRichMemo won't be usable on Linux. > To a certain extent, yes - not all features are available thanks, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From bo.berglund at gmail.com Thu Dec 9 15:05:15 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Thu, 09 Dec 2021 15:05:15 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: On Wed, 8 Dec 2021 23:28:23 +0200, "R.Smith via lazarus" wrote: >> Is there some non-standard package I could install (from OLPM or similar) in >> order to get it working again? > >Not really. The problem is TRichEdit was just a wrapper for the MS >Windows RichEdit component, which was born before the days of HTML >componenets, so far inferior to anything new. It does not exist anywhere >else in the world and so not useful on a cross-platform development >system. Not sure if anybody reverse-engineered the RichEdit >functionality, but if Google doesn't know then I am relatively sure >there isn't. > OK, I do not think I need a TRichEdit just to show the data, a TMemo would probably suffice as long as I can set a font like Licida Console or other fixed width font. But I also need to be able to set the text selection in code... The application is used to display a panel of the hex representation of the binary data loaded from file. When the user selects a data item in a separate item list the corresponding hex values in the panel shall be selected. The amount of data varies from item to item and this is the reason for the display. Also, if a user clicks on a value in the hex list the corresponding data item in the data list shall be selected and the full scope of the data item selected in the hex view. So I need a display component that can handle this: - an address panel to the left of the main panel, not part of the list - a main panel showing the hex data in rows of 16 values each - the data lines shall scroll in sync with the address panel - the hex values of a data item shall be selected in the main panel - if the user clicks in the hex panel the clicked item shall be selected Problems: Can TMemo handle this? If not is there another Lazarus component that can be used? -- Bo Berglund Developer in Sweden From luca at wetron.es Thu Dec 9 16:01:01 2021 From: luca at wetron.es (Luca Olivetti) Date: Thu, 9 Dec 2021 16:01:01 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> Message-ID: <6739e10a-d8f8-8a8d-30c2-1821d3d6cf16@wetron.es> El 9/12/21 a les 15:05, Bo Berglund via lazarus ha escrit: > > If not is there another Lazarus component that can be used? Maybe you could use TMPHexEditorEx https://github.com/michalgw/mphexeditor Bye -- Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007 From bo.berglund at gmail.com Thu Dec 9 16:19:52 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Thu, 09 Dec 2021 16:19:52 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <6739e10a-d8f8-8a8d-30c2-1821d3d6cf16@wetron.es> Message-ID: On Thu, 9 Dec 2021 16:01:01 +0100, Luca Olivetti via lazarus wrote: >El 9/12/21 a les 15:05, Bo Berglund via lazarus ha escrit: > >> >> If not is there another Lazarus component that can be used? > >Maybe you could use TMPHexEditorEx > >https://github.com/michalgw/mphexeditor > I went there but I found no documentation... There is a doc/MPHexEditor.chm file but it does not display anything except headings on my Windows-10 PC... How can I use this? Notice that I do not want to *edit* any data just display them in a matrix where the bytes corresponding to a data item are highlighted. -- Bo Berglund Developer in Sweden From luca at wetron.es Thu Dec 9 17:30:34 2021 From: luca at wetron.es (Luca Olivetti) Date: Thu, 9 Dec 2021 17:30:34 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <6739e10a-d8f8-8a8d-30c2-1821d3d6cf16@wetron.es> Message-ID: El 9/12/21 a les 16:19, Bo Berglund via lazarus ha escrit: > On Thu, 9 Dec 2021 16:01:01 +0100, Luca Olivetti via lazarus > wrote: > >> El 9/12/21 a les 15:05, Bo Berglund via lazarus ha escrit: >> >>> >>> If not is there another Lazarus component that can be used? >> >> Maybe you could use TMPHexEditorEx >> >> https://github.com/michalgw/mphexeditor >> > > I went there but I found no documentation... there's a chm file (which I didn't use) and a couple of examples. Then there's google, duckduckgo and other search engines. > > There is a doc/MPHexEditor.chm file but it does not display anything except > headings on my Windows-10 PC... Ah, Ok, as I didn't use it (I didn't even see it was there) I cannot tell, I just used the examples and looked at the properties in the object inspector. > > How can I use this? > > Notice that I do not want to *edit* any data just display them in a matrix where > the bytes corresponding to a data item are highlighted. It's up to you to check it out. If, by adjusting some properties, can do what you want, good, if not you'll have to look for something else :-( I can only say that it has some bugs: in my case, I tried to adjust the BytesPerColumn property at design time and it made the form unloadable. I just set it during FormShow and it's fine (for my needs). Bye -- Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007 From werner.pamler at freenet.de Thu Dec 9 17:57:35 2021 From: werner.pamler at freenet.de (Werner Pamler) Date: Thu, 9 Dec 2021 17:57:35 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <6739e10a-d8f8-8a8d-30c2-1821d3d6cf16@wetron.es> Message-ID: Am 09.12.2021 um 16:19 schrieb Bo Berglund via lazarus: > How can I use this? Very easy: just load the file procedure TForm1.FormCreate(Sender: TObject); begin   hex := TMpHexEditor.Create(self);   hex.Parent := self;   hex.Align := alClient;   hex.LoadFromFile('Project1.exe'); end; From skalogryz.lists at gmail.com Thu Dec 9 18:56:28 2021 From: skalogryz.lists at gmail.com (Dmitry Boyarintsev) Date: Thu, 9 Dec 2021 12:56:28 -0500 Subject: [Lazarus] How to convert Delphi program with TRichEdit? In-Reply-To: <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> Message-ID: On Thu, Dec 9, 2021 at 5:23 AM John Landmesser via lazarus < lazarus at lists.lazarus-ide.org> wrote: > If i remember correct TRichMemo won't be usable on Linux. > According to Bo's needs (showing colored Hex values) RichMemo will work just fine for Linux. (Both Qt and Gtk2) thanks, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From bo.berglund at gmail.com Thu Dec 9 21:49:28 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Thu, 09 Dec 2021 21:49:28 +0100 Subject: [Lazarus] How to convert Delphi program with TRichEdit? References: <0q42rg9oop3td3hhoovn1gbkjd46901i5c@4ax.com> <4905a456-02cb-304f-d10b-c43b74da1487@gmx.de> Message-ID: On Wed, 8 Dec 2021 16:39:55 -0500, Dmitry Boyarintsev via lazarus wrote: >> Is there some non-standard package I could install (from OLPM or similar) >> in >> order to get it working again? >> > you might want to try TRichMemo > >it provides some sort of compatibility with TRichEdit >https://wiki.freepascal.org/RichMemo#TRichEditForMemo > >thanks, >Dmitry I looked at the wiki page: https://wiki.freepascal.org/RichMemo And the Download section says: "The easiest way to install the stable version of the component is via the Online-Package-Manager" THis was the best way, so I did this and then I replaced the TRichEdit panels with the TRichMemo instead. With some small tweaks it seems like it is now working! :-) THANK YOU for your pointer towards this component! Much obliged! -- Bo Berglund Developer in Sweden From bo.berglund at gmail.com Mon Dec 13 21:01:05 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Mon, 13 Dec 2021 21:01:05 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? Message-ID: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> I am constantly using strip -s to reduce the size of the application binary and now I wonder if there is a way to make Lazarus NOT include the symbols in the first place? It seems like these are only used when debugging and that is started by clicking the green arrow button, so Lazarus should be able to figure out when the symbols are needed (for debug) and not add them when not needed... The reason I have for finding a way to do this is that the binary will be added to version control and then it may happen that one forgets the strip -s step before committing the state, thus sending an overly bloated binary into subversion. -- Bo Berglund Developer in Sweden From d.ioannidis at nephelae.eu Mon Dec 13 21:34:55 2021 From: d.ioannidis at nephelae.eu (Dimitrios Chr. Ioannidis) Date: Mon, 13 Dec 2021 22:34:55 +0200 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? In-Reply-To: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> Message-ID: <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Hi, Στις 13/12/2021 10:01 μ.μ., ο/η Bo Berglund via lazarus έγραψε: > I am constantly using strip -s to reduce the size of the application binary and > now I wonder if there is a way to make Lazarus NOT include the symbols in the > first place? > > It seems like these are only used when debugging and that is started by clicking > the green arrow button, so Lazarus should be able to figure out when the symbols > are needed (for debug) and not add them when not needed... > > The reason I have for finding a way to do this is that the binary will be added > to version control and then it may happen that one forgets the strip -s step > before committing the state, thus sending an overly bloated binary into > subversion. I suppose one way is to use Lazarus's build modes. i.e. Add a "release" build mode to your project with the settings you want including strip -s support. Then make a script /batch which builds it using that build mode (i.e.  lazbuild with --build-mode= or --bm= ) and copies the "release" stripped to the relevant location ( i.e. local repocitory) before commits it . Or if you don't want a script / batch, I think that you can add the copy functionality for that "release" build mode to the "Project Options -> Compiler  Options -> Compiler Commands -> Execute after -> Command" and leave only the build checkbox checked. With such a configuration, every time you build that mode you'll have a stripped "release" binary which will be copied to the location after the build ends . regards, -- Dimitrios Chr. Ioannidis regards, -- Dimitrios Chr. Ioannidis From bo.berglund at gmail.com Mon Dec 13 22:29:15 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Mon, 13 Dec 2021 22:29:15 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: On Mon, 13 Dec 2021 22:34:55 +0200, "Dimitrios Chr. Ioannidis via lazarus" wrote: >Hi, > >???? 13/12/2021 10:01 ?.?., ?/? Bo Berglund via lazarus ??????: >> I am constantly using strip -s to reduce the size of the application binary and >> now I wonder if there is a way to make Lazarus NOT include the symbols in the >> first place? >> >> It seems like these are only used when debugging and that is started by clicking >> the green arrow button, so Lazarus should be able to figure out when the symbols >> are needed (for debug) and not add them when not needed... >> >> The reason I have for finding a way to do this is that the binary will be added >> to version control and then it may happen that one forgets the strip -s step >> before committing the state, thus sending an overly bloated binary into >> subversion. > >I suppose one way is to use Lazarus's build modes. > >i.e. > >Or if you don't want a script / batch, I think that you can add the copy >functionality for that "release" build mode to the "Project Options -> >Compiler  Options -> Compiler Commands -> Execute after -> Command" and >leave only the build checkbox checked. With such a configuration, every >time you build that mode you'll have a stripped "release" binary which >will be copied to the location after the build ends . > This sounds like what I need. I looked at the project options and found that there is one box to enter the Execute after command. And checkboxes to set when it is used. But I would need *two* commands: - copy to the svn location - strip -s on that copy Does this mean I will have to use a batch file (on Windows) to perform both steps? How do I then transfer the name of the executable into that command? And will Lazarus use the standard binary or the one created in the release folder when debugging? -- Bo Berglund Developer in Sweden From bartjunk64 at gmail.com Mon Dec 13 22:44:16 2021 From: bartjunk64 at gmail.com (Bart) Date: Mon, 13 Dec 2021 22:44:16 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? In-Reply-To: References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: On Mon, Dec 13, 2021 at 10:29 PM Bo Berglund via lazarus wrote: > This sounds like what I need. > I looked at the project options and found that there is one box to enter the > Execute after command. And checkboxes to set when it is used. > > But I would need *two* commands: > - copy to the svn location > - strip -s on that copy > In compiler options you can set various debugging related options as well. One is to strip symbols. - uncheck: "generate infor for the debugger" - check: Strip symbols from executable You apply those debugging options to your "Release" build mode. You keep all relevant debugging options checked for your "Debug" build-mode. Now you develop in "Debug" build-mode. Once you're ready to release: swicth build-mode to "Release" and compile. (You might want a "Release and copy to svn" build mode with the "Execute after command", sou you can also test your release mode build (if you use e.g. different optimizations, some yet unnoticed bug may crop up.) -- Bart From juha.manninen62 at gmail.com Mon Dec 13 23:16:47 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Tue, 14 Dec 2021 00:16:47 +0200 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? In-Reply-To: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> Message-ID: On Mon, Dec 13, 2021 at 10:01 PM Bo Berglund via lazarus < lazarus at lists.lazarus-ide.org> wrote: > ... the binary will be added to version control ... You are misusing a version (or revision) control system, Subversion in your case if I understand right. Generated binaries should not be added there. It is only for source files. A revision control system stores diffs between revisions. A diff between 2 binaries is not useful information. You can share the binaries through a file system, FTP server, web page or something like that. Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bo.berglund at gmail.com Tue Dec 14 09:29:54 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Tue, 14 Dec 2021 09:29:54 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: On Mon, 13 Dec 2021 22:44:16 +0100, Bart via lazarus wrote: >On Mon, Dec 13, 2021 at 10:29 PM Bo Berglund via lazarus > wrote: > > >> This sounds like what I need. >> I looked at the project options and found that there is one box to enter the >> Execute after command. And checkboxes to set when it is used. >> >> But I would need *two* commands: >> - copy to the svn location >> - strip -s on that copy >> > >In compiler options you can set various debugging related options as well. >One is to strip symbols. >- uncheck: "generate infor for the debugger" >- check: Strip symbols from executable >You apply those debugging options to your "Release" build mode. >You keep all relevant debugging options checked for your "Debug" build-mode. > >Now you develop in "Debug" build-mode. >Once you're ready to release: swicth build-mode to "Release" and compile. > >(You might want a "Release and copy to svn" build mode with the >"Execute after command", sou you can also test your release mode build >(if you use e.g. different optimizations, some yet unnoticed bug may >crop up.) > Thanks, I went there and found where I could add the release mode. I did not even need to enter a copy command because I could set the "Target filename" for Release to include the subdirectory bin. So now my Default mode saves the executable to the project dir and the Release saves to subdir bin. This is exactly what I needed, thanks for explaining it! :) -- Bo Berglund Developer in Sweden From d.ioannidis at nephelae.eu Tue Dec 14 10:32:25 2021 From: d.ioannidis at nephelae.eu (d.ioannidis at nephelae.eu) Date: Tue, 14 Dec 2021 11:32:25 +0200 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? In-Reply-To: References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: <297db4e677eb2ac2458a6531d26d0b98@nephelae.eu> On 2021-12-14 10:29, Bo Berglund via lazarus wrote: > On Mon, 13 Dec 2021 22:44:16 +0100, Bart via lazarus < snip > >> (You might want a "Release and copy to svn" build mode with the >> "Execute after command", sou you can also test your release mode build >> (if you use e.g. different optimizations, some yet unnoticed bug may >> crop up.) >> > > Thanks, I went there and found where I could add the release mode. > I did not even need to enter a copy command because I could set the > "Target > filename" for Release to include the subdirectory bin. > So now my Default mode saves the executable to the project dir and the > Release > saves to subdir bin. > This is exactly what I needed, thanks for explaining it! :) < snip > Just FYI, I usually put in the "Target filname" this : bin\$(TargetCPU)-$(TargetOS)\$(BuildMode)\projectname and in the "Unit output directory" this : lib\$(TargetCPU)-$(TargetOS)\$(BuildMode) The result is that I have a dir structure dependent on OS, CPU and BuildMode values . i.e. CPU=i386 OS=win32 BuildMode=debug bin\i386-win32\debug with lib\i386-win32\debug CPU=i386 OS=win32 BuildMode=release bin\i386-win32\release with lib\i386-win32\release CPU=x86_64 OS=win64 BuildMode=test_for_scm bin\x86_64-win64\test_for_scm with lib\x86_64-win64\test_for_scm etc regards, -- Dimitrios Chr. Ioannidis From wkitty42 at windstream.net Tue Dec 14 10:49:08 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Tue, 14 Dec 2021 04:49:08 -0500 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? In-Reply-To: References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: <8cd88dc3-242c-9a02-f044-37451320d484@windstream.net> On 12/14/21 3:29 AM, Bo Berglund via lazarus wrote: > This is exactly what I needed, thanks for explaining it! :) don't you love it when a plan comes together? :) -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From bo.berglund at gmail.com Tue Dec 14 11:55:45 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Tue, 14 Dec 2021 11:55:45 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: On Tue, 14 Dec 2021 09:29:54 +0100, Bo Berglund via lazarus wrote: >Thanks, I went there and found where I could add the release mode. Forgot to say this about my modes creation: When I only had a Default mode then when I clicked the [..] button there was a button to the upper left that says "Create Debug and Release modes", which I used rather than the + button to add a Release mode. This resulted in two new modes and I could set the Release one as described before to my liking. But when I tested the new Debug and Release modes the Debug one worked in a different way from the Default mode in that it creates two files on compile, app.exe and app.dbg. The exe file in this case is not very much different from the release exe in size, only some 10:s of kb. But the app.dbg file (wich was not present before) is quite big, larger than the exe file. Since I did not know what to do about that I removed the Debug mode from the list and use Default when writing code and test run, then switch to Release to make the final output. Question1: ---------- What did Lazarus do to the Debug mode it created automatically? Next: I also tested on a RaspberryPi project on an RPi3B (using 2.0.12 + 3.2.0): In this case I have a service application (non-GUI app), which I basically never run in the Lazarus debugger. Its mode has never been dealt with so it was the out-of-the-box default. Now I added the Release mode by just clicking the + button and naming it Release. Then I did as before and disbled the "Generate info for the debugger" item. I also checked the "Strip symbols from executable" box. The result was a bit surprising: Using Default Binary size: 9,572,660 after strip: 4,547,712 Using Release: Binary size: 6,311,022 after strip: 6,311,020 So in this case the original binary size was a lot larger than the stripped result from the Default mode, why was this? And every time I now switch mode for compliling my app it also recompiles the Indy10 package, why? Question 2: ----------- Does my action to create the Release mode as a copy of Default using the + button affect the way the Release mode is actually configured? On my Windows project discussed earlier I used the "Create Debug and Release modes" button and then deleted the Debug mode and modified the Release a little concerning the debug and strip. And on RPi I used the + button instead, but did the same config on debug settings. I will need to document this behaviour and note the "correct" way to do it... -- Bo Berglund Developer in Sweden From bo.berglund at gmail.com Tue Dec 14 13:04:51 2021 From: bo.berglund at gmail.com (Bo Berglund) Date: Tue, 14 Dec 2021 13:04:51 +0100 Subject: [Lazarus] Can I configure Lazarus (or the project) to only add symbols on debug? References: <879frg9npei73sr0s5n28pbnti8vcfqgpj@4ax.com> <92adac4d-37bd-fce3-6060-14c81d4599ed@nephelae.eu> Message-ID: <8p1hrglkcjmbvpf4a63cf0peuuej69jhr5@4ax.com> On Tue, 14 Dec 2021 11:55:45 +0100, Bo Berglund via lazarus wrote: >I also tested on a RaspberryPi project on an RPi3B (using 2.0.12 + 3.2.0): >In this case I have a service application (non-GUI app), which I basically never >run in the Lazarus debugger. Its mode has never been dealt with so it was the >out-of-the-box default. >Now I added the Release mode by just clicking the + button and naming it >Release. >Then I did as before and disbled the "Generate info for the debugger" item. >I also checked the "Strip symbols from executable" box. > >The result was a bit surprising: >Using Default >Binary size: 9,572,660 after strip: 4,547,712 > >Using Release: >Binary size: 6,311,022 after strip: 6,311,020 > >So in this case the original binary size was a lot larger than the stripped >result from the Default mode, why was this? > >And every time I now switch mode for compliling my app it also recompiles the >Indy10 package, why? I went back to he RPi and removed the Release mode I created from the + button, then used the "Create Debug and Release modes" button to create two new modes. Deleted the Debug mode and configured Release as described above. Now my compile results in a file of size 3,251,612 bytes and when going back to the Default mode it is (after recompiling the Indy10 package) 9,572,660 Strange... -- Bo Berglund Developer in Sweden From sysrpl at gmail.com Wed Dec 15 09:40:53 2021 From: sysrpl at gmail.com (Anthony Walter) Date: Wed, 15 Dec 2021 03:40:53 -0500 Subject: [Lazarus] Programming Mechanical Simulations Message-ID: I wanted to share this . I can't stop writing code. I was inspired to design a program that can convert any SVG drawing into a simulation of a physical machine. I am just getting started writing such a program, but have had some promising initial results. See the link at the top of this message for context. -------------- next part -------------- An HTML attachment was scrubbed... URL: From christo.crause at gmail.com Wed Dec 15 10:58:57 2021 From: christo.crause at gmail.com (Christo Crause) Date: Wed, 15 Dec 2021 11:58:57 +0200 Subject: [Lazarus] Programming Mechanical Simulations In-Reply-To: References: Message-ID: On Wed, Dec 15, 2021 at 10:41 AM Anthony Walter via lazarus < lazarus at lists.lazarus-ide.org> wrote: > I wanted to share this > . > > I can't stop writing code. I was inspired to design a program that can > convert any SVG drawing into a simulation of a physical machine. I am just > getting started writing such a program, but have had some promising initial > results. See the link at the top of this message for context. > Well done, looks impressive! -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.ioannidis at nephelae.eu Wed Dec 15 11:56:58 2021 From: d.ioannidis at nephelae.eu (d.ioannidis at nephelae.eu) Date: Wed, 15 Dec 2021 12:56:58 +0200 Subject: [Lazarus] Source file license Message-ID: Hi, currently I'm writing a utility which I would like to be included in Lazarus and / or FPC ( if it accepted of course ). Usually I license my open source software under MIT license ( see PS ). After reading some forum threads regarding licenses I am a little lost . So I ask, what licenses are accepted by the Lazarus project ? MIT ? GPL v2 ? GPL v3 ? regards, PS: This MIT license is what I usually use : Inside units : Licensed under the MIT License (MIT). See licence file in root directory. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. And in the project root a license file : MIT License Copyright (c) 2021 Dimitrios Chr. Ioannidis Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -- Dimitrios Chr. Ioannidis From michael at freepascal.org Wed Dec 15 12:17:21 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Wed, 15 Dec 2021 12:17:21 +0100 (CET) Subject: [Lazarus] Source file license In-Reply-To: References: Message-ID: On Wed, 15 Dec 2021, Dimitrios Chr. Ioannidis via lazarus wrote: > Hi, > > currently I'm writing a utility which I would like to be included in > Lazarus and / or FPC ( if it accepted of course ). > > Usually I license my open source software under MIT license ( see PS > ). After reading some forum threads regarding licenses I am a little > lost . So I ask, what licenses are accepted by the Lazarus project ? > > MIT ? GPL v2 ? GPL v3 ? For inclusion in FPC we require the FPC license, which is essentially LGPL V2 with linking exception. Michael. From juha.manninen62 at gmail.com Wed Dec 15 12:53:07 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Wed, 15 Dec 2021 13:53:07 +0200 Subject: [Lazarus] Source file license In-Reply-To: References: Message-ID: On Wed, Dec 15, 2021 at 12:57 PM Dimitrios Chr. Ioannidis via lazarus < lazarus at lists.lazarus-ide.org> wrote: > what licenses are accepted by the Lazarus project ? > MIT ? GPL v2 ? GPL v3 ? > The Lazarus IDE itself is GPL v2. Most packages are LGPL v2. If your utility is to be installed into the IDE, LGPL v2 is the best. Executables outside the IDE can have other licenses but GPL or LGPL is still recommended. Regards, Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From lazarus at mfriebe.de Wed Dec 15 13:24:27 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Wed, 15 Dec 2021 13:24:27 +0100 Subject: [Lazarus] Source file license In-Reply-To: References: Message-ID: <941c574e-120f-b796-016b-fa20d44b64a3@mfriebe.de> On 15/12/2021 12:53, Juha Manninen via lazarus wrote: > On Wed, Dec 15, 2021 at 12:57 PM Dimitrios Chr. Ioannidis via lazarus > wrote: > > what licenses are accepted by the Lazarus project ? >    MIT ? GPL v2 ? GPL v3 ? > > > The Lazarus IDE itself is GPL v2. Most packages are LGPL v2. > If your utility is to be installed into the IDE, LGPL v2 is the best. > Executables outside the IDE can have other licenses but GPL or LGPL is > still recommended. > IMHO: It is also acceptable to have a "mixed license". E.g.    Licensed [L]GPL 2 [OR: "2 or later"] or MIT at the users choice.    Anyone using/modifying this code must keep at least either one of the two Licenses, but is free to remove the other. We inherited a case like this in SynEdit. You can check there how exactly it is described. ----- As Juha said: - If it is an IDE plugin, to be compiled into the IDE - and to be included in our repo / shipped with the installer then it should really include/be [L]GPL at least including version 2. If it is to be run from the IDE, lazbuild, our makefiles, or the like, then it should be permissive enough not to burden us with displaying special notes. Of course a mention in the "about" dialog can be required. (not sure if Lazbuild currently has any provisions to print such things as "attribution notes". ----- As with all things that evolve over time, we already have to many Licenses, on different parts of the code. This imposes extra work on users, who have to check what the can use in their projects. By including [L]GPL 2 (with/without the LCL linking permission) a common base would be kept. Additional/Optional Licenses should be no issue. -------------- next part -------------- An HTML attachment was scrubbed... URL: From m.e.sanliturk at gmail.com Wed Dec 15 13:54:58 2021 From: m.e.sanliturk at gmail.com (Mehmet Erol Sanliturk) Date: Wed, 15 Dec 2021 15:54:58 +0300 Subject: [Lazarus] Source file license In-Reply-To: <941c574e-120f-b796-016b-fa20d44b64a3@mfriebe.de> References: <941c574e-120f-b796-016b-fa20d44b64a3@mfriebe.de> Message-ID: On Wed, Dec 15, 2021 at 3:24 PM Martin Frb via lazarus < lazarus at lists.lazarus-ide.org> wrote: > On 15/12/2021 12:53, Juha Manninen via lazarus wrote: > > On Wed, Dec 15, 2021 at 12:57 PM Dimitrios Chr. Ioannidis via lazarus < > lazarus at lists.lazarus-ide.org> wrote: > >> what licenses are accepted by the Lazarus project ? >> MIT ? GPL v2 ? GPL v3 ? >> > > The Lazarus IDE itself is GPL v2. Most packages are LGPL v2. > If your utility is to be installed into the IDE, LGPL v2 is the best. > Executables outside the IDE can have other licenses but GPL or LGPL is > still recommended. > > IMHO: > > It is also acceptable to have a "mixed license". E.g. > Licensed [L]GPL 2 [OR: "2 or later"] or MIT at the users choice. > Anyone using/modifying this code must keep at least either one of the > two Licenses, but is free to remove the other. > > We inherited a case like this in SynEdit. You can check there how exactly > it is described. > > LGPL is a Copy-left , MIT is a Permissive license . Use of such a two licenses will be more suitable for ( real or legal ) persons to select one most suitable for their needs . In turn , this will increase the number of users of the related software . With my best wishes , Mehmet Erol Sanliturk > ----- > As Juha said: > - If it is an IDE plugin, to be compiled into the IDE > - and to be included in our repo / shipped with the installer > then it should really include/be [L]GPL at least including version 2. > > If it is to be run from the IDE, lazbuild, our makefiles, or the like, > then it should be permissive enough not to burden us with displaying > special notes. > Of course a mention in the "about" dialog can be required. (not sure if > Lazbuild currently has any provisions to print such things as "attribution > notes". > > ----- > As with all things that evolve over time, we already have to many > Licenses, on different parts of the code. > > This imposes extra work on users, who have to check what the can use in > their projects. > By including [L]GPL 2 (with/without the LCL linking permission) a common > base would be kept. > Additional/Optional Licenses should be no issue. > > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juha.manninen62 at gmail.com Wed Dec 15 14:57:22 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Wed, 15 Dec 2021 15:57:22 +0200 Subject: [Lazarus] Source file license In-Reply-To: <941c574e-120f-b796-016b-fa20d44b64a3@mfriebe.de> References: <941c574e-120f-b796-016b-fa20d44b64a3@mfriebe.de> Message-ID: On Wed, Dec 15, 2021 at 2:24 PM Martin Frb via lazarus < lazarus at lists.lazarus-ide.org> wrote: > It is also acceptable to have a "mixed license". E.g. > Licensed [L]GPL 2 [OR: "2 or later"] or MIT at the users choice. > Anyone using/modifying this code must keep at least either one of the > two Licenses, but is free to remove the other. > Yes, a dual license. I forgot to mention that. Lazarus sources include a fork of the Jedi Code Format package which has a dual license: Mozilla Public License v1.1 or GNU General Public License v2 Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From christo.crause at gmail.com Thu Dec 16 12:10:03 2021 From: christo.crause at gmail.com (Christo Crause) Date: Thu, 16 Dec 2021 13:10:03 +0200 Subject: [Lazarus] Making fpdebug more cross-platform friendly Message-ID: Currently there are a few instances where x86 specific assumptions/code are used [1] in what I consider as the target agnostic level of fpdebug. My proposal is to move these instances to a hardware target specific unit so that these target specific handling is clearly separated. One location for the target specific code is perhaps the fpdbgdisas* units, since it already provides a number of hardware specific concepts such as identifying call & return instructions and some stack frame analysis. Adding for example IsSoftwareBreakInstruction seems like a logical addition. These changes are hindering some functionality on AVR and Xtensa targets, e.g. some instructions are incorrectly disassembled because they start with $CC, which is incorrectly interpreted as a software break. Also the StepOut functionality currently assumes the x86 ABI to locate the return address, which obviously fails for other targets. While these methods could in principle be hidden by new target specific overrides, it is not aligned with proper OOP principles. I will start with some concepts to address these soon. Similarly, but possibly a bit more tricky, is to separate the OS code from the OS view of the hardware. This would imply for example that the register handling in fpdbglinuxclasses needs to be separated from the rest of the code so that one could re-use the Linux layer, but swap out the hardware specific layer x86_Linux. Similarly for Windows, the WOW64_CONTEXT could be part of an x86_Windows unit, so that it is possible to also define and use _ARM64_NT_CONTEXT for Windows on ARM64. Any thoughts, critique or suggestions are welcome. Best wishes, Christo [1] Examples of x86 specific code not in a x86 specific unit fpdbgclasses.pas: checking for int3 and its x86 encoding in TDbgProcess, TDbgThread and TBreakLocationMap fpdbgcontroller.pas: TDbgControllerCallRoutineCmd uses x86 call specific encoding in InsertCallInstructionCode and RestoreInstructionPointer, StoreRoutineResult using x86 register specific assumptions, TDbgControllerStepOutCmd.SetReturnAdressBreakpoint assumes return address is stored according to x86 stack frame layout. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lazarus at mfriebe.de Thu Dec 16 13:18:34 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Thu, 16 Dec 2021 13:18:34 +0100 Subject: [Lazarus] Making fpdebug more cross-platform friendly In-Reply-To: References: Message-ID: On 16/12/2021 12:10, Christo Crause via lazarus wrote: >   One location for the target specific code is perhaps the fpdbgdisas* > units, since it already provides a number of hardware specific > concepts such as identifying call & return instructions and some stack > frame analysis.  Adding for example IsSoftwareBreakInstruction seems > like a logical addition. The current Frame-analysis is not really in good shape. It is more likely that the central entry point for hardware and/or OS specifics will be the subclassed TDbgProcess (or potentially in some cases the subclassed Thread, but I can't think of a case for that....). The process class can then decide to use the asm class, or have other means... Dividing this between hardware and OS will be a story of its own. > > These changes are hindering some functionality on AVR and Xtensa > targets, e.g. some instructions are incorrectly disassembled because > they start with $CC, which is incorrectly  interpreted as a software > break. So that needs to be controlled by the process class then. > Also the StepOut functionality currently assumes the x86 ABI to locate > the return address, which obviously fails for other targets.  While > these methods could in principle be hidden by new target specific > overrides, it is not aligned with proper OOP principles.  I will start > with some concepts to address these soon. Probably there will be some need for delegate classes.... Btw, this is a big one, and I have some todo on this too. Currently the entire handling of win SEH is in LazDebuggerFp => that should move. But it needs exactly the above ability of having target specific functions. Even one step further: => those are not even default for win, because they assume fpc generated code [1]. So they need to be added, similar like the fpc additions for dwarf (separate unit, with detection of compiler....) So it's likely they will undergo a big rework. [1] They should be fixed to be generic. > > Similarly, but possibly a bit more tricky, is to separate the OS code > from the OS view of the hardware.  This would imply for example that > the register handling in fpdbglinuxclasses needs to be separated from > the rest of the code so that one could re-use the Linux layer, but > swap out the hardware specific layer x86_Linux.  Similarly for > Windows, the WOW64_CONTEXT could be part of an x86_Windows unit, so > that it is possible to also define and use _ARM64_NT_CONTEXT for > Windows on ARM64. > > [1] Examples of x86 specific code not in a x86 specific unit > fpdbgclasses.pas: checking for int3 and its x86 encoding in > TDbgProcess, TDbgThread and TBreakLocationMap The above will probably best go into a delegation class. But the extend needs to be evaluated. The first 2 actually should allow to override (so that is not the desired solution)... > fpdbgcontroller.pas: TDbgControllerCallRoutineCmd uses x86 call > specific encoding in InsertCallInstructionCode and > RestoreInstructionPointer, StoreRoutineResult using x86 register > specific assumptions, > TDbgControllerStepOutCmd.SetReturnAdressBreakpoint assumes return > address is stored according to x86 stack frame layout. As I said, there will have to be a big rework. Until then, quickfix it by overriding it in the backend (like LazDebuggerFp does). More later.... From lazarus at mfriebe.de Thu Dec 16 23:26:57 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Thu, 16 Dec 2021 23:26:57 +0100 Subject: [Lazarus] Making fpdebug more cross-platform friendly In-Reply-To: References: Message-ID: <3ab0c248-1b26-33b4-2145-61c116ef53e4@mfriebe.de> Ok, for now limiting this to a smaller audience, to get some brainstorming done. => Original mail at the bottom. Some initial thoughts. *** First, I would suggest, that all calls go to T[Foo]DbgProccess *** T[Foo]DbgProccess can then forward them. Outside classes should not have knowledge, nor access to the delegates. (Better start restricted, so we have more flexibility for future changes) ---------- I can currently see the following areas, that could be separated as needed. API:  Calls to the OS => start/stop/mem-access/register/access ABI:  Conventions about register usage, and stack layout         (registers are known by either name or dwarf-index) CPU: Identifying specific asm instructions  (Breakpoint / ret / call ....) IHMO: - API remains hardcoded in T[Foo]DbgProccess. - The others are private delegate classes. - Your idea about WOW64_CONTEXT / _ARM64_NT_CONTEXT is imho API.    It can not be CPU or ABI. If it were, we would need to subclass them for each API. - Identifying breakpoint is CPU (not necessarily the current asm class / could be a new class) - unwinding stack => could be either. (there may be an API of the OS, otherwise ABI, maybe with help of CPU) Only, question is, if we need the ABI? - Do win64 and linux64 share the same ABI? (or win32 <> linux32)?   Afaik SEH is win specific (and probably would be best of in the ABI?).   But there could still be a base class that is shared between Win/Linux. Similar about ABI vs CPU. Only here we definitely have a shared base class for inter 32/64bit (currently it's only one class / no subclasses for the bitness). So IMHO there is a point in splitting them. ------- Mind that, FPC specific units could provide specialized sub-classes of any of the delegate classes. E.g. dealing with $fin$MyProcedure subroutines. (Win-SEH) For some purposes, finally blocks need to look like they are part of the main-procedure, not sub-routines of their own. ------- With the above some of the issues can be solved by combining different subclasses of each. And then we would end up with combinations like this: (I am not sure, if the ABI for Win and Linux are actually different..., but there would be win-seh) TDbgWinProcess with:    x64-win ABI     /  x64 CPU TDbgWinProcess with:    i386-win-ABI    /  x32 CPU TDbgWinProcess with:    arm-ABI    /  arm CPU TDbgLinuxProcess with:    x64-linux-ABI    /  x64 CPU TDbgLinuxProcess with:    i386-linux-ABI    /  x32 CPU ... There is potential for some re-usable code there. Since all access would be through T[Foo]DbgProccess code can be moved gradually. ------- However, setting Breakpoints for example may be a bit more complex. For example: - if there is a limit to the amount, or only certain locations can be set. - If breakpoints are not set to memory, but like watchpoints on intel. In those cases, it needs to be decided how much power to give to the CPU class. Should the CPU class be able to make calls to read/write mem and/or change registers? (That is the point, where it would be a separate class from the asm class, which at most reads mem) Even, if the CPU class can handle it all by itself, it may still end up needing support in the main class (T[Foo]DbgProccess). - There can be errors due to restrictions (user watch points have been set to the register needed for the breakpoint). - There may be different requirement on the relative timing of those actions and other actions outside the CPU class.   Those timings are not known by the CPU class. So we end up with a tighter coupling than desired. ---------- Comments? Ideas? On 16/12/2021 12:10, Christo Crause via lazarus wrote: > Currently there are a few instances where x86 specific > assumptions/code are used [1] in what I consider as the target > agnostic level of fpdebug.  My proposal is to move these instances to > a hardware target specific unit so that these target specific handling > is clearly separated.  One location for the target specific code is > perhaps the fpdbgdisas* units, since it already provides a number of > hardware specific concepts such as identifying call & return > instructions and some stack frame analysis.  Adding for example > IsSoftwareBreakInstruction seems like a logical addition. > > These changes are hindering some functionality on AVR and Xtensa > targets, e.g. some instructions are incorrectly disassembled because > they start with $CC, which is incorrectly  interpreted as a software > break.  Also the StepOut functionality currently assumes the x86 ABI > to locate the return address, which obviously fails for other targets. > While these methods could in principle be hidden by new target > specific overrides, it is not aligned with proper OOP principles.  I > will start with some concepts to address these soon. > > Similarly, but possibly a bit more tricky, is to separate the OS code > from the OS view of the hardware.  This would imply for example that > the register handling in fpdbglinuxclasses needs to be separated from > the rest of the code so that one could re-use the Linux layer, but > swap out the hardware specific layer x86_Linux.  Similarly for > Windows, the WOW64_CONTEXT could be part of an x86_Windows unit, so > that it is possible to also define and use _ARM64_NT_CONTEXT for > Windows on ARM64. > > Any thoughts, critique or suggestions are welcome. > > Best wishes, > Christo > > [1] Examples of x86 specific code not in a x86 specific unit > fpdbgclasses.pas: checking for int3 and its x86 encoding in > TDbgProcess, TDbgThread and TBreakLocationMap > fpdbgcontroller.pas: TDbgControllerCallRoutineCmd uses x86 call > specific encoding in InsertCallInstructionCode and > RestoreInstructionPointer, StoreRoutineResult using x86 register > specific assumptions, > TDbgControllerStepOutCmd.SetReturnAdressBreakpoint assumes return > address is stored according to x86 stack frame layout. > From pascaldragon at googlemail.com Fri Dec 17 08:02:55 2021 From: pascaldragon at googlemail.com (Sven Barth) Date: Fri, 17 Dec 2021 08:02:55 +0100 Subject: [Lazarus] Making fpdebug more cross-platform friendly In-Reply-To: <3ab0c248-1b26-33b4-2145-61c116ef53e4@mfriebe.de> References: <3ab0c248-1b26-33b4-2145-61c116ef53e4@mfriebe.de> Message-ID: Martin Frb via lazarus schrieb am Do., 16. Dez. 2021, 23:27: > Only, question is, if we need the ABI? > - Do win64 and linux64 share the same ABI? (or win32 <> linux32)? > Afaik SEH is win specific (and probably would be best of in the ABI?). > But there could still be a base class that is shared between Win/Linux. > Yes, SEH is Windows specific. The ABI is mostly CPU specific however. E.g. all the i386 calling conventions can be used on all OSes. On the x86_64 systems one usually uses either the SysV or the Win64 ABI, but both FPC and Clang allow the use of the other calling convention as well. On the other hand the register usage on m68k for example differs between Amiga and non-Amiga (the stack pointer is a different register and the stack alignment is 2 instead of 4 Byte) and this can't be changed. So in my opinion the design should allow for as much code sharing/reuse as possible, e.g. with a general x86_64 ABI with OS specifics (e.g. SEH) on top. Regards, Sven > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kadissov.e at gmail.com Mon Dec 20 08:46:45 2021 From: kadissov.e at gmail.com (=?UTF-8?B?0JXQstCz0LXQvdC40Lkg0JrQsNC00LjRgdC+0LI=?=) Date: Mon, 20 Dec 2021 10:46:45 +0300 Subject: [Lazarus] Question Message-ID: Dear lazarus programmers. After the change of the Lazarus version a command "find" fails when it must find a word written with russian letters. What must I do? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkitty42 at windstream.net Mon Dec 20 11:58:41 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Mon, 20 Dec 2021 05:58:41 -0500 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: On 12/20/21 2:46 AM, Евгений Кадисов via lazarus wrote: > Dear lazarus programmers. After the change of the Lazarus version a command > "find" fails when it must find a word written with russian letters. What must I do? what must you do? maybe provide a little more information? what version of laz were you using? are you using UTF-8 or something else? what format are the russian glyphs written in? -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From kadissov.e at gmail.com Mon Dec 20 12:35:52 2021 From: kadissov.e at gmail.com (=?UTF-8?B?0JXQstCz0LXQvdC40Lkg0JrQsNC00LjRgdC+0LI=?=) Date: Mon, 20 Dec 2021 14:35:52 +0300 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: Thank you. It seems I have found the reason. пн, 20 дек. 2021 г. в 13:58, wkitty42--- via lazarus < lazarus at lists.lazarus-ide.org>: > On 12/20/21 2:46 AM, Евгений Кадисов via lazarus wrote: > > Dear lazarus programmers. After the change of the Lazarus version a > command > > "find" fails when it must find a word written with russian letters. What > must I do? > > > what must you do? maybe provide a little more information? > > what version of laz were you using? > are you using UTF-8 or something else? > what format are the russian glyphs written in? > > > -- > NOTE: No off-list assistance is given without prior approval. > *Please keep mailing list traffic on the list where it belongs!* > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkitty42 at windstream.net Mon Dec 20 16:21:09 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Mon, 20 Dec 2021 10:21:09 -0500 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: On 12/20/21 6:35 AM, Евгений Кадисов via lazarus wrote: > Thank you. It seems I have found the reason. ok... and the reason/solution is/was? i ask so that others might find it if they have similar problems... > пн, 20 дек. 2021 г. в 13:58, wkitty42--- via lazarus > >: > > On 12/20/21 2:46 AM, Евгений Кадисов via lazarus wrote: > > Dear lazarus programmers. After the change of the Lazarus version a command > > "find" fails when it must find a word written with russian letters. What > must I do? > > > what must you do? maybe provide a little more information? > > what version of laz were you using? > are you using UTF-8 or something else? > what format are the russian glyphs written in? -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From kadissov.e at gmail.com Tue Dec 21 09:15:28 2021 From: kadissov.e at gmail.com (=?UTF-8?B?0JXQstCz0LXQvdC40Lkg0JrQsNC00LjRgdC+0LI=?=) Date: Tue, 21 Dec 2021 11:15:28 +0300 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: Thank you once more. It turned out that the problem was not solved. When I have installed new version of Lazarus (2.0.12 instead of 2.0.10), my program failed to work. I want to return old version of Lazrus but I do not know how to do it. пн, 20 дек. 2021 г. в 18:21, wkitty42--- via lazarus < lazarus at lists.lazarus-ide.org>: > On 12/20/21 6:35 AM, Евгений Кадисов via lazarus wrote: > > Thank you. It seems I have found the reason. > > ok... and the reason/solution is/was? i ask so that others might find it > if they > have similar problems... > > > пн, 20 дек. 2021 г. в 13:58, wkitty42--- via lazarus > > >: > > > > On 12/20/21 2:46 AM, Евгений Кадисов via lazarus wrote: > > > Dear lazarus programmers. After the change of the Lazarus version > a command > > > "find" fails when it must find a word written with russian > letters. What > > must I do? > > > > > > what must you do? maybe provide a little more information? > > работа со строками lazarus > > what version of laz were you using? > > are you using UTF-8 or something else? > > what format are the russian glyphs written in? > > > -- > NOTE: No off-list assistance is given without prior approval. > *Please keep mailing list traffic on the list where it belongs!* > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jmlandmesser at gmx.de Tue Dec 21 11:12:00 2021 From: jmlandmesser at gmx.de (John Landmesser) Date: Tue, 21 Dec 2021 11:12:00 +0100 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: <557e85b4-98dd-8d87-828e-fbee41aa52fa@gmx.de> You find old versions here: https://sourceforge.net/projects/lazarus/files/ Am 21.12.21 um 09:15 schrieb Евгений Кадисов via lazarus: > Thank you once more. It turned out that the problem was not solved. > When I have installed new version of Lazarus (2.0.12 instead of > 2.0.10), my program failed to work. I want to return old version of > Lazrus but I do not know how to do it. > > пн, 20 дек. 2021 г. в 18:21, wkitty42--- via lazarus > : > > On 12/20/21 6:35 AM, Евгений Кадисов via lazarus wrote: > > Thank you. It seems I have found the reason. > > ok... and the reason/solution is/was? i ask so that others might > find it if they > have similar problems... > > > пн, 20 дек. 2021 г. в 13:58, wkitty42--- via lazarus > > >: > > > >     On 12/20/21 2:46 AM, Евгений Кадисов via lazarus wrote: > >      > Dear lazarus programmers. After the change of the Lazarus > version a command > >      > "find" fails when it must find a word written with > russian letters. What > >     must I do? > > > > > >     what must you do? maybe provide a little more information? > > работа со строками lazarus > >     what version of laz were you using? > >     are you using UTF-8 or something else? > >     what format are the russian glyphs written in? > > > -- >   NOTE: No off-list assistance is given without prior approval. >         *Please keep mailing list traffic on the list where it > belongs!* > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wkitty42 at windstream.net Tue Dec 21 14:28:21 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Tue, 21 Dec 2021 08:28:21 -0500 Subject: [Lazarus] Question In-Reply-To: References: Message-ID: <5425e337-83a2-0d11-07bf-09defeabc29e@windstream.net> On 12/21/21 3:15 AM, Евгений Кадисов via lazarus wrote: > Thank you once more. It turned out that the problem was not solved. When I have > (2.0.12 instead of 2.0.10) [...] perhaps looking at the changelogs for the differences between these two versions will help you find what you need to change in your program so it will work properly for your expected output? i've spent about 15 minutes looking around so i might provide you a link to the changelogs but i'm unable to easily find them... sadly, the changes to the debian/changelog file provide no real information other than the version number being changed... so i've dug deeper... looking on the lazarus SF site, i was finally able to find the commits for those two versions... with this information, we should be able diff them to find the difference(s) that may be causing your program troubles... ae9017 2.0.10 871307 2.0.12 if you have a full git repository, you can diff these two with git diff ae9017..871307 however, the following command will list all of the commits between ae9017 and 871307, inclusive... this may be an easier route to go to find what commit has lead to your situation... git rev-list --ancestry-path ae9017^..871307 this gives me, on my fresh clone, the following raw commit hash list of 76 commits... they are listed from newer to older... the bottom one is the one that set the version to 2.0.10 (where you were) and the top one is the one that set the version to 2.0.12 (where you ended up)... 87130707e7b820ae7167900041f9ea21087f2015 9c41034dda9369ea03e24d1cc87d919a43ba5699 0501f61b9bd61bd0af43802cf6ae8478b4540e19 6a91ceb313ed2d98e5997ecbb44a354dd928fe5d 0fc756f057705612f85b619a09faa39074137ba2 128960ee5dc7b506f6085a32670e642e659a07d4 815e3dc6d67ed3b82cfed51d8e6ee68c19dbf996 48ffd9739cfee3aa6a2ca5fb0fd4fee7a6702459 f7aa0270f144dcf23dac499ca64cfc1d8cde32d9 17f21607562f81144161a19da7c7a57e1bc73dda 77ecaa085694f99f38422dbacd01cb366364c314 65acd97bcd3bf2c7f936a9dde4c66342a582f236 8ae7b2bb457f6e820d9830eae2dbe6b0fcc82be8 240db20a1df4b627eb08fcf4173e18157b739019 2961d09e9e1c037cdfccec76a497e676925ddc08 2c03a634c538289b4adb59b46a421e23dc97d92f 561d01f2e6e12099448bd1137ede5d2aadeb4df2 7c386df181505370de67a479a80783ff5fe8c4a1 9b78caaf21fae9724ed9f9114457b6efef969cc5 c00cb12cca382e57035586b67838feb44538e0ef ee6fa154ed0c2af13f82250d274eb14af058e1ba 19aad3af455164815b82d4bc37bdc2659cb30b72 77b7cb7d6171ba101a43957ec3d88d7eff7aa6b3 69c095d1e2adc7f7f9e5a697547f2c13a5f441e3 3e0f2f5e68b19313aa17f9d400368d7f390eb729 c476a94462550eaff34b932a47797798a0231523 8bf9b7f784eabf68bf9a40a87a34a93e49ed5f94 8a0f57666f8e6c687b6b1c7dfed535cbfd9cca72 2889401bd2fd5203e783c6dfdaaa5351bb10493b f29f662333bbd277b242d3f82d58b141257f26cc 770bee475c8fda95eec53dc82a5c35ea88351aff f464a500fc618514ab1a5dd24b2e95e70b57539f 36319f66d58c2b646db6b6a4d0e8d14167b80768 a6e6e910f0eec6862de16e6aab0fcfeffb3ac671 18ff8c47009eb874541e3bb69c0e2f54a80f4941 90cbe5eed994dd1743b0e36c8180894c633cc34a c3e8aa2edff6559abf45f43df5bde550b97d1200 3bb8fe51b51eca80430b07b6a7c3181659496e9e 5d06959c361fdea9b32a0823ebc8b3f7f18343f1 8ddcd102ee193cb4289ec2e2aa85a54e6c2bdf94 c910891f1b64865ca203a16636ce810809be7474 ebab00eb5b8ac37b2d3daf60d85318c187a17dbd 886ab5f015ba66d71cf24f7773195400f0bd10cd d6eb3f3a05bff9ec84e98b5a27f3609698997be1 f7c69baac6b2c088660674d178d682af744f1d0c 2838eb631edeb4bc450043e8a747b51227796b12 0c525d92226f4b22e0f77b35d7e18b22d433f193 7b2dec7cc7e6ae765ccf5fc750eb4fe874902b58 99021f5b35481decda4130b18d18375b34948e56 d7ef338ff6a6c709afd6f214828336034d787906 8e84940d5c2fbb4c498aa62cd95195fb00b26f10 b85943b8636152ca286cbaed38bbc605f8a87083 7382f9906d344397b37c8c583acc8842349006f1 b1e194c219563707d0fb161e6d71fdabb50abe75 036336f93dde2058b1c3afd6ad53a759b6cf9092 5a9fdda475102679a45bb8afa8be569864a4a356 ed18ef848631e0c5bbad38f5bf9ffef3a72ab5cd 7b87b59dda416573e4796d080027ffab0348dd5f e400755e2bf56c5dd6ddebca5270da61a1e6cb98 08002f607e7451f104b1e46f529c3bfec7cf9cee 80f42df3c7a5aefc348bccfc1b431de21c457fff 918ae6195764b5f060f7c891996f98d6d604bf28 9a2223b8b20f21a82fa60a8550cf006410e5e0ae d6c010f5a8c0c31ec1a1516589c1b92f69816a81 e544c76cd565c32fd329751e18951cd0b77f32e5 d4e099a4f3e962760f254210cdc5dd3bbd52811b bda8742a72427fc72a333cc145d9a20b5de7dfe2 764bbbc59c6db0564166e4d61f729501a0ee2870 efa348b3c513888855b79671efa7c8832f318838 53025b32f9736f37b3feb0967d4fc42b129738bf 0e06f0d9e1fa5b6f761d6bd4bbd4d3bf8f422b7a bee446707f7308b9c2772085512030cce92c89b1 90bde32386685bf9675c506c9479a054ceed4b63 7d81afac5da07c0c5a47503fe30682bd36e818cf eb4b5f979ecd00f9e0856090ee2855aa750dad21 ae9017d9a9b821dfed143169da8b59a0613335c8 assuming you are using UTF-8, i do find a few UTF-8 related commits and some of those are related to the version of FPC being used... so that brings the question of which FPC are you using behind your Lazarus? using this command line, i find these 5 commits which may lead you in the right direction... git log --pretty=oneline ae9017^..871307 | grep -Ei -e "UTF-?8" d6eb3f3a05bff9ec84e98b5a27f3609698997be1 TUTF8Process: workaround for the poNoConsole->poDetached change. Better hacks for old FPC versions without TProcessClassTemplate. Issue #35991 f7c69baac6b2c088660674d178d682af744f1d0c LazUtils: Dont override TProcess.Execute for TProcessUTF8 starting from FPC 3.2. Issue #35991 2838eb631edeb4bc450043e8a747b51227796b12 LazUtils: Delete very old code for TProcessUTF8 that was needed for FPC 2.x. 7b2dec7cc7e6ae765ccf5fc750eb4fe874902b58 lazutils: Try to minimize (NOT a fix) the risks of TProcessUtf8 crashes. Crash introduced in r50595 #d0dace2bf5 for Issue #28991 d7ef338ff6a6c709afd6f214828336034d787906 Merged revision(s) part of 63978 from trunk: FpDebug: fix opening files with utf8 name on windows. Issue #37884 ........ *NOTE:* i am not a git guru... the above commands were found via uncle google... i may be missing some commits in my list(s) above... but i think not when i look at the repo using a graphical git tool (giggle and gitk) that shows me the tree of commits in the Lazarus main (svn trunk?) branch... i can only hope that this helps somehow and made lead to a solution for you without having to try to drop back to an older version of Lazarus (and possibly FPC)... -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From wkitty42 at windstream.net Tue Dec 21 14:34:35 2021 From: wkitty42 at windstream.net (wkitty42 at windstream.net) Date: Tue, 21 Dec 2021 08:34:35 -0500 Subject: [Lazarus] Question In-Reply-To: <5425e337-83a2-0d11-07bf-09defeabc29e@windstream.net> References: <5425e337-83a2-0d11-07bf-09defeabc29e@windstream.net> Message-ID: <3c8637fb-da99-47cd-0bff-37d66dd90271@windstream.net> On 12/21/21 8:28 AM, wkitty42 at windstream.net wrote: >   git log --pretty=oneline ae9017^..871307 | grep -Ei -e "UTF-?8" i forgot to also add this list of commits that appear to be specific to various FPC versions used behind Lazarus or used to compile Lazarus... git log --pretty=oneline ae9017^..871307 | grep -Ei -e "fpc" c476a94462550eaff34b932a47797798a0231523 IDE: fixed compile with fpc 3.0.4 90cbe5eed994dd1743b0e36c8180894c633cc34a lazwiki: fixed compile with fpc 3.2 d6eb3f3a05bff9ec84e98b5a27f3609698997be1 TUTF8Process: workaround for the poNoConsole->poDetached change. Better hacks for old FPC versions without TProcessClassTemplate. Issue #35991 f7c69baac6b2c088660674d178d682af744f1d0c LazUtils: Dont override TProcess.Execute for TProcessUTF8 starting from FPC 3.2. Issue #35991 2838eb631edeb4bc450043e8a747b51227796b12 LazUtils: Delete very old code for TProcessUTF8 that was needed for FPC 2.x. 90bde32386685bf9675c506c9479a054ceed4b63 chmhelp: fixed compile fpc 3.0.4 -- NOTE: No off-list assistance is given without prior approval. *Please keep mailing list traffic on the list where it belongs!* From kadissov.e at gmail.com Wed Dec 22 05:56:52 2021 From: kadissov.e at gmail.com (=?UTF-8?B?0JXQstCz0LXQvdC40Lkg0JrQsNC00LjRgdC+0LI=?=) Date: Wed, 22 Dec 2021 07:56:52 +0300 Subject: [Lazarus] Question In-Reply-To: <3c8637fb-da99-47cd-0bff-37d66dd90271@windstream.net> References: <5425e337-83a2-0d11-07bf-09defeabc29e@windstream.net> <3c8637fb-da99-47cd-0bff-37d66dd90271@windstream.net> Message-ID: Thank you for your help. I have found the cause. In version 2.0.12 one can not use TStream.GetSize. Now my program works fine. вт, 21 дек. 2021 г. в 16:34, wkitty42--- via lazarus < lazarus at lists.lazarus-ide.org>: > On 12/21/21 8:28 AM, wkitty42 at windstream.net wrote: > > git log --pretty=oneline ae9017^..871307 | grep -Ei -e "UTF-?8" > > > i forgot to also add this list of commits that appear to be specific to > various > FPC versions used behind Lazarus or used to compile Lazarus... > > git log --pretty=oneline ae9017^..871307 | grep -Ei -e "fpc" > > c476a94462550eaff34b932a47797798a0231523 IDE: fixed compile with fpc > 3.0.4 > > 90cbe5eed994dd1743b0e36c8180894c633cc34a lazwiki: fixed compile with > fpc 3.2 > > d6eb3f3a05bff9ec84e98b5a27f3609698997be1 TUTF8Process: workaround for > the > poNoConsole->poDetached change. Better hacks for old FPC versions without > TProcessClassTemplate. Issue #35991 > > f7c69baac6b2c088660674d178d682af744f1d0c LazUtils: Dont override > TProcess.Execute for TProcessUTF8 starting from FPC 3.2. Issue #35991 > > 2838eb631edeb4bc450043e8a747b51227796b12 LazUtils: Delete very old code > for > TProcessUTF8 that was needed for FPC 2.x. > > 90bde32386685bf9675c506c9479a054ceed4b63 chmhelp: fixed compile fpc > 3.0.4 > > > > > -- > NOTE: No off-list assistance is given without prior approval. > *Please keep mailing list traffic on the list where it belongs!* > -- > _______________________________________________ > lazarus mailing list > lazarus at lists.lazarus-ide.org > https://lists.lazarus-ide.org/listinfo/lazarus > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mlnglsts at bgss.hu Wed Dec 22 20:01:21 2021 From: mlnglsts at bgss.hu (Gabor Boros) Date: Wed, 22 Dec 2021 20:01:21 +0100 Subject: [Lazarus] Codetools - "Error: identifier not found: Boolean8" Message-ID: <913ba0dc-9557-2a8b-1131-90f3938ecce8@bgss.hu> Hi All, FPC main and Lazarus fixes_2_2 or main. If double click on the new project's main form got "identifier not found: Boolean8". Is this a Lazarus or FPC problem? Gabor From michael at freepascal.org Wed Dec 22 22:59:00 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Wed, 22 Dec 2021 22:59:00 +0100 (CET) Subject: [Lazarus] Codetools - "Error: identifier not found: Boolean8" In-Reply-To: <913ba0dc-9557-2a8b-1131-90f3938ecce8@bgss.hu> References: <913ba0dc-9557-2a8b-1131-90f3938ecce8@bgss.hu> Message-ID: On Wed, 22 Dec 2021, Gabor Boros via lazarus wrote: > Hi All, > > FPC main and Lazarus fixes_2_2 or main. If double click on the new > project's main form got "identifier not found: Boolean8". > > Is this a Lazarus or FPC problem? Lazarus code tools issue. I have the same problem since some time. Michael. From nc-gaertnma at netcologne.de Wed Dec 22 23:59:42 2021 From: nc-gaertnma at netcologne.de (Mattias Gaertner) Date: Wed, 22 Dec 2021 23:59:42 +0100 Subject: [Lazarus] Codetools - "Error: identifier not found: Boolean8" In-Reply-To: References: <913ba0dc-9557-2a8b-1131-90f3938ecce8@bgss.hu> Message-ID: <20211222235942.110a8169@limapholos.matflo.wg> On Wed, 22 Dec 2021 22:59:00 +0100 (CET) Michael Van Canneyt via lazarus wrote: > On Wed, 22 Dec 2021, Gabor Boros via lazarus wrote: > > > Hi All, > > > > FPC main and Lazarus fixes_2_2 or main. If double click on the new > > project's main form got "identifier not found: Boolean8". > > > > Is this a Lazarus or FPC problem? > > Lazarus code tools issue. > > I have the same problem since some time. Fixed. Mattias From michael at freepascal.org Thu Dec 23 00:22:53 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Thu, 23 Dec 2021 00:22:53 +0100 (CET) Subject: [Lazarus] Codetools - "Error: identifier not found: Boolean8" In-Reply-To: <20211222235942.110a8169@limapholos.matflo.wg> References: <913ba0dc-9557-2a8b-1131-90f3938ecce8@bgss.hu> <20211222235942.110a8169@limapholos.matflo.wg> Message-ID: On Wed, 22 Dec 2021, Mattias Gaertner via lazarus wrote: > On Wed, 22 Dec 2021 22:59:00 +0100 (CET) > Michael Van Canneyt via lazarus wrote: > >> On Wed, 22 Dec 2021, Gabor Boros via lazarus wrote: >> >> > Hi All, >> > >> > FPC main and Lazarus fixes_2_2 or main. If double click on the new >> > project's main form got "identifier not found: Boolean8". >> > >> > Is this a Lazarus or FPC problem? >> >> Lazarus code tools issue. >> >> I have the same problem since some time. > > Fixed. Thank you ! Michael. From marc at dommelstein.nl Fri Dec 24 08:30:53 2021 From: marc at dommelstein.nl (Marc Weustink) Date: Fri, 24 Dec 2021 08:30:53 +0100 Subject: [Lazarus] Lazarus server maintenance Message-ID: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> Hi, On Monday 27 December 9.00 CET (8.00 GMT) the Lazarus server will be down for maintenance. This affects the following services: * Lazarus website * Lazarus mailinglists * Lazarus online package manager * Lazarus and FreePascal forum Thanks, Marc From michael at freepascal.org Fri Dec 24 14:13:24 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Fri, 24 Dec 2021 14:13:24 +0100 (CET) Subject: [Lazarus] FPC and WebAssembly Message-ID: Hello, I'm glad to anncounce that FPC now covers working with webassembly in the browser. Because pictures say more than a thousand words, 3 demos: A modest "hello, world" variation: https://www.freepascal.org/~michael/pas2js-demos/wasienv/simple/ A (limited) terminal emulation: https://www.freepascal.org/~michael/pas2js-demos/wasienv/terminal/ And yes, we can do graphics too: https://www.freepascal.org/~michael/pas2js-demos/wasienv/terminal/ In all 3 cases: - The webassembly program was created with the FPC webassembly target. - The hosting environment is created using pas2js. This is just initial hosting support, there is lots of work to be done. The webassembly compiler works very well, and is integrated in the compiler testsuite. More info can be found in: https://wiki.freepascal.org/WebAssembly/Compiler https://wiki.freepascal.org/WebAssembly/Roadmap Our little contribution to a Merry Christmas for everyone... Michael. From michael at freepascal.org Fri Dec 24 14:16:01 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Fri, 24 Dec 2021 14:16:01 +0100 (CET) Subject: [Lazarus] [Pas2js] FPC and WebAssembly In-Reply-To: References: Message-ID: > And yes, we can do graphics too: > > https://www.freepascal.org/~michael/pas2js-demos/wasienv/terminal/ That should obviously be: https://www.freepascal.org/~michael/pas2js-demos/wasienv/canvas/ My apologies for the mistake. Michael. From jean.suzineau at wanadoo.fr Fri Dec 24 14:46:26 2021 From: jean.suzineau at wanadoo.fr (Jean SUZINEAU) Date: Fri, 24 Dec 2021 14:46:26 +0100 Subject: [Lazarus] FPC and WebAssembly In-Reply-To: References: Message-ID: Wonderful ! Thank you very much for all your work and all the work of the FPC / Lazarus team ! And Merry Christmas for all ! From coppolastudio at gmail.com Sat Dec 25 15:22:27 2021 From: coppolastudio at gmail.com (Salvatore Coppola) Date: Sat, 25 Dec 2021 15:22:27 +0100 Subject: [Lazarus] FPC and WebAssembly In-Reply-To: References: Message-ID: <8bcf56f8-f758-4c1b-b9ed-8f159aa1c55c@gmail.com> Merry Christmas prof. and all of you ⁣Ottieni BlueMail per Android ​ Il giorno 24 Dic 2021, 14:13, alle ore 14:13, Michael Van Canneyt via lazarus ha scritto: > >Hello, > >I'm glad to anncounce that FPC now covers working with webassembly in >the >browser. > >Because pictures say more than a thousand words, 3 demos: > >A modest "hello, world" variation: > >https://www.freepascal.org/~michael/pas2js-demos/wasienv/simple/ > >A (limited) terminal emulation: > >https://www.freepascal.org/~michael/pas2js-demos/wasienv/terminal/ > >And yes, we can do graphics too: > >https://www.freepascal.org/~michael/pas2js-demos/wasienv/terminal/ > >In all 3 cases: >- The webassembly program was created with the FPC webassembly target. > >- The hosting environment is created using pas2js. > >This is just initial hosting support, there is lots of work to be done. > >The webassembly compiler works very well, and is integrated in the >compiler testsuite. > >More info can be found in: >https://wiki.freepascal.org/WebAssembly/Compiler >https://wiki.freepascal.org/WebAssembly/Roadmap > >Our little contribution to a Merry Christmas for everyone... > >Michael. >-- >_______________________________________________ >lazarus mailing list >lazarus at lists.lazarus-ide.org >https://lists.lazarus-ide.org/listinfo/lazarus -------------- next part -------------- An HTML attachment was scrubbed... URL: From badsectoracula at gmail.com Sat Dec 25 17:01:03 2021 From: badsectoracula at gmail.com (Kostas Michalopoulos) Date: Sat, 25 Dec 2021 18:01:03 +0200 Subject: [Lazarus] FPC and WebAssembly In-Reply-To: References: Message-ID: <670a4c7b-2fcc-2b7a-da3f-d285fbf0534a@gmail.com> Hello, > I'm glad to anncounce that FPC now covers working with webassembly in the > browser. Sounds great. I did a quick compile for the wasi target and seemed to work fine with a simple helloworld-style test. So i decided to try and port a game i made last year[0] - initially made for an MS-DOS game jam, but added a bunch of ports later and wanted to make a web-based one too. I decided to go with a similar approach to the one i used for Android, with the game compiled into a library (or a wasm file for here) and a "stub" that loads the library and passes events, etc to it. However no matter what i try, it always seems to fail when (i guess) tries to link or assemble the first unit (the unit doesn't do anything, just defines an entity class). By default it gets this error which happens after it compiles everything (SndEmit.o is the object file for the unit, it is just the first unit it tries to link/compile): wasm-ld: error: ./Engine/SndEmit.o: invalid relocation function index Error: Error while linking If i pass -Awabt to use the wasa assembler (thinking that there might be something wrong with the object files), it then has the error: SndEmit.pas(171) Fatal: Internal error 2019093001 (The unit is 171 lines long) Also there are a bunch of "steam read error" errors, often after "assembling someunitname" (though not all "assembling someunitname" have such an error after them). If i pass -Allvm-mc to use LLVM's assembler, i get a bunch of errors from a different unit (Misc): Compiling ./Engine/Misc.pas Assembling misc ./Engine/Misc.wat:5082:3: error: symbol MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT missing .functype call MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT ^ ./Engine/Misc.wat:5412:3: error: symbol MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT missing .functype call MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT ^ ./Engine/Misc.wat:5606:3: error: symbol MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT missing .functype call MISC$_$TSTRINGKEYVALUESET_$__$$_INDEXOFKEY$ANSISTRING$$LONGINT ^ ./Engine/Misc.wat:6023:3: error: 3 superfluous return values end_function ^ ./Engine/Misc.wat:6272:3: error: 1 superfluous return values end_function ^ ./Engine/Misc.wat:6496:3: error: 1 superfluous return values end_function ^ ./Engine/Misc.wat:6681:3: error: 1 superfluous return values end_function ^ Misc.pas(150) Error: Error while assembling exitcode 1 Note that i have LLVM 13 installed but also tried removing it and installing LLVM 11 and didn't make any difference. Also FWIW i'm calling the ppcrosswasm32 executable directly and have it installed in ~/Apps/fpc, though it shouldn't make a difference (and the simple hello world code compiled anyway so the compiler setup should be fine). Is there anything i am missing/not setting up correctly (e.g. i didn't tried with the "embedded" target - i tried to build FPC with it first but got some errors so i didn't explore further and went with "wasi" instead that seemed to work)? AFAIK i do not use any of the functionality that isn't implemented in the Roadmap page in the wiki. Kostas [0] https://bad-sector.itch.io/post-apocalyptic-petra From noelduffy at xtra.co.nz Mon Dec 27 00:43:56 2021 From: noelduffy at xtra.co.nz (Noel Duffy) Date: Mon, 27 Dec 2021 12:43:56 +1300 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) Message-ID: I need some help getting to the root of a problem with incorrect results on Apple hardware (M1, aarch64) for the function UTF8LengthFast in lazutf8. On MacOS, when given a string containing one or more UTF8 characters, UTF8LengthFast returns wildly incorrect results. On Fedora, the function returns the correct answer. On Apple, I'm using fpc 3.3.1, and Lazarus is 2.2.0RC3. On Fedora, Lazarus is 2.0.12-2, and fpc is 3.2.2-1. The following small program demonstrates the problem here. % cat utf8len.pas program utf8len; {$mode objfpc}{$H+} {$CODEPAGE UTF8} uses SysUtils, lazutf8; const s = '€'; var n: PtrInt; begin n := UTF8LengthFast(s); writeln('Len='+inttostr(n)); end. % file utf8len.pas utf8len.pas: Unicode text, UTF-8 text To compile this, on MacOS I use this: % ~/fpc3.3.1/bin/fpc -Sh -Cro -O3 -XX -vewbq -FU. -Fu/usr/local/share/lazarus/components/lazutils/lib/aarch64-darwin utf8len.pas On Fedora, with this: $ /usr/bin/fpc -Sh -Cro -O3 -XX -vewbq -FU. -Fu/usr/lib64/lazarus/components/lazutils utf8len.pas Then run it: On MacOS: % ./utf8len Len=-100663283 On Fedora: $ ./utf8len Len=1 On MacOS, I built fpc from source, compiling 3.3.1 with version 3.2.2. I then compiled Lazarus using fpc 3.3.1. Because I built fpc and then Lazarus, I'm considering the possibility I introduced an error or a bug somewhere, so I want to eliminate that possibility if possible by asking if anyone else can reproduce this problem? I have traced through the code using a debugger on both platforms. The same path through the function UTF8LengthFast is followed, but the final loop involving boolean shifting of bytes produces different results. I don't understand well enough the algorithm that the function uses to easily see what's going on. From juha.manninen62 at gmail.com Mon Dec 27 13:47:54 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Mon, 27 Dec 2021 14:47:54 +0200 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On Mon, Dec 27, 2021 at 1:44 AM Noel Duffy via lazarus < lazarus at lists.lazarus-ide.org> wrote: > I need some help getting to the root of a problem with incorrect results > on Apple hardware (M1, aarch64) for the function UTF8LengthFast in lazutf8. > > On MacOS, when given a string containing one or more UTF8 characters, > UTF8LengthFast returns wildly incorrect results. On Fedora, the function > returns the correct answer. > You mean both MacOS and Fedora run on the same aarch64 CPU? It must be a Big endian / Little endian issue. IIRC it can be adjusted in ARM CPUs. Why do MacOS and Linux use a different setting there? I have no idea. If somebody can figure out how to port the function, good. Otherwise we can make UTF8LengthFast call the standard UTF8Length using an IFDEF when needed. What is the correct define then? Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartjunk64 at gmail.com Mon Dec 27 13:28:23 2021 From: bartjunk64 at gmail.com (Bart) Date: Mon, 27 Dec 2021 13:28:23 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On Mon, Dec 27, 2021 at 12:44 AM Noel Duffy via lazarus wrote: > I need some help getting to the root of a problem with incorrect results > on Apple hardware (M1, aarch64) for the function UTF8LengthFast in lazutf8. Your M1 architecture is BigEndian perhaps? (I really have no idea) -- Bart From marc at dommelstein.nl Mon Dec 27 15:31:32 2021 From: marc at dommelstein.nl (Marc Weustink) Date: Mon, 27 Dec 2021 15:31:32 +0100 Subject: [Lazarus] Lazarus server maintenance In-Reply-To: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> References: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> Message-ID: <6c917291-509a-2eb7-00ef-c397b7be7c75@dommelstein.nl> The mailing lists should be available again Marc On 24-12-2021 08:30, Marc Weustink via lazarus wrote: > Hi, > > On Monday 27 December 9.00 CET (8.00 GMT) the Lazarus server will be > down for maintenance. This affects the following services: > > * Lazarus website > * Lazarus mailinglists > * Lazarus online package manager > * Lazarus and FreePascal forum > > Thanks, > Marc > From michael at freepascal.org Mon Dec 27 12:00:44 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Mon, 27 Dec 2021 12:00:44 +0100 (CET) Subject: [Lazarus] SQLDBRestBridge memory consumption attn Michael In-Reply-To: <2ecbf680-7f92-bf6c-8d6b-32bfd3c995f5@holobit.hr> References: <2ecbf680-7f92-bf6c-8d6b-32bfd3c995f5@holobit.hr> Message-ID: On Mon, 27 Dec 2021, zeljko wrote: > Hi all, > > fpc-3.2.2, lazarus trunk...found that sqldbrestdispatcher does not free some > unused resources, so sqldbrestserver memory alloc grows and after few > thousand connections eats more than 1GB of memory. > Just found in TSQLDBRestDispatcher.HandleResourceRequest that > H: TSQLDBRestDBHandler is created for each request but freed when dispatcher > is destroyed. So, dispatcher in this case can contain thousands of > TSQLDBRestDBHandler unused components (could not find that dispatcher reuse > any of this). > I've added H.Free at last place in finally section and it seem that memory > does not grow too much at all (still grows but slower, probably there's still > unused resources around). Solution would be to reuse TSQLDBRestDBHandler's or > free them. Pls look at > https://forum.lazarus.freepascal.org/index.php/topic,51938.msg382021.html#msg382021 Forum is down, but I distinctly remember fixing this memleak. It was not on my main development machine, but in a Windows VM. Probably I forgot to commit that at the time :/ I applied a fix (free the handler), and fixed some compiler hints/warnings as well. I did a test, and I can't reproduce memleaks with the current demo application (the -m option exists for this very purpose), but that doesn't mean there are no memleaks left, there are many different execution paths :/ Michael. From bartjunk64 at gmail.com Mon Dec 27 16:39:05 2021 From: bartjunk64 at gmail.com (Bart) Date: Mon, 27 Dec 2021 16:39:05 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On Mon, Dec 27, 2021 at 3:41 PM Juha Manninen via lazarus wrote: > It must be a Big endian / Little endian issue. IIRC it can be adjusted in ARM CPUs. > Why do MacOS and Linux use a different setting there? I have no idea. On second thought: if the function returns grabage for just a single '?', the code for that should not enter the pasrt where it handles blocks of size PtrInt and does masking with EIGHTYMASK etc. (The part of the code that might be endianness dependant). It should go to one of the 2 loops that simply does: Result += (pn8^ shr 7) and ((not pn8^) shr 6); That part should not depend on endianness at all. On Win32 a sigle '?' will result in something like this: pn8^ =11100010 //first byte (pn8^ shr 7) =11111111 //<<-- I would have expected that to be 00000001 ? (not pn8^) =00011101 (not pn8^) shr 6 =00000000 Add: (pn8^ shr 7) and ((not pn8^) shr 6)=0 pn8^ =10000010 //second byte (pn8^ shr 7) =11111111 (not pn8^) =01111101 (not pn8^) shr 6 =00000001 Add: (pn8^ shr 7) and ((not pn8^) shr 6)=1 pn8^ =10101100 //third and last byte of '?' (pn8^ shr 7) =11111111 (not pn8^) =01010011 (not pn8^) shr 6 =00000001 Add: (pn8^ shr 7) and ((not pn8^) shr 6)=1 B.t.w. I find the code in Utf8LengthFast difficult to read. Personally I dislike the C-ism of += and >> (even more so if both >> and shr is used). -- Bart From florian at freepascal.org Mon Dec 27 16:53:38 2021 From: florian at freepascal.org (=?UTF-8?Q?Florian_Kl=c3=a4mpfl?=) Date: Mon, 27 Dec 2021 16:53:38 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: <4d8d0650-86bb-4cd4-b5ca-826f0ef6d1fd@freepascal.org> Am 27.12.2021 um 13:28 schrieb Bart via lazarus: > On Mon, Dec 27, 2021 at 12:44 AM Noel Duffy via lazarus > wrote: > >> I need some help getting to the root of a problem with incorrect results >> on Apple hardware (M1, aarch64) for the function UTF8LengthFast in lazutf8. > > Your M1 architecture is BigEndian perhaps? > (I really have no idea) > No. It is little endian. From fpc at pascalprogramming.org Mon Dec 27 18:34:57 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Mon, 27 Dec 2021 18:34:57 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Op 12/27/2021 om 4:39 PM schreef Bart via lazarus: > pn8^ =11100010 //first byte > (pn8^ shr 7) =11111111 //<<-- I would have expected that to be 00000001 ? Depends on if pn8^ is signed or not, for a signed shift it makes sense. The definition as pint8 (instead of puint8) is an odd choice. The expression seems to be 1 when the top bits are 10? iow when it is a follow bytes of utf8, that is what the comment says, and I as far as I can see the signedness doesn't matter. Basically to me that seems to be a branchless version of if (p[i] and %11000000)=%10000000 then ?? inc(result); ...which counts all utf8 follow bytes, and then subtracts it from the number of bytes in a string to find the number of utf8 sequences/codepoints. Maybe the absolute stuff confuses somehow? Also make sure the input is 100% the same by printing the values of the bytes of the input string. From bartjunk64 at gmail.com Mon Dec 27 19:21:51 2021 From: bartjunk64 at gmail.com (Bart) Date: Mon, 27 Dec 2021 19:21:51 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Mon, Dec 27, 2021 at 6:35 PM Marco van de Voort via lazarus wrote: > The expression seems to be 1 when the top bits are 10 iow when it is a > follow bytes of utf8, that is what the comment says, and I as far as I > can see the signedness doesn't matter. > > Basically to me that seems to be a branchless version of > > if (p[i] and %11000000)=%10000000 then > > inc(result); This is how I understood that paert of the code as well. Just a side node: all this assumes that the UTF8 is correct (in the strict sense). Now for the part that tries to do calculations on blocks of 32 or 64 bits. It uses a multiplication in that part. That seems a bit odd as this code tries to do evrything with ands, nots, and shifts. Would this approach (in that part of the code) also work? X = 32 or 64 bit block 1: AND X with EIGHTYMASK 2: SHR 7: gives (A) 3: NOT X 4: SHR 6: gives (B) 5: (A) AND (B): gives (C) Basically we do the same as for a 1-byte block; Now we have a pattern where any 1 tells us this byte was a following byte A 32-bit example: (Invalid sequence but that does not matter here) X = %11xxxxxx %01xxxxxx %00xxxxxx %10xxxxxx (Leading-ASCII-ASCII-Following, so count of following bytes must be 1) AND with EIGTHYMASK gives %10000000 %00000000 %00000000 %10000000 SHR 7 gives: %00000001 %00000000 %00000000 %00000001 (A) NOT %11xxxxxx %01xxxxxx %00xxxxxx %10xxxxxx = %00yyyyyy %10yyyyyy %11yyyyyy %01yyyyyy SHR 6 gives: %00000000 %yyyyyy10 %yyyyyy11 %yyyyyy01 (B) (A) and (B) gives: %00000000 %00000000 %00000000 %00000001 (C) All non-following bytes turn into %00000000 The count of following bytes is PopCnt(C) As long as PopCnt is available as a machine instruction, this will be faster than the (nx * ONEMASK) calculation I think. (I would hope this is the case for all platforms Lazarus supports, if not the call to PopCnt could be ifdef-ed.) So basically change this: nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} into nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); Result := Result + PopCnt(PtrUInt(nx)); /PopCnt only accepts unsigend parameter Since bit manipulating is not my strongpoint, please comment. -- Bart From noelduffy at xtra.co.nz Mon Dec 27 22:02:04 2021 From: noelduffy at xtra.co.nz (Noel Duffy) Date: Tue, 28 Dec 2021 10:02:04 +1300 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On 28/12/21 04:39, Bart via lazarus wrote: > On Mon, Dec 27, 2021 at 3:41 PM Juha Manninen via lazarus > wrote: > >> It must be a Big endian / Little endian issue. IIRC it can be adjusted in ARM CPUs. >> Why do MacOS and Linux use a different setting there? I have no idea. > > On second thought: if the function returns grabage for just a single > '?', the code for that should not enter the pasrt where it handles > blocks of size PtrInt and does masking with EIGHTYMASK etc. It's not just the euro, though. It's any utf-8 sequence. From noelduffy at xtra.co.nz Mon Dec 27 22:10:51 2021 From: noelduffy at xtra.co.nz (Noel Duffy) Date: Tue, 28 Dec 2021 10:10:51 +1300 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On 28/12/21 01:47, Juha Manninen via lazarus wrote: > On Mon, Dec 27, 2021 at 1:44 AM Noel Duffy via lazarus < > lazarus at lists.lazarus-ide.org> wrote: > >> I need some help getting to the root of a problem with incorrect results >> on Apple hardware (M1, aarch64) for the function UTF8LengthFast in lazutf8. >> >> On MacOS, when given a string containing one or more UTF8 characters, >> UTF8LengthFast returns wildly incorrect results. On Fedora, the function >> returns the correct answer. >> > > You mean both MacOS and Fedora run on the same aarch64 CPU? Oh no, Fedora runs on an Intel x86_64. I don't think there's a Fedora that runs on aarch64. I included the results from Fedora just to show that the code does work in some places. > It must be a Big endian / Little endian issue. IIRC it can be adjusted in > ARM CPUs. > Why do MacOS and Linux use a different setting there? I have no idea. Well, as Florian said above, the M1 is little-endian. So it doesn't appear to be an endian issue. https://developer.apple.com/documentation/apple-silicon/porting-your-macos-apps-to-apple-silicon From bartjunk64 at gmail.com Mon Dec 27 23:43:01 2021 From: bartjunk64 at gmail.com (Bart) Date: Mon, 27 Dec 2021 23:43:01 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On Mon, Dec 27, 2021 at 10:02 PM Noel Duffy via lazarus wrote: > It's not just the euro, though. It's any utf-8 sequence. What I meant was that a single '?' (or any other single UTF8 "character") will not enter the mentioned block. Can you add some debug statements to display the values of the it uses in the calculation like I did in the 4th message in this thread? function UTF8LengthFast(p: PChar; ByteCount: PtrInt): PtrInt; const {$ifdef CPU32} ONEMASK =$01010101; EIGHTYMASK=$80808080; {$endif} {$ifdef CPU64} ONEMASK =$0101010101010101; EIGHTYMASK=$8080808080808080; {$endif} var pnx: PPtrInt absolute p; // To get contents of text in PtrInt blocks. x refers to 32 or 64 bits pn8: pint8 absolute pnx; // To read text as Int8 in the initial and final loops ix: PtrInt absolute pnx; // To read text as PtrInt in the block loop nx: PtrInt; // values processed in block loop i,cnt,e: PtrInt; begin Result := 0; e := ix+ByteCount; // End marker // Handle any initial misaligned bytes. cnt := (not (ix-1)) and (sizeof(PtrInt)-1); if cnt>ByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? writeln('pn8^ = ',byte(pn8^).ToBinString); writeln('pn8^ shr 7 = ',Byte(Byte(pn8^) shr 7).ToBinString); writeln('not pn8^ = ',Byte(not pn8^).ToBinString); writeln('(not pn8^) shr 6 = ',Byte((not pn8^) shr 6).ToBinString); writeln; Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrUInt) do begin // Count bytes which are NOT the first byte of a character. { nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} } nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); Result := Result + PopCnt(PtrUInt(nx)); inc(pnx); end; // Take care of any left-over bytes. while ix References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On 28/12/21 07:21, Bart via lazarus wrote: > On Mon, Dec 27, 2021 at 6:35 PM Marco van de Voort via lazarus > wrote: > >> The expression seems to be 1 when the top bits are 10 iow when it is a >> follow bytes of utf8, that is what the comment says, and I as far as I >> can see the signedness doesn't matter. >> >> Basically to me that seems to be a branchless version of >> >> if (p[i] and %11000000)=%10000000 then >> >> inc(result); > > This is how I understood that paert of the code as well. > > Just a side node: all this assumes that the UTF8 is correct (in the > strict sense). > > Now for the part that tries to do calculations on blocks of 32 or 64 bits. > It uses a multiplication in that part. > That seems a bit odd as this code tries to do evrything with ands, > nots, and shifts. > > Would this approach (in that part of the code) also work? > X = 32 or 64 bit block > > 1: AND X with EIGHTYMASK > 2: SHR 7: gives (A) > > 3: NOT X > 4: SHR 6: gives (B) > > 5: (A) AND (B): gives (C) > > Basically we do the same as for a 1-byte block; > Now we have a pattern where any 1 tells us this byte was a following byte > > A 32-bit example: > (Invalid sequence but that does not matter here) > > X = %11xxxxxx %01xxxxxx %00xxxxxx %10xxxxxx > (Leading-ASCII-ASCII-Following, so count of following bytes must be 1) > > AND with EIGTHYMASK gives %10000000 %00000000 %00000000 %10000000 > SHR 7 gives: %00000001 %00000000 %00000000 %00000001 (A) > > NOT %11xxxxxx %01xxxxxx %00xxxxxx %10xxxxxx = %00yyyyyy %10yyyyyy > %11yyyyyy %01yyyyyy > SHR 6 gives: %00000000 %yyyyyy10 %yyyyyy11 %yyyyyy01 (B) > > (A) and (B) gives: %00000000 %00000000 %00000000 %00000001 (C) > All non-following bytes turn into %00000000 > > The count of following bytes is PopCnt(C) > > As long as PopCnt is available as a machine instruction, this will be > faster than the (nx * ONEMASK) calculation I think. > (I would hope this is the case for all platforms Lazarus supports, if > not the call to PopCnt could be ifdef-ed.) > > So basically change this: > > nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); > {$push}{$overflowchecks off} // "nx * ONEMASK" causes an > arithmetic overflow. > Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); > {$pop} > > into > > nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); > Result := Result + PopCnt(PtrUInt(nx)); /PopCnt only accepts > unsigend parameter > > > Since bit manipulating is not my strongpoint, please comment. > I ran my code that calls UTF8LengthFast with a euro through the debugger to see what path through UTF8LengthFast was followed. The debugger skipped over all the loops until the very last one: // Take care of any left-over bytes. while ix References: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> Message-ID: <49c29315-0cc1-0113-5ef1-9d1866bff6c2@dommelstein.nl> Hi all, It took a bit longer than expected, but I'm happy to inform you that the Lazarus services are back online. For those interested in why it took longer, I'll explain at the end of the message. Marc On 24-12-2021 08:30, Marc Weustink wrote: > Hi, > > On Monday 27 December 9.00 CET (8.00 GMT) the Lazarus server will be > down for maintenance. This affects the following services: > > * Lazarus website > * Lazarus mailinglists > * Lazarus online package manager > * Lazarus and FreePascal forum > > Thanks, > Marc The story.... The server was running Ubuntu 16 LTR, so it's support ended on April this year. An attempt at that time failed since Ubuntu decided to switch to systemd-resolve which resulted in a server not even able to resolve its own name, let alone other hosts. When Googling about it, you learn that it doesn't work, it's a half backed solution, not a full DNS etc. At that time it became clear that I wouldn't be able to solve that in an evening. Luckily I could try this on a cloned server (we had to rent for another issue) so I parked the upgrade till I could spend a full day. Yesterday I found the "correct" solution to this which appeared to work. So I continued to upgrade to Ubuntu 18 and finally to Ubuntu 20 LTR. Everything seemed to work until I enabled the mailserver. It couldn't resolve any mailserver for a given domain. What the f.. 'host -t mx freepascal.org' resolves, why can't postfix resolve it. Again after some Googling, postfix needs a /etc/resolf.conf. However one step of the DNS solution was to remove the /etc/resolf.conf symlink, so I tried to restore the original link to some systemd-resolve generated one. This one pointed to their internal resolver. Still no luck since I hadn't configured systemd-resolve which DNS itself should use. After doing so, the generated resolf.conf became empty ????? After more Googling I found that systemd-resolve generates another conf where you also can link to. No clue why there have to exist another version, but this one works. So this part of the server upgrade got finished around 14:00. The Lazarus mailing list and main website were back online. Another wish we had was to change the database backend of the forum. It appeared over time that when doing a search on the database, mysql blocks updates, so browsing the forum becomes unresponsive. The current version of SMF supports different databases so we decided to go for PostgreSQL (I'm using them at work for years now). Migrating MySql data to PostgreSQL seemed easy with pgLoader. The documentation about is was initially a bit sparse, but I could start a conversion with some commandline options. Unfortunately it got killed after 15 mins of import. After two more attempts it became clear, out of memory :( Reducing the memory requirements was a build time option so I didn't want to go that way. Another solution was to convert only a few tables at a time. That required however a configuration file which has more options than the command line. Most of the examples I found on the web failed, since they lack the semi colon at the end if the configuration. So the parser barfs with some abracadabra, initially not giving a clue. Fast forward, on 18:00 all but the messages table were converted. On 22:00 the messages table was converted using 9 parts. Then I realized that I didn't have a php-pgsql driver installed. After doing so, I discovered that the Lazarus main site also showed the forum maintenance message ??? Those sites are running on two different virtual servers. How can the contents of an index.php of one site have influence on the index.php of another site. What did go wrong when I installed the driver ??? After an hour investigating I decided to enable the forum first and investigate the issue later. Poof.... the forum results in a bunch of errors. What we didn't think of when switching backend, is that we use SMF (which is PostgreSQL capable) and TinyPortal (TP) to have the menus at the sides. And TP is full of MySQL only statements. Luckily there is someone who created all those missing functions for SMF and I created them on the database. After adjusting some TP php files (PostgreSQL requires a true boolean and not some integer <> 0), the forum started without errors. But it didn't show any boards. So there is still something wrong under the hood. Meanwhile it was 24:00 and I decided to continue using MySQL and called it a day. This morning I reverted my TP changes and put the MySQL database back online. To be continued... Marc From nc-gaertnma at netcologne.de Tue Dec 28 10:04:00 2021 From: nc-gaertnma at netcologne.de (Mattias Gaertner) Date: Tue, 28 Dec 2021 10:04:00 +0100 Subject: [Lazarus] Lazarus server back online In-Reply-To: <49c29315-0cc1-0113-5ef1-9d1866bff6c2@dommelstein.nl> References: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> <49c29315-0cc1-0113-5ef1-9d1866bff6c2@dommelstein.nl> Message-ID: <20211228100400.3a5e03a3@limapholos.matflo.wg> On Tue, 28 Dec 2021 09:41:03 +0100 Marc Weustink via lazarus wrote: >[...] > To be continued... Oh dear. Thanks for the all the work! Mattias From michael at freepascal.org Tue Dec 28 10:36:46 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Tue, 28 Dec 2021 10:36:46 +0100 (CET) Subject: [Lazarus] Lazarus server back online In-Reply-To: <49c29315-0cc1-0113-5ef1-9d1866bff6c2@dommelstein.nl> References: <767773f6-5fac-4612-c562-5ca79f355570@dommelstein.nl> <49c29315-0cc1-0113-5ef1-9d1866bff6c2@dommelstein.nl> Message-ID: On Tue, 28 Dec 2021, Marc Weustink via lazarus wrote: > Hi all, > > It took a bit longer than expected, but I'm happy to inform you that the > Lazarus services are back online. > For those interested in why it took longer, I'll explain at the end of the > message. > [snip] > Meanwhile it was 24:00 and I decided to continue using MySQL and called it a > day. > This morning I reverted my TP changes and put the MySQL database back online. And that is why I no longer wish to maintain something like Mantis. Similar problems, every time you upgrade. But upgrade you must. Michael. From juha.manninen62 at gmail.com Tue Dec 28 11:52:19 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Tue, 28 Dec 2021 12:52:19 +0200 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 3:29 AM Noel Duffy via lazarus < lazarus at lists.lazarus-ide.org> wrote: > So it appears to me that an unsigned pointer type is required in > UTFLengthFast. > Can you please create a patch for UTFLengthFast. You can upload it here or create a merge request in GitLab or anything. Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From lazarus at mfriebe.de Tue Dec 28 12:08:30 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Tue, 28 Dec 2021 12:08:30 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On 28/12/2021 11:52, Juha Manninen via lazarus wrote: > On Tue, Dec 28, 2021 at 3:29 AM Noel Duffy via lazarus > wrote: > > So it appears to me that an unsigned pointer type is required in > UTFLengthFast. > > > Can you please create a patch for UTFLengthFast. You can upload it > here or create a merge request in GitLab or anything. > I would like to see the generates assembler on M1, if that is possible?? (for code with optimization off, as well as code with whatever optimization was used so far) Thanks None the less, switching to "puint8" might be a good idea. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartjunk64 at gmail.com Tue Dec 28 12:41:43 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 12:41:43 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 11:52 AM Juha Manninen via lazarus wrote: > Can you please create a patch for UTFLengthFast. You can upload it here or create a merge request in GitLab or anything. @Juha: can you please comment on my possible improvement using PopCnt instead of a multiplication with ONEMASK. I will patch the function using unsigned types where applicable. I will keep the loop variables unsigned though. -- Bart From juha.manninen62 at gmail.com Tue Dec 28 13:09:33 2021 From: juha.manninen62 at gmail.com (Juha Manninen) Date: Tue, 28 Dec 2021 14:09:33 +0200 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 1:45 PM Bart via lazarus < lazarus at lists.lazarus-ide.org> wrote: > @Juha: can you please comment on my possible improvement using PopCnt > instead of a multiplication with ONEMASK. > I confess I didn't remember what PopCnt does. I checked from the net. FPC implements it as internproc. function PopCnt(Const AValue : QWord): QWord;[internproc:fpc_in_popcnt_x]; I guess it translates to one x86_64 instruction. Is it implemented for all CPUs? I found this: https://gitlab.com/freepascal.org/fpc/source/-/issues/38729 If it works everywhere, good. It looks like another good optimization for this highly optimized function. I will patch the function using unsigned types where applicable. > I will keep the loop variables unsigned though. > Yes, thank you. Regards, Juha -------------- next part -------------- An HTML attachment was scrubbed... URL: From bartjunk64 at gmail.com Tue Dec 28 13:26:56 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 13:26:56 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 12:08 PM Martin Frb via lazarus wrote: > I would like to see the generates assembler on M1, if that is possible? (for code with optimization off, as well as code with whatever optimization was used so far) @Noel: Here's example code (standalone) you can use to test both signed and unsigned versions. Save this code as ulen.pas In order to get assembler output compile with: fpc -al ulen.pas This will produce the file ulen.s You can attach or copy that here. ============= program ulen; {$mode objfpc}{$H+} {$optimization off} {$codepage utf8} uses SysUtils; function UTF8LengthFast_Signed(p: PChar; ByteCount: PtrInt): PtrInt; const {$ifdef CPU32} ONEMASK =$01010101; EIGHTYMASK=$80808080; {$endif} {$ifdef CPU64} ONEMASK =$0101010101010101; EIGHTYMASK=$8080808080808080; {$endif} var pnx: PPtrInt absolute p; // To get contents of text in PtrInt blocks. x refers to 32 or 64 bits pn8: pint8 absolute pnx; // To read text as Int8 in the initial and final loops ix: PtrInt absolute pnx; // To read text as PtrInt in the block loop nx: PtrInt; // values processed in block loop i,cnt,e: PtrInt; begin Result := 0; e := ix+ByteCount; // End marker // Handle any initial misaligned bytes. cnt := (not (ix-1)) and (sizeof(PtrInt)-1); if cnt>ByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? //writeln('pn8^ = ',byte(pn8^).ToBinString); //writeln('pn8^ shr 7 = ',Byte(Byte(pn8^) shr 7).ToBinString); //writeln('not pn8^ = ',Byte(not pn8^).ToBinString); //writeln('(not pn8^) shr 6 = ',Byte((not pn8^) shr 6).ToBinString); //writeln; Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrUInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? //writeln('pn8^ = ',byte(pn8^).ToBinString); //writeln('pn8^ shr 7 = ',Byte(Byte(pn8^) shr 7).ToBinString); //writeln('not pn8^ = ',Byte(not pn8^).ToBinString); //writeln('(not pn8^) shr 6 = ',Byte((not pn8^) shr 6).ToBinString); //writeln; Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrUInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ix FPC implements it as internproc. > function PopCnt(Const AValue : QWord): QWord;[internproc:fpc_in_popcnt_x]; > I guess it translates to one x86_64 instruction. > Is it implemented for all CPUs? I found this: > https://gitlab.com/freepascal.org/fpc/source/-/issues/38729 I just tested PopCnt vs Multiplication on win32 and win64. The version with PopCnt is appr. 3 times slower on both 32 and 64 bit! C:\Users\Bart\LazarusProjecten\bugs\Utf8\ulenfast>fpc ulen.lpr Free Pascal Compiler version 3.3.1 [2021/12/08] for i386 Copyright (c) 1993-2021 by Florian Klaempfl and others Target OS: Win32 for i386 C:\Users\Bart\LazarusProjecten\bugs\Utf8\ulenfast>ulen Unsigned version with multiplication: 1344 Unsigned version with PopCnt : 3563 C:\Users\Bart\LazarusProjecten\bugs\Utf8\ulenfast>fpc ulen.lpr -Px86_64 Free Pascal Compiler version 3.3.1 [2021/12/08] for x86_64 Copyright (c) 1993-2021 by Florian Klaempfl and others Target OS: Win64 for x64 C:\Users\Bart\LazarusProjecten\bugs\Utf8\ulenfast>ulen Unsigned version with multiplication: 656 Unsigned version with PopCnt : 3797 It looks like PopCnt on these platforms at least calls the generic version (/rtl/inc/generic.inc). -- Bart From bartjunk64 at gmail.com Tue Dec 28 14:30:42 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 14:30:42 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 1:09 PM Juha Manninen via lazarus wrote: >> I will patch the function using unsigned types where applicable. >> I will keep the loop variables unsigned though. > > > Yes, thank you. Done. Should that be merged to fixes? -- Bart From fpc at pascalprogramming.org Tue Dec 28 14:45:56 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Tue, 28 Dec 2021 14:45:56 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> Op 12/28/2021 om 1:53 PM schreef Bart via lazarus: > I just tested PopCnt vs Multiplication on win32 and win64. > The version with PopCnt is appr. 3 times slower on both 32 and 64 bit! You need an appropriate minimal CPU with -Cp Try e.g. -Cpcoreavx? for core 3000 series and higher From bartjunk64 at gmail.com Tue Dec 28 15:01:50 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 15:01:50 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 2:46 PM Marco van de Voort via lazarus wrote: > You need an appropriate minimal CPU with -Cp > > > Try e.g. -Cpcoreavx for core 3000 series and higher Thanks for that. Up to PENTIUMM: PopCnt slower COREI : approximately equally fast COREAVX PopCnt slightly faster COREAVX2 PopCnt slightly faster Most likely not worth bothering. In code can we check (at compile time) for which instructionset the code was compiled? -- Bart From florian at freepascal.org Tue Dec 28 15:31:05 2021 From: florian at freepascal.org (=?UTF-8?Q?Florian_Kl=c3=a4mpfl?=) Date: Tue, 28 Dec 2021 15:31:05 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> Message-ID: <8d7ba7a2-c1fa-ae73-195f-3a9ddeac24d8@freepascal.org> Am 28.12.2021 um 15:01 schrieb Bart via lazarus: > On Tue, Dec 28, 2021 at 2:46 PM Marco van de Voort via lazarus > wrote: > > >> You need an appropriate minimal CPU with -Cp >> >> >> Try e.g. -Cpcoreavx for core 3000 series and higher > > Thanks for that. > > Up to PENTIUMM: PopCnt slower > COREI : approximately equally fast > COREAVX PopCnt slightly faster > COREAVX2 PopCnt slightly faster > > Most likely not worth bothering. > In code can we check (at compile time) for which instructionset the > code was compiled? > For X86, check for the define CPUX86_HAS_POPCNT (compile time!). From fpc at pascalprogramming.org Tue Dec 28 15:39:34 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Tue, 28 Dec 2021 15:39:34 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> Message-ID: <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> Op 12/28/2021 om 3:01 PM schreef Bart via lazarus: > -Cpcoreavx for core 3000 series and higher > Thanks for that. > > Up to PENTIUMM: PopCnt slower > COREI : approximately equally fast > COREAVX PopCnt slightly faster > COREAVX2 PopCnt slightly faster On what machine did you test? The settings if for the generated code, but the actual processor determines the effective speed. > Most likely not worth bothering. > In code can we check (at compile time) for which instructionset the > code was compiled? Not that I know. Moreover with AMD etc there would be no simple "x or greater"? kind of criterium other than having popcnt From bartjunk64 at gmail.com Tue Dec 28 15:36:55 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 15:36:55 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: <8d7ba7a2-c1fa-ae73-195f-3a9ddeac24d8@freepascal.org> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <8d7ba7a2-c1fa-ae73-195f-3a9ddeac24d8@freepascal.org> Message-ID: On Tue, Dec 28, 2021 at 3:31 PM Florian Kl?mpfl via lazarus wrote: > For X86, check for the define CPUX86_HAS_POPCNT (compile time!). Thanks. -- Bart From bartjunk64 at gmail.com Tue Dec 28 15:50:15 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 15:50:15 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> Message-ID: On Tue, Dec 28, 2021 at 3:39 PM Marco van de Voort via lazarus wrote: > On what machine did you test? The settings if for the generated code, > but the actual processor determines the effective speed. I have a Intel i5 7th generation on my Win10-64 laptop from approx. 2017 (so, it's really old for more modern folks than me). Compiled for 32-bit: With -CpCOREI Unsigned version with multiplication: 1359 Unsigned version with PopCnt : 1282 Compiled for 32-bit: With -CpCOREAVX2 Unsigned version with multiplication: 1312 Unsigned version with PopCnt : 1297 Compiled for 32-bit No -Cp switch Unsigned version with multiplication: 1329 Unsigned version with PopCnt : 3546 B.t.w. what happens if I compile for e.g. CoreAVX2 but my processor does not support that instructionset. Will the compilation/build fail, or will the executable just error out? -- Bart From florian at freepascal.org Tue Dec 28 15:56:52 2021 From: florian at freepascal.org (=?UTF-8?Q?Florian_Kl=c3=a4mpfl?=) Date: Tue, 28 Dec 2021 15:56:52 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> Message-ID: <0cc9add8-4ce6-b40d-6dff-21c564a58daf@freepascal.org> Am 28.12.2021 um 15:50 schrieb Bart via lazarus: > On Tue, Dec 28, 2021 at 3:39 PM Marco van de Voort via lazarus > wrote: > >> On what machine did you test? The settings if for the generated code, >> but the actual processor determines the effective speed. > > I have a Intel i5 7th generation on my Win10-64 laptop from approx. > 2017 (so, it's really old for more modern folks than me). > > Compiled for 32-bit: > With -CpCOREI > Unsigned version with multiplication: 1359 > Unsigned version with PopCnt : 1282 > > Compiled for 32-bit: > With -CpCOREAVX2 > Unsigned version with multiplication: 1312 > Unsigned version with PopCnt : 1297 > > Compiled for 32-bit > No -Cp switch > Unsigned version with multiplication: 1329 > Unsigned version with PopCnt : 3546 > > B.t.w. what happens if I compile for e.g. CoreAVX2 but my processor > does not support that instructionset. > Will the compilation/build fail, or will the executable just error out? > Crash at run time with sigill. Popcnt was introduced with Nehalem, so >10 years ago. From bartjunk64 at gmail.com Tue Dec 28 15:59:44 2021 From: bartjunk64 at gmail.com (Bart) Date: Tue, 28 Dec 2021 15:59:44 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: <0cc9add8-4ce6-b40d-6dff-21c564a58daf@freepascal.org> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <0cc9add8-4ce6-b40d-6dff-21c564a58daf@freepascal.org> Message-ID: On Tue, Dec 28, 2021 at 3:56 PM Florian Kl?mpfl via lazarus wrote: > > Crash at run time with sigill. Popcnt was introduced with Nehalem, so >10 years ago. Thanks. Any other CPU's support something like this? -- Bart From michael at freepascal.org Tue Dec 28 16:25:45 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Tue, 28 Dec 2021 16:25:45 +0100 (CET) Subject: [Lazarus] Improved FPC JSON-RPC support Message-ID: Hello, Thanks to the magic of RTTI and Invoke(), creating a JSON-RPC server has just become significantly easier ! Given an interface definition: IMyOtherInterface = interface ['{4D52BEE3-F709-44AC-BD31-870CBFF44632}'] Function SayHello : string; function Echo(args : TStringArray) : String; end; Creating a JSON-RPC server for this is now as simple as: // Create a class that implements the interface Type TIntf2Impl = class(TInterfacedObject, IMyOtherInterface) public function Echo(args: TStringArray): String; function SayHello: string; end; function TIntf2Impl.Echo(args: TStringArray): String; var S : String; begin Result:=''; For S in Args do begin if Result<>'' then Result:=Result+' '; Result:=Result+S; end end; function TIntf2Impl.SayHello: string; begin Result:='Hello, World!'; end; // Register the class using an interface factory: Function GetMyOtherInterface(Const aName : string) : IInterface; begin Result:=TIntf2Impl.Create as IInterface; end; initialization RTTIJSONRPCRegistry.Add(TypeInfo(IMyOtherInterface), at GetMyOtherInterface,'Service2'); end. And calling it from a client program is even more simple: var client: IMyOtherInterface; aRPCClient: TFPRPCClient; begin // Register interface with name 'Service2' RPCServiceRegistry.Add(TypeInfo(IMyOtherInterface),'Service2'); // Create client aRPCClient:=TFPRPCClient.Create(Nil); aRPCClient.BaseURL:='http://localhost:8080/RPC'; // Just typecast the client to the desired interface Client:=aRPCClient as IMyotherInterface; // or explitly create using the registered service name // Client:=RPC.Specialize CreateService('Service2'); Writeln('Sayhello: ',client.SayHello); Writeln('Sayhello: ',client.Echo(['This','is','Sparta'])); end. The service can also be consumed by pas2js. The support for various argument types is still limited, but this will improve soon. Currently simple types and arrays of simple types are improved. Proper support for records and other structured types needs extended RTTI... An example server and client programs have been committed to the FPC repo. (packages/fcl-web/examples/jsonrpc/rtti) With many thanks to Sven Barth for putting me on the right track... Enjoy! Michael. From marc at dommelstein.nl Tue Dec 28 18:11:41 2021 From: marc at dommelstein.nl (Marc Weustink) Date: Tue, 28 Dec 2021 18:11:41 +0100 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: Message-ID: On 27-12-2021 22:10, Noel Duffy via lazarus wrote: > On 28/12/21 01:47, Juha Manninen via lazarus wrote: >> On Mon, Dec 27, 2021 at 1:44 AM Noel Duffy via lazarus < >> lazarus at lists.lazarus-ide.org> wrote: >> >>> I need some help getting to the root of a problem with incorrect results >>> on Apple hardware (M1, aarch64) for the function UTF8LengthFast in >>> lazutf8. >>> >>> On MacOS, when given a string containing one or more UTF8 characters, >>> UTF8LengthFast returns wildly incorrect results. On Fedora, the function >>> returns the correct answer. >>> >> >> You mean both MacOS and Fedora run on the same aarch64 CPU? > > Oh no, Fedora runs on an Intel x86_64. I don't think there's a Fedora > that runs on aarch64. I included the results from Fedora just to show > that the code does work in some places. I've tried on pine64 aarch64 linux little-endian fpc 3.2.0 and it showed the correct results using your example code in another mail Marc > >> It must be a Big endian / Little endian issue. IIRC it can be adjusted in >> ARM CPUs. >> Why do MacOS and Linux use a different setting there? I have no idea. > > Well, as Florian said above, the M1 is little-endian. So it doesn't > appear to be an endian issue. > > https://developer.apple.com/documentation/apple-silicon/porting-your-macos-apps-to-apple-silicon > > > From noelduffy at xtra.co.nz Tue Dec 28 22:05:23 2021 From: noelduffy at xtra.co.nz (Noel Duffy) Date: Wed, 29 Dec 2021 10:05:23 +1300 Subject: [Lazarus] UTF8LengthFast returning incorrect results on AARCH64 (MacOS) In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> Message-ID: On 29/12/21 01:26, Bart via lazarus wrote: > fpc -al ulen.pas > This will produce the file ulen.s > You can attach or copy that here. File is attached. The output from running this program is: % ./ulen Signed version Len = -100663283 Unsigned version Len = 1 To add another wrinkle to this, the signed and unsigned versions of the code both work if compiled with fpc 3.2.2. It's only when compiled with 3.3.1 that I get an incorrect result. -------------- next part -------------- # Begin asmlist al_procedures .text .align 4 .globl _P$ULEN_$$_UTF8LENGTHFAST_SIGNED$PCHAR$INT64$$INT64 _P$ULEN_$$_UTF8LENGTHFAST_SIGNED$PCHAR$INT64$$INT64: # [ulen.pas] # [28] begin stp x29,x30,[sp, #-16]! mov x29,sp sub sp,sp,#64 # Var p located at sp+0, size=OS_64 # Var ByteCount located at sp+8, size=OS_S64 # Var $result located at sp+16, size=OS_S64 # Var nx located at sp+24, size=OS_S64 # Var i located at sp+32, size=OS_S64 # Var cnt located at sp+40, size=OS_S64 # Var e located at sp+48, size=OS_S64 str x0,[sp] str x1,[sp, #8] # [29] Result := 0; str xzr,[sp, #16] # [30] e := ix+ByteCount; // End marker ldr x0,[sp] ldr x1,[sp, #8] add x0,x1,x0 str x0,[sp, #48] # [32] cnt := (not (ix-1)) and (sizeof(PtrInt)-1); ldr x0,[sp] sub x0,x0,#1 mvn x0,x0 and x0,x0,#7 str x0,[sp, #40] # [33] if cnt>ByteCount then ldr x0,[sp, #40] ldr x1,[sp, #8] cmp x0,x1 b.gt Lj7 b Lj8 Lj7: # [34] cnt := ByteCount; ldr x0,[sp, #8] str x0,[sp, #40] Lj8: # [35] for i := 1 to cnt do ldr x1,[sp, #40] cmp x1,#1 b.ge Lj9 b Lj10 Lj9: str xzr,[sp, #32] Lj11: ldr x0,[sp, #32] add x0,x0,#1 str x0,[sp, #32] # [43] Result += (pn8^ shr 7) and ((not pn8^) shr 6); ldr x0,[sp] ldrsb w0,[x0] eor w0,w0,#255 lsr w0,w0,#6 ldr x2,[sp] ldrsb w2,[x2] lsr w2,w2,#7 and w0,w2,w0 sxtw x0,w0 ldr x2,[sp, #16] add x0,x2,x0 str x0,[sp, #16] # [44] inc(pn8); ldr x0,[sp] add x0,x0,#1 str x0,[sp] ldr x0,[sp, #32] cmp x1,x0 b.le Lj13 b Lj11 Lj13: Lj10: # [47] for i := 1 to (ByteCount-cnt) div sizeof(PtrUInt) do ldr x0,[sp, #8] ldr x1,[sp, #40] sub x0,x0,x1 asr x1,x0,#63 add x0,x0,x1,lsr #61 asr x0,x0,#3 cmp x0,#1 b.ge Lj14 b Lj15 Lj14: str xzr,[sp, #32] Lj16: ldr x1,[sp, #32] add x1,x1,#1 str x1,[sp, #32] # [50] nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); ldr x1,[sp] ldr x1,[x1] and x1,x1,#0x8080808080808080 lsr x1,x1,#7 ldr x2,[sp] ldr x2,[x2] mvn x2,x2 lsr x2,x2,#6 and x1,x2,x1 str x1,[sp, #24] # [52] Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); ldr x1,[sp, #24] mov x2,#72340172838076673 mul x1,x1,x2 lsr x1,x1,#56 ldr x2,[sp, #16] add x1,x2,x1 str x1,[sp, #16] # [54] inc(pnx); ldr x1,[sp] add x1,x1,#8 str x1,[sp] ldr x1,[sp, #32] cmp x0,x1 b.le Lj18 b Lj16 Lj18: Lj15: # [57] while ixByteCount then ldr x0,[sp, #40] ldr x1,[sp, #8] cmp x0,x1 b.gt Lj24 b Lj25 Lj24: # [94] cnt := ByteCount; ldr x0,[sp, #8] str x0,[sp, #40] Lj25: # [95] for i := 1 to cnt do ldr x1,[sp, #40] cmp x1,#1 b.ge Lj26 b Lj27 Lj26: str xzr,[sp, #32] Lj28: ldr x0,[sp, #32] add x0,x0,#1 str x0,[sp, #32] # [103] Result += (pn8^ shr 7) and ((not pn8^) shr 6); ldr x0,[sp] ldrb w0,[x0] eor w0,w0,#255 lsr w0,w0,#6 ldr x2,[sp] ldrb w2,[x2] lsr w2,w2,#7 and w0,w2,w0 ldr x2,[sp, #16] add x0,x2,x0 str x0,[sp, #16] # [104] inc(pn8); ldr x0,[sp] add x0,x0,#1 str x0,[sp] ldr x0,[sp, #32] cmp x1,x0 b.le Lj30 b Lj28 Lj30: Lj27: # [107] for i := 1 to (ByteCount-cnt) div sizeof(PtrUInt) do ldr x0,[sp, #8] ldr x1,[sp, #40] sub x0,x0,x1 asr x1,x0,#63 add x0,x0,x1,lsr #61 asr x0,x0,#3 cmp x0,#1 b.ge Lj31 b Lj32 Lj31: str xzr,[sp, #32] Lj33: ldr x1,[sp, #32] add x1,x1,#1 str x1,[sp, #32] # [110] nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); ldr x1,[sp] ldr x1,[x1] and x1,x1,#0x8080808080808080 lsr x1,x1,#7 ldr x2,[sp] ldr x2,[x2] mvn x2,x2 lsr x2,x2,#6 and x1,x2,x1 str x1,[sp, #24] # [112] Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); ldr x1,[sp, #24] mov x2,#72340172838076673 mul x1,x1,x2 lsr x1,x1,#56 ldr x2,[sp, #16] add x1,x2,x1 str x1,[sp, #16] # [114] inc(pnx); ldr x1,[sp] add x1,x1,#8 str x1,[sp] ldr x1,[sp, #32] cmp x0,x1 b.le Lj35 b Lj33 Lj35: Lj32: # [117] while ix References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> Message-ID: <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> On 28/12/2021 15:50, Bart via lazarus wrote: > On Tue, Dec 28, 2021 at 3:39 PM Marco van de Voort via lazarus > wrote: > >> On what machine did you test? The settings if for the generated code, >> but the actual processor determines the effective speed. > I have a Intel i5 7th generation on my Win10-64 laptop from approx. > 2017 (so, it's really old for more modern folks than me). > > Compiled for 32-bit: > With -CpCOREI > Unsigned version with multiplication: 1359 > Unsigned version with PopCnt : 1282 > I have a core I7-8600 The diff between the old code and popcnt is less significant. old: 715 pop: 695 But there is a 3rd way, that is faster. add: 610 "nx" has a single "1" in each of the 8 bytes in a Qword (based on 64bit). If we regard each of this bytes as an entity of its own, then we can keep adding those "1". We could add the 1 of up to 255 iteration, before an overflow can happen. The example only does 128, as this avoids the "div" and "mod" operations. The full routine / incl benchmark for all 3 versions is attached For 64 bit: ? bc := (ByteCount-cnt) div sizeof(PtrInt); ? for j := 1 to bc >> 7 do begin ??? nx := 0; ??? for i := 0 to 127 do ??? begin ????? // Count bytes which are NOT the first byte of a character. ????? nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); ????? inc(pnx); ??? end; ??? nx := (nx and $00FF00FF00FF00FF) + ((nx >>? 8) and $00FF00FF00FF00FF); ??? nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); ??? nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); ??? Result := Result + nx; ? end; ? if (bc and 127) > 0 then begin ? nx := 0; ? for i := 1 to bc and 127 do ? begin ??? // Count bytes which are NOT the first byte of a character. ??? nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); ??? inc(pnx); ? end; ??? nx := (nx and $00FF00FF00FF00FF) + ((nx >>? 8) and $00FF00FF00FF00FF); ??? nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); ??? nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); ??? Result := Result + nx; ? end; -------------- next part -------------- program Project1; {$mode objfpc}{$H+} uses SysUtils; function UTF8LengthFast(p: PChar; ByteCount: PtrInt): PtrInt; const {$ifdef CPU32} ONEMASK =$01010101; EIGHTYMASK=$80808080; {$endif} {$ifdef CPU64} ONEMASK =$0101010101010101; EIGHTYMASK=$8080808080808080; {$endif} var pnx: PPtrInt absolute p; // To get contents of text in PtrInt blocks. x refers to 32 or 64 bits pn8: pint8 absolute pnx; // To read text as Int8 in the initial and final loops ix: PtrInt absolute pnx; // To read text as PtrInt in the block loop nx: PtrInt; // values processed in block loop i,cnt,e: PtrInt; begin Result := 0; e := ix+ByteCount; // End marker // Handle any initial misaligned bytes. cnt := (not (ix-1)) and (sizeof(PtrInt)-1); if cnt>ByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. //Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); Result += PopCnt(qword(nx)); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks bc := (ByteCount-cnt) div sizeof(PtrInt); for j := 1 to bc >> 7 do begin nx := 0; for i := 0 to 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; if (bc and 127) > 0 then begin nx := 0; for i := 1 to bc and 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; // Take care of any left-over bytes. while ix References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <01da83b2-a978-6153-28ad-92dd5e92943c@mfriebe.de> Message-ID: <8c375950-9400-0aa7-fc44-69fb3fc9862d@mfriebe.de> On 28/12/2021 23:18, Noel Duffy via lazarus wrote: > > The assembler produced by 3.2.2 looks like this: > > # [43] Result += (pn8^ shr 7) and ((not pn8^) shr 6); > ????ldr??? x0,[sp] > ????ldrsb??? w0,[x0] > ????mvn??? w0,w0 mvn => bitwise not. And that applies to the whole register. So I guess "eor??? w0,w0,#255 " is meant to be some optimization, but comes with a bug. From bartjunk64 at gmail.com Wed Dec 29 00:00:55 2021 From: bartjunk64 at gmail.com (Bart) Date: Wed, 29 Dec 2021 00:00:55 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: On Tue, Dec 28, 2021 at 11:35 PM Martin Frb via lazarus wrote: > I have a core I7-8600 > The diff between the old code and popcnt is less significant. > > old: 715 > pop: 695 > > But there is a 3rd way, that is faster. > add: 610 Not surprising that you should come up with a faster solution. IIRC you won both speed contests I had on the forum ;-) Feel free to implement it in LazUtf8. -- Bart From fpc at pascalprogramming.org Wed Dec 29 02:10:41 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Wed, 29 Dec 2021 02:10:41 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: On 28-12-2021 23:35, Martin Frb via lazarus wrote: > > > "nx" has a single "1" in each of the 8 bytes in a Qword (based on 64bit). > If we regard each of this bytes as an entity of its own, then we can > keep adding those "1". I also was thinking in that direction, but more about how to optimize that loop using SSE2 Some simple masking achieves the same (an 1 for each byte that starts with %10 bits) in 5 instructions, the load inclusive. Since 64-bit always supports SSE2, this could work: {$mode objfpc}{$H+} {$asmmode intel} uses sysutils,strutils; Type int128 = array[0..1] of int64; const ???? mask3?????? :? array[0..15] of byte? = ( $C0,$C0,$C0,$C0, ???????????????????????????????????????????????? $C0,$C0,$C0,$C0, ???????????????????????????????????????????????? $C0,$C0,$C0,$C0, ???????????????????????????????????????????????? $C0,$C0,$C0,$C0); ????? mask4?????? :? array[0..15] of byte? = (?? $80,$80,$80,$80, ???????????????????????????????????????????????? $80,$80,$80,$80, ???????????????????????????????????????????????? $80,$80,$80,$80, ???????????????????????????????????????????????? $80,$80,$80,$80); ????? mask2?????? :? array[0..15] of byte? = ( $1,$1,$1,$1, ?? ??? ??? ??? ??? ??? ? $1,$1,$1,$1, ???????????????????????????????????????????????? $1,$1,$1,$1, ???????????????????????????????????????????????? $1,$1,$1,$1); function utf8length(const s : pchar;var res:int128;len:integer):integer; // len is number of 16-byte counts to accumulate, max 255 I think // stores 16 bytes worth of counts in "res" begin ?asm ? movdqu xmm1,[rip+mask3]???????? // unaligned is SSE3, doesn't work on original X86_64 clawhammer? ? movdqu xmm2,[rip+mask4] ? movdqu xmm3,[rip+mask2] ? pxor xmm4,xmm4 @lbl: ? movdqu xmm0, [rcx] ? pand? xmm0,xmm1????? // mask out top 2 bits? ($C0) ? pcmpeqb xmm0,xmm2??? // compare with $80. sets byte to 11111111 or 00000000 ? pand? xmm0,xmm3????? // change to lsb (1/0) per byte only. ? paddb? xmm4,xmm0???? // add to cumulative ? add rcx,16 ? dec r8 ? jne @lbl ? movdqu [rdx],xmm4 end; // no volatile registers used. end; function countmask(nx:int64):integer; // Martin's routine that should be replaced by some punpkl magic, but it is too late now. begin ?? nx := (nx and $00FF00FF00FF00FF) + ((nx >>? 8) and $00FF00FF00FF00FF); ?? nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); ?? result := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); end; // one of each pattern. const pattern : array[0..3] of char = (chr(%11001001),chr(%10001001), chr(%00001001),chr(%01001001)); const testblocks = 5; var s : string; ??? i,j,cnt : integer; ??? r : int128; begin ? randomize; ? setlength(s,testblocks*16); ? // random string but keep a count of bytes with high value %10 ? cnt:=0; ? for i:=0 to testblocks*16-1 do ??? begin ????? j:=random(4); ????? if j=1 then inc(cnt); ????? s[i+1]:=pattern[j]; ??? end; ? utf8length(pchar(s),r,testblocks+1); ? writeln(cnt,' = ',countmask(r[0])+countmask(r[1])); //? writeln(inttohex(r[0],16)); //? writeln(inttohex(r[1],16)); end. From lazarus at mfriebe.de Wed Dec 29 10:16:39 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Wed, 29 Dec 2021 10:16:39 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: On 29/12/2021 02:10, Marco van de Voort via lazarus wrote: > > On 28-12-2021 23:35, Martin Frb via lazarus wrote: >> >> >> "nx" has a single "1" in each of the 8 bytes in a Qword (based on >> 64bit). >> If we regard each of this bytes as an entity of its own, then we can >> keep adding those "1". > > I also was thinking in that direction, but more about how to optimize > that loop using SSE2 good idea... > > // Martin's routine that should be replaced by some punpkl magic, but > it is too late now. Why too late? There is a place for both. My routine works fine for cpu. (soon as a 32 bit (and maybe 16 bit) variant are added). Then for known cpu, special handling can be added. From fpc at pascalprogramming.org Wed Dec 29 13:42:31 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Wed, 29 Dec 2021 13:42:31 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: On 29-12-2021 10:16, Martin Frb via lazarus wrote: > >> // Martin's routine that should be replaced by some punpkl magic, but >> it is too late now. > > Why too late? See datetime stamp.? 02:10 AM. I don't know how it is with you Lazarus devels, but we FPC devels need our beauty sleep from time to time. So I'll be working on it some more today, possible things to do are: -? fold countmask into the asm, and -? maybe do another outer loop to process all bytes in one go. -? special case for only a few bytes - maybe align to 16-byte (easier/cheaper for old machines than all the movdqu). - Maybe test the code on godbolt for performance At the very least I hope the punpkl code for countmask will be more compact since it doesn't require literals. > There is a place for both. My routine works fine for cpu. (soon as a > 32 bit (and maybe 16 bit) variant are added). > > Then for known cpu, special handling can be added. Could you post full source if you haven't already? For a bit of benchmarking. I just wrote it from the top of my head, and I assumed 5 instructions for 16-byte would win any time, but haven't verified anything yet. For 32-bit this can also be done, but you'd need a procedure variable to test for SSE support, and only run this for long strings (>200-500 or so) p.s. is there a workaround for git worktree to work on the same branch? E.g. trunk for 32-bit and trunk for 64-bit ? :-) From florian at freepascal.org Wed Dec 29 14:30:28 2021 From: florian at freepascal.org (=?UTF-8?Q?Florian_Kl=c3=a4mpfl?=) Date: Wed, 29 Dec 2021 14:30:28 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: <07b860af-a12e-f462-0b4f-cee01f05715b@freepascal.org> Am 29.12.2021 um 13:42 schrieb Marco van de Voort via lazarus: > > p.s. is there a workaround for git worktree to work on the same branch? E.g. trunk for 32-bit and trunk for 64-bit ? :-) > No. You cannot checkout the same branch in two worktrees. But you can do the following: create a new branch in both worktrees and check them out (main_32bit and main_64bit), do in both directories: work, commit, pull, checkout main, merge the branch, push: cd git/fpc/main1 git checkout -b main_32bit git worktree add ../main2 -b main_64bit cd main1 git checkout main git pull git merge main_32bit git push cd ../main2 git checkout main git pull git merge main_64bit git push From lazarus at mfriebe.de Wed Dec 29 16:30:30 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Wed, 29 Dec 2021 16:30:30 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: <2d9d71d8-8b94-9609-6ed8-dbefc7e15918@mfriebe.de> On 29/12/2021 13:42, Marco van de Voort via lazarus wrote: > > On 29-12-2021 10:16, Martin Frb via lazarus wrote: >> >>> // Martin's routine that should be replaced by some punpkl magic, >>> but it is too late now. >> >> Why too late? > > See datetime stamp.? 02:10 AM. I don't know how it is with you Lazarus > devels, but we FPC devels need our beauty sleep from time to time. Oh, too late for the day. I thought. Too late, that ship has sailed. > > Could you post full source if you haven't already? For a bit of > benchmarking. I just wrote it from the top of my head, and I assumed 5 > instructions for 16-byte would win any time, but haven't verified > anything yet. I had it attached on my last mail. Attached it again here. (3rd procedure / "Utf8LengthAdd") It is only 64bit for now. (And not cleaned up in any way). Also changing "bc >> 7" and "bc and 127" to "moddiv(bc, 255, full, remain)" might save a few more ms. But probably needs larger data to benchmark. If you do work on this, feel free to integrate my code as the baseline for cpu without SSE. Otherwise, it might be a bit until I get to it. -------------- next part -------------- A non-text attachment was scrubbed... Name: Project1.pas Type: application/x-unknown-content-type Size: 6800 bytes Desc: not available URL: From fpc at pascalprogramming.org Wed Dec 29 17:19:44 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Wed, 29 Dec 2021 17:19:44 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <2d9d71d8-8b94-9609-6ed8-dbefc7e15918@mfriebe.de> References: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <2d9d71d8-8b94-9609-6ed8-dbefc7e15918@mfriebe.de> Message-ID: <4b3c1970-e5ad-4405-393c-a1ff7720a44d@pascalprogramming.org> On 29-12-2021 16:30, Martin Frb via lazarus wrote: > > >> >> Could you post full source if you haven't already? For a bit of >> benchmarking. I just wrote it from the top of my head, and I assumed >> 5 instructions for 16-byte would win any time, but haven't verified >> anything yet. > I had it attached on my last mail. Attached it again here. (3rd > procedure / "Utf8LengthAdd") > > It is only 64bit for now. (And not cleaned up in any way). > > Also changing "bc >> 7" and "bc and 127" > to "moddiv(bc, 255, full, remain)" might save a few more ms. But > probably needs larger data to benchmark. > > If you do work on this, feel free to integrate my code as the baseline > for cpu without SSE. > Otherwise, it might be a bit until I get to it. > First results: (on an ageing i7-3770, trunk FPC -O4 -Cpcoreavx) fst 781 fst 781 fst 797 fst 766 pop 656 pop 641 pop 640 pop 641 add 562 add 578 add 563 add 594 asm 297 asm 296 asm 297 asm 297 Asm is nearly fully functional and working, more importantly the remaining issues are constant time and single instruction work, shouldn't influence benchmarking for anything than the shortest sequences. I'll finish up and post the whole shebang, since more eyes could help, I'm an asm amateur in some regards. From fpc at pascalprogramming.org Thu Dec 30 01:29:14 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 30 Dec 2021 01:29:14 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: On 29-12-2021 00:00, Bart via lazarus wrote: > On Tue, Dec 28, 2021 at 11:35 PM Martin Frb via lazarus > wrote: > >> I have a core I7-8600 >> The diff between the old code and popcnt is less significant. >> >> old: 715 >> pop: 695 >> >> But there is a 3rd way, that is faster. >> add: 610 > Not surprising that you should come up with a faster solution. > IIRC you won both speed contests I had on the forum ;-) > > Feel free to implement it in LazUtf8. New unit test, with Martin's integrated. If I play with godbolt, Ryzen zen3 (ryzen 5x00X) is nearly twice as fast in cycles as my Ivy Bridge, so I would like to see some benchmarks from various processors. Also from very old ones (P4 and Clawhammers) to test instruction sets. I use unaligned loads which afaik on older ( pre Core 1st or 2nd generation) CPUs are costly. (because it loads two caches lines per time) -------------- next part -------------- // // (C) 2021 Martin Friebe and Marco van de Voort. // attempt to accelerate utf8lengthfast which is a length(s) in utf8 codepoints without integrity checking // // 4 versions. // - Original, // - with popcount and // - the "add" variant that accumulates 127 iterations of ptrints and only adds // the intermeidates outside that loop // - a SSE2 version loosely inspired by the add variant combined with // the core of an existing (branchless) binarization routine for the main loop. {$mode objfpc}{$H+} {$asmmode intel} {define asmdebug} uses SysUtils,StrUtils; const mask3 : array[0..15] of byte = ( $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0); mask4 : array[0..15] of byte = ( $80,$80,$80,$80, $80,$80,$80,$80, $80,$80,$80,$80, $80,$80,$80,$80); mask2 : array[0..15] of byte = ( $1,$1,$1,$1, $1,$1,$1,$1, $1,$1,$1,$1, $1,$1,$1,$1); // Integer arguments are passed in registers RCX, RDX, R8, and R9. // Floating point arguments are passed in XMM0L, XMM1L, XMM2L, and XMM3L. // volatile: RAX, RCX, RDX, R8, R9, R10, R11 // nonvolatile RBX, RBP, RDI, RSI, RSP, R12, R13, R14, and R15 are considered nonvolatile // volatile xmm0-xmm3 (params) en xmm4,5 // https://msdn.microsoft.com/en-us/library/ms235286.aspx {$ifdef asmdebug} function asmutf8length(const s : pchar;res:pbyte;len:integer):int64; {$else} function asmutf8length(const s : pchar;len:integer):int64; {$endif} begin asm {$ifndef asmdebug} mov r8,rdx {$endif} // using broadcast etc raises requirements? mov rax,r8 mov r9,r8 // tuning for short strings: // ------ // test rax,rax // je @theend // cmp r9,128 // difference between long and short // jl @restbytes and r9,15 shr r8,4 pxor xmm5,xmm5 // always zero pxor xmm6,xmm6 // dword counts movdqu xmm1,[rip+mask3] movdqu xmm2,[rip+mask4] movdqu xmm3,[rip+mask2] test r8,r8 je @restbytes @outer: mov r10,127 // max iterations cmp r10,r8 // more or less left? jl @last // more mov r10,r8 // less @last: sub r8,r10 // iterations left - iterations to do pxor xmm4,xmm4 @inner: movdqu xmm0, [rcx] pand xmm0,xmm1 // mask out top 2 bits pcmpeqb xmm0,xmm2 // compare with $80. pand xmm0,xmm3 // change to $1 per byte. paddb xmm4,xmm0 // add to cumulative add rcx,16 dec r10 jne @inner // process 127 iterations movdqa xmm0,xmm4 PUNPCKLBW xmm0,xmm5 // zero extend to words PUNPCKHBW xmm4,xmm5 paddw xmm0,xmm4 // add, now 8 16-bit words. movdqa xmm4,xmm0 PUNPCKLWD xmm0,xmm5 // zero extend to dwords paddd xmm6,xmm0 PUNPCKHWD xmm4,xmm5 paddd xmm6,xmm4 // add both to cumulative test r8,r8 jne @outer MOVHLPS xmm4,xmm6 // move high 8 bytes to low (float->int penalty?) paddd xmm6,xmm4 // add both 2*dwords (high doesn't matter) pshufd xmm4,xmm6,1 // mov 2nd dword in xmm6 to first in xmm4 paddd xmm6,xmm4 // add movd r8d,xmm6 // to int alu reg sub rax,r8 // subtract from length in bytes. @restbytes: test r9,r9 je @theend // Done! @restloop: movzx r8d, byte [rcx] // unaligned bytes after sse loop mov r10,r8 shr r10,7 not r8 shr r8,6 and r10,r8 sub rax,r10 inc rcx dec r9 jne @restloop @theend: end['xmm5','xmm6']; // volatile registers used. end; function countmask(nx:int64):integer; begin nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); result := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); end; function UTF8LengthFast(p: PChar; ByteCount: PtrInt): PtrInt; const {$ifdef CPU32} ONEMASK =$01010101; EIGHTYMASK=$80808080; {$endif} {$ifdef CPU64} ONEMASK =$0101010101010101; EIGHTYMASK=$8080808080808080; {$endif} var pnx: PPtrInt absolute p; // To get contents of text in PtrInt blocks. x refers to 32 or 64 bits pn8: pint8 absolute pnx; // To read text as Int8 in the initial and final loops ix: PtrInt absolute pnx; // To read text as PtrInt in the block loop nx: PtrInt; // values processed in block loop i,cnt,e: PtrInt; begin Result := 0; e := ix+ByteCount; // End marker // Handle any initial misaligned bytes. cnt := (not (ix-1)) and (sizeof(PtrInt)-1); if cnt>ByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. //Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); Result += PopCnt(qword(nx)); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks bc := (ByteCount-cnt) div sizeof(PtrInt); for j := 1 to bc >> 7 do begin nx := 0; for i := 0 to 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; if (bc and 127) > 0 then begin nx := 0; for i := 1 to bc and 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; // Take care of any left-over bytes. while ixnil then // runtime debugger detection {$endif} readln; end. From fpc at pascalprogramming.org Thu Dec 30 02:07:01 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 30 Dec 2021 02:07:01 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: <97a78afa-0366-5d12-cc64-abf2a68a2d0e@pascalprogramming.org> On 30-12-2021 01:29, Marco van de Voort via lazarus wrote: > > > ?(P4 and Clawhammers) to test instruction sets. 64-bit supporting Pentium 4's of course, since this is a 64-bit only test. Claw/Sledgehammer is the first iteration of Athlon64 and of the x86_64 architecture as a whole and misses some instructions (like SSE3) that all other 64-bit capable CPUs have. From support at uvviewsoft.com Thu Dec 30 08:23:35 2021 From: support at uvviewsoft.com (Alexey Tor.) Date: Thu, 30 Dec 2021 10:23:35 +0300 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> Message-ID: <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> > New unit test, with Martin's integrated. If I play with godbolt, Ryzen > zen3 (ryzen 5x00X) is nearly twice as fast in cycles as my Ivy Bridge, > so I would like to see some benchmarks from various processors. Also > from very old ones (P4 and Clawhammers) to test instruction sets. Project utf8lentest raised exception class 'External: SIGSEGV'. ?In file 'utf8lentest.lpr' at line 89: movdqu xmm0, [rcx] OS: Linux x64. CPU: ?? vendor_id = "GenuineIntel" ????? (simple synth)? = Intel Core (unknown type) (Sandy Bridge D2/J1/Q0) {Sandy Bridge}, 32nm From florian at freepascal.org Thu Dec 30 10:15:25 2021 From: florian at freepascal.org (=?UTF-8?Q?Florian_Kl=c3=a4mpfl?=) Date: Thu, 30 Dec 2021 10:15:25 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> References: <53fc2dc3-b786-4a81-425f-74d2391b9051@pascalprogramming.org> <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> Message-ID: Am 30.12.21 um 08:23 schrieb Alexey Tor. via lazarus: > >> New unit test, with Martin's integrated. If I play with godbolt, Ryzen >> zen3 (ryzen 5x00X) is nearly twice as fast in cycles as my Ivy Bridge, >> so I would like to see some benchmarks from various processors. Also >> from very old ones (P4 and Clawhammers) to test instruction sets. > Project utf8lentest raised exception class 'External: SIGSEGV'. > > ?In file 'utf8lentest.lpr' at line 89: > > movdqu xmm0, [rcx] > > OS: Linux x64. CPU: Linux uses different calling conventions, please check with the patch below. > > ?? vendor_id = "GenuineIntel" > ????? (simple synth)? = Intel Core (unknown type) (Sandy Bridge > D2/J1/Q0) {Sandy Bridge}, 32nm > 15c15 < {define asmdebug} --- > { $define asmdebug} 46c46 < function asmutf8length(const s : pchar;len:integer):int64; --- > function asmutf8length(const s : pchar;len:int64):int64;assembler;nostackframe; 49d48 < begin 52c51 < mov r8,rdx --- > mov r8,len 89c88 < movdqu xmm0, [rcx] --- > movdqu xmm0, oword ptr [s] 95c94 < add rcx,16 --- > add s,16 128c127 < movzx r8d, byte [rcx] // unaligned bytes after sse loop --- > movzx r8d, byte [s] // unaligned bytes after sse loop 135c134 < inc rcx --- > inc s 140c139 < end['xmm5','xmm6']; // volatile registers used. --- > ret From fpc at pascalprogramming.org Thu Dec 30 13:58:21 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 30 Dec 2021 13:58:21 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> Message-ID: On 30-12-2021 10:15, Florian Kl?mpfl via lazarus wrote: > > Linux uses different calling conventions, please check with the patch > below. > Linux is quite generous with the volatile registers, so luckily it matches quite closely. I first tried the approach of your patch, but [s] has problems on windows, so would require ifdef on every "s"use, so I simply move [s] to rcx ? {$ifndef Windows} ? // we can't use [s] as an alias for the pointer parameter, because the non assembler procedure on Windows ?// changes that into a stack reference. FPC doesn't support non volatile frame management for assembler procs like Delphi does. ? mov rcx,s???????? // rdi ? mov edx,len?????? // rsi ? {$endif} and the ifdeffing of the assembler procedure on linux vs inline asm block on Windows. Then it works on Linux x86_64. Funnily, our server AMD Athlon 200GE (Zen1, 3.2GHz?) nearly the exact same timings as my i7-3770 3.4GHz I did some other minor work after last post, so here is now the entire program: -------------- next part -------------- // // (C) 2021 Martin Friebe and Marco van de Voort. // attempt to accelerate utf8lengthfast which is a length(s) in utf8 codepoints without integrity checking // // 4 versions. // - Original, // - with popcount and // - the "add" variant that accumulates 127 iterations of ptrints and only adds // the intermeidates outside that loop // - a SSE2 version loosely inspired by the add variant combined with // the core of an existing (branchless) binarization routine for the main loop. {$mode objfpc}{$H+} {$asmmode intel} {$coperators on} {define asmdebug} uses SysUtils,StrUtils; const mask3 : array[0..15] of byte = ( $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0, $C0,$C0,$C0,$C0); mask4 : array[0..15] of byte = ( $80,$80,$80,$80, $80,$80,$80,$80, $80,$80,$80,$80, $80,$80,$80,$80); mask2 : array[0..15] of byte = ( $1,$1,$1,$1, $1,$1,$1,$1, $1,$1,$1,$1, $1,$1,$1,$1); // Integer arguments are passed in registers RCX, RDX, R8, and R9. // Floating point arguments are passed in XMM0L, XMM1L, XMM2L, and XMM3L. // volatile: RAX, RCX, RDX, R8, R9, R10, R11 // nonvolatile RBX, RBP, RDI, RSI, RSP, R12, R13, R14, and R15 are considered nonvolatile // volatile xmm0-xmm3 (params) en xmm4,5 // https://msdn.microsoft.com/en-us/library/ms235286.aspx {$ifdef asmdebug} function asmutf8length(const s : pchar;len:integer,res:pbyte):int64; {$else} function asmutf8length(const s : pchar;len:integer):int64;{$ifndef Windows}assembler; nostackframe;{$endif} {$endif} {$ifdef Windows} begin {$endif} asm // tuning for short strings: // ------ {$ifndef Windows} // we can't use [s] as an alias for the pointer parameter, because the non assembler procedure on Windows // changes that into a stack reference. FPC doesn't support non volatile frame management for assembler procs like Delphi does. mov rcx,s // rdi mov edx,len // rsi {$endif} test rax,rax je @theend cmp rdx,128 // threshold between long and short. jl @restbytes mov rax,rdx mov r10,rcx and r10,15 mov r9,16 sub r9,r10 and r9,15 test r9,r9 je @nopreloop sub rdx,r9 @preloop: // roughly 2 cycles per iteration on ivy bridge movzx r11d, byte [rcx] // unaligned bytes after sse loop mov r10,r11 shr r10,7 not r11 shr r11,6 and r10,r11 sub rax,r10 inc rcx dec r9 jne @preloop @nopreloop: mov r9,rdx and r9,15 shr rdx,4 pxor xmm5,xmm5 // always zero pxor xmm6,xmm6 // dword counts // using broadcast etc raises requirements? -> use constant loads. movdqu xmm1,[rip+mask3] movdqu xmm2,[rip+mask4] movdqu xmm3,[rip+mask2] test rdx,rdx je @restbytes @outer: mov r10,127 // max iterations per inner loop cmp r10,rdx // more or less left? jl @last // more mov r10,rdx // less @last: sub rdx,r10 // iterations left - iterations to do pxor xmm4,xmm4 // process 127 iterations (limit of signed int8) @inner: // +/- 2.2 cycles per iteration for 16 bytes on ivy bridge movdqu xmm0, [rcx] pand xmm0,xmm1 // mask out top 2 bits pcmpeqb xmm0,xmm2 // compare with $80. pand xmm0,xmm3 // change to $1 per byte. paddb xmm4,xmm0 // add to cumulative add rcx,16 dec r10 jne @inner // SSSE3 vertical adds might help this, but increase CPU reqs. movdqa xmm0,xmm4 PUNPCKLBW xmm0,xmm5 // zero extend to words PUNPCKHBW xmm4,xmm5 paddw xmm0,xmm4 // add, now 8 16-bit words. movdqa xmm4,xmm0 PUNPCKLWD xmm0,xmm5 // zero extend to dwords paddd xmm6,xmm0 PUNPCKHWD xmm4,xmm5 paddd xmm6,xmm4 // add both L and H to cumulative 4x dword xmm6 reg test rdx,rdx jne @outer MOVHLPS xmm4,xmm6 // move high 8 bytes to low (no float->int penalty for move only?) paddd xmm6,xmm4 // add both 2*dwords (high doesn't matter) pshufd xmm4,xmm6,1 // mov 2nd dword in xmm6 to first in xmm4 paddd xmm6,xmm4 // add movd edx,xmm6 // to int alu reg sub rax,rdx // subtract from length in bytes. @restbytes: test r9,r9 je @theend // Done! @restloop: movzx edx, byte [rcx] // unaligned bytes after sse loop mov r10,rdx shr r10,7 not rdx shr rdx,6 and r10,rdx sub rax,r10 inc rcx dec r9 jne @restloop @theend: {$ifdef Windows} end['xmm6']; // volatile registers used. {$endif} end; function countmask(nx:int64):integer; begin nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); result := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); end; function UTF8LengthFast(p: PChar; ByteCount: PtrInt): PtrInt; const {$ifdef CPU32} ONEMASK =$01010101; EIGHTYMASK=$80808080; {$endif} {$ifdef CPU64} ONEMASK =$0101010101010101; EIGHTYMASK=$8080808080808080; {$endif} var pnx: PPtrInt absolute p; // To get contents of text in PtrInt blocks. x refers to 32 or 64 bits pn8: pint8 absolute pnx; // To read text as Int8 in the initial and final loops ix: PtrInt absolute pnx; // To read text as PtrInt in the block loop nx: PtrInt; // values processed in block loop i,cnt,e: PtrInt; begin Result := 0; e := ix+ByteCount; // End marker // Handle any initial misaligned bytes. cnt := (not (ix-1)) and (sizeof(PtrInt)-1); if cnt>ByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks for i := 1 to (ByteCount-cnt) div sizeof(PtrInt) do begin // Count bytes which are NOT the first byte of a character. nx := ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); {$push}{$overflowchecks off} // "nx * ONEMASK" causes an arithmetic overflow. //Result += (nx * ONEMASK) >> ((sizeof(PtrInt) - 1) * 8); Result += PopCnt(qword(nx)); {$pop} inc(pnx); end; // Take care of any left-over bytes. while ixByteCount then cnt := ByteCount; for i := 1 to cnt do begin // Is this byte NOT the first byte of a character? Result += (pn8^ shr 7) and ((not pn8^) shr 6); inc(pn8); end; // Handle complete blocks bc := (ByteCount-cnt) div sizeof(PtrInt); for j := 1 to bc >> 7 do begin nx := 0; for i := 0 to 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; if (bc and 127) > 0 then begin nx := 0; for i := 1 to bc and 127 do begin // Count bytes which are NOT the first byte of a character. nx += ((pnx^ and EIGHTYMASK) shr 7) and ((not pnx^) shr 6); inc(pnx); end; nx := (nx and $00FF00FF00FF00FF) + ((nx >> 8) and $00FF00FF00FF00FF); nx := (nx and $0000FFFF0000FFFF) + ((nx >> 16) and $0000FFFF0000FFFF); nx := (nx and $00000000FFFFFFFF) + ((nx >> 32) and $00000000FFFFFFFF); Result := Result + nx; end; // Take care of any left-over bytes. while ixnil then // runtime debugger detection {$endif} readln; end. From jmlandmesser at gmx.de Thu Dec 30 14:17:40 2021 From: jmlandmesser at gmx.de (John Landmesser) Date: Thu, 30 Dec 2021 14:17:40 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: References: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> Message-ID: <5db0de09-b8c9-7a62-9f34-2343173cd55b@gmx.de> Perhaps usefui test information from my PC: ****************************************** [john1 at manjaro sdb2]$ ./utf8lentest 234526968 fst:128406168 pop:128406168 add:128406168 asm:128406168 29315871 fst 1365 fst 1367 fst 1366 fst 1366 pop 9990 pop 9990 pop 9997 pop 9981 add 1386 add 1382 add 1386 add 1390 asm 346 asm 346 asm 346 asm 349 fst 1357 fst 1368 fst 1372 fst 1371 pop 10681 pop 6886 pop 6895 pop 6916 add 1247 add 1248 add 1250 add 1248 asm 295 asm 291 asm 291 asm 293 [john1 at manjaro sdb2]$ [john1 at manjaro sdb2]$ inxi -F System: ? Host: manjaro Kernel: 5.10.84-1-MANJARO x86_64 bits: 64 ??? Desktop: Xfce 4.16.0 Distro: Manjaro Linux Machine: ? Type: Laptop System: LENOVO product: 81RS v: Lenovo Yoga S740-14IIL ??? serial: ? Mobo: LENOVO model: LNVNB161216 v: SDK0J40709 WIN ??? serial: UEFI: LENOVO v: BYCN39WW date: 05/28/2021 Battery: ? ID-1: BAT0 charge: 62.4 Wh (95.6%) condition: 65.3/62.0 Wh (105.3%) CPU: ? Info: quad core model: Intel Core i7-1065G7 bits: 64 type: MT MCP cache: ??? L2: 2 MiB ? Speed (MHz): avg: 3520 min/max: 400/3900 cores: 1: 3543 2: 3890 3: 2319 ??? 4: 3513 5: 3709 6: 3650 7: 3792 8: 3749 Graphics: ? Device-1: Intel Iris Plus Graphics G7 driver: i915 v: kernel ? Device-2: NVIDIA GP108M [GeForce MX250] driver: nvidia v: 495.44 ? Device-3: Chicony Integrated Camera type: USB driver: uvcvideo ? Display: x11 server: X.Org 1.21.1.2 driver: loaded: modesetting,nvidia ??? unloaded: nouveau resolution: 1: 1920x1080~60Hz 2: 1920x1080~60Hz ? Message: Unable to show advanced data. Required tool glxinfo missing. Audio: ? Device-1: Intel Ice Lake-LP Smart Sound Audio driver: sof-audio-pci ? Sound Server-1: ALSA v: k5.10.84-1-MANJARO running: yes ? Sound Server-2: PipeWire v: 0.3.40 running: yes Network: ? Device-1: Intel Ice Lake-LP PCH CNVi WiFi driver: iwlwifi ? IF: wlp0s20f3 state: up mac: 04:33:c2:02:de:51 ? Device-2: Realtek RTL8153 Gigabit Ethernet Adapter type: USB ??? driver: r8152 ? IF: enp0s13f0u1u4 state: up speed: 1000 Mbps duplex: full ??? mac: 4c:e1:73:42:1f:6b ? IF-ID-1: pan1 state: down mac: 7a:5c:6a:f4:06:56 Bluetooth: ? Device-1: Intel AX201 Bluetooth type: USB driver: btusb ? Report: rfkill ID: hci0 state: up address: see --recommends Drives: ? Local Storage: total: 1.86 TiB used: 317.16 GiB (16.7%) ? ID-1: /dev/nvme0n1 vendor: Micron model: MTFDHBA1T0TCK size: 953.87 GiB ? ID-2: /dev/sda type: USB vendor: Western Digital model: WD10EARX-00N0YB0 ??? size: 931.51 GiB ? ID-3: /dev/sdb type: USB vendor: Kingston model: DataTraveler 2.0 ??? size: 14.54 GiB Partition: ? ID-1: / size: 57.9 GiB used: 35.88 GiB (62.0%) fs: ext4 dev: /dev/nvme0n1p8 ? ID-2: /boot/efi size: 259.5 MiB used: 114.1 MiB (44.0%) fs: vfat ??? dev: /dev/nvme0n1p1 Swap: ? ID-1: swap-1 type: partition size: 16.67 GiB used: 0 KiB (0.0%) ??? dev: /dev/nvme0n1p9 Sensors: ? System Temperatures: cpu: 58.0 C mobo: N/A ? Fan Speeds (RPM): N/A Info: ? Processes: 289 Uptime: 9m Memory: 15.2 GiB used: 2.19 GiB (14.4%) ? Shell: Bash inxi: 3.3.11 ***************************************** Am 30.12.21 um 13:58 schrieb Marco van de Voort via lazarus: > > On 30-12-2021 10:15, Florian Kl?mpfl via lazarus wrote: >> >> Linux uses different calling conventions, please check with the patch >> below. >> > Linux is quite generous with the volatile registers, so luckily it > matches quite closely. > > I first tried the approach of your patch, but [s] has problems on > windows, so would require ifdef on every "s"use, so I simply move [s] > to rcx > > ? {$ifndef Windows} > ? // we can't use [s] as an alias for the pointer parameter, because > the non assembler procedure on Windows > ?// changes that into a stack reference. FPC doesn't support non > volatile frame management for assembler procs like Delphi does. > ? mov rcx,s???????? // rdi > ? mov edx,len?????? // rsi > ? {$endif} > > and the ifdeffing of the assembler procedure on linux vs inline asm > block on Windows. Then it works on Linux x86_64. > > Funnily, our server AMD Athlon 200GE (Zen1, 3.2GHz?) nearly the exact > same timings as my i7-3770 3.4GHz > > I did some other minor work after last post, so here is now the entire > program: > From fpc at pascalprogramming.org Thu Dec 30 14:43:19 2021 From: fpc at pascalprogramming.org (Marco van de Voort) Date: Thu, 30 Dec 2021 14:43:19 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <5db0de09-b8c9-7a62-9f34-2343173cd55b@gmx.de> References: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> <5db0de09-b8c9-7a62-9f34-2343173cd55b@gmx.de> Message-ID: <54dd8cdc-c40b-75b3-d9c2-950d36b6137e@pascalprogramming.org> On 30-12-2021 14:17, John Landmesser via lazarus wrote: > Perhaps usefui test information from my PC: 77 > Compile with -O4 -Cpcoreavx2 , the others (non asm) will become faster, my guess is? "add" will be about double of asm. Also, on windows "high performance" as power scheme.? On non windows try to disable power saving for a short while another way. If the first and second runs deviate it is probably the CPU throttling up. From lazarus at mfriebe.de Thu Dec 30 15:09:04 2021 From: lazarus at mfriebe.de (Martin Frb) Date: Thu, 30 Dec 2021 15:09:04 +0100 Subject: [Lazarus] Faster than popcnt [[Re: UTF8LengthFast returning incorrect results on AARCH64 (MacOS)]] In-Reply-To: <54dd8cdc-c40b-75b3-d9c2-950d36b6137e@pascalprogramming.org> References: <7cb2d9da-633a-532a-f7f2-e512957f26de@pascalprogramming.org> <351c698e-6a69-30fb-42de-ccb483e7ce16@pascalprogramming.org> <3aca5903-38c8-f267-4feb-c6c24f20875f@mfriebe.de> <6c309bb5-270d-5279-4c38-21cc5c200ce1@uvviewsoft.com> <5db0de09-b8c9-7a62-9f34-2343173cd55b@gmx.de> <54dd8cdc-c40b-75b3-d9c2-950d36b6137e@pascalprogramming.org> Message-ID: On 30/12/2021 14:43, Marco van de Voort via lazarus wrote: > Compile with -O4 -Cpcoreavx2 , the others (non asm) will become > faster, my guess is? "add" will be about double of asm. Core I7 8700K 3.3.1 from Dec 10th 3.2.3 from Dec 9th With fpc 3.3.1: - fst is worse? - add gets better -O4 -Cpcoreavx2 fpc 3.2.3 /?? fpc 3.3.1 fst 594?????? fst 688 fst 578?????? fst 703 fst 578?????? fst 687 fst 562?????? fst 688 pop 485?????? pop 485 pop 500?????? pop 500 pop 500?????? pop 484 pop 484?????? pop 500 add 594?????? add 422 add 578?????? add 438 add 578?????? add 437 add 594?????? add 453 asm 250?????? asm 250 asm 250?????? asm 250 asm 250?????? asm 250 asm 250?????? asm 266 fpc 3.2.3 -O4 -Cpcoreavx?????????? -O4 -CpCOREI fst 594????????????????? fst 593 fst 578????????????????? fst 579 fst 578????????????????? fst 562 fst 594????????????????? fst 578 pop 500????????????????? pop 500 pop 515????????????????? pop 500 pop 500????????????????? pop 500 pop 485????????????????? pop 485 add 593????????????????? add 593 add 579????????????????? add 578 add 578????????????????? add 594 add 593????????????????? add 594 asm 250????????????????? asm 250 asm 250????????????????? asm 250 asm 235????????????????? asm 250 asm 250????????????????? asm 250 From michael at freepascal.org Thu Dec 30 16:39:36 2021 From: michael at freepascal.org (Michael Van Canneyt) Date: Thu, 30 Dec 2021 16:39:36 +0100 (CET) Subject: [Lazarus] More Webassembly... Message-ID: Hello, I've just committed to the FPC git repo the wasmtime unit. This unit can be used to load and use the wasmtime library. Using this library you can compile, load and run Webassembly files embedded in your FPC program on all FPC & webassembly supported native platforms. There are 4 demo programs that show how this can be done. This means that you can now have a FPC-compiled webassembly module and run it in the browser or in a native binary. Additionally, all webassembly modules out there should also be callable from both native FPC and Pas2JS programs. Enjoy ! Michael.