[Lazarus] Parser

[Mattias Gaertner]

On Fri, 02 Jul 2010 10:02:20 +0300
Adem <listmember at letterboxes.org> wrote:

>   On 2010-07-02 09:23, Mattias Gaertner wrote:
> > On Fri, 02 Jul 2010 07:49:25 +0300
> > Adem<listmember at letterboxes.org>  wrote
> >> [...]
> >> Well.. the whole point is to *not* maintain any more than *one* parser
> >> --and, that includes current n parsers (I am not sure what the value of
> >> 'n' is).
> >>
> >> That is the target.
> > Means?
> > I still don't understand what do you really want to achieve.
> Call it some form of perfectionism, if you like. Or 'unionism', for a 
> slightly better term. (reasons below)
> > There are currently several different pascal parsers with different abilities for different purposes. For example the fpc one for compiling, synedit for highlighting, codetools has several for directives, declarations,  indentations, fcl has one basic and extendable parser and so forth.
> > Each one has abilities that the others do not have.
> That's just it: There are simply too many of them neither one is as 
> complete or up-to-date as the original.
> 
> And, the reason there are too many simply boils down to the fact that 
> the real/genuine/true one wasn't/isn't available.

No.
It's about speed and memory structures.
Just think about a general parser. It must handle macros, directives, include files,
spaces, encodings, comments, line endings, syntax errors. It must work
for code snippets and needs a bunch of flags what to ignore while
parsing.
It creates an output with a lot of different nodes and
every code that accesses this output must handle this vast amount of
information and must be updated whenever the parser is changed.

 
> I mean, it's not as if you need a completely different parser for IDE 
> purposes; the fact that you have to write and maintain one is because 
> you have to do it.
> 
> And, as expected, each one of those baby-parsers are merely partial 
> emulators of the actual one.

No. 
As I said: Every parser has features that the others don't have.
For example the synedit parser can parse code snippets
very fast, because it can save/restore its whole state very fast.
The fpc parser can not do that.

 
> I am surprised you (plural) don't see this as waste of resources, but 
> instead tell *me* it would be a waste of time to refactor the fpc-code 
> so that its parser can be used (or extended, as the case may be) as a 
> blackbox module downstream in all those projects which shouldn't have 
> had to write parsers.
> > What do you want to use the fpc parser for?
> It's not 'fpc parser' in the sense that it cannot be used anywhere else; 
> it is a parser that is currently used solely within free-pascal 
> compiler. I would like it to be available to other people too.

It is available, it is open source.
You want to change it, so it can be used for other things than
compiling. So the question remains: For what other things?


> Isn't this a good enough reason?

It is, but please don't ignore the good reason against it. Speed,
memory, zoo of flags, how to .

 
> Do I have to have an ulterior motive? Shall I invent one :)
> >> [...]
> >> So, could I now ask for some constructive --instead of discouraging-- criticism.
> > The fear of slowing down the compiler and its development without seeing the gain is discouraging.
> I do sympathize with those fears; but, as you'll agree, worries about 
> speed degradation can only be meaningfully addressed (put to rest) when 
> the actual code is available --no amount of talk or assurances can 
> help/change that.
> 
> Now, about 'gain':
> 
> I think you're overlooking medium/long term benefits.
> 
> When you turn the parser into a module (for the purposes of usage 
> downstream, in IDE etc.), what you will have actually done is to make 
> that 'parser module' replaceable too.
> 
> That alone can be worth its weight in gold, in the sense that from then 
> on, you can use other 'parser module's --such as, you name it, 'C 
> module', 'Modula module', 'Java module' etc. etc.

Please take a look at projects that have replaceable parsers, like swig,
highlighter systems, ctags. Swig was rewritten three times from
scratch. Highlighters and ctags use only very small subsets of the
languages and often not correct ones, just to fit into a general
parser api.

 
> And, that expands horizons, brings in more talent.
> 
> Which cannot be bad for FPC, can it?

There is not one solution for everything.


Mattias