[lazarus] Clean build from nothing

[Marco van de Voort]

> My education continues....

And you need it ;-)
 
> I thought for sure that I saw some C files.  OK so I need FPC installed
> before I install FPC. I understand that now. I'll start again from scratch
> tonight and make notes of what I do. (and were I had to backtrack). I'm
> going to try for fpc 1.0.8 from the CVS since it is so near release any
> differences between what I get and what the release is should presumably be
> small.

Ok, and true. Most of the changes will be very internal. Most bugfixes
in the 1.0.x branch usually are.

> I am shocked that FPC doesn't use libc at all. 

That's normal. The usual documentation sketches a quite rosy picture, and
most people react that way. There are good reasons though, see below for 
a selection.  Most people consider them reasonably

> More then that FPC doesn't use any C code to build itself. 

Why would it need it? C is not "special" in anyway. Pascal is better even.

> Is this by design? 

Of course it is. 50-100 MB source don't fall out of thin air. The project
has been running for a decade.

> Why avoid all that good code?

Are you so sure it actually _is_ good code? :-) And more important, from the
perspective of somebody not really interested in C?

The main reasons for not doing a gcc port:

- _Historical_ gcc was not stable and cleaned up enough when FPC was concieved.

- _Non Unix origin_  FPC originated on Dos, and got cleaned up after. Unix
   _and_ platforms are just like the others. Choices made for Unix can be hard
   on other platforms.  Emulations make non Unix ports feel "fat", non-native
   and emulation like. Example: cygwin.  Dos and Windows are really major
   ports. OS/2 also still has a steady following

- _manpower_ The gcc integration costs a lot of work.    Adapting to other
   peoples changes, working with patches, maintaining compability with a
   very complicated build process, not breaking other people work and the
   complexity of regression testing that comes with that etc.  FPC has grown
   till about 10 main developpers and several hardworking people submitting
   units, maintaining OS ports, building snapshots daily etc. It did most 
   of its existance with less than a third of that manpower.

- _language_ Pascal is what we do. Modding gcc would mean the core team
   would _have to_ work in C. No option, at least not until I get paid.
   The Pascal code (think big here. 50 till over 100 MB FPC compilable code
   in CVS and in some related projects) is also a kind of testsuite. It forces
   one to use the compiler.

- _Bootstrapping_ and crosscompiling issues are much easier than most
  outsiders expect. Often even easier due to smaller dependancies (both
  libraries and tools used in the build). You saw yourself, essentially you
  need as,ld ,make and a start compiler to build the entire tree. And some
  platforms even eliminate AS. 
- _bloat_ gcc has a lot of unnecessary features, code and provisions. This again
  costs a lot of maintenance, raises some limitations etc.

- _multi language?_ gcc is primary a C compiler, the rest is duct-taped on.
  Provisions like units are alien to it. 
  This also goes for the separation of parts of the suite too (preprocessor
  - fronten - backend - backend assembler - backend linker), buildprocess
  etc all tailored to C and C'ish languages.  E.g. we are unable to
  smartlink lazarus, simply because the linker eats an estimated amount of
  1.5 Gb to generate the 10-20 MB magnitude Lazarus binary. C does only some
  minor smartlinking, so it isn't very optimised :-) We would have more of
  this tool related problems in the gcc case.

- _control_  using gcc is giving-up control. For a pascal specific major
   	  change you have to negotiate with really big commercially funded
   	  teams (with fulltime people calling the shots), from a totally
   	  inferior negotiating position. (you don't have anything crucial
   	  they want, and the C crowd is even _Pascal unfriendly_, and that is
   	  an understatement). If you go the way people want it, add libc,
   	  cpp autoconf/make, yacc lex, and whole deal more also to this list
   	  of "negotiating partners".  This point shouldn't be
   	  underestimated, and for me might be the most important one,
   	  together with *nix centic view of the gcc scheme. Even though I'm
   	  a BSD user for over a decade now.
 
          Example: imagine if you _really_ need some change in the main gcc
          code in order to fix or create some major part of your
          infrastucture. Assume you are in say about 1/2 a year before the
          head branch of gcc splits of the next release branch. e.g. one
          branch will receive new major features and changes to become gcc5
          in time, the other one will stabilize to become gcc4 in say about
          a year. If they don't wnat to incorporate your code for gcc4
          because of stability reasons, schedules or simply being
          uninterested, you are stuck with it till gcc5 comes out. (say
          minimally 3 years from now), that time you are stuck with patches
          or hopeless workarounds severely hampering your development.

All this also hampers having a working version early, which is useful to get
contributors. (the ones that really put in manyears are the ones that are
the most rare)

Also compare the state of GPC (GNU Pascal which is gcc based) with FPC, and
realise that GPC is 1 and 1/2 times as old, a bit more even, and has full GNU
backing. This is partially due to their non standard Pascal dialect choice
also, but still...

There are also disadvantages:
- the "good" code in gcc that simply isn't worth the trouble, but still has
  to be reimplemented: codegenerators for a lot of processors, optimizers.
- inability to interface to headers of the other gcc languages natively,
  most notably C, C++ and Objective C. (the last one because a major Mac OS
  X api is based on it)

----------
The libc issue

The libc thing is a bit different. I still think choosing syscalls was a good choice at
the time because

- libc activity instability and libc->glibc transition.
- different versions on a lot of distro's
- again buildprocess complications.
- programs that don't use libraries are only slightly larger, and 
- It's not just libc, but also libm, libgcc etc. However this is a trend
  that might be a reason to do it in the end.
- Headers have to be translated and  kept up to date. The few functions we
  use from libc in some cases are much easier to maintain. Binary interfaces
  like the kernel rarely change.
- other unices use other libc's. That means a header translation per OS, not
  one for all with a few ifdef's.
- the total lack of support to make such usage easier for non C compiler
  derivates or C like languages Most unices fail to have a master set of
  headers in a abstract, easy parsable format from which other headers could
  be easily generated.  Even Windows does this a bit by providing a version
  of its newer modules in IDL.

This might change in near-middle the future (by setting a define when
recompiling FPC) to switch the default to use libc as back end on *nix.

Main reasons:
- the free unices are more stable, there is some unification and
  standarisation process going on.
- OS X. For various reasons it was decided to keep to libc, mainly because
  Apple can change the kernel interfaces easier. This also means we have
  some basic infrastructure for the change in place.
- the tendancy to put more functionality  in (merge libs with) libc. This is bad IMHO, however
  a fact. (think userland threading libs, NIS and other multi-machine
  authentication support, unicode, internationalisation).
- FPC is more advanced, the avg programs have become more complex, and
  longterm quite a lot will link to C libraries and therefore libc anyway.
  (e.g. lazarus because of gtk.)
- keep the syscall interface, because it makes small stuff and bootstrapping
  so much easier.

> I assume there is a provision for FPC to interface with C isn't there?

Of course it can. How do you think lazarus interfaces with gtk?

But the exact answer depends how you see it. You can't use C headers and
simply try to call the functions listed in them. You'd have to convert the
headers to their pascal equivalent, and there are some semi automatic ways.
Then you can link to C. Look for the existing headers (like the gtk
interface that lazarus uses)

The fact that library and OS teams don't have any abstract (so non C)
headers that can be easy auto-ocnverted makes this unnecessary painful. However that
is not our choice, but the relevant teams.

So it is like having a mold (headers to use with C programs), but not be
able to use it to make some special form (Pascal programs). However by
looking at the original mold, you can make a better fitting one for the
special form.

Linking directly (as opposed to via C or the C++ CEXTERN or what its called
way ) with C++ and Objective C is not possible yet. Some other languages
might be possible through their C compability.

You can of course try to revert to asm to write the interface. It is the
only other language that FPC directly supports.

The fact that C and C++  are such horrible languages to implement also
prohibits the possibility to make a simple C subset compiler part of FPC,
just to get at the headers.

___

(Marco V is really tired now)

I hope I showed you a bit of what is behind the commonly presented
idealistic view of Linux and Unix in general.

If you have more questions related to this, I'd be happy to answer them.