[Lazarus] RE : RE : Console App Development

[Martin]

On 14/08/2011 08:56, Ludo Brands wrote:
>> So the memory is probably available, but due to
>> fragmentation, and small
>> pieces still being allocated, it can't be returned to the OS.
>>
>>
>>
> Clearly you haven't looked at the valgrind massif data I posted earlier.
Sorry, no I didn't. Also i do not know enough about fpc memory 
management (and maybe it even depends on the memory manager.

Does fpc hand calls for memory directly through to the OS? I thought fpc 
does allocate some meory, and then return slices of this when the app 
requires it? the heap trace for example definetly keeps some memory 
around, as it is checksummed, and later tested for modifications by 
invalid memory access.

Actually, I thing my figures where on a Laz compiled with heap trace => 
so memory would not be freed for real.

Does valgrind know about the fpc internal mem handling

> They clearly show that a lot of data is staying in memory. It'll require a
> lot of digging to figure out exactly where the problems are. One I found out
> is that synedit creates an image list with gutter symbols for every page!
Well there is a synedit for  every page, every synedit is acting 
completely alone currently. (except for sharing the highlighter.)

Well yes, actually, the highlighter keeps some memory during the entire 
IDE lifetime. However, as long as you keep opening pascal files most of 
the structures in this memory are re-used.

> That adds up to 3 M for the 450 file, which apparenltly aren't released when

Hm, strange, not sure which one you refer too.
BookmarOptions has an image list (the digits for bookmarks), but the 
image list is actually a single instance  created by SourceEditorMarks)

I haven't gone through all the rest of code, but I just checked by 
debugging the IDE. SynEdit itself definitely gets destroyed when the 
page is closed. And I have not yet seen a memory leak in regard to this...


> closing the page. An other one is in zlib when reading lfm resources (there
> aren't any in the 450 tes files afaik). Another few meg's hanging around.
>
>


is zlib used on windows? my figures where on windows?


anyway: we have 200 MB - 50 that were already allocated = 150

16.7 MB real text, but split into sting by line. I haven't counted the 
lines, but they a line is avg 20 bytes => 800K lines, each of them a 
string (4 bytes pointer, mem alloc overhead, len => at least 12 bytes) 
another 8 MB, synEdit needs other per line info, for highlight and 
similar things. estimate another 8 MB

Codetool may keep separate copies of the text or parts of it, as it uses 
a different way of storage, another up to 16 MB (+ mem manager overhead)

So far that makes 50 MB + 3MB image lists => still a lot missing.

I am not sure if we look at any bitmap memory for synedits paint area. 
SynEdits handle are only created when you switch to the tab => before 
this, I would not expect bitmaps to be allocated? But I do not know for 
sure. The memory however is gone, before I ever switch a tab, that is 
only one synedit  got a Handle allocated.

-----------
Now doing the same test without heaptrace

- Lazarus just started 36.8 MB
- added the 450 univint files 137.8 MB / significantly less than the 200 
that I had with heap trc
- but yet no memory freed (to OS) when closing project

Whil the lack of freeing memory may in parts be caused by Lazarus,  (do 
not know), it is not entirely caused by Lazarus.
There is definetely some memory freed.   And this freed mem is not going 
back to the OS for a reason outside the control of Lazarus.

I do not know, if valgrind would detect such mem as freed or not.