[Lazarus] custom thread reading file does not respond properly

[Andrea Mauri]

> Note that his example works on my Mac with or without
> sleep. Maybe if Andrea posts his full example we may find out more.
>
> Mattias
>

Unfortunately I cannot post the real case, it is included in a bigger 
unit involving other stuffs.
Anyway the example I posted is quite the same.

In order to better clarify my issue:
- I have to manage text files including some logical blocks (items), 
these files can be huge (gigabytes) and they can store millions of items 
(some file types define an item using a single line). My application 
deals with these files enabling the user to visualize the items in a 
grid and rapidly move along the file.
I encounter it in two different threads:
1. a thread that I launch when the user load one of these files. This 
thread simply reads the file, counts the items and stores the position 
of the items (streambuffer positions).
2. a thread reading a file and looking for a substring.

1. source

procedure TCountItemsThread.Execute;
var
   streamreader: TStreamReader;
   streampos, streamsize: Int64;
   bufferpos, nitems: Integer;
begin
   _itemfile:= TItemsFile.Create(_filename, _fileformat);
   try
     streamreader:= _itemfile._streamreader;
     streamreader.Reset;
     streamsize:= streamreader.BaseStream.Size;
     _itempos.Clear;
     _itembufpos.Clear;
     nitems:= 0;
     while not Terminated and not streamreader.Eof do
     begin
       // stores items source position in file as Position and 
BufferPosition
       streampos:= streamreader.Position;
       if streamreader.BufferRead < (BUFFER_SIZE - 1) then
         streampos:= streampos - (streamreader.BufferRead)
       else
         streampos:= streampos - (BUFFER_SIZE - 1);
       bufferpos:= streamreader.BufferPosition;
       if (bufferpos = 0) and (streampos > 0 ) then
       begin
         streampos:= streampos - (BUFFER_SIZE - 1);
         bufferpos:= BUFFER_SIZE - 1;
       end;
       if _itemsfile.nextItem then begin
         _itempos.Add(streampos);
         _itembufpos.Add(bufferpos);
         _progress:= round((streampos / streamsize) * 100);
         Inc(nitems);
         if (_itempos.Count mod 1000) = 0 then begin
           Synchronize(@CountingItems);
         end;
       end;
     end;
     Synchronize(@EndCountItems);
   finally
     _itemfile.Free;
   end;
end;


nextItem simply parse the file and move the stream to the next item 
position. If the file has single line items I cannot interact with my 
app even on win.

2. source

procedure TFilterNameThread.Execute;
var
   substr, str1: string;
   i, step, occurrences: Integer;
   clonedfile: TItemFile;
begin
   // if substr is empty do not perform search
   FFindItems.Size:= FItemFile.CountItems;
   if Trim(FindString) = '' then begin
     for i:= 0 to FItemFile.CountItems - 1 do begin
       FFindItems[i]:= True;
     end;
     _itemidx:= FItemFile.CountItems;
     Synchronize(@ShowProgress);
     Exit;
   end;
   // store original FItemFile.Selected[] and sets FItemFile.Selected[] 
to False
   _itemidx:= 0;
   FFindItems.Clearall;
   Synchronize(@ShowProgress);
   clonedfile:= FItemFile.CloneFile;
   try
     step:= clonedfile.CountItems div 100;
     if step = 0 then
       step:= 1;

     substr:= FindString;
     if not CaseSensitive then
       substr:= upcase(substr);
     occurrences:= 0;
     while (not Terminated) and (_itemidx < clonedfile.CountItems) do begin
       if CheckAll or Selected[_itemidx] then
       begin
         str1:= clonedfile.ItemName[_itemidx];
         if not CaseSensitive then
           str1:= upcase(str1);
         if Pos(substr, str1) > 0 then begin
           FFindItems[_itemidx]:= True;
           Synchronize(@ShowProgress);
           Inc(occurrences);
         end;
       end;
       if (_itemidx mod step) = (step - 1) then begin
         Sleep(5);
         Synchronize(@ShowProgress);
       end;
       Inc(_itemidx);
     end;
     Synchronize(@ShowProgress);
   finally
     clonedfile.Free;
   end;
end;