[Lazarus] Multi-threading support in IDE

[Martin]

The loop in outputfilter can be simplified quite a bit

All the extra handling of the asyncprocess can be reduced to the 
following lines (the event handlers can be removed):

        if fProcess is TAsyncProcess then begin
          Count:=TAsyncProcess(fProcess).NumBytesAvailable;
          if Count > 0 then
            Count:=fProcess.Output.Read(Buf[1], Min(Count, length(Buf)));
        end
        else
          // using a blocking TProcess
          Count:=fProcess.Output.Read(Buf[1], length(Buf));
        if (Count=0) and (not fProcess.Running) then
          break
        else if count = 0 then
          sleep(30);

Works on windows / shouldn't change anything on *nix


Martin wrote:
> Vincent Snijders wrote:
>> I have looked a bit more why outputfilter is so slow at parsing a lot 
>> of compiler output (e.g. compiling a simple LCL app with -va). The 
>> pipe buffer is rather small, NumBytesAvailable is not bigger than 
>> 1280 bytes. So OnAsyncReadData reads only 1280 bytes at a time. 
>> OnAsyncReadData is only called after an application.handlemessage. So 
>> on line 319 of outputfilter.pas, the buffer is most times empty, but 
>> NumBytesAvailable = 1280, and we are sleeping until 
>> Application.HandleMessage is called.
>>
>> I had some success with the following patch. It does a sleep(0) to 
>> give the compiler the chance to write to the pipe again and tries to 
>> read it again.
>> --- ide/outputfilter.pas
>> +++ ide/outputfilter.pas
>> @@ -1206,9 +1206,12 @@ var
>>    Count: LongWord;
>>  begin
>>    if fProcess=nil then exit;
>> -  Count:=TAsyncProcess(fProcess).NumBytesAvailable;
>> -  if Count>0 then
>> -    FAsyncOutput.Push(TStream(TAsyncProcess(fProcess).Output),Count);
>> +  repeat
>> +    Count:=TAsyncProcess(fProcess).NumBytesAvailable;
>> +    if Count>0 then
>> +      FAsyncOutput.Push(TStream(TAsyncProcess(fProcess).Output),Count);
>> +    sleep(0);
>> +  until Count=0;
>>  end;
>>
>>  function TOutputFilter.CreateScanners(ScannerOptions: TStrings): 
>> boolean;
> If I understand it correctly the problem is that TAsyncProcess, 
> currently only reads data, when the event is triggered, which is in a 
> ProcessMessages. Ouch.
>
> Either TAsyncProcess.OnReadData is transferred into the main loop, in 
> a similar way as for the normal TProcess.
> I have not tested this, but it should work, as NumBytesAvailable seems 
> to be none blocking.
> => This could simplify the overall loop as the handling of the 2 
> projects would be more or less the same.
>
> Alternatively, before we sleep(3), we could check for data, and 
> process it.
>        end else begin
>          // no new input, but process still running
>          if TheAsyncProcess.NumBytesAvailable > 0 then
>            Application.ProcessMessages
>          else
>            Sleep(30);
>        end;
>
>
>
> -- 
> _______________________________________________
> Lazarus mailing list
> Lazarus at lists.lazarus.freepascal.org
> http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus