[Lazarus] WinINet STDCALL callback crash

[Ludo Brands]

On 02/23/2013 09:49 PM, matthew at accordancebible.com wrote:
> Hello Lazarus Community,
> 
> I am implementing asynchronous file download using WinINet's STDCALL callback.  However, when the callback function recursively calls itself, it generates a crash around fpc_popaddrstack.  I hypothesize that Pascal does not understand how to handle callbacks that use the stdcall convention (which I must use since the callback interfaces with the Win32 API and is itself set via the Win32 WinINet procedure InternetSetStatusCallback).  
> 
> I was wondering if anyone might graciously offer guidance on how to implement asynchronous download to file using the Win32 API or associated libraries only.  Note that I have tried WinHTTP and a different implementation of WinINet using InternetConnect/InternetOpenRequest/InternetSendRequest/InternetReadFile).  In the previous two implementations I have encountered the same segmentation fault around fpc_popaddrstack.  Based on my experience with the below code and other STDCALL implementations, I do not think that Free Pascal can handle the case where a STDCALL function interrupts itself recursively. In the STDCALL convention, the callee is responsible for cleaning up the stack upon function exit.  However, because the function in question (HttpCallback) has been interrupted by a second call to it from the Windows API, it has not had a chance to clean up the stack (because it has not had a chance to exit and thereby execute the epilogue code that cleans up the stack).  The
  conditio
n that causes the callback to interrupt itself is when InternetReadFile errors out with ERROR_IO_PENDING.  In this circumstance, InternetReadFile also sends back an INTERNET_STATUS_REQUEST_COMPLETE message which interrupts the currently executing HttpCallback function.  
> 
> To state the exact circumstance of the crash:  when not stepping through, InternetReadFile below errors out frequently with ERROR_IO_PENDING.  This is fine except that InternetReadFile ALSO recursively triggers the callback, which after the second recursive call causes a crash.
> 
> My current code is as follows.  I've added inline comments around the parts that seem to crash.  Thank you very much for any asssistance you can provide.  
> 

You are trying to read the complete HTTP reply inside the callback which
will not work. ERROR_IO_PENDING just means that the data for the next
buffer is not there yet. The callback function is called for every
buffer you receive from the server and you should have only one
InternetReadFile per INTERNET_STATUS_REQUEST_COMPLETE. So when the HTTP
reply takes many buffers you will get many callbacks and in every
callback you are supposed to read the buffer received with one single
InternetReadFile,

Take a look at this example on how to set up async processing:
http://msdn.microsoft.com/en-us/library/windows/desktop/cc185684(v=vs.85).aspx.
You'll notice the AcquireRequestHandle function that uses a
EnterCriticalSection to lock the context. In your code you are not
locking anything and your "recursive" entry in the callback is just
using the same context which is the probable cause of your crash.

STDCALL callbacks are used commonly in Freepascal and are not a problem
at all.

If you just want to download an url to file the easy way using the
windows framework take a look at this post
http://www.lazarus.freepascal.org/index.php?topic=18506.15#msg105226

Ludo