[Lazarus] db grid/dataset components that only load currently visible records?

[Michael Van Canneyt]

On Fri, 24 Feb 2023, Luca Olivetti via lazarus wrote:

> El 23/2/23 a les 17:16, Michael Van Canneyt ha escrit:
>
> write a TDataset descendant tailor made for this
>>> application, any pointer on how to write a minimal descendant?
>>> I'm looking at the TMemDataset implementation as an example, is it a good 
>>> one?
>> 
>> As good as any other.
>> 
>>> The idea is to substitute open and scrolling (next/prior/locate/etc.) with 
>>> the appropriate sql statements to just retrieve a small buffer of data 
>>> around the current record (I can do that easily enough), though I'm not 
>>> sure that would be enough to fool the TDBGrid.
>> 
>> I don't know what the grid exactly does, so I cannot advise.
>
> I did some tests and the grid is not a problem (it only fetches the records 
> that are visible), the TDataset is:  either I am blind or the only way for 
> the TDataset to realize there is data is to fetch every record and allocate 
> the corresponding buffers.
> That's because FRecordCount/FBof/FEof are private

FRecordCount is the number of records in internal buffers, 
and has nothing to do with the total number of records.

The total RecordCount must be given by TDataset descendents,
but defaults to FRecordCount in TDataset itself.

The terminology is confusing, I know.

We should probably rename FRecordCount to FBufferRecordCount.

>
> procedure TDataSet.DoInternalOpen;
>
> begin
>  InternalOpen;
>  FInternalOpenComplete := True;
> {$ifdef dsdebug}
>  Writeln ('Calling internal open');
> {$endif}
> {$ifdef dsdebug}
>  Writeln ('Calling RecalcBufListSize');
> {$endif}
>  FRecordCount := 0;
>  RecalcBufListSize; <-- this will fetch all the records
>  FBOF := True;
>  FEOF := (FRecordCount = 0); <--- FRecordCount must be set
> end;

No.

RecalcBufListSize does not necessarily fetch all the records. It fetches as
much records as needed: 10 by default, but classes such as TDBGrid may set
the number of needed records to the number of visible grid lines.

It can be that TBufDataset fetches all records on open, there is a property that
controls this. But it can also fetch on an as-neede basis.

FEOF is simply initialized from the first batch. 
But it is maintained based on the result of GetNextRecord. 
Check the MoveBy procedure.

Only when GetNextRecord returns false, is FEOF set to True (line 2005).

> There's no point overriding 
> GetNextRecords/GetNextRecord/GetPriorRecords/GetPriorRecord (called by 
> RecalcBufListSize) since I cannot directly modify the private fields of the 
> TDataSet.

You don't need to modify them. They are automatically maintained based on
the result of GetNextRecord. TDataset is responsible for maintaining a
correct state based on what descendants return in the methods they override.

Exposing these internal state fields for modification means TDataset cannot
guarantee the correct state, and therefore that will not happen.

>
> Any idea?


Personally, I believe you are on the wrong path by wanting to modify TDataset or
its descendants.

But if you want to pursue that track, study TDataset a little more and ask questions. 
It is a complex class, one of the most complex in the whole FCL.

Michael.