[Lazarus] TDBEdit, TStringField Size, DataSize, DisplayWidth and MaxLength

[Tony Whyman]

An IBX user came to me with a problem and the problem seems to be a deep 
seated disconnect between multi-byte character sets, TStringField.Size 
and TDBEdit.MaxLength. Something needs to give - but I am not sure what 
should.

Firstly documentation:

If you go back to Delphi, TField.DataSize is the memory needed to hold 
the Field's value. The DisplayWidth is the number of characters to be 
displayed, and Size is, for datatype ftstring, "the maximum number of 
characters in the string". How literally this last definition should be 
taken, I'm not sure, as it may well have been written assuming a single 
byte character set.

On the other hand, the FPC documentation is consistent with Delphi for 
DisplayWidth and DataSize, but more opaque for TField.Size where it is 
the "logical size" - whatever that means, although TStringField is more 
definitive by saying it is the maximum size (in characters) - their 
brackets not mine.

That seems to be consistent with TDBEdit.Maxlength which should be the 
maximum number of characters that can appear in the control and, if you 
look at the code, TDBEdit will source the default value from 
FDatalink.Size (And also seems to ignore DisplayWidth).

The problem comes when you look at the code for TStringField.GetValue, 
where it starts off as:

function TStringField.GetValue(var AValue: string): Boolean;

var Buf, TBuf : TStringFieldBuffer;
     DynBuf, TDynBuf : Array of char;

begin
   if DataSize <= dsMaxStringSize then
     begin
     Result:=GetData(@Buf);
     Buf[Size]:=#0;  //limit string to Size
     If Result then
       begin
...

If nothing else, this is a "bug in waiting". TStringField.GetDataSize 
always returns "Size+1", so "Buff[Size]:=#0; should work - but only as 
long as the virtual method "GetDataSize" is not overridden (GetValue is 
non-virtual) and Size is the byte length of the string!

There is a built-in assumption here that "Size" is the byte length of 
the string and not the character length. If you have a multi-byte 
character set and set size to the number of characters and DataSize to 
e.g. for UTF8 4*(no of characters)+1, then you will get string 
corruption as a result of the above.

IBX handles multi-byte character sets and does so by defining 
TIBStringField as a subclass of TStringFIeld and setting size to the 
byte length and the Default DisplayWidth to the character width. This is 
compatible with TStringField as it works today. It also seems to be 
compatible with TDBGrid, which uses Field.DisplayWidth. However, it does 
result in TDBEdit accepting too many characters.

What should be done?

It's a problem. Ideally, the TStringField code should be aligned with 
the documentation. However, that could break existing code and would 
need to handled carefully. TStringFIeld also needs fixing i.e. to 
Buf[DataSize-1]:=#0 in order to make this a reality.

Alternatively, the documentation could be amended to reflect the 
implementation. This means that TDBEdit (and maybe more) have to be 
updated - but why doesn't TDBEdit respect the DisplayWidth property anyway?

Perhaps, it is also about time that TStringField got a characterWidth 
property to hold the maximum number of bytes for each character. That 
would at least allow the DataSize to be automatically computed from the 
character width.

If I had to write a bug report today, I would write it to avoid changes 
to IBX - but then is that the right answer?

Tony Whyman

MWA