delphi 7.0, delphi 2010 and binary files

Question

My software developed under Delphi 7.0 has been upgraded and developed under Delphi 2010 RAD Studio XE. It also saves or writes user's settings into binary files. The problem I am running into is that my Delphi 2010 application or software is expected to read binary files created by Delphi 7.0 application, but Delphi 2010 application has problem reading the binary file.Both of these software are copy of each other.

I did set the Record Field Alignment option for the project at 1 on Delphi 7.0 and byte on Delphi 2010.

I am always running into exception "read beyond end of line," when the software reads the binary file.

I read and write a binary file by doing the following.

Writing into a binary file:

procedure WriteUnitFile;
var
  unitFile: File;
  x:integer;
  FileHeader:TFileHeader;
begin
  if not DemoMode then
  begin
    FileHeader.id := 'UnitFile';
    FileHeader.version := 7;

    if fileexists(basedir+unitfilename) then
      BackupFile(basedir+unitfilename);

    AssignFile(unitFile,baseDir+unitfilename);

    if UnitList.Count > 0 then
    begin
      Rewrite(unitFile,1);
      BlockWrite(unitFile, FileHeader, SizeOf(FileHeader));
      for x := 0 to UnitList.Count - 1 do
      begin
        TUnit(UnitList[x]).Write(unitFile);
      end;
    end
    else
      DeleteFile(baseDir+unitfilename);

    CloseFile(unitFile);
  end;
end;

Reading from a binary file:

procedure ReadUnitFile;
var
  unitFile:File;
  newUnit,simUnit:TUnit;
  ut,TypeCard:TUnitType;
  New_Ver_File:TFileHeader;
  Address:SmallInt;
  State:TUnitState;
  SubAddress:SmallInt;
  MyfDefs:array[1..20] of SmallInt;
  fDefs:array[1..16] of SmallInt;
begin

  if FileExists(baseDir + unitfilename) then
  begin
    oneUnit := false;
    AssignFile(unitFile,baseDir+unitfilename);
    AssignFile(newUnitFile,baseDir+Dummyfilename);
    Reset(unitFile,1);

    BlockRead(unitFile,New_Ver_File,SizeOf(New_Ver_File));

    Reset(UnitFile,1);
    BlockRead(UnitFile,New_Ver_File,SizeOf(New_Ver_File));

    while not Eof(UnitFile) do
    begin
      BlockRead(UnitFile,ut,SizeOf(ut));
      case ut of
        tutSimulator:newUnit := TSimulator.Create;
        tutDX2202:newUnit := TDX2202.Create;
        tutDX8884CS:newUnit:=TDX8884CS.Create;
        tutDX8F44F: newUnit := TDX8F44F.Create;
        tutDX0008:newUnit := TDX0008.Create;
        tutDX0800:newUnit := TDX0800.Create;
        tutDX8000:newUnit := TDX8000.Create;
        tutDX1000:newUnit := TDX1000.Create;
        tutDX0100:newUnit := TDX0100.Create;
        tutDX4404:newUnit := TDX4404.Create;
        tutDX0020:newUnit := TDX0020.Create;
        tutDX0080:newUnit := TDX0080.Create;        
        tutDX8814:newUnit := TDX8814.Create;
        tutDX8814CS:newUnit := TDX8814CS.Create;
        tutDX8884:newUnit := TDX8884.Create;
        else
                newUnit := TUnit.Create;
      end;
      newUnit.Read(unitFile);
      if DemoMode then
      begin
        if oneUnit = true then
        begin
          simUnit := TSimulator.Create;
          simUnit.Assign(newUnit);
          newUnit.Free;
          newUnit := simUnit;
        end
        else
        begin
          oneUnit := true;
        end;
      end;
      unitList.Add(newUnit);
    end;
    CloseFile(unitfile);
    UnitsDlg.UnitGrid.RowCount := unitList.Count+1;
    if UnitsDlg.UnitGrid.RowCount > 1 then
      UnitsDlg.UnitGrid.FixedRows := 1;
    UnitsDlg.FillIn;
  end
  else
  begin
       UnitsDlg.UnitGrid.RowCount := 1;
  end;
end;

UPDATE: Here is the user-defined datatypes:

  TFileHeader = record
    id:string[32];
    version:SmallInt;
  end;

TUnitType = (tutUnused,tutSimulator,tutDX2202,tutDX0008,tutDX0800,tutDX8000,
                 tutDX1000,tutDX4404,tutDX0020,tutDX8814,tutDX8814CS,
                 tutDX8884,tutDX8884CS,tutDX8f44f,tutDX0080,tutDX0100,tutLast);

TUnitState = (tusDisabled,tusEnabled);

TUnit = class(TObject)
  changeLink:TUnit;
  Address: SmallInt;
  SubAddress:SmallInt;
  State:TUnitState;
  uType:TUnitType;
  RegCnt:integer;
  fRegs:array[1..20] of SmallInt;
  fDefs:array[1..20] of SmallInt;
  MsgCnt:LongWord;
  RxCnt:LongWord;
  BreakCnt: LongWord;
  LineErrCnt: LongWord;
  OtherErrCnt:LongWord;
  TimeoutCnt: LongWord;
  BadMsgErrCnt:LongWord;
  XSumErrCnt:LongWord;
  OutsFlag:Boolean;
  CommBits:LongWord;
  OfflineCnt:integer;
  Online:Boolean;
  CurReg:integer;
  selectedonce,usedIP:Boolean;
  LastDigitalOut,LastDigitalIn,
  CurRegOut,umsglen,urmsglen,
  dummycount,Unitlocation,
  CommLocation:integer;
private

public
   followed by list of procedures and functions...
end;

Is there something I am missing or not understanding very well?

One thing I did forget to mention is that Delphi 2010 application reads and writes into binary files just fine without making any changes to the code, but it has problem only when reading files created by Delphi 7.0. application.

Obviously, this question is related to Record Field Alignment. So, what should I set it on my Delphi 2010 (Byte, Word, Double Word, Quad Word), if Delphi 7.0 is set at 1.

Thanks in advance.

Answer 1

The key is in the TFileHeader declaration (and maybe the TUnit and TUnitType declaration). You should add these definitions to your question.

If you define a pure string in either of these records, you need to use ShortString instead of String in Delphi 2010 to make it work.

String in Delphi 7 = 1 byte per char.
String in Delphi 2010 = 2 bytes per char.

Update:

Your last update doesn't really reveal any new information, but the fDefs variable in the ReadUnitFile procedure is defined as an array[1..16] of SmallInt; , but in your TUnit class, fDefs is an array[1..20] of SmallInt; ?

The fDefs variable doesn't seem to be used, but this might be the difference that causes the EOL error?

You should define arrays like this as a common Type , anyway, to ensure that they can be passed around and assigned to each other as compatible types (eg. as parameters in methods).

Yet an update:

The error is not in the strings, but in the Size of TObject .

In Delphi 2009, TObject increased the size from 4 bytes to 8 bytes.
The first 4 bytes is a pointer to the objects VMT (as it has been for a long time), and in the last 4 bytes (IIRC), it is possible to contain a reference to a synchronization monitor.

Have a look at this nice article: http://blogs.teamb.com/craigstuntz/2009/03/25/38138/

Answer 2

Mark all your records you write to file in blocks with packed keyword both in Delphi 7 and 2010.

Answer 3

I don't know the answer, but I can give you some directions on how to properly identify the problem yourself. Once the problem is identified, the solution would usually immediately follow.

How to debug the issue

Since you're getting "read beyond end of line file" errors, you're obviously dealing with changing record size. A number of things can cause that:

• Alignment changes. Unless the record is defined as packed record the compiler assumes it's only used internally so it's free to change it's alignment. Since CPU's (and OS'es) changed allot in the D7 - D2010 time frame, is reasonable to expect changes in alignment between the two compilers.
• Underlying data type size changes. Delphi 2009 made the change to Unicode, so D2009+ (including D2010) has two-bytes Chars. If your record contains Char data, that can cause the change in record size.

Back to debugging. You can write a bit of trivial code that displays the size of the record itself, the size of each of it's fields and the offset for each of the fields. Run this code from both D7 and D2010, make a note of all the numbers and address the possible differences:

program Project1;

{$APPTYPE CONSOLE}

uses
  SysUtils;

type

  TTestRecord = record
    IntField: Integer;
    ShortStrField1: string[6];
    ShortStrField2: string[5];
    D: Double;
  end;

var R: TTestRecord;

begin
  WriteLn('TTestRecord size: ', SizeOf(R));
  WriteLn('IntField size: ', SizeOf(R.IntField), ', offset: ', Integer(@R.IntField) - Integer(@R));
  WriteLn('ShortStrField1 size: ', SizeOf(R.ShortStrField1), ', offset: ', Integer(@R.ShortStrField1) - Integer(@R));
  WriteLn('ShortStrField2 size: ', SizeOf(R.ShortStrField2), ', offset: ', Integer(@R.ShortStrField2) - Integer(@R));
  WriteLn('D size: ', SizeOf(R.D), ', offset: ', Integer(@R.D) - Integer(@R));
  ReadLn;
end.

Once you get this information you can change the record so it looks the same on both Delphi 7 and Delphi 2010. I'd start on the Delphi 7 platform; I'd first change the definition to packed record and then add extra padding bytes to the record to maintain field offsets. The new type definition would look like this:

TTestRecord = packed record
  IntField: Integer;
  ShortStrField1: string[6];
  ShortStrField2: string[5];
  _DummyPadding: array[0..6] of Byte; // Added to fix D's alignment after adding "packed"
  D: Double;
end;

Once this is done move to Delphi 2010, keeping the packed modifier and all the manually added padding. Run the code that displays field size and alignment, pay special attention to the size of the individual fields: if you've got any Char fields in your record, you'll need to change them to AnsiChar . Fortunately both Delphi 7 and Delphi 2010 know about AnsiChar and consider it to be 1 byte long.