I am reading blkXXX.dat files. I am facing the problem that in my blk00970.dat I can read at some position the magicNumber of the block. but the block is empty because after that the magicNumber is again contained in the file which should be the version.
workaround: if version == MAGICNUMBER: new block starts here, this stupid workaround works until blk00976.dat where somwhere the script signature length data entry of an input is suddenly 3052914961……
does somebody know why?
The blockfiles (blkXXX.dat) are not an API. Use the interface at your own risk… it may change.
The block-files does contain the 4byte net magic, the size of the block and the serialised block data, and those three elements per block. You may need to have a closer look at transaction deserialisation to avoid an accidental offset.
In some cases there are gaps between the blocks with arbitrary junk (which sometimes looks like partical blocks). This shouldn’t be the case with newly-downloaded blockchains, but in the past there have been some bugs which resulted in this.
Your code needs to be robust against invalid blocks when parsing the .dat files - this is why our own import logic scans over the file and considers every valid block, and if there is a parse error retries from the next possible offset; e.g. see the rewind logic in https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L4029 Or for a python example: https://github.com/bitcoin/bitcoin/blob/master/contrib/linearize/linearize-data.py#L199
