build: add --enable-determinism configure flag #18135

This adds a --enable-determinsm configure flag, which if used, will enable additional compile / link time flags to make subsequent builds of bitcoind deterministic.

The first flag enabled is --no-insert-timestamp. This prevents the linker from embedding timestamps, and makes consecutive builds of bitcoind.exe deterministic. This will likely also be used for Guix Windows builds.

--no-insert-timestamp:

The option --no-insert-timestamp can be used to insert a zero value for the timestamp, this ensuring that binaries produced from identical sources will compare identically.

Diff of consecutive builds of master:

--- bitcoind.exe.1
+++ bitcoind.exe.2
@@ -2,20 +2,20 @@
 00000060: 7420 6265 2072 756e 2069 6e20 444f 5320  t be run in DOS 
 00000070: 6d6f 6465 2e0d 0d0a 2400 0000 0000 0000  mode....$.......
-00000080: 5045 0000 6486 1400 57e8 445e 00da 6900  PE..d...W.D^..i.
+00000080: 5045 0000 6486 1400 e3e9 445e 00da 6900  PE..d.....D^..i.
 00000090: e015 0100 f000 2600 0b02 021f 00de 4900  ......&.......I.
 000000a0: 00b0 5b00 008a 0000 e014 0000 0010 0000  ..[.............
 000000b0: 0000 4000 0000 0000 0010 0000 0002 0000  ..@.............
 000000c0: 0400 0000 0000 0000 0500 0200 0000 0000  ................
-000000d0: 00f0 6a00 0006 0000 bd31 af00 0300 6001  ..j......1....`.
+000000d0: 00f0 6a00 0006 0000 d434 af00 0300 6001  ..j......4....`.
 000000e0: 0000 2000 0000 0000 0010 0000 0000 0000  .. .............
@@ -373594,15 +373594,15 @@
 005b35f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-005b3600: 0000 0000 57e8 445e 0000 0000 7ce1 5b00  ....W.D^....|.[.
+005b3600: 0000 0000 e2e9 445e 0000 0000 7ce1 5b00  ......D^....|.[.
 005b3610: 0100 0000 2200 0000 2200 0000 28e0 5b00  ...."..."...(.[.

Concept ACK: deterministic is better than non-deterministic

Thanks for doing this!

ACK 73b33c116486f6d052e6ddff2e6e95c7d70addc9 -- patch looks correct

Might roll this into an --enable-determinism type build option instead.

<!--a722867cd34abeea1fadc8d60700f111-->

Gitian builds

@fanquake Don't we want determinism by default? :)

Concept NACK, build preferences should be set by the user.

(--enable-determinism seems fair though)

I've updated this to be an --enable-determinism configure flag instead.

ACK 3d9b41ecc02a69daa19df2bc850d60bf9917b724 -- patch looks correct

<!--e57a25ab6845829454e8d69fc972939a-->

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

<!--174a7506f384e20aa4161008e828411d-->

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #12557 ([WIP] 64 bit iOS device support by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

How is this different from libfaketime, which we use in the gitian builds?

How is this different from libfaketime,

This bypasses the need for libfaketime, and I'd much prefer be able to pass an additional configure flag, than having to install, setup and use it. There are a couple other flags I plan on adding here too, such as passing -D to libtool on macOS, rather than having to set ZERO_AR_DATE.

Yes, the idea is to not need faketime anymore at some point. It was always a hack because so many of the GNU and Qt tools embedded timestamps in all kind of unexpected places. We've slowly been working to get rid of those.

I like putting this under an option instead of making it configure default.

ACK 3d9b41ecc02a69daa19df2bc850d60bf9917b724

So we can remove libfaketime from gitian as well and replace it with this configure flag and/or appropriate SOURCE_DATE_EPOCH exports?

@MarcoFalke

So we can remove libfaketime from gitian as well and replace it with this configure flag and/or appropriate SOURCE_DATE_EPOCH exports?

Mind looking into #19627?