guix: Pin Guix using guix time-machine #17933

1. 
   dongcarl commented at 5:20 pm on January 15, 2020: member

   An alternative to #16519, pinning our version of Guix and eliminating a guix pull and changing the default Guix profile of builders.

   I think this method might be superior, as it:

   • Eliminates the possibility of future changes to the guix environment command line interface breaking our builds
   • Eliminates the need to set up a separate channel repo

   It is a more general pinning solution than #16519.

   ---

   The reason why I didn’t originally propose this is because guix time-machine is a recent addition to Guix, only available since f675f8dec73d02e319e607559ed2316c299ae8c7

2. dongcarl added the label Build system on Jan 15, 2020
3. dongcarl added this to the "PRs" column in a project

   ---

4. 
   MarcoFalke commented at 5:32 pm on January 15, 2020: member

   Eliminates the need to set up a separate channel repo

   Wasn’t the conclusion that the channel could be in the same repo? See #16519 (comment)

   Though, I guess the time machine still makes sense when we can switch it to vanilla upstream and not use dongcarl/guix?

5. 
   dongcarl commented at 6:37 pm on January 15, 2020: member

   Wasn’t the conclusion that the channel could be in the same repo? See #16519 (comment)

   Though, I guess the time machine still makes sense when we can switch it to vanilla upstream and not use dongcarl/guix?

   Ah sorry, what I meant to say is that using time-machine eliminates the need for any channel/inferior whatsoever! You define some extra packages in the manifest, and just point directly to the version of Guix you want. This is why this diff is so small.

   Previously in #16519, we had to point to the channel (which contained some extra packages), which in turn pointed to the version of Guix we wanted.

   Side benefit: if we used channels in bitcoin/bitcoin or in bitcoin-core/packaging, we would need a .guix-channel file at the root of the tree. This method doesn’t.

6. 
   MarcoFalke commented at 7:30 pm on January 15, 2020: member

   Approach ACK 4763171ff7ad03e654ccd9f78b5ac5ab17aecd25

   Didn’t test or review the diff to guix upstream

7. 
   DrahtBot commented at 7:47 pm on January 15, 2020: member

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Conflicts

   Reviewers, this pull request conflicts with the following ones:

   • #17920 (guix: Build support for macOS by dongcarl)
   • #17595 (guix: Enable building for x86_64-w64-mingw32 target by dongcarl)

   If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

8. 
   fanquake commented at 9:12 am on January 16, 2020: member

   Concept ACK

   Side benefit: if we used channels in bitcoin/bitcoin or in bitcoin-core/packaging, we would need a .guix-channel file at the root of the tree. This method doesn’t.

   Big ++ if this means the project doesn’t have to maintain more build / packaging infrastructure.

   I’ve completed a Guix build using this PR (https://github.com/bitcoin/bitcoin/pull/17933/commits/4763171ff7ad03e654ccd9f78b5ac5ab17aecd25) and this Docker container:

    0env PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" guix describe
    1  guix 65b510b
    2    repository URL: https://git.savannah.gnu.org/git/guix.git
    3    branch: master
    4    commit: 65b510bbc4f2a9ce5bfe3355e6006e9d08f14532
    5
    6git rev-parse HEAD
    74763171ff7ad03e654ccd9f78b5ac5ab17aecd25
    8
    9find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
   10edf4db52c0835a89dec1b9c90d7fe43bf94a7ffaa0a9cf54b528bf97283af256  output/bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz
   11816d5a669333ee84a665b8ec8873a288123f4f0008b2943461393f27aacbb7bc  output/bitcoin-0.19.99-aarch64-linux-gnu.tar.gz
   123b211b8c7056c81afba457f56f5fbc8a93d819880fac5e464d9a49d82aa87d60  output/bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz
   137ae65539e3866af01e843b1008bd604dccf0c327a811a5fbff749a0a0ac3ebbe  output/bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz
   140215604cd6706a58a798112bd5284ee2b4d8c941a2da2ba129eefd6f9b615fb1  output/bitcoin-0.19.99-i686-linux-gnu-debug.tar.gz
   151a19ffc68a191c5246e8ed30c34548c9ac4351bcf4070b29799144e60cf43476  output/bitcoin-0.19.99-i686-linux-gnu.tar.gz
   166632680da7511d941e40a7663246d167b5ce5a58924c9923790988707d71f628  output/bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz
   17b3acdb8db33d77cabe99198e4db2b94757b5bfb5b68b29568ffd432a6d6380d6  output/bitcoin-0.19.99-riscv64-linux-gnu.tar.gz
   18dab9bf161c283d996b4bf168c7365d966375b434d2a329fbfeb02ee252dba824  output/bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz
   197ba0242c970eb466a20cfc015f88fb472efba08df5c25b9d668878515a420d07  output/bitcoin-0.19.99-x86_64-linux-gnu.tar.gz
   204ac1efb49459a1bb98216580ca3f05c367b8628440505141e924c8de6691639e  output/src/bitcoin-0.19.99.tar.gz
   

9. 
   MarcoFalke commented at 9:52 pm on January 23, 2020: member

   Why is this a draft?
10. 
    fanquake commented at 5:58 am on January 24, 2020: member

    The reason why I didn’t originally propose this is because guix time-machine is a recent addition to Guix, only available since f675f8dec73d02e319e607559ed2316c299ae8c7

    Given that Guix is a new addition to the project, and there’s only a few people doing builds, I’ve got no issue with us requiring Guix master for now.

    I’ve completed time-machine builds for Linux and Windows. macOS builds are also working as far as I know; I haven’t done a build yet, because I haven’t got an SDK with the headers packaged.

    I’ve taken a quick look at the 2019-10-bitcoin-staging-GREAT-CORE-UPDATES-MERGE branch as well. I can see the changes you’ve made match some upstream discussion on the Guix mailing list here.

    It also seems that your commit with David Truby’s fix for libstdc++ has now made it into Guix master 496bbeee7a0afbf1c7cbcc0de8c33c2111cb661f. So I guess will be dropped when you pull upstream. @dongcarl I’d be pretty happy for us to merge this change. Will ACK once you mark as ready for review.

11. 
    dongcarl commented at 7:48 pm on January 27, 2020: member

    I’ve completed a Guix build using this PR (4763171) and this Docker container :

     0env PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" guix describe
     1  guix 65b510b
     2    repository URL: https://git.savannah.gnu.org/git/guix.git
     3    branch: master
     4    commit: 65b510bbc4f2a9ce5bfe3355e6006e9d08f14532
     5
     6git rev-parse HEAD
     74763171ff7ad03e654ccd9f78b5ac5ab17aecd25
     8
     9find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
    10edf4db52c0835a89dec1b9c90d7fe43bf94a7ffaa0a9cf54b528bf97283af256  output/bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz
    11816d5a669333ee84a665b8ec8873a288123f4f0008b2943461393f27aacbb7bc  output/bitcoin-0.19.99-aarch64-linux-gnu.tar.gz
    123b211b8c7056c81afba457f56f5fbc8a93d819880fac5e464d9a49d82aa87d60  output/bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz
    137ae65539e3866af01e843b1008bd604dccf0c327a811a5fbff749a0a0ac3ebbe  output/bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz
    140215604cd6706a58a798112bd5284ee2b4d8c941a2da2ba129eefd6f9b615fb1  output/bitcoin-0.19.99-i686-linux-gnu-debug.tar.gz
    151a19ffc68a191c5246e8ed30c34548c9ac4351bcf4070b29799144e60cf43476  output/bitcoin-0.19.99-i686-linux-gnu.tar.gz
    166632680da7511d941e40a7663246d167b5ce5a58924c9923790988707d71f628  output/bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz
    17b3acdb8db33d77cabe99198e4db2b94757b5bfb5b68b29568ffd432a6d6380d6  output/bitcoin-0.19.99-riscv64-linux-gnu.tar.gz
    18dab9bf161c283d996b4bf168c7365d966375b434d2a329fbfeb02ee252dba824  output/bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz
    197ba0242c970eb466a20cfc015f88fb472efba08df5c25b9d668878515a420d07  output/bitcoin-0.19.99-x86_64-linux-gnu.tar.gz
    204ac1efb49459a1bb98216580ca3f05c367b8628440505141e924c8de6691639e  output/src/bitcoin-0.19.99.tar.gz
    

    Fantastic, see my matching hashes below. Note that even though our guix describe’s differ, we get the same results! guix time-machine FTW!

     0$ guix describe
     1Generation 36   Jan 14 2020 14:50:02    (current)
     2  guix 09bad3c
     3    repository URL: https://git.savannah.gnu.org/git/guix.git
     4    branch: master
     5    commit: 09bad3cd42ca51974f3697f04ce841bf2fe6786c
     6
     7$ git rev-parse HEAD
     84763171ff7ad03e654ccd9f78b5ac5ab17aecd25
     9
    10$ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
    11edf4db52c0835a89dec1b9c90d7fe43bf94a7ffaa0a9cf54b528bf97283af256  output/bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz
    12816d5a669333ee84a665b8ec8873a288123f4f0008b2943461393f27aacbb7bc  output/bitcoin-0.19.99-aarch64-linux-gnu.tar.gz
    133b211b8c7056c81afba457f56f5fbc8a93d819880fac5e464d9a49d82aa87d60  output/bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz
    147ae65539e3866af01e843b1008bd604dccf0c327a811a5fbff749a0a0ac3ebbe  output/bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz
    150215604cd6706a58a798112bd5284ee2b4d8c941a2da2ba129eefd6f9b615fb1  output/bitcoin-0.19.99-i686-linux-gnu-debug.tar.gz
    161a19ffc68a191c5246e8ed30c34548c9ac4351bcf4070b29799144e60cf43476  output/bitcoin-0.19.99-i686-linux-gnu.tar.gz
    176632680da7511d941e40a7663246d167b5ce5a58924c9923790988707d71f628  output/bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz
    18b3acdb8db33d77cabe99198e4db2b94757b5bfb5b68b29568ffd432a6d6380d6  output/bitcoin-0.19.99-riscv64-linux-gnu.tar.gz
    19dab9bf161c283d996b4bf168c7365d966375b434d2a329fbfeb02ee252dba824  output/bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz
    207ba0242c970eb466a20cfc015f88fb472efba08df5c25b9d668878515a420d07  output/bitcoin-0.19.99-x86_64-linux-gnu.tar.gz
    214ac1efb49459a1bb98216580ca3f05c367b8628440505141e924c8de6691639e  output/src/bitcoin-0.19.99.tar.gz
    

12. guix: Pin Guix using `guix time-machine` e6050884fd
13. guix: Update documentation for time-machine

    Wait a minute, doc. Are you telling me you built a time machine... Out
    of a functional package manager?

    88c83636d5
14. dongcarl force-pushed on Jan 27, 2020
15. 
    dongcarl commented at 10:39 pm on January 27, 2020: member

    Marking as “Ready for review”: 88c83636d5a56bd9551577139786bdd3e74852c2

     0$ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
     139608e90c90be13c10488c25fb4bfa3969c5304a4fcde21ce8c57acc0947c9d8  output/bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz
     20540ab5aba8959c45255b60077ee7498333852f3d2067610357d811925762f61  output/bitcoin-0.19.99-aarch64-linux-gnu.tar.gz
     3cc1fcc9dd71162baa0f8d85ae9b514bdbf7f92001bfbd45998dd6db92260edc0  output/bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz
     4f978a4496dd37aa7588e7868ae52607b7e8849d86d27b82f2007ea369cddd1fc  output/bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz
     592da6f327f187cc44bd30c31852111bf09879640efac630303bad2f040e20b93  output/bitcoin-0.19.99-i686-linux-gnu-debug.tar.gz
     6db4654620e0dfe03c4f3a3570b1a9cace15c28c9f68497614710156b7a4a1318  output/bitcoin-0.19.99-i686-linux-gnu.tar.gz
     7d841618b51ce5e9a83ca789b89e86b617c646fee2c9d15487e521e29741f4e72  output/bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz
     8879e526aa5139eea666c949036d811652c3c6f7e3601a53a7617bc21738baf1a  output/bitcoin-0.19.99-riscv64-linux-gnu.tar.gz
     95cde323bbcf1c3331be294b562aab477a72adfad141d5e019ff10e1753178de8  output/bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz
    108cf0240185043fdcfb41500058bbc60c7ec61d15ec30230cc9a33c8e99d73955  output/bitcoin-0.19.99-x86_64-linux-gnu.tar.gz
    11843f958009812087fcb68fab8117d7710db19bcc64d36966121975718609dced  output/src/bitcoin-0.19.99.tar.gz
    

    Changes since fanquake’s last build

    1. Updated my Guix branch to have clearer commit messages.
    2. Added commit adjusting the contrib/guix/README.md according to this PR.

    Some context on my Guix branch

    Outstanding commits between my Guix branch and Guix master: https://github.com/dongcarl/guix/compare/377606238f0260a849a1f529d94e1546276a5db6...b3a7c72c8b2425f8ddb0fc6e3b1caeed40f86dee

    Proof that 3776062 is in Guix master: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=377606238f0260a849a1f529d94e1546276a5db6

16. dongcarl marked this as ready for review on Jan 27, 2020
17. fanquake approved
18. 
    fanquake commented at 8:56 am on January 28, 2020: member

    ACK 88c83636d5a56bd9551577139786bdd3e74852c2

    Ran another build and seeing (new) matching results:

     0bash-5.0# env PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" guix describe
     1  guix 7ee8acb
     2    repository URL: https://git.savannah.gnu.org/git/guix.git
     3    branch: master
     4    commit: 7ee8acbb76bd440a8985a4b2a20b75521c6853ed
     5
     6bash-5.0# git rev-parse HEAD
     788c83636d5a56bd9551577139786bdd3e74852c2
     8
     9bash-5.0# find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
    1039608e90c90be13c10488c25fb4bfa3969c5304a4fcde21ce8c57acc0947c9d8  output/bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz
    110540ab5aba8959c45255b60077ee7498333852f3d2067610357d811925762f61  output/bitcoin-0.19.99-aarch64-linux-gnu.tar.gz
    12cc1fcc9dd71162baa0f8d85ae9b514bdbf7f92001bfbd45998dd6db92260edc0  output/bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz
    13f978a4496dd37aa7588e7868ae52607b7e8849d86d27b82f2007ea369cddd1fc  output/bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz
    1492da6f327f187cc44bd30c31852111bf09879640efac630303bad2f040e20b93  output/bitcoin-0.19.99-i686-linux-gnu-debug.tar.gz
    15db4654620e0dfe03c4f3a3570b1a9cace15c28c9f68497614710156b7a4a1318  output/bitcoin-0.19.99-i686-linux-gnu.tar.gz
    16d841618b51ce5e9a83ca789b89e86b617c646fee2c9d15487e521e29741f4e72  output/bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz
    17879e526aa5139eea666c949036d811652c3c6f7e3601a53a7617bc21738baf1a  output/bitcoin-0.19.99-riscv64-linux-gnu.tar.gz
    185cde323bbcf1c3331be294b562aab477a72adfad141d5e019ff10e1753178de8  output/bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz
    198cf0240185043fdcfb41500058bbc60c7ec61d15ec30230cc9a33c8e99d73955  output/bitcoin-0.19.99-x86_64-linux-gnu.tar.gz
    20843f958009812087fcb68fab8117d7710db19bcc64d36966121975718609dced  output/src/bitcoin-0.19.99.tar.gz
    

    Had another look over your branch guix branch as well. Do you have any sort of timeline in regards to your patches being upstreamed? We should probably update #16496.

    I’ve also updated the Guix instructions in core-review to pull from master rather than your repo.

19. fanquake referenced this in commit 3774281327 on Jan 28, 2020
20. fanquake merged this on Jan 28, 2020
21. fanquake closed this on Jan 28, 2020

    ---

22. in contrib/guix/README.md:223 in 88c83636d5

    221@@ -224,6 +222,7 @@ repository and will likely put one up soon.
    222 [guix/substitute-server-auth]: https://www.gnu.org/software/guix/manual/en/html_node/Substitute-Server-Authorization.html
    223 [guix/inferiors]: https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html
    

    ---

    ---

    

    MarcoFalke commented at 11:41 pm on January 28, 2020:

    Can remove unused link now?

    ---

    

    fanquake commented at 0:38 am on January 30, 2020:

    Made a note to fix this in 17595.
23. fanquake removed this from the "PRs" column in a project

    ---

24. fanquake referenced this in commit ed3b8eada8 on Apr 12, 2020
25. sidhujag referenced this in commit 9543357506 on Apr 13, 2020
26. PhotoshiNakamoto referenced this in commit ad0ab16751 on Dec 13, 2021
27. DrahtBot locked this on Feb 15, 2022

Contributors
dongcarl MarcoFalke DrahtBot fanquake

Labels
Build system