Add instructions for parallel gitian builds. #10968

1. 
   coblee commented at 6:31 AM on August 1, 2017: contributor

   This makes builds much faster if you have a multi-core machine.

2. 
   laanwj commented at 9:11 AM on August 1, 2017: member

   utACK https://github.com/bitcoin/bitcoin/pull/10968/commits/36928b069561501c9d239830ed59d27dd9814325

   (that's awful, it only picks a single-core VM by default?)

3. laanwj added the label Docs and Output on Aug 1, 2017
4. 
   coblee commented at 9:15 AM on August 1, 2017: contributor

   Yes by default, it's only a single-core VM. And gbuild defaults to "-j 2". Now builds are 4 times faster on my machine.

5. in doc/release-process.md:114 in 36928b0695 outdated

    110 | @@ -111,16 +111,16 @@ The gbuild invocations below <b>DO NOT DO THIS</b> by default.
    111 |  ### Build and sign Bitcoin Core for Linux, Windows, and OS X:
    112 |  
    113 |      pushd ./gitian-builder
    114 | -    ./bin/gbuild --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
    115 | +    ./bin/gbuild --num-make `nproc` --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
   

   ---

   ---

   

   laanwj commented at 7:05 AM on August 2, 2017:

   This is a bit dangerous - in many cases, number of concurrent C++ compiles is limited by memory, not number of cores

   ---

   

   PRabahy commented at 3:05 AM on August 3, 2017:

   https://github.com/bitcoin/bitcoin/blob/master/contrib/gitian-build.sh appears to use -j instead of --num-make. Is there a difference?

   ---

   

   coblee commented at 3:15 AM on August 3, 2017:

   -j and --num-make are the same thing. @laanwj what's a better way then? I do know that builds are much faster on my quad-core mbp when I run it with "--num-make 4"

   ---

   

   laanwj commented at 6:18 AM on August 3, 2017:

   You could just provide the default of --num-make 2 here, which works with --memory 3000 Doubling one of the numbers means you have to double the other one too, at least for KVM, for LXC the memory limit is ignored which is why you didn't notice anything.

6. coblee force-pushed on Aug 3, 2017
7. 
   coblee commented at 6:37 AM on August 3, 2017: contributor

   @laanwj I made the fix you recommended. Change num-make to 2

8. 
   jonasschnelli commented at 9:47 AM on August 3, 2017: contributor

   utACK 393e77637c65f73c65de8462454d2582a5e359a7

9. 
   fanquake commented at 8:14 AM on August 9, 2017: member

   utACK 393e776

10. 
    MarcoFalke commented at 10:59 AM on August 9, 2017: member

    utACK 393e776, but I am not entirely happy about the binary files in our source code repo. At least it seems pngcrush can reduce the size by 30k... Would you mind fixing that up? (c.f. 7a482e4ac)

11. 
    fanquake commented at 1:36 PM on August 12, 2017: member

    ping @coblee Could you make Marcos change so we can merge this?

12. coblee force-pushed on Aug 13, 2017
13. 
    coblee commented at 9:59 AM on August 13, 2017: contributor

    Done and pngcrush'd 2 other images.

14. Add instructions for multi-processor gitian builds e93ff718c5
15. MarcoFalke force-pushed on Aug 13, 2017
16. 
    MarcoFalke commented at 10:32 AM on August 13, 2017: member

    Looks like my compression was slightly more efficient. (I used contrib/devtools/optimize-pngs.py)

    Also, rewriting existing files in the git repo will keep the old versions around. Probably not worth trying to reduce the size...

    I reverted the change to the existing files and force pushed the commit to the litecoin repo. I hope this is ok.

17. MarcoFalke added this to the milestone 0.15.0 on Aug 13, 2017
18. 
    coblee commented at 5:48 PM on August 13, 2017: contributor

    Sure, np

19. 
    fanquake commented at 12:34 AM on August 14, 2017: member

    ACK e93ff71

20. laanwj merged this on Aug 14, 2017
21. laanwj closed this on Aug 14, 2017

    ---

22. laanwj referenced this in commit 98aa3f6d5c on Aug 14, 2017
23. thrasher- referenced this in commit d991b65bba on Aug 17, 2017
24. thrasher- referenced this in commit 27287cd946 on Aug 17, 2017
25. thrasher- deleted the branch on Aug 17, 2017
26. PastaPastaPasta referenced this in commit d0d2d3fb9c on Aug 6, 2019
27. PastaPastaPasta referenced this in commit 4b67772174 on Aug 6, 2019
28. PastaPastaPasta referenced this in commit 4329905876 on Aug 6, 2019
29. PastaPastaPasta referenced this in commit fb2774d7d2 on Aug 7, 2019
30. PastaPastaPasta referenced this in commit c3446d68a1 on Aug 8, 2019
31. PastaPastaPasta referenced this in commit 19a17a58d2 on Aug 12, 2019
32. barrystyle referenced this in commit 014c94c6e7 on Jan 22, 2020
33. DrahtBot locked this on Sep 8, 2021

Contributors

cobleelaanwjPRabahyjonasschnellifanquakeMarcoFalke

Milestone
0.15.0