gitian: Improve error handling #15549

pull laanwj wants to merge 1 commits into bitcoin:master from laanwj:2019_03_gitian_error_handling changing 6 files +12 −3
  1. laanwj commented at 5:29 pm on March 6, 2019: member

    Improve error handling in gitian builds:

    • Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
    • Also use xargs instead of find -exec, because find will not propagate errors in the executed command, but xargs will.

    This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).

  2. laanwj added this to the milestone 0.18.0 on Mar 6, 2019
  3. laanwj force-pushed on Mar 6, 2019
  4. MarcoFalke added the label Needs backport on Mar 6, 2019
  5. MarcoFalke added the label Build system on Mar 6, 2019
  6. hebasto commented at 6:23 pm on March 6, 2019: member
    Concept ACK.
  7. laanwj requested review from theuni on Mar 6, 2019
  8. MarcoFalke added the label Needs gitian build on Mar 6, 2019
  9. practicalswift commented at 9:12 pm on March 6, 2019: contributor
    Concept ACK
  10. laanwj commented at 0:17 am on March 7, 2019: member

    #15541 (comment) :

    I see the default size for the LXC disk is 10240MB maybe that’s no longer enough

    Independently of this we should probably increase the default LXC base VM image size (here https://github.com/devrandom/gitian-builder/blob/master/bin/make-base-vm#L267 ), or make the descriptor clean up the intermediate result after each host build. I think I have a slight preference for the latter as it doesn’t require everyone to regenerate—or resize—their base images.

  11. luke-jr commented at 7:06 am on March 7, 2019: member
    Related: https://github.com/devrandom/gitian-builder/pull/172
  12. DrahtBot removed the label Needs gitian build on Mar 7, 2019
  13. DrahtBot commented at 12:29 pm on March 7, 2019: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #15554 (docs: binary tar improvement by cisba)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  14. in contrib/devtools/split-debug.sh.in:2 in 9793a5ca75 outdated 
    0@@ -1,5 +1,5 @@
1 #!/bin/sh
2-
3+set -e -o pipefail
    MarcoFalke commented at 3:05 pm on March 7, 2019:

    DrahtBot fails with

    0../contrib/devtools/split-debug.sh: 2: set: Illegal option -o pipefail
    MarcoFalke commented at 3:07 pm on March 7, 2019:

    On Ubuntu the default shell is dash (aka Debian Almquist Shell), to which /bin/sh is symlink. When your shell script is run with #!/bin/sh, you are effectively trying to run it with the default shell. However, dash doesn’t have the pipefail option, which is why you’re getting the error.

    https://askubuntu.com/a/886540

    laanwj commented at 4:17 pm on March 7, 2019:
    Oh crap. I hate shellscript programming so much.
    laanwj commented at 4:20 pm on March 7, 2019:

    Oh wait, is this only about split-debug.sh.in, not the main gitian scripts? I don’t think the pipefail matters there at all as the script doesn’t use pipes.

    Edit: removed it here, please retry

  15. laanwj closed this on Mar 7, 2019
  16. laanwj reopened this on Mar 7, 2019
  17. gitian: Improve error handling 32da92bdf6
  18. laanwj force-pushed on Mar 7, 2019
  19. MarcoFalke deleted a comment on Mar 7, 2019
  20. MarcoFalke added the label Needs gitian build on Mar 7, 2019
  21. DrahtBot commented at 10:49 pm on March 8, 2019: member

    Gitian builds for commit d211edb34982aedbcd514ba3723396f2369ff286 (master):

    Gitian builds for commit 6eb1a44d8d316c144d96924ca5c55dac35bc0ca3 (master and this pull):

  22. DrahtBot removed the label Needs gitian build on Mar 8, 2019
  23. MarcoFalke commented at 11:18 pm on March 8, 2019: member
    utACK 32da92bdf6bb55d6d312b0f85797d439cc942db5
  24. laanwj referenced this in commit f810f14cf6 on Mar 9, 2019
  25. laanwj merged this on Mar 9, 2019
  26. laanwj closed this on Mar 9, 2019
  27. laanwj referenced this in commit 12408d33c6 on Mar 9, 2019
  28. fanquake removed the label Needs backport on Mar 9, 2019
  29. fanquake commented at 5:49 am on March 9, 2019: member
    This has been backported to 0.18.0 in https://github.com/bitcoin/bitcoin/commit/f810f14cf61db3968b905f84c3eea3056f89c2a8.
  30. HashUnlimited referenced this in commit f22c67f72f on Mar 10, 2019
  31. hebasto commented at 5:56 pm on March 14, 2019: member

    Just noted that -e option is already set in bin/gbuild:

    0 script.puts "set -e"

    So set -o pipefail (without -e) could be sufficient.

  32. PastaPastaPasta referenced this in commit 7e64f9a6fe on Jun 27, 2021
  33. PastaPastaPasta referenced this in commit f8134bc157 on Jun 28, 2021
  34. PastaPastaPasta referenced this in commit 6ae817a5b7 on Jun 29, 2021
  35. PastaPastaPasta referenced this in commit 009543c188 on Jul 1, 2021
  36. PastaPastaPasta referenced this in commit 9a144a5e55 on Jul 1, 2021
  37. PastaPastaPasta referenced this in commit 2a9cc9afdc on Jul 8, 2021
  38. PastaPastaPasta referenced this in commit a33756c72e on Jul 10, 2021
  39. DrahtBot locked this on Dec 16, 2021


laanwj hebasto practicalswift luke-jr DrahtBot MarcoFalke fanquake


theuni

Labels
Build system

Milestone
0.18.0

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-19 09:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me