ci: set docker run –ulimit to workaround Valgrind assertion #27364

1. 
   fanquake commented at 4:15 pm on March 29, 2023: member

   Running the native_fuzz_with_valgrind_job, on aarch64 (Fedora 37), I’ve seen the following:

   0Run addr_info_deserialize with args ['valgrind', '--quiet', '--error-exitcode=1', '/home/fedora/ci_scratch/ci/scratch/build/bitcoin-aarch64-unknown-linux-gnu/src/test/fuzz/fuzz', '-runs=1', '/home/fedora/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/addr_info_deserialize']
   1valgrind: m_libcfile.c:66 (vgPlain_safe_fd): Assertion 'newfd >= VG_(fd_hard_limit)' failed.
   2
   3
   4valgrind: m_libcfile.c:66 (vgPlain_safe_fd): Assertion 'newfd >= VG_(fd_hard_limit)' failed.
   5
   6Target "valgrind --quiet --error-exitcode=1 /home/fedora/ci_scratch/ci/scratch/build/bitcoin-aarch64-unknown-linux-gnu/src/test/fuzz/fuzz -runs=1 /home/fedora/ci_scratch/ci/scratch/qa-assets/fuzz_seed_corpus/addr_info_deserialize" failed with exit code -11
   7./ci/test/04_install.sh: line 98: pop_var_context: head of shell_variables not a function context
   

   This was first reported as a Valgrind bug, https://bugs.kde.org/show_bug.cgi?id=465435, however:

   I really think that the problem is with Docker. It’s advertising some ridiculously high value for ulimit -n like 1048576. Valgrind wants to put its own files in the top 12 of those slots, and is trying to to a fcntl(oldfd, F_DUPFD, 1048576-12) - note that 1048576-12 matches the 1048564 that you get from the patch message. Then Docker fails to honour its promised file descriptor limit and the fcntl fails.

   So the easiest thing to do here might just be to set some sane ulimit values (during docker run), that still work for all other jobs, and avoid the Valgrind assertion (which should become a more useful error message at some point?).

   Opening a PR for discussion/brainstorming. The changes in this PR (from the bug report) “fix” this particular issue, but I haven’t yet tested all jobs etc. Maybe we’d rather only do this on the affected test.

2. fanquake added the label Brainstorming on Mar 29, 2023
3. fanquake added the label Tests on Mar 29, 2023
4. 
   DrahtBot commented at 4:15 pm on March 29, 2023: contributor

   The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

   Reviews

   See the guideline for information on the review process. A summary of reviews will appear here.

5. 
   Sjors commented at 1:57 pm on March 30, 2023: member

   I was able to run this, at least to get past addr_info_deserialize, on x86_64 Ubuntu 22.10 with Docker version 23.0.2.

   Potential caveat: if you’re running Docker in rootless mode then —ulimit seems to be ignored by default. See https://docs.docker.com/engine/security/rootless/#limiting-resources (the instructions there worked for me)

6. 
   maflcko commented at 7:10 am on March 31, 2023: member

   Have you tried podman? The podman-docker on Ubuntu/Debian might help (haven’t tried it), but not sure if it exists on your Fedora.
7. fanquake force-pushed on Apr 13, 2023
8. ci: try workaround file descsriptor issue with Valgrind

   More info: https://bugs.kde.org/show_bug.cgi?id=465435.

   c20a3b8156
9. fanquake force-pushed on Apr 13, 2023
10. 
    maflcko commented at 2:32 pm on April 13, 2023: member

    I tried to reproduce on fedora 37 on current master, and it passed with podman
11. 
    fanquake commented at 12:57 pm on May 4, 2023: member

    Migrated to podman on 38.
12. fanquake closed this on May 4, 2023

    ---

13. fanquake deleted the branch on May 4, 2023
14. bitcoin locked this on May 3, 2024

Contributors
fanquake DrahtBot Sjors maflcko

Labels
Brainstorming Tests