flatfile_tests fail on alpine linux #15624

1. 
   MarcoFalke commented at 1:59 am on March 20, 2019: member

   File size checks fails and I haven’t figured out why, but next I’d look into how posix_fallocate works on alpine linux.

   Log:

    0$ cat /etc/alpine-release 
    13.9.2
    2
    3$ git log --oneline -1
    4e45b7f20e Merge [#15618](/bitcoin-bitcoin/15618/): refactor: Remove unused function
    5
    6$ ./src/test/test_bitcoin -t flatfile_tests
    7Running 4 test cases...
    8Version using posix_fallocate
    9test/flatfile_tests.cpp(94): error: in "flatfile_tests/flatfile_allocate": check fs::file_size(seq.FileName(FlatFilePos(0, 0))) == 100 has failed [0 != 100]
   10test/flatfile_tests.cpp(98): error: in "flatfile_tests/flatfile_allocate": check fs::file_size(seq.FileName(FlatFilePos(0, 99))) == 100 has failed [0 != 100]
   11Version using posix_fallocate
   12test/flatfile_tests.cpp(102): error: in "flatfile_tests/flatfile_allocate": check fs::file_size(seq.FileName(FlatFilePos(0, 99))) == 200 has failed [0 != 200]
   13Version using posix_fallocate
   14test/flatfile_tests.cpp(116): error: in "flatfile_tests/flatfile_flush": check fs::file_size(seq.FileName(FlatFilePos(0, 1))) == 100 has failed [0 != 100]
   15
   16*** 4 failures are detected in the test module "Bitcoin Core Test Suite"
   

2. MarcoFalke added the label Linux/Unix on Mar 20, 2019
3. MarcoFalke added the label Tests on Mar 20, 2019
4. MarcoFalke added the label Up for grabs on Mar 20, 2019
5. MarcoFalke added the label good first issue on Mar 21, 2019
6. 
   lucayepa commented at 2:33 pm on March 22, 2019: contributor

   I’ve tried to replicate it, but it seems to work.

   Log:

    0$ cat /etc/alpine-release 
    13.9.2
    2
    3$ git log --oneline -1
    4e45b7f20e (HEAD) Merge [#15618](/bitcoin-bitcoin/15618/): refactor: Remove unused function
    5
    6$ ./src/test/test_bitcoin -t flatfile_tests
    7Running 4 test cases...
    8
    9*** No errors detected
   10
   11$ uname -srv
   12Linux 4.19.26-0-virt [#1](/bitcoin-bitcoin/1/)-Alpine SMP Thu Feb 28 21:29:54 UTC 2019
   

7. 
   MarcoFalke commented at 3:33 pm on March 22, 2019: member

   Hmm, I was running alpine in podman.

   0$ uname -srv
   1Linux 4.20.14-200.fc29.x86_64 [#1](/bitcoin-bitcoin/1/) SMP Tue Mar 5 19:55:32 UTC 2019
   

8. 
   lucayepa commented at 9:47 pm on March 22, 2019: contributor

   Maybe it is related to filesystem. Are you using unionfs or aufs? They don’t support posix_fallocate. What do you get, if you try from command line:

   0sudo apk add util-linux
   1fallocate -l 10M filename
   

9. 
   MarcoFalke commented at 0:18 am on March 23, 2019: member

   Indeed

   0fallocate -l 10M filename
   1fallocate: fallocate failed: Not supported
   

10. 
    lucayepa commented at 2:32 am on March 23, 2019: contributor

    This is still strange, because, when there is no posix_fallocate, there is an emulation in glibc. Maybe some versions of glibc do not provide this emulation.

    If the underlying filesystem does not support fallocate(2), then the operation is emulated with the following caveats:

    • The emulation is inefficient.
    • There is a race condition where concurrent writes from another thread or process could be overwritten with null bytes.
    • There is a race condition where concurrent file size increases by another thread or process could result in a file whose size is smaller than expected.
    • If fd has been opened with the O_APPEND or O_WRONLY flags, the function will fail with the error EBADF. In general, the emulation is not MT-safe. On Linux, applications may use fallocate(2) if they cannot tolerate the emulation caveats. In general, this is only recommended if the application plans to terminate the operation if EOPNOTSUPP is returned, otherwise the application itself will need to implement a fallback with all the same problems as the emulation provided by glibc.

    Since we open the files rb+, it should be fine.

11. 
    lucayepa commented at 4:28 am on March 23, 2019: contributor

    We can see the error on strace:

    0$ strace test/test_bitcoin -t flatfile_tests 2>&1 | grep fallocate
    1fallocate(3, 0, 0, 10000)               = -1 EOPNOTSUPP (Operation not supported)
    

12. MarcoFalke closed this on May 2, 2019

    ---

13. MarcoFalke referenced this in commit c4560a7dfe on May 2, 2019
14. sidhujag referenced this in commit b5fe08a951 on May 2, 2019
15. fanquake removed the label Up for grabs on May 8, 2019
16. PastaPastaPasta referenced this in commit b504998b77 on Jun 27, 2021
17. PastaPastaPasta referenced this in commit 7d35d71797 on Jun 28, 2021
18. PastaPastaPasta referenced this in commit a3d5e832a4 on Jun 29, 2021
19. PastaPastaPasta referenced this in commit c29a551053 on Jul 1, 2021
20. PastaPastaPasta referenced this in commit 02534d43d5 on Jul 1, 2021
21. PastaPastaPasta referenced this in commit 43a027d45e on Jul 8, 2021
22. PastaPastaPasta referenced this in commit 0199129fdc on Jul 10, 2021
23. MarcoFalke locked this on Dec 16, 2021

Contributors
MarcoFalke lucayepa

Labels
Linux/Unix Tests good first issue