getrawtransaction xxxxxx…. 2 causes a segfault #28986

issue techy2 openend this issue on December 2, 2023
  1. techy2 commented at 7:43 am on December 2, 2023: none

    Is there an existing issue for this?

    • I have searched the existing issues

    Current behaviour

    ubuntu 18.02 with updated clib (20.02) bitcoincore v25.0

    cli command getrawtransaction 1160bb49db71df577deff983d590ea1eab182a71ada56c6720748b6f2c234128 2
    this combination caused a segfault

    [26881892.674102] b-bitcoin-httpw[7582]: segfault at fffffffffffffff8 ip 000055d902af340e sp 00007fbcf4ff8210 error 5 in kyanite-qt[55d9023a7000+28b0000] [26947689.966408] QThread[20975]: segfault at 18 ip 0000555f4a30a590 sp 00007f6786402ea0 error 4 in kyanite-qt[555f49e71000+28b2000] [26948783.316634] QThread[21670]: segfault at 18 ip 000055f023872590 sp 00007efece402ea0 error 4 in kyanite-qt[55f0233d9000+28b2000] [26951195.452935] QThread[22625]: segfault at 18 ip 0000560e866ce570 sp 00007fdfeec03ea0 error 4 in kyanite-qt[560e86235000+28b2000] [26952087.602076] QThread[23237]: segfault at 18 ip 000055647fce2590 sp 00007f8a01cc4ea0 error 4 in kyanite-qt[55647f849000+28b2000] [26960790.449760] traps: b-bitcoin-httpw[25191] trap divide error ip:558940947530 sp:7fc5f97f8fa0 error:0 in kyanite-qt[5589401f9000+28b2000] [27049939.313930] QThread[3280]: segfault at fffffffffffffff0 ip 0000555cb73207b5 sp 00007feb4762b130 error 5 in kyanite-qt[555cb6bd3000+28b2000] [27049991.320420] QThread[3331]: segfault at fffffffffffffff0 ip 000056397be317b5 sp 00007f8a3e687130 error 5 in kyanite-qt[56397b6e4000+28b2000] [27050370.120192] QThread[4086]: segfault at fffffffffffffff0 ip 0000559ecadfc7a9 sp 00007f8344da0130 error 5 in kyanite-qt[559eca6af000+28b2000] [27050392.997822] QThread[4137]: segfault at fffffffffffffff0 ip 000056501b88e7a9 sp 00007f4fafffc130 error 5 in kyanite-qt[56501b141000+28b2000] [27050465.409277] QThread[4195]: segfault at fffffffffffffff0 ip 000055999c4927b5 sp 00007fdd86e88130 error 5 in kyanite-qt[55999bd45000+28b2000] [27050633.394804] QThread[4279]: segfault at fffffffffffffff0 ip 000056469500f7b5 sp 00007f230cd28130 error 5 in kyanite-qt[5646948c2000+28b2000] [27050737.425313] QThread[4833]: segfault at fffffffffffffff0 ip 0000563fd74fa34b sp 00007f8dda4030f0 error 5 in kyanite-qt[563fd6dad000+28b3000] [27051213.398570] QThread[5455]: segfault at fffffffffffffff0 ip 000055f61a06d3b7 sp 00007fa056c04110 error 5 in kyanite-qt[55f619920000+28b2000] [27088846.545005] traps: b-bitcoin-httpw[6610] trap divide error ip:556835ecea4f sp:7f23f37fcec0 error:0 in kyanite-qt[55683577f000+28b4000] [27313624.022634] traps: b-bitcoin-httpw[19465] trap divide error ip:55fa7dc943f8 sp:7f24cd7f9330 error:0 in kyanite-qt[55fa7d67a000+28b2000] [27351765.886069] traps: b-bitcoin-httpw[20870] trap divide error ip:558c00298509 sp:7fc6917f9210 error:0 in kyanite-qt[558bffc7d000+28b3000] [27752239.427215] mce: [Hardware Error]: Machine check events logged [27752239.427228] EDAC MC0: 1 CE error on CPU#0Channel#0_DIMM#0 (channel:0 slot:0 page:0x0 offset:0x0 grain:8 syndrome:0x0) [27887096.610673] b-bitcoin-httpw[4290]: segfault at 30 ip 00005587078b7681 sp 00007fd0ffffe080 error 4 in kyanite-qt[5587071fb000+28b6000] [28069835.670800] b-bitcoin-httpw[17514]: segfault at 18 ip 0000561df279c6f1 sp 00007f1cf1ff9b30 error 4 in kyanite-qt[561df20e4000+28b5000] [28165624.544731] b-bitcoin-httpw[900]: segfault at 18 ip 0000560af5e179b8 sp 00007f9aecff7b50 error 4 in kyanite-qt[560af575f000+28b5000] [28169813.703180] b-bitcoin-httpw[2709]: segfault at 18 ip 0000559cae76552b sp 00007f234effb740 error 4 in kyanite-qt[559cae0ab000+28b9000] [28173443.842854] b-bitcoin-httpw[3435]: segfault at 18 ip 0000556944ac4273 sp 00007fac2bffd520 error 4 in kyanite-qt[556944409000+28ba000] [28175729.904671] b-bitcoin-httpw[4640]: segfault at 18 ip 000055a563954b80 sp 00007fde95ff9290 error 4 in kyanite-qt[55a56360a000+28bb000]

    Expected behaviour

    did not expect a crash cli help getrawtransaction indicates there is a second verbose mode = 2 ??? obviously not

    Steps to reproduce

    as shown in “current behavior” above, exactly as shown

    built from src with depends ./configure –enable-glibc-back-compat –prefix=$(pwd)/depends/x86_64-pc-linux-gnu LDFLAGS="-static-libstdc++" –enable-cxx –enable-static –disable-shared –disable-debug –disable-tests –disable-bench –with-pic CPPFLAGS="-fPIC -O3 –param ggc-min-expand=1 –param ggc-min-heapsize=32768" CXXFLAGS="-fPIC -O3 –param ggc-min-expand=1 –param ggc-min-heapsize=32768"

    make

    Relevant log output

    see the “current behavior” section above

    How did you obtain Bitcoin Core

    Compiled from source

    What version of Bitcoin Core are you using?

    v25.0 commit 8105bce5b384c72cf08b25b7c5343622754e7337 (HEAD, tag: v25.0)

    Operating system and version

    Ubuntu 18.04.4 LTS

    Machine specifications

    : Intel(R) Xeon(R) CPU X5690 @ 3.47GHz 12 real cores, 24 hyperthreads 192G memory 2T managed raid

  2. techy2 commented at 7:50 am on December 2, 2023: none

    configurations commands for the build shown above are incorrect, should be….

    ./configure –enable-glibc-back-compat –prefix=$(pwd)/depends/x86_64-pc-linux-gnu LDFLAGS="-static-libstdc++" –enable-cxx –enable-static –disable-shared –disable-debug –disable-tests –disable-bench –with-pic CPPFLAGS="-fPIC -O2 –param ggc-min-expand=1 –param ggc-min-heapsize=32768" CXXFLAGS="-fPIC -O2 –param ggc-min-expand=1 –param ggc-min-heapsize=32768"

  3. techy2 commented at 7:53 am on December 2, 2023: none
    When I tried getrawtransaction xxxx…. 1 It worked as expected with a “2” produced the segfault
  4. maflcko commented at 2:25 pm on December 2, 2023: member
    The log you posted says “Hardware Error”, which lead me to believe that this is a hardware error and not a software error?
  5. maflcko commented at 2:26 pm on December 2, 2023: member

    Bitcoin Core makes heavy use of CPU, RAM and disk IO. Hardware defects might only become visible when running Bitcoin Core. You might want to check your hardware for defects.

    • memtest86 to check your RAM
    • to check the CPU behaviour under load, use linpack or Prime95
    • to test your storage device use smartctl or CrystalDiskInfo

    Source: https://bitcoin.stackexchange.com/a/12206

  6. maflcko commented at 2:26 pm on December 2, 2023: member
    Otherwise, if you believe this is a software error, can you try in valgrind or another debugger, to get more information about the cause?
  7. mzumsande commented at 8:19 pm on December 3, 2023: contributor
    Are you using txindex or was the transaction in your mempool at the time of querying?
  8. maflcko added the label Data corruption on Dec 4, 2023
  9. maflcko added the label Linux/Unix on Dec 4, 2023
  10. maflcko added the label GUI on Dec 4, 2023
  11. fanquake commented at 10:39 am on December 4, 2023: member

    configurations commands for the build shown above are incorrect, should be….

    Note that you are passing options to configure that will have no effect. --enable-glibc-back-compat no-longer exists, and will do nothing. --enable-cxx is not one of our configure options, so will also do nothing (might have been copy-pasted from a BDB configure?). I would suggest switching --prefix=xxx for CONFIG_SITE=xxx ./configure your-other-options.

    CPPFLAGS="-fPIC -O2 –param ggc-min-expand=1 –param ggc-min-heapsize=32768"

    Is this meant to be CFLAGS? Otherwise, these are compile options, but are being passed to the preprocessor.

  12. techy2 commented at 2:10 am on December 5, 2023: none

    computer is an HP Proliant GL 380 G7 that has been up for over a year, it is very reliable and runs many other tasks. Don’t think it is a hardware issue or I would think there would be other problems as well and there are not.

    -tindex is off

    The transaction is not one in my wallet, just a random tx from the network I picked off the block explorer, I was doing some testing and picked it just to have a look at the transaction structure as reported by the daemon. In hindsight, because it was a recent tx, it must have been in the mempool.

    First attempt with “verbosity” not set and then with “verbosity” set = 1 went smoothly Blew up when “verbosity” set = 2

    So I’m guessing that in between those two requests the tx exited the mempool but the rpc command did not notice or did not check that it was not present and executed something with a missing/corrupted pointer.

    After restarting the daemon and since then including now, the rpc command correctly tells me that: “No such mempool transaction. Use -txindex or provide a block hash to enable blockchain transaction queries. Use gettransaction for wallet transactions.”

    So it looks like a software bug in “rawtransaction.cpp” or one of the calls used.

    I think there is small window in node/transaction.cpp for a slip up. Maybe my analysis misses something

    for “verbosity” = 2 in rawtransaction.cpp around line 280 we have const CTransactionRef tx = GetTransaction(blockindex, node.mempool.get(), hash, chainman.GetConsensus(), hash_block);

    however, in node/transaction.cpp around line 127 if (mempool && !block_index) { CTransactionRef ptx = mempool->get(hash); if (ptx) return ptx; } There is the opportunity to return without setting hashBlock (hash_block in rawtransaction.cpp)

    hash_block should be checked for NULLPTR prior to this I think

    when “verbosity” = 2, that allows the code to fall all the way to TxToJSON with a bad pointer in hash_block where it will not do the right thing.

  13. maflcko commented at 7:59 am on December 5, 2023: member

    when “verbosity” = 2, that allows the code to fall all the way to TxToJSON with a bad pointer in hash_block where it will not do the right thing.

    hash_block is not a pointer. If it was unset, but read, it would just be all-zero.

    After restarting the daemon and since then including now, the rpc command correctly tells me that: “No such mempool transaction. Use -txindex or provide a block hash to enable blockchain transaction queries. Use gettransaction for wallet transactions.”

    Can you try again, providing the block hash the transaction was included in, in the RPC? Ideally, running in valgrind or another debugger, to get more information about the crash, if it happens again.

  14. mzumsande commented at 5:02 pm on December 5, 2023: contributor
    I found the issue and could reproduce it, it only happens with a pruned node. This line calls IsBlockPruned(blockindex), and if the tx is in the mempool, blockindex is a nullptr. If also m_have_pruned is true, dereferencing pblockindex->nStatus will cause a crash. Will open a fix.
  15. maflcko added this to the milestone 25.2 on Dec 5, 2023
  16. maflcko removed the label GUI on Dec 5, 2023
  17. maflcko removed the label Linux/Unix on Dec 5, 2023
  18. maflcko removed the label Data corruption on Dec 5, 2023
  19. maflcko added the label Bug on Dec 5, 2023
  20. maflcko added the label RPC/REST/ZMQ on Dec 5, 2023
  21. maflcko added the label Needs backport (25.x) on Dec 5, 2023
  22. techy2 commented at 5:26 pm on December 5, 2023: none
    relevant parts of config daemon=1 #txindex=1 server=1 #listen=1 shrinkdebuglog=1 prune=2000 mempoolfullrbf=1
  23. mzumsande commented at 6:31 pm on December 5, 2023: contributor
    See #29003 for a fix - thanks a lot for reporting, I’m really surprised that no one else has run into this before!
  24. techy2 commented at 9:27 pm on December 5, 2023: none
    Over the years I have found a lot of obscure bugs in all kinds of code. I think karma pushes them towards me LOL I’m 80+ and still finding them. I seem to have a knack for it… purely by accident.
  25. fanquake added the label Needs backport (26.x) on Dec 6, 2023
  26. fanquake closed this on Dec 6, 2023
  27. fanquake referenced this in commit dde7ac5c70 on Dec 6, 2023
  28. fanquake removed the label Needs backport (26.x) on Dec 6, 2023
  29. fanquake removed the label Needs backport (25.x) on Dec 6, 2023


techy2 maflcko mzumsande fanquake

Labels
Bug RPC/REST/ZMQ

Milestone
25.2

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-28 22:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me