Crash: b-httpworker.2[2027] general protection fault #19571

issue github12101 opened this issue on July 23, 2020
  1. github12101 commented at 8:16 AM on July 23, 2020: none

    <!-- Describe the issue -->

    Crash while running Bitcoin Core-Qt with watch-only wallet, v0.20.0

    <!--- What behavior did you expect? -->

    No crash

    <!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->

    Window disappeared completely

    <!--- How reliably can you reproduce the issue, what are the steps to do so? -->

    Open Bitcoin Core and let it run for days Do nothing Window suddenly disappeared after hours/days of uptime

    System information

    <!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->

    bitcoin-qt v0.20.0 binary downloaded from website, sha256sum bitcoin-qt fd6c2aa5904325083d40edbc0af4668b24a2cf2dbe256a6319648b579de36d2d (I verified it by downloading bitcoin-0.20.0-x86_64-linux-gnu.tar.gz again on another computer, binary sha256sum matches)

    <!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->

    OS: Debian GNU/Linux 10 (buster) x86_64 Kernel: 5.6.0-0.bpo.2-amd64 CPU: AMD Ryzen 7 1700 (16) @ 3.000GHz HDD: mdadm RAID1 consisting of 2x 2TB drives, very good throughput

    <!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->

    DE: MATE WM: Metacity (Marco) WM Theme: Menta Shell: bash 5.0.3

    <!-- Any extra information that might be useful in the debugging process. -->

    <!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

    GUI dissapeared completely, so no traces there dmesg says: [Thu Jul 23 08:06:35 2020] traps: b-httpworker.2[2027] general protection fault ip:55c5648aa040 sp:7f36f9ff9190 error:0 in bitcoin-qt[55c564488000+1dee000] debug.log says nothing, look at timestamps comparing to dmesg:

    2020-07-23T06:43:59Z Socks5() connect to vibsdpohu3atcimw.onion:8333 failed: host unreachable
2020-07-23T06:48:47Z UpdateTip: new best=00000000000000000002452497d3a72f18778c843af3a05c994b08b9f534621f height=640383 version=0x20000000 log2_work=92.138711 tx=551189631 date='2020-07-23T06:48:37Z' progress=1.000000 cache=116.7MiB(838970txo) warning='62 of last 100 blocks have unexpected version.'
2020-07-23T06:51:30Z Pre-allocating up to position 0xd00000 in rev02163.dat
2020-07-23T06:51:31Z UpdateTip: new best=0000000000000000000ee02285dafe68a0fc329a495fc853b0f325fead22f8cd height=640384 version=0x20800000 log2_work=92.138731 tx=551191584 date='2020-07-23T06:51:15Z' progress=1.000000 cache=116.8MiB(839933txo) warning='62 of last 100 blocks have unexpected version.'
2020-07-23T06:52:19Z [default wallet] Submitting wtx (edited for privacy) to mempool for relay
2020-07-23T06:52:19Z [default wallet] ResendWalletTransactions: resubmit 1 unconfirmed transactions
2020-07-23T06:55:39Z UpdateTip: new best=0000000000000000000680fcdd48c25550fe0f1110bb58d2f521e1f415d3407a height=640385 version=0x20c00000 log2_work=92.138751 tx=551193516 date='2020-07-23T06:55:03Z' progress=1.000000 cache=117.1MiB(842826txo) warning='62 of last 100 blocks have unexpected version.'
2020-07-23T06:59:26Z [default wallet] Submitting wtx (edited for privacy) to mempool for relay
2020-07-23T06:59:26Z [default wallet] ResendWalletTransactions: resubmit 1 unconfirmed transactions


[crash happened here on 2020-07-23T07:06:35Z)
Bitcoin Core started again:

2020-07-23T07:27:49Z Bitcoin Core version v0.20.0 (release build)
2020-07-23T07:27:49Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
2020-07-23T07:27:49Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
2020-07-23T07:27:49Z Qt 5.9.8 (static), plugin=xcb (static)
2020-07-23T07:27:49Z System: Debian GNU/Linux 10 (buster), x86_64-little_endian-lp64
2020-07-23T07:27:49Z Screen: default 1280x1024, pixel ratio=1.0
2020-07-23T07:27:49Z Assuming ancestors of block 0000000000000000000f2adce67e49b0b6bdeb9de8b7c3d7e93b21e7fc1e819d have valid signatures.
2020-07-23T07:27:49Z Setting nMinimumChainWork=00000000000000000000000000000000000000000e1ab5ec9348e9f4b8eb8154
2020-07-23T07:27:49Z Using the 'shani(1way,2way)' SHA256 implementation
2020-07-23T07:27:49Z Using RdSeed as additional entropy source
2020-07-23T07:27:49Z Using RdRand as an additional entropy source
2020-07-23T07:27:49Z Default data directory /home/(edited for privacy)/.bitcoin
2020-07-23T07:27:49Z Using data directory /home/(edited for privacy)/BitcoinCore/Blockchain
2020-07-23T07:27:49Z Config file: /home/(edited for privacy)/BitcoinCore/Blockchain/bitcoin.conf
2020-07-23T07:27:49Z Config file arg: addresstype="p2sh-segwit"
2020-07-23T07:27:49Z Config file arg: alertnotify="curl -sI --connect-timeout 1 http://localhost:62602/alertnotify?%s"
2020-07-23T07:27:49Z Config file arg: bind="127.0.0.1"
2020-07-23T07:27:49Z Config file arg: dbcache="8192"
2020-07-23T07:27:49Z Config file arg: deprecatedrpc="signrawtransaction"
2020-07-23T07:27:49Z Config file arg: disablewallet="0"
2020-07-23T07:27:49Z Config file arg: listen="1"
2020-07-23T07:27:49Z Config file arg: maxmempool="1024"
2020-07-23T07:27:49Z Config file arg: maxorphantx="1000"
2020-07-23T07:27:49Z Config file arg: maxuploadtarget="25313"
2020-07-23T07:27:49Z Config file arg: mempoolexpiry="744"
2020-07-23T07:27:49Z Config file arg: onlynet="onion"
2020-07-23T07:27:49Z Config file arg: par="16"
2020-07-23T07:27:49Z Config file arg: proxy="127.0.0.1:9050"
2020-07-23T07:27:49Z Config file arg: rpcpassword=****
2020-07-23T07:27:49Z Config file arg: rpcuser=****
2020-07-23T07:27:49Z Config file arg: rpcworkqueue="32"
2020-07-23T07:27:49Z Config file arg: server="1"
2020-07-23T07:27:49Z Config file arg: testnet="0"
2020-07-23T07:27:49Z Config file arg: torpassword=****
2020-07-23T07:27:49Z Config file arg: txindex="1"
2020-07-23T07:27:49Z Config file arg: walletnotify="curl -sI --connect-timeout 1 http://localhost:62602/walletnotify?%s"
2020-07-23T07:27:49Z Config file arg: zmqpubrawblock="tcp://127.0.0.1:29000"
2020-07-23T07:27:49Z Config file arg: zmqpubrawtx="tcp://127.0.0.1:29000"
2020-07-23T07:27:49Z Command-line arg: datadir="/home/(edited for privacy)/BitcoinCore/Blockchain"
2020-07-23T07:27:49Z Using at most 125 automatic connections (1024 file descriptors available)
2020-07-23T07:27:49Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2020-07-23T07:27:49Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2020-07-23T07:27:49Z Script verification uses 15 additional threads
2020-07-23T07:27:49Z scheduler thread start
2020-07-23T07:27:49Z HTTP: creating work queue of depth 32
2020-07-23T07:27:49Z Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcauth for rpcauth auth generation.
2020-07-23T07:27:49Z HTTP: starting 4 worker threads
2020-07-23T07:27:49Z Using wallet directory /home/(edited for privacy)/BitcoinCore/Blockchain
2020-07-23T07:27:49Z init message: Verifying wallet(s) ...
2020-07-23T07:27:49Z Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2020-07-23T07:27:49Z Using wallet /home/(edited for privacy)/BitcoinCore/Blockchain
2020-07-23T07:27:49Z BerkeleyEnvironment::Open: LogDir=/home/(edited for privacy)/BitcoinCore/Blockchain/database ErrorFile=/home/(edited for privacy)/BitcoinCore/Blockchain/db.log
2020-07-23T07:27:49Z init message: Loading banlist ...
2020-07-23T07:27:49Z Using /16 prefix for IP bucketing
2020-07-23T07:27:49Z Cache configuration:
2020-07-23T07:27:49Z * Using 2.0 MiB for block index database
2020-07-23T07:27:49Z * Using 1023.8 MiB for transaction index database
2020-07-23T07:27:49Z * Using 8.0 MiB for chain state database
2020-07-23T07:27:49Z * Using 7158.2 MiB for in-memory UTXO set (plus up to 976.6 MiB of unused mempool space)
2020-07-23T07:27:49Z init message: Loading block index ...

    I've edited few things (like tx and username of the machine) for privacy. Included debug.log just before crash (Bitcoin Core uses timestamp in GMT timezone looks like), then dmesg output (in my timezone which is GMT+1), and then debug.log output again from new start, so you can see my config and Core's configuration. I am using Tor only network and I added extra threads and memory to Core, as I have plenty of computing power and RAM. Bitcoin Core can run for days or weeks, but today it crashed.

    Hopefully it helps :) Thanks.

  2. github12101 added the label Bug on Jul 23, 2020
  3. promag commented at 8:23 AM on July 23, 2020: member

    I don't know what "general protection fault ip" means. Anyway, are you running bitcoin-qt with -server=1?

    Edit: I'll answer it myself 😅 yes, server=1 is set. Do you have anything making RPC or REST requests?

  4. github12101 commented at 8:25 AM on July 23, 2020: none

    I don't know what "general protection fault ip" means. Anyway, are you running bitcoin-qt with -server=1?

    Yes, I am running with server=1 in config file, please look at debug.log output. I have ElectrumX connected to this Bitcoin Core.

  5. promag commented at 8:27 AM on July 23, 2020: member

    I have ElectrumX connected to this Bitcoin Core.

    Does it make a difference if you run bitcoind instead?

  6. github12101 commented at 8:28 AM on July 23, 2020: none

    I could try. Will it load all my settings I set in config file, just as bitcoin-qt does? So I can just switch between qt and non-qt without any difference?

  7. promag commented at 8:32 AM on July 23, 2020: member

    Yes. If you can try running with gdb.

  8. github12101 commented at 8:38 AM on July 23, 2020: none

    I tried bitcoind and it works perfectly, so now I can switch to bitcoind. Also just installed gdb. Could you tell me what command to run to debug with gdb?

  9. promag commented at 8:42 AM on July 23, 2020: member

    There's lot of guides for that, for instance https://stackoverflow.com/a/43364504/1978589, but you can search for "debug crash with gdb" or something like that.

  10. github12101 commented at 8:57 AM on July 23, 2020: none

    I started bitcoind normally, then on another console started gdb and attached it to bitcoind pid. I'll leave it like that and will wait now if crash ever happens again.

  11. promag commented at 9:03 AM on July 23, 2020: member

    If nothing happens, then try debugging with bitcoin-qt so we can try to fix that crash.

  12. github12101 commented at 9:26 AM on July 23, 2020: none

    Unfortunately using attach [pid] from gdb stales bitcoin process, it's "frozen", no output and no communication. When I detach, bitcoin process gets alive again. I need to find another way to attach gdb to bitcoin process. i will search on internet. If you have exact command to run, please share.

  13. github12101 commented at 12:27 PM on August 18, 2020: none

    Crash happened again today on bitcoind. Dmesg:

    [Tue Aug 18 04:18:53 2020] b-httpworker.1[1268]: segfault at 4cbdd ip 0000558274347f57 sp 00007f7f41ff9180 error 4 in bitcoind[558274103000+9b5000]
[Tue Aug 18 04:18:53 2020] Code: 18 31 c0 eb 19 66 0f 1f 44 00 00 48 83 eb 08 84 c0 0f 84 d4 00 00 00 48 8b 03 48 89 43 08 48 8b 6b f8 49 8b 4c 24 78 49 89 dd <48> 8b 55 78 48 39 d1 0f 92 c0 75 d5 66 0f ef d2 48 8b 75 38 48 8b

    So this issue persists and it's on both Bitcoin-Qt and bitcoind. bitcoind had been running for 10h45min and then it crashed. Please also notice that this time error reported is segfault, not general protection fault.

  14. github12101 commented at 9:20 PM on September 4, 2020: none

    bitcoind been running for 5 days and crashed:

    [Fri Sep  4 22:01:28 2020] b-httpworker.2[926]: segfault at 0 ip 0000558345006510 sp 00007f5707ffd1b0 error 4 in bitcoind[558344ba6000+9b5000]
[Fri Sep  4 22:01:28 2020] Code: e6 75 ef 4c 8b 65 40 4d 85 e4 74 08 4c 89 e7 e8 06 5b 28 00 4c 8b 75 30 4c 8b 65 28 4d 39 e6 74 29 66 0f 1f 84 00 00 00 00 00 <49> 8b 3c 24 49 8d 44 24 10 48 39 c7 74 05 e8 dd 5a 28 00 49 83 c4

    Just again, my bitcoin version:

    $ bitcoin-cli --version
Bitcoin Core RPC client version v0.20.1

    And current config:

    2020-09-04T21:01:31Z Config file: /home/nv01/.bitcoin/bitcoin.conf
2020-09-04T21:01:31Z Config file arg: addresstype="p2sh-segwit"
2020-09-04T21:01:31Z Config file arg: alertnotify="curl -sI --connect-timeout 1 http://localhost:62602/alertnotify?%s"
2020-09-04T21:01:31Z Config file arg: bind="127.0.0.1"
2020-09-04T21:01:31Z Config file arg: dbcache="8192"
2020-09-04T21:01:31Z Config file arg: deprecatedrpc="signrawtransaction"
2020-09-04T21:01:31Z Config file arg: disablewallet="0"
2020-09-04T21:01:31Z Config file arg: listen="1"
2020-09-04T21:01:31Z Config file arg: maxmempool="1024"
2020-09-04T21:01:31Z Config file arg: maxorphantx="1000"
2020-09-04T21:01:31Z Config file arg: maxuploadtarget="25313"
2020-09-04T21:01:31Z Config file arg: onlynet="onion"
2020-09-04T21:01:31Z Config file arg: par="16"
2020-09-04T21:01:31Z Config file arg: proxy="127.0.0.1:9050"
2020-09-04T21:01:31Z Config file arg: rpcpassword=****
2020-09-04T21:01:31Z Config file arg: rpcuser=****
2020-09-04T21:01:31Z Config file arg: rpcworkqueue="32"
2020-09-04T21:01:31Z Config file arg: server="1"
2020-09-04T21:01:31Z Config file arg: testnet="0"
2020-09-04T21:01:31Z Config file arg: torpassword=****
2020-09-04T21:01:31Z Config file arg: txindex="1"
2020-09-04T21:01:31Z Config file arg: walletnotify="curl -sI --connect-timeout 1 http://localhost:62602/walletnotify?%s"
2020-09-04T21:01:31Z Config file arg: zmqpubrawblock="tcp://127.0.0.1:29000"
2020-09-04T21:01:31Z Config file arg: zmqpubrawtx="tcp://127.0.0.1:29000"

    Originally when created this issue I was using Bitcoin-Qt, but since I moved to bitcoind, upgraded to v0.20.1, but this problem persists.

    OS: Debian GNU/Linux 10 (buster) x86_64 (fully updated with backports) Kernel: 5.7.0-0.bpo.2-amd64 (upgraded from 5.6 since creating this issue)

  15. MarcoFalke commented at 9:28 AM on September 5, 2020: member

    Unfortunately using attach [pid] from gdb stales bitcoin process, it's "frozen", no output and no communication. When I detach, bitcoin process gets alive again.

    When you attach gdb, you might have to type continue

  16. github12101 commented at 9:56 AM on September 5, 2020: none

    Fantastic, thanks. Now I started gdb, attached to pid and continued. bitcoind works normally. Also I edited my previous post - I am using v0.20.1, not v0.20.0.

  17. github12101 commented at 7:37 PM on September 12, 2020: none

    bitcoind has been online for 7 days and continues to operate. gdb says:

    (...)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f475992cbf0 in __GI___nanosleep (requested_time=0x7ffe1088c6d0, remaining=0x7ffe1088c6d0) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
28      ../sysdeps/unix/sysv/linux/nanosleep.c: No such file or directory.
(gdb) continue
Continuing.
[New Thread 0x7f462eb41700 (LWP 951382)]
[Detaching after fork from child process 951383]
[Thread 0x7f462eb41700 (LWP 951382) exited]
(gdb) [New Thread 0x7f462eb41700 (LWP 3790577)]
[Detaching after fork from child process 3790578]
[Thread 0x7f462eb41700 (LWP 3790577) exited]
[New Thread 0x7f462eb41700 (LWP 3793663)]
[Detaching after fork from child process 3793664]
[Thread 0x7f462eb41700 (LWP 3793663) exited]
[New Thread 0x7f462eb41700 (LWP 4067590)]
[Detaching after fork from child process 4067591]
[Thread 0x7f462eb41700 (LWP 4067590) exited]
[New Thread 0x7f462eb41700 (LWP 4075366)]
[Detaching after fork from child process 4075367]
[Thread 0x7f462eb41700 (LWP 4075366) exited]
(gdb) [New Thread 0x7f462eb41700 (LWP 2815996)]
[Detaching after fork from child process 2815997]
[Thread 0x7f462eb41700 (LWP 2815996) exited]
[New Thread 0x7f462eb41700 (LWP 3436644)]
[Detaching after fork from child process 3436645]
[Thread 0x7f462eb41700 (LWP 3436644) exited]
(gdb) [New Thread 0x7f462eb41700 (LWP 614817)]
[Detaching after fork from child process 614818]
[Thread 0x7f462eb41700 (LWP 614817) exited]
[New Thread 0x7f462eb41700 (LWP 627796)]
[Detaching after fork from child process 627797]
[Thread 0x7f462eb41700 (LWP 627796) exited]
[New Thread 0x7f462eb41700 (LWP 735205)]
[Detaching after fork from child process 735206]
[Thread 0x7f462eb41700 (LWP 735205) exited]
[New Thread 0x7f462eb41700 (LWP 737834)]
[Detaching after fork from child process 737835]
[Thread 0x7f462eb41700 (LWP 737834) exited]

    I hope it will catch error there very soon. Will report once something happens.

  18. MarcoFalke commented at 11:55 AM on October 2, 2020: member

    I hope it will catch error there very soon. Will report once something happens.

    Ok, closing for now until we have more info to work with

  19. MarcoFalke closed this on Oct 2, 2020
  20. github12101 commented at 1:44 PM on October 2, 2020: none

    Bitcoin Core stats: Downloaded: 8190 MB Uploaded: 42116 MB Connections: 21 Onion address: b..y.onion:8333 Onion service score: 1515 Uptime: 26d:23h:12m

    Bitcoind has been stable for 26 days now. Please do not close too early. I am still waiting for a bug to catch it.

  21. MarcoFalke commented at 1:51 PM on October 2, 2020: member

    I'll reopen as soon as you hit the bug again

  22. MarcoFalke locked this on Feb 15, 2022
Contributors
github12101promagMarcoFalke
Labels
Bug
github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-25 00:14 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me