<!-- This issue tracker is only for technical issues related to Bitcoin Core. General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com. For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/. If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! -->
<!-- Describe the issue -->
With a binary built at master @ https://github.com/bitcoin/bitcoin/commit/a04440, bitcoind crashes with segfault on startup on one of my nodes.
<!--- What behavior did you expect? -->
Expected: No segfault.
<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->
Actual: Segfault, with no more info in debug.log, even with -debug set (see below).
<!--- How reliably can you reproduce the issue, what are the steps to do so? -->
Crash seems reproducible, but only on this node. I'm recompiling with --enable-debug to see if that gives more info.
<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->
Self-compiled at https://github.com/bitcoin/bitcoin/commit/a04440. Switching to the version built at https://github.com/bitcoin/bitcoin/commit/6324c6 that the node was running earlier starts up fine.
<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->
Linux, armv7l, SSD disks.
<!-- Any extra information that might be useful in the debugging process. -->
<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->
Debug logs:
$ bitcoind -debug
2018-04-11T11:57:14Z
2018-04-11T11:57:14Z Bitcoin Core version v0.16.99.0-a04440feb (release build)
2018-04-11T11:57:14Z InitParameterInteraction: parameter interaction: -blocksonly=1 -> setting -whitelistrelay=0
2018-04-11T11:57:14Z Assuming ancestors of block 0000000000000000005214481d2d96f898e3d5416e43359c145944a909d242e0 have valid signatures.
2018-04-11T11:57:14Z Setting nMinimumChainWork=000000000000000000000000000000000000000000f91c579d57cad4bc5278cc
2018-04-11T11:57:14Z ParameterInteraction: parameter interaction: -blocksonly=1 -> setting -walletbroadcast=0
2018-04-11T11:57:14Z Using the 'standard' SHA256 implementation
2018-04-11T11:57:14Z Default data directory /home/bitcoin/.bitcoin
2018-04-11T11:57:14Z Using data directory /home/bitcoin/.bitcoin
2018-04-11T11:57:14Z Using config file /home/bitcoin/.bitcoin/bitcoin.conf
2018-04-11T11:57:14Z Using at most 16 automatic connections (1048576 file descriptors available)
2018-04-11T11:57:14Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2018-04-11T11:57:14Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2018-04-11T11:57:14Z Using 4 threads for script verification
2018-04-11T11:57:14Z scheduler thread start
2018-04-11T11:57:14Z Allowing HTTP connections from: 127.0.0.0/8 ::1/128 163.172.184.153/32 10.4.2.0/24
2018-04-11T11:57:14Z Binding RPC on address :: port 8332
2018-04-11T11:57:14Z Binding RPC on address 0.0.0.0 port 8332
2018-04-11T11:57:14Z Binding RPC on address 0.0.0.0 port 8332 failed.
2018-04-11T11:57:14Z Initialized HTTP server
2018-04-11T11:57:14Z HTTP: creating work queue of depth 16
2018-04-11T11:57:14Z Starting RPC
2018-04-11T11:57:14Z Starting HTTP RPC server
2018-04-11T11:57:14Z No rpcpassword set - using random cookie authentication
2018-04-11T11:57:14Z Generated RPC authentication cookie /home/bitcoin/.bitcoin/.cookie
2018-04-11T11:57:14Z Registering HTTP handler for / (exactmatch 1)
2018-04-11T11:57:14Z Registering HTTP handler for /wallet/ (exactmatch 0)
2018-04-11T11:57:14Z Starting HTTP server
2018-04-11T11:57:14Z HTTP: starting 4 worker threads
2018-04-11T11:57:14Z Using wallet directory /home/bitcoin/.bitcoin
2018-04-11T11:57:14Z init message: Verifying wallet(s)...
2018-04-11T11:57:14Z Entering http event loop
2018-04-11T11:57:14Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-04-11T11:57:14Z Using wallet wallet.dat
2018-04-11T11:57:14Z CDBEnv::Open: LogDir=/home/bitcoin/.bitcoin/database ErrorFile=/home/bitcoin/.bitcoin/db.log
2018-04-11T11:57:14Z net: setting try another outbound peer=false
2018-04-11T11:57:14Z Cache configuration:
2018-04-11T11:57:14Z * Using 12.5MiB for block index database
2018-04-11T11:57:14Z * Using 8.0MiB for chain state database
2018-04-11T11:57:14Z * Using 79.5MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2018-04-11T11:57:14Z init message: Loading block index...
2018-04-11T11:57:14Z LevelDB using max_open_files=64 (default=1000)
2018-04-11T11:57:14Z Opening LevelDB in /home/bitcoin/.bitcoin/blocks/index
2018-04-11T11:57:14Z leveldb: Recovering log [#109195](/bitcoin-bitcoin/109195/)
2018-04-11T11:57:14Z leveldb: Level-0 table [#109210](/bitcoin-bitcoin/109210/): started
2018-04-11T11:57:14Z leveldb: Level-0 table [#109210](/bitcoin-bitcoin/109210/): 179 bytes OK
2018-04-11T11:57:14Z leveldb: Delete type=0 [#109195](/bitcoin-bitcoin/109195/)
2018-04-11T11:57:14Z leveldb: Delete type=3 [#109193](/bitcoin-bitcoin/109193/)
2018-04-11T11:57:14Z Opened LevelDB successfully
2018-04-11T11:57:14Z Using obfuscation key for /home/bitcoin/.bitcoin/blocks/index: 0000000000000000
2018-04-11T11:57:37Z LoadBlockIndexDB: last block file = 1230
2018-04-11T11:57:37Z leveldb: Compacting 1@3 + 13@4 files
2018-04-11T11:57:37Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=196, size=119299357, heights=517496...517691, time=2018-04-10...2018-04-11)
2018-04-11T11:57:37Z Checking all blk files are present...
2018-04-11T11:57:37Z leveldb: Generated table [#109212](/bitcoin-bitcoin/109212/)@3: 14790 keys, 2141156 bytes
2018-04-11T11:57:37Z leveldb: Generated table [#109213](/bitcoin-bitcoin/109213/)@3: 494 keys, 71674 bytes
2018-04-11T11:57:38Z leveldb: Generated table [#109214](/bitcoin-bitcoin/109214/)@3: 14790 keys, 2141044 bytes
2018-04-11T11:57:38Z LoadBlockIndexDB: transaction index enabled
2018-04-11T11:57:38Z LevelDB using max_open_files=64 (default=1000)
2018-04-11T11:57:38Z Opening LevelDB in /home/bitcoin/.bitcoin/chainstate
2018-04-11T11:57:38Z leveldb: Generated table [#109215](/bitcoin-bitcoin/109215/)@3: 313 keys, 45397 bytes
2018-04-11T11:57:38Z leveldb: Recovering log [#118330](/bitcoin-bitcoin/118330/)
2018-04-11T11:57:38Z leveldb: Level-0 table [#118367](/bitcoin-bitcoin/118367/): started
2018-04-11T11:57:38Z leveldb: Level-0 table [#118367](/bitcoin-bitcoin/118367/): 308 bytes OK
2018-04-11T11:57:38Z leveldb: Delete type=3 [#118328](/bitcoin-bitcoin/118328/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118175](/bitcoin-bitcoin/118175/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118069](/bitcoin-bitcoin/118069/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118062](/bitcoin-bitcoin/118062/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118068](/bitcoin-bitcoin/118068/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118061](/bitcoin-bitcoin/118061/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118066](/bitcoin-bitcoin/118066/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118064](/bitcoin-bitcoin/118064/)
2018-04-11T11:57:38Z leveldb: Delete type=0 [#118330](/bitcoin-bitcoin/118330/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118032](/bitcoin-bitcoin/118032/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118063](/bitcoin-bitcoin/118063/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118060](/bitcoin-bitcoin/118060/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118065](/bitcoin-bitcoin/118065/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118368](/bitcoin-bitcoin/118368/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118366](/bitcoin-bitcoin/118366/)
2018-04-11T11:57:38Z leveldb: Delete type=2 [#118067](/bitcoin-bitcoin/118067/)
2018-04-11T11:57:38Z Opened LevelDB successfully
2018-04-11T11:57:38Z Using obfuscation key for /home/bitcoin/.bitcoin/chainstate: 72e7c68d386988b3
2018-04-11T11:57:38Z leveldb: Generated table [#109216](/bitcoin-bitcoin/109216/)@3: 42165 keys, 2173766 bytes
2018-04-11T11:57:38Z Loaded best chain: hashBestChain=0000000000000000003f6dfa302cd20c2721555a8004ef8c84864f64e68df67c height=517691 date=2018-04-11T11:45:00Z progress=0.999991
2018-04-11T11:57:38Z init message: Rewinding blocks...
2018-04-11T11:57:38Z leveldb: Generated table [#109217](/bitcoin-bitcoin/109217/)@3: 42164 keys, 2173669 bytes
2018-04-11T11:57:39Z leveldb: Generated table [#109218](/bitcoin-bitcoin/109218/)@3: 15988 keys, 823981 bytes
2018-04-11T11:57:39Z leveldb: Generated table [#109219](/bitcoin-bitcoin/109219/)@3: 42164 keys, 2173673 bytes
2018-04-11T11:57:39Z leveldb: Generated table [#109220](/bitcoin-bitcoin/109220/)@3: 42164 keys, 2173039 bytes
2018-04-11T11:57:40Z leveldb: Generated table [#109221](/bitcoin-bitcoin/109221/)@3: 42164 keys, 2173419 bytes
2018-04-11T11:57:40Z leveldb: Generated table [#109222](/bitcoin-bitcoin/109222/)@3: 26480 keys, 1364917 bytes
2018-04-11T11:57:40Z leveldb: Generated table [#109223](/bitcoin-bitcoin/109223/)@3: 42164 keys, 2173787 bytes
2018-04-11T11:57:41Z leveldb: Generated table [#109224](/bitcoin-bitcoin/109224/)@3: 37474 keys, 1931396 bytes
2018-04-11T11:57:41Z leveldb: Compacted 1@3 + 13@4 files => 21560918 bytes
2018-04-11T11:57:41Z leveldb: compacted to: files[ 3 7 62 725 5024 2064 0 ]
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109137](/bitcoin-bitcoin/109137/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109145](/bitcoin-bitcoin/109145/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109144](/bitcoin-bitcoin/109144/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109139](/bitcoin-bitcoin/109139/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109142](/bitcoin-bitcoin/109142/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109143](/bitcoin-bitcoin/109143/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109134](/bitcoin-bitcoin/109134/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109146](/bitcoin-bitcoin/109146/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109136](/bitcoin-bitcoin/109136/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109141](/bitcoin-bitcoin/109141/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109135](/bitcoin-bitcoin/109135/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109140](/bitcoin-bitcoin/109140/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#108990](/bitcoin-bitcoin/108990/)
2018-04-11T11:57:41Z leveldb: Delete type=2 [#109196](/bitcoin-bitcoin/109196/)
2018-04-11T11:57:45Z WriteBatch memory usage: db=index, before=0.1MiB, after=0.1MiB
2018-04-11T11:57:45Z Writing final batch of 0.00 MiB
2018-04-11T11:57:45Z WriteBatch memory usage: db=chainstate, before=0.0MiB, after=0.0MiB
2018-04-11T11:57:45Z Committed 0 changed transaction outputs (out of 0) to coin database...
2018-04-11T11:57:45Z init message: Verifying blocks...
2018-04-11T11:57:45Z Verifying last 6 blocks at level 3
2018-04-11T11:57:45Z [0%]...leveldb: Moved [#118367](/bitcoin-bitcoin/118367/) to level-1 308 bytes OK: files[ 0 1 24 362 1179 0 0 ]
2018-04-11T11:57:45Z leveldb: Compacting 1@1 + 24@2 files
2018-04-11T11:57:45Z leveldb: Generated table [#118369](/bitcoin-bitcoin/118369/)@1: 1 keys, 207 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118370](/bitcoin-bitcoin/118370/)@1: 4217 keys, 210255 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118371](/bitcoin-bitcoin/118371/)@1: 4078 keys, 200014 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118372](/bitcoin-bitcoin/118372/)@1: 4087 keys, 203922 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118373](/bitcoin-bitcoin/118373/)@1: 4531 keys, 216005 bytes
2018-04-11T11:57:45Z [16%]...leveldb: Generated table [#118374](/bitcoin-bitcoin/118374/)@1: 4119 keys, 199759 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118375](/bitcoin-bitcoin/118375/)@1: 4318 keys, 216798 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118376](/bitcoin-bitcoin/118376/)@1: 98 keys, 4814 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118377](/bitcoin-bitcoin/118377/)@1: 4387 keys, 209206 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118378](/bitcoin-bitcoin/118378/)@1: 4345 keys, 211747 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118379](/bitcoin-bitcoin/118379/)@1: 4036 keys, 200497 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118380](/bitcoin-bitcoin/118380/)@1: 4146 keys, 202248 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118381](/bitcoin-bitcoin/118381/)@1: 4196 keys, 198305 bytes
2018-04-11T11:57:45Z leveldb: Generated table [#118382](/bitcoin-bitcoin/118382/)@1: 3832 keys, 190471 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118383](/bitcoin-bitcoin/118383/)@1: 4051 keys, 201361 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118384](/bitcoin-bitcoin/118384/)@1: 4464 keys, 221466 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118385](/bitcoin-bitcoin/118385/)@1: 4096 keys, 198885 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118386](/bitcoin-bitcoin/118386/)@1: 4382 keys, 217587 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118387](/bitcoin-bitcoin/118387/)@1: 4917 keys, 229917 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118388](/bitcoin-bitcoin/118388/)@1: 4300 keys, 211286 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118389](/bitcoin-bitcoin/118389/)@1: 4003 keys, 196340 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118390](/bitcoin-bitcoin/118390/)@1: 4157 keys, 206832 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118391](/bitcoin-bitcoin/118391/)@1: 4596 keys, 215723 bytes
2018-04-11T11:57:46Z leveldb: Generated table [#118392](/bitcoin-bitcoin/118392/)@1: 1466 keys, 71785 bytes
2018-04-11T11:57:46Z leveldb: Compacted 1@1 + 24@2 files => 4435430 bytes
2018-04-11T11:57:46Z leveldb: compacted to: files[ 0 0 24 362 1179 0 0 ]
2018-04-11T11:57:46Z [33%]...[50%]...[66%]...leveldb: Compacting 1@3 + 10@4 files
2018-04-11T11:57:50Z [83%]...leveldb: Generated table [#118393](/bitcoin-bitcoin/118393/)@3: 38486 keys, 2174479 bytes
2018-04-11T11:57:51Z leveldb: Generated table [#118394](/bitcoin-bitcoin/118394/)@3: 37469 keys, 2173916 bytes
2018-04-11T11:57:51Z leveldb: Generated table [#118395](/bitcoin-bitcoin/118395/)@3: 38878 keys, 2175990 bytes
2018-04-11T11:57:51Z leveldb: Generated table [#118396](/bitcoin-bitcoin/118396/)@3: 38694 keys, 2174535 bytes
2018-04-11T11:57:52Z leveldb: Generated table [#118397](/bitcoin-bitcoin/118397/)@3: 37903 keys, 2174138 bytes
2018-04-11T11:57:52Z leveldb: Generated table [#118398](/bitcoin-bitcoin/118398/)@3: 38108 keys, 2174691 bytes
2018-04-11T11:57:52Z leveldb: Generated table [#118399](/bitcoin-bitcoin/118399/)@3: 38036 keys, 2173630 bytes
2018-04-11T11:57:53Z [99%]...leveldb: Generated table [#118400](/bitcoin-bitcoin/118400/)@3: 38108 keys, 2173306 bytes
2018-04-11T11:57:53Z [DONE].
2018-04-11T11:57:53Z No coin database inconsistencies in last 7 blocks (7607 transactions)
2018-04-11T11:57:53Z block index 38962ms
2018-04-11T11:57:53Z Reading estimates: 190 buckets counting confirms up to 48 blocks
2018-04-11T11:57:53Z Reading estimates: 190 buckets counting confirms up to 12 blocks
2018-04-11T11:57:53Z Reading estimates: 190 buckets counting confirms up to 1008 blocks
2018-04-11T11:57:53Z init message: Loading wallet...
2018-04-11T11:57:53Z leveldb: Generated table [#118401](/bitcoin-bitcoin/118401/)@3: 38342 keys, 2173156 bytes
2018-04-11T11:57:54Z nFileVersion = 169900
2018-04-11T11:57:54Z Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total
2018-04-11T11:57:54Z leveldb: Generated table [#118402](/bitcoin-bitcoin/118402/)@3: 38613 keys, 2173466 bytes
2018-04-11T11:57:54Z wallet 374ms
2018-04-11T11:57:54Z setKeyPool.size() = 2000
2018-04-11T11:57:54Z mapWallet.size() = 0
2018-04-11T11:57:54Z mapAddressBook.size() = 0
Segmentation fault (core dumped)
Backtrace with gdb:
2018-04-11T12:21:29Z Loaded best chain: hashBestChain=0000000000000000003f6dfa302cd20c2721555a8004ef8c84864f64e68df67c height=517691 date=2018-04-11T11:45:00Z progress=0.999975
2018-04-11T12:21:29Z init message: Rewinding blocks...
2018-04-11T12:21:35Z init message: Verifying blocks...
2018-04-11T12:21:35Z Verifying last 6 blocks at level 3
2018-04-11T12:21:35Z [0%]...[New Thread 0xaf12d450 (LWP 57)]
[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
2018-04-11T12:21:46Z No coin database inconsistencies in last 7 blocks (7607 transactions)
2018-04-11T12:21:46Z block index 40569ms
2018-04-11T12:21:46Z init message: Loading wallet...
2018-04-11T12:21:46Z nFileVersion = 169900
2018-04-11T12:21:46Z Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total
2018-04-11T12:21:46Z wallet 351ms
2018-04-11T12:21:46Z setKeyPool.size() = 2000
2018-04-11T12:21:46Z mapWallet.size() = 0
2018-04-11T12:21:46Z mapAddressBook.size() = 0
[New Thread 0xae92c450 (LWP 58)]
2018-04-11T12:21:46Z mapBlockIndex.size() = 517694
2018-04-11T12:21:46Z nBestHeight = 517691
[New Thread 0xae12b450 (LWP 59)]
Thread 12 "bitcoin-loadblk" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xae92c450 (LWP 58)]
0xb6f81868 in pthread_setschedparam () from /usr/lib/libpthread.so.0
(gdb) backtrace
[#0](/bitcoin-bitcoin/0/) 0xb6f81868 in pthread_setschedparam () from /usr/lib/libpthread.so.0
[#1](/bitcoin-bitcoin/1/) 0x0075b29c in ScheduleBatchPriority() ()
[#2](/bitcoin-bitcoin/2/) 0x0047509c in ThreadImport(std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >) ()
[#3](/bitcoin-bitcoin/3/) 0x00494f40 in boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >), boost::_bi::list1<boost:
:_bi::value<std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> > > > > >::run() ()
[#4](/bitcoin-bitcoin/4/) 0x0082aed4 in thread_proxy ()
[#5](/bitcoin-bitcoin/5/) 0xb6f7fee4 in start_thread () from /usr/lib/libpthread.so.0
[#6](/bitcoin-bitcoin/6/) 0xb6d04328 in ?? () from /usr/lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)
