Restart causes bitcoind 0.18 to re-validate whole blockchain history #16749

I am playing around with RaspiBolt recently. Running Raspbian Buster with Raspberry Pi 3, bitcoind 0.18, LND 0.7.0-beta. It had catched up with the latest block height and everything was working, had some channels open with LND on mainnet. Then changed some bitcoin.conf settings and did sudo systemctl restart bitcoind and now bitcoind for some reason is re-validating whole blockchain from the block height 0 (which will take some month on that hardware). Bitcoin datadir is on external USB HDD, but that itself is kinda working without any issues.

Here's excerpt from debug.log output:

2019-08-28T01:12:29Z UpdateTip: new best=0000000000000000001a2136bb2f460972be231142acc07e05577dedf582232e height=592048 version=0x20000000 log2_work=91.013342 tx=449294398 date='2019-08-28T01
:12:08Z' progress=1.000000 cache=119.5MiB(1044308txo) warning='42 of last 100 blocks have unexpected version'
2019-08-28T01:16:23Z UpdateTip: new best=00000000000000000014d3bc502b10f719f2e4b664cd5c219c4342d211a3306f height=592049 version=0x20000000 log2_work=91.013368 tx=449295686 date='2019-08-28T01
:15:52Z' progress=1.000000 cache=120.6MiB(1054191txo) warning='42 of last 100 blocks have unexpected version'
2019-08-28T01:17:43Z UpdateTip: new best=000000000000000000025cdd78962fa6027f662045347be11e5a18f8fccc093c height=592050 version=0x20400000 log2_work=91.013393 tx=449296119 date='2019-08-28T01
:17:35Z' progress=1.000000 cache=120.6MiB(1054739txo) warning='42 of last 100 blocks have unexpected version'
2019-08-28T01:25:19Z tor: Thread interrupt
2019-08-28T01:25:19Z opencon thread exit
2019-08-28T01:25:19Z addcon thread exit
2019-08-28T01:25:19Z Shutdown: In progress...
2019-08-28T01:25:19Z RemoveLocal(myonionaddr.onion:8333)
2019-08-28T01:25:19Z torcontrol thread exit
2019-08-28T01:25:20Z net thread exit
2019-08-28T01:25:20Z msghand thread exit
2019-08-28T01:25:21Z scheduler thread interrupt
2019-08-28T01:25:21Z Dumped mempool: 0.020371s to copy, 0.090668s to dump
2019-08-28T01:25:45Z [default wallet] Releasing wallet
2019-08-28T01:25:45Z Shutdown: done
2019-08-28T01:26:17Z 




2019-08-28T01:26:17Z Bitcoin Core version v0.18.0 (release build)
2019-08-28T01:26:17Z Assuming ancestors of block 0000000000000000000f1c54590ee18d15ec70e68c8cd4cfbadb1b4f11697eee have valid signatures.
2019-08-28T01:26:17Z Setting nMinimumChainWork=0000000000000000000000000000000000000000051dc8b82f450202ecb3d471
2019-08-28T01:26:17Z Using the 'standard' SHA256 implementation
2019-08-28T01:26:18Z Default data directory /home/bitcoin/.bitcoin
2019-08-28T01:26:18Z Using data directory /home/bitcoin/.bitcoin
2019-08-28T01:26:18Z Config file: /home/bitcoin/.bitcoin/bitcoin.conf
2019-08-28T01:26:18Z Using at most 40 automatic connections (1024 file descriptors available)
2019-08-28T01:26:18Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
2019-08-28T01:26:18Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
2019-08-28T01:26:18Z Using 4 threads for script verification
2019-08-28T01:26:18Z scheduler thread start
2019-08-28T01:26:18Z HTTP: creating work queue of depth 16
2019-08-28T01:26:18Z Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcauth for rpcauth auth generation.
2019-08-28T01:26:18Z HTTP: starting 4 worker threads
2019-08-28T01:26:18Z Using wallet directory /home/bitcoin/.bitcoin
2019-08-28T01:26:18Z init message: Verifying wallet(s)...
2019-08-28T01:26:18Z Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
2019-08-28T01:26:18Z Using wallet /home/bitcoin/.bitcoin
2019-08-28T01:26:18Z BerkeleyEnvironment::Open: LogDir=/home/bitcoin/.bitcoin/database ErrorFile=/home/bitcoin/.bitcoin/db.log
2019-08-28T01:26:18Z init message: Loading banlist...
2019-08-28T01:26:18Z Cache configuration:
2019-08-28T01:26:18Z * Using 2.0 MiB for block index database
2019-08-28T01:26:18Z * Using 8.0 MiB for chain state database
2019-08-28T01:26:18Z * Using 90.0 MiB for in-memory UTXO set (plus up to 47.7 MiB of unused mempool space)
2019-08-28T01:26:18Z init message: Loading block index...
2019-08-28T01:26:18Z Wiping LevelDB in /home/bitcoin/.bitcoin/blocks/index
2019-08-28T01:26:19Z Opening LevelDB in /home/bitcoin/.bitcoin/blocks/index
2019-08-28T01:26:19Z Opened LevelDB successfully
2019-08-28T01:26:19Z Using obfuscation key for /home/bitcoin/.bitcoin/blocks/index: 0000000000000000
2019-08-28T01:26:19Z Initializing databases...
2019-08-28T01:26:19Z Wiping LevelDB in /home/bitcoin/.bitcoin/chainstate
2019-08-28T01:26:19Z Opening LevelDB in /home/bitcoin/.bitcoin/chainstate
2019-08-28T01:26:19Z Opened LevelDB successfully
2019-08-28T01:26:19Z Wrote new obfuscate key for /home/bitcoin/.bitcoin/chainstate: 5a1a5205288550bc
2019-08-28T01:26:19Z Using obfuscation key for /home/bitcoin/.bitcoin/chainstate: 5a1a5205288550bc
2019-08-28T01:26:19Z  block index             964ms
2019-08-28T01:26:19Z init message: Loading wallet...
2019-08-28T01:26:19Z BerkeleyEnvironment::Open: LogDir=/home/bitcoin/.bitcoin/database ErrorFile=/home/bitcoin/.bitcoin/db.log
2019-08-28T01:26:20Z [default wallet] nFileVersion = 180000
2019-08-28T01:26:20Z [default wallet] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
2019-08-28T01:26:20Z [default wallet] Wallet completed loading in             536ms
2019-08-28T01:26:20Z [default wallet] setKeyPool.size() = 2000
2019-08-28T01:26:20Z [default wallet] mapWallet.size() = 0
2019-08-28T01:26:20Z [default wallet] mapAddressBook.size() = 0
2019-08-28T01:26:20Z Reindexing block file blk00000.dat...
2019-08-28T01:26:20Z UpdateTip: new best=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f height=0 version=0x00000001 log2_work=32.000022 tx=1 date='2009-01-03T18:15:05Z' progress=0.000000 cache=0.0MiB(0txo)
...

Here's bitcoin.conf (but don't think there is anything special):

# RaspiBolt: bitcoind configuration
# /home/bitcoin/.bitcoin/bitcoin.conf

# remove the following line to enable Bitcoin mainnet
#testnet=1

#temp
reindex=1

# Bitcoind options
server=1
daemon=1
peerbloomfilters=0

# Connection settings
rpcuser=user
rpcpassword=pass

onlynet=onion
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28333

dnsseed=0
dns=0
# someothermynode
addnode=someothermynode.onion:8333
# someothermynode
addnode=someothermynode.onion:8333

# Raspberry Pi optimizations
dbcache=100
maxorphantx=10
maxmempool=50
maxconnections=40
maxuploadtarget=5000

Any ideas?

This is the problem:

#temp reindex=1

That causes bitcoind to reindex every time you restart.

Ohh, damn, by bad! Dunno why it was there in the first place and why I didn't notice it.