unexpected behavior when specify datadir in config file #15745

issue sumary

I specificed datadir in conf file and run bitcoind with -conf option, but data is writing to default ~/.bitcoin dir , not the dir I want

steps to reproduce

bitcoind version 0.17.1

root@MNG-BC ➜  .bitcoin bitcoind --version
Bitcoin Core Daemon version v0.17.1.0-gef70f9b52b851c7997a9f1a0834714e3eebc1fd8

step1: configrations

bitcoind conf file path /root/.bitcoin/bitcoin.conf datadir for testnet : datadir=/mnt/data/bitcoin

# [core]
# Specify the location of the configuration file. To use non-default location, create a default location config file containing this setting.
conf=/root/.bitcoin/bitcoin.conf

# [debug]
# Enable debug logging for all categories.
#debug=1
# Log IP Addresses in debug output.
#logips=1
# Run this node on the Bitcoin Test Network.
#testnet=1

# [rpc]
# Accept command line and JSON-RPC commands.
server=1
# Accept public REST requests.
rest=1
# RPC user
rpcuser=rpcuser
# RPC password
rpcpassword=rpcpass
# RPC allow ip (allow all)
rpcallowip=0.0.0.0/0
# RPC bind
#rpcbind=127.0.0.1
# get all transaction
txindex=1


# [Sections]
# Most options automatically apply to mainnet, testnet, and regtest networks.
# If you want to confine an option to just one network, you should add it in the relevant section.
# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
# only apply to mainnet unless they appear in the appropriate section below.

# Options only for mainnet
[main]
# datadir
datadir=/mnt/data/bitcoin/mainnet
# Listen for incoming connections on non-default port.(default: 8333 or testnet: 18333)
port=8333
# RPC port(default:8332 or testnet:18332)
rpcport=8332

# Options only for testnet
[test]
# datadir
datadir=/mnt/data/bitcoin
# Listen for incoming connections on non-default port.(default: 8333 or testnet: 18333)
port=18333
# RPC port(default:8332 or testnet:18332)
rpcport=18332

# Options only for regtest
[regtest]

step2 : run bitcoind

run bitcoind with below command

bitcoind -conf=/root/.bitcoin/bitcoin.conf -testnet=1

step3 : check the data

expected all the data write to /mnt/data/bitcoin/testnet3
but actually block data is been wroted to /root/.bitcoin/

drwx------  2 root root 4096 Apr  4 16:51 blocks
drwx------  3 root root 4096 Apr  4 16:51 testnet3

more info about the contents in /root/.bitcoin

root@MNG-BC ➜  .bitcoin du --max-depth=1 -h
1.1G    ./testnet3
4.0K    ./blocks
1.1G    .
root@MNG-BC ➜  .bitcoin cd testnet3
root@MNG-BC ➜  testnet3 ls
blocks
root@MNG-BC ➜  testnet3 cd blocks
root@MNG-BC ➜  blocks ls
blk00000.dat  blk00001.dat  blk00002.dat  blk00003.dat  blk00004.dat  blk00005.dat  blk00006.dat  blk00007.dat  index  rev00000.dat  rev00001.dat  rev00002.dat  rev00003.dat  rev00004.dat  rev00005.dat  rev00006.dat  rev00007.dat

question

Why datadir is not taken effect in conf? Is it a bug or my conf is wrong? Should I specific -datadir option in the command line?

@huahuayu Please don't tag multiple people just to draw attention to your issue.

There is a circular relationship of sorts between the config file and the data dir. The configuration file can specify a data directory (using -datadir), while the default configuration file bitcoin.conf is in the data directory.

This potential circularity is broken by ignoring -datadir in the default configuration file, so when -conf is driven by the data directory.

However, when -conf is specified on the command line it should be possible to set a -datadir in that configuration file and have it be used. The purpose of this is to be able to have the configuration file in a read-only directory, e.g. /etc/bitcoin.conf, specifying where the read/write data is.

I've tested this and with master this seems to work as expected:

$ mkdir /tmp/conf /tmp/datadir
$ echo "datadir=/tmp/datadir" > /tmp/conf/test1.conf
$ src/bitcoind --conf=/tmp/conf/test1.conf -printtoconsole
2019-04-04T11:00:28Z Bitcoin Core version v0.18.99.0-79c345a0114c9a83fd40e01150519373c017b130 (release build)
2019-04-04T11:00:28Z Default data directory /home/.../.bitcoin
2019-04-04T11:00:28Z Using data directory /tmp/datadir
2019-04-04T11:00:28Z Config file: /tmp/conf/test1.conf
...

Same for 0.17.1. Cannot reproduct this.

$ mkdir /tmp/conf /tmp/datadir
$ echo "datadir=/tmp/datadir" > /tmp/conf/test1.conf
$ src/bitcoind --conf=/tmp/conf/test1.conf -printtoconsole
2019-04-04T11:34:53Z Bitcoin Core version v0.17.1.0-6cf81b01b48e871a35072dd08e7735c3ba42528c (release build)
2019-04-04T11:34:53Z InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2019-04-04T11:34:53Z Assuming ancestors of block 0000000000000000002e63058c023a9a1de233554f28c7b21380b6c9003f36a8 have valid signatures.
2019-04-04T11:34:53Z Setting nMinimumChainWork=0000000000000000000000000000000000000000028822fef1c230963535a90d
2019-04-04T11:34:53Z Using the 'standard' SHA256 implementation
2019-04-04T11:34:53Z Default data directory /home/.../.bitcoin
2019-04-04T11:34:53Z Using data directory /tmp/datadir
2019-04-04T11:34:53Z Using config file /tmp/conf/test1.conf

hi @laanwj , sorry for the late reply, I tried again, if you followed my steps you will definitely reproduce the problem. Please not only to see the log but check the dirs actually created.

Env info

system info

$ uname -a
Linux MNG-BC 4.4.0-142-generic [#168](/bitcoin-bitcoin/168/)-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

bitcoind version

$ bitcoind --version
Bitcoin Core Daemon version v0.17.1.0-gef70f9b52b851c7997a9f1a0834714e3eebc1fd8

Steps to reproduce

step1

vi ~/.bitcoind/bitcoin.conf add below contents (change line 44/53 to your datadir)

# This config should be placed in following path:
# /root/.bitcoin/bitcoin.conf

# [core]
# Specify the location of the configuration file. To use non-default location, create a default location config file containing this setting.
conf=/root/.bitcoin/bitcoin.conf

# [debug]
# Enable debug logging for all categories.
#debug=1
# Log IP Addresses in debug output.
#logips=1
# Run this node on the Bitcoin Test Network.
#testnet=1

# [rpc]
# Accept command line and JSON-RPC commands.
server=1
# Accept public REST requests.
rest=1
# RPC user
rpcuser=rpcuser
# RPC password
rpcpassword=rpcpass
# RPC allow ip (allow all)
rpcallowip=0.0.0.0/0
# RPC bind
#rpcbind=127.0.0.1
# get all transaction
txindex=1


# [Sections]
# Most options automatically apply to mainnet, testnet, and regtest networks.
# If you want to confine an option to just one network, you should add it in the relevant section.
# EXCEPTIONS: The options addnode, connect, port, bind, rpcport, rpcbind and wallet
# only apply to mainnet unless they appear in the appropriate section below.

# Options only for mainnet
[main]
# datadir
datadir=/mnt/data/bitcoin/mainnet
# Listen for incoming connections on non-default port.(default: 8333 or testnet: 18333)
port=8333
# RPC port(default:8332 or testnet:18332)
rpcport=8332

# Options only for testnet
[test]
# datadir
datadir=/mnt/data/bitcoin
# Listen for incoming connections on non-default port.(default: 8333 or testnet: 18333)
port=18333
# RPC port(default:8332 or testnet:18332)
rpcport=18332

# Options only for regtest
[regtest]

step2

run bitcoind with below command

bitcoind -conf=/root/.bitcoin/bitcoin.conf -testnet=1 -daemon

step3

1. check the dir /root/.bitcoin/ it will have two dirs created: blocks and testnet3 which is not expected
2. check the datadir you specified (for me it's /mnt/data/bitcoin/) , it will have a dir created: testnet3, the files inside are as below

root@MNG-BC ➜  testnet3 ls
banlist.dat  chainstate  debug.log  fee_estimates.dat  indexes  mempool.dat  peers.dat  wallets

Problem summary

After bitcoind run, data write to two different dirs

1. wallet and some other files write to the expected datadir which is good
2. but block data write to /root/.bitcoin/ is not expected, it should in datadir too. It runs out of my disk space in system volume.

@huahuayu does #15864 fix this issue?

Should we just check the special case -conf is specified and avoid GetDefaultDataDir in that case?

@hebasto let me check

@huahuayu Did #15864 fix your issue?