This means that the "util library" will be compiled twice?
Yes, as it stands currently at least these few files of the util library will be compiled twice.
Maybe it could make sense to split libutil out from libcommon and make libkernel depend on libutil?
Unfortunately, there are plenty of util/*.cpp that libbitcoinkernel doesn't use, and I wish to prevent any futher coupling in that regard especially since the files under util/*.cpp are largely unrelated.
Maybe it could make sense to remove consensus stuff from libnode and instead make libnode depend on libkernel?
Unfortunately bitcoin-chainstate or libbitcoinkernel requires quite a lot of .cpp files that libnode doesn't, and also vice versa (both ends of the venn diagram are non-empty). So there's no clear hierarchy that we can establish without linking in a large number of unnecessary .cpp files which I want to prevent.
The following invocation (in a shell that supports process substitution) will print all the .cpp that bitcoin-chainstate/libbitcoinkernel requires but libnode doesn't:
comm -23 <(make --no-print-directory -C src print-bitcoin_chainstate_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u) \
<(make --no-print-directory -C src print-libbitcoin_node_a_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u)
arith_uint256.cpp
bitcoin-chainstate.cpp
chainparamsbase.cpp
chainparams.cpp
clientversion.cpp
coins.cpp
compat/glibcxx_sanity.cpp
compressor.cpp
consensus/merkle.cpp
consensus/tx_check.cpp
core_read.cpp
deploymentinfo.cpp
fs.cpp
hash.cpp
init/common.cpp
key.cpp
logging.cpp
netaddress.cpp
policy/feerate.cpp
policy/policy.cpp
primitives/block.cpp
primitives/transaction.cpp
pubkey.cpp
random.cpp
randomenv.cpp
scheduler.cpp
script/interpreter.cpp
script/script.cpp
script/script_error.cpp
script/standard.cpp
support/cleanse.cpp
support/lockedpool.cpp
sync.cpp
threadinterrupt.cpp
uint256.cpp
util/asmap.cpp
util/bytevectorhash.cpp
util/getuniquepath.cpp
util/hasher.cpp
util/moneystr.cpp
util/rbf.cpp
util/serfloat.cpp
util/settings.cpp
util/strencodings.cpp
util/syscall_sandbox.cpp
util/system.cpp
util/thread.cpp
util/threadnames.cpp
util/time.cpp
util/tokenpipe.cpp
warnings.cpp
All the .cpp that libnode requires but bitcoin-chainstate/libbitcoinkernel doesn't:
comm -13 <(make --no-print-directory -C src print-bitcoin_chainstate_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u) \
<(make --no-print-directory -C src print-libbitcoin_node_a_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u)
addrdb.cpp
addrman.cpp
banman.cpp
blockencodings.cpp
httprpc.cpp
httpserver.cpp
i2p.cpp
index/txindex.cpp
init.cpp
mapport.cpp
net.cpp
net_processing.cpp
node/caches.cpp
node/coin.cpp
node/context.cpp
node/interfaces.cpp
node/miner.cpp
node/minisketchwrapper.cpp
node/psbt.cpp
node/transaction.cpp
noui.cpp
rest.cpp
rpc/blockchain.cpp
rpc/mining.cpp
rpc/misc.cpp
rpc/net.cpp
rpc/rawtransaction.cpp
rpc/server.cpp
rpc/server_util.cpp
torcontrol.cpp
txorphanage.cpp
txrequest.cpp
wallet/init.cpp
However, if we perform a similar analysis on libbitcoinconsensus.la we'll see that it compiles the crypto/*.cpp files, which can be eliminated by https://github.com/bitcoin/bitcoin/pull/24322/commits/7c2b41e9f81e52cb30efc2d90ebb4b414b97617f, after that change there's just script/bitcoinconsensus.cpp left.
Potential followup PR: Extract out a libbitcoinconensus_internal.la which doesn't include script/bitcoinconsensus.cpp that libbitcoinkernel can link against and avoid unnecessary compilation.
comm -13 <(make --no-print-directory -C src print-bitcoin_chainstate_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u) \
<(make --no-print-directory -C src print-libbitcoinconsensus_la_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u)
crypto/aes.cpp
crypto/chacha20.cpp
crypto/chacha_poly_aead.cpp
crypto/hkdf_sha256_32.cpp
crypto/hmac_sha256.cpp
crypto/hmac_sha512.cpp
crypto/muhash.cpp
crypto/poly1305.cpp
crypto/ripemd160.cpp
crypto/sha1.cpp
crypto/sha256.cpp
crypto/sha256_sse4.cpp
crypto/sha3.cpp
crypto/sha512.cpp
crypto/siphash.cpp
script/bitcoinconsensus.cpp
Appendix
For reference, here's the .cpp comparison for libcommon:
All the .cpp that bitcoin-chainstate/libbitcoinkernel requires but libcommon doesn't:
comm -23 <(make --no-print-directory -C src print-bitcoin_chainstate_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u) \
<(make --no-print-directory -C src print-libbitcoin_common_a_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u)
arith_uint256.cpp
bitcoin-chainstate.cpp
blockfilter.cpp
chain.cpp
chainparamsbase.cpp
clientversion.cpp
compat/glibcxx_sanity.cpp
consensus/merkle.cpp
consensus/tx_check.cpp
consensus/tx_verify.cpp
dbwrapper.cpp
deploymentstatus.cpp
flatfile.cpp
fs.cpp
hash.cpp
index/base.cpp
index/blockfilterindex.cpp
index/coinstatsindex.cpp
logging.cpp
node/blockstorage.cpp
node/chainstate.cpp
node/coinstats.cpp
node/ui_interface.cpp
policy/fees.cpp
policy/packages.cpp
policy/rbf.cpp
policy/settings.cpp
pow.cpp
primitives/block.cpp
primitives/transaction.cpp
pubkey.cpp
random.cpp
randomenv.cpp
script/interpreter.cpp
script/script.cpp
script/script_error.cpp
script/sigcache.cpp
shutdown.cpp
signet.cpp
support/cleanse.cpp
support/lockedpool.cpp
sync.cpp
threadinterrupt.cpp
timedata.cpp
txdb.cpp
txmempool.cpp
uint256.cpp
util/asmap.cpp
util/bytevectorhash.cpp
util/getuniquepath.cpp
util/hasher.cpp
util/moneystr.cpp
util/rbf.cpp
util/serfloat.cpp
util/settings.cpp
util/strencodings.cpp
util/syscall_sandbox.cpp
util/system.cpp
util/thread.cpp
util/threadnames.cpp
util/time.cpp
util/tokenpipe.cpp
validation.cpp
validationinterface.cpp
versionbits.cpp
All the .cpp that libcommon requires but bitcoin-chainstate/libbitcoinkernel doesn't:
comm -13 <(make --no-print-directory -C src print-bitcoin_chainstate_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u) \
<(make --no-print-directory -C src print-libbitcoin_common_a_SOURCES | cut -d= -f2 | tr ' ' '\n' | grep -v '\.h$' | sort -u)
base58.cpp
bech32.cpp
common/bloom.cpp
core_write.cpp
external_signer.cpp
key_io.cpp
merkleblock.cpp
netbase.cpp
net_permissions.cpp
net_types.cpp
outputtype.cpp
protocol.cpp
psbt.cpp
rpc/external_signer.cpp
rpc/rawtransaction_util.cpp
rpc/util.cpp
script/descriptor.cpp
script/sign.cpp
script/signingprovider.cpp
