This replaces the only use of thread_local with a mutex-protected map.
This leaks per thread, but because bitcoin’s threads are static and permanent, this is not a problem in practice. Add a note to the thread rename function to only call it for permanent threads, just in case.
Also removes associated build system functionality.
Closes #18678.
This replaces the only use of `thread_local` with a mutex-protected map.
This leaks per thread, but because bitcoin's threads are static and
permanent, this is not a problem in practice. Add a note to the thread
rename function to only call it for permanent threads, just in case.
Also removes associated build system functionality.
Closes #18678.
If this doesn’t solve the problem, then the problem isn’t thread_local but looking up thread names in the first place, and we should remove thread name logging completely (as well as thread_local).
A workaround then that would avoid the need for this bookkeeping at all would be:
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
Testing on ODROID-HC1:
0$ uname -srm
1Linux 4.14.173-173 armv7l
2$ ./autogen.sh && ./configure --disable-wallet --disable-tests --disable-bench --disable-zmq --disable-man --disable-util-tx --disable-util-wallet --without-miniupnpc --without-libs --without-gui CC=clang-9 CXX=clang++-9 && make clean && time taskset --cpu-list 4-7 make --jobs=4
UPDATE: sorry, wrong branch compiled
Here are results of three consequent tests on ODROID-HC1:
0$ uname -srm
1Linux 4.14.173-173 armv7l
2$ date; time bitcoin-cli -rpcclienttimeout=10000 gettxoutsetinfoSat May 2 23:18:12 UTC 2020
3{
4 "height": 628635,
5 "bestblock": "0000000000000000000210066becbfb2ebfdd957d3bdaec7190f46d87ce59b7a",
6 "transactions": 39898110,
7 "txouts": 66828188,
8 "bogosize": 5019712922,
9 "hash_serialized_2": "78c80aed68a14274a1dbadd9213ff6da55618d29c2b077c8f502ce5f900fcd03",
10 "disk_size": 4027535280,
11 "total_amount": 18357767.32081895
12}
13
14real 6m32.011s
15user 0m0.030s
16sys 0m0.020s
17$ date; time bitcoin-cli -rpcclienttimeout=10000 gettxoutsetinfoSat May 2 23:31:50 UTC 2020
18{
19 "height": 628635,
20 "bestblock": "0000000000000000000210066becbfb2ebfdd957d3bdaec7190f46d87ce59b7a",
21 "transactions": 39898110,
22 "txouts": 66828188,
23 "bogosize": 5019712922,
24 "hash_serialized_2": "78c80aed68a14274a1dbadd9213ff6da55618d29c2b077c8f502ce5f900fcd03",
25 "disk_size": 4035885849,
26 "total_amount": 18357767.32081895
27}
28
29real 9m48.031s
30user 0m0.004s
31sys 0m0.040s
32$ date; time bitcoin-cli -rpcclienttimeout=10000 gettxoutsetinfoSat May 2 23:44:47 UTC 2020
33{
34 "height": 628636,
35 "bestblock": "0000000000000000000e91c7a666359c09a405e3a5294181100a6aeb7d9d6427",
36 "transactions": 39898996,
37 "txouts": 66829911,
38 "bogosize": 5019841104,
39 "hash_serialized_2": "065cb1955ffc90c37ef293d8bee8e2ca9af9f9d87b0a50f8d58b4b874b642a82",
40 "disk_size": 4030980759,
41 "total_amount": 18357779.82081895
42}
43
44real 53m16.860s
45user 0m0.008s
46sys 0m0.035s
The related log:
02020-05-02T21:11:03Z [init] Bitcoin Core version v0.20.99.0-c7933899e (release build)
12020-05-02T21:11:03Z [init] InitParameterInteraction: parameter interaction: -externalip set -> setting -discover=0
22020-05-02T21:11:03Z [init] Assuming ancestors of block 0000000000000000000f2adce67e49b0b6bdeb9de8b7c3d7e93b21e7fc1e819d have valid signatures.
32020-05-02T21:11:03Z [init] Setting nMinimumChainWork=00000000000000000000000000000000000000000e1ab5ec9348e9f4b8eb8154
42020-05-02T21:11:03Z [init] Using the 'standard' SHA256 implementation
52020-05-02T21:11:03Z [init] Default data directory /home/core/.bitcoin
62020-05-02T21:11:03Z [init] Using data directory /var/lib/bitcoind
72020-05-02T21:11:03Z [init] Config file: /etc/bitcoin/bitcoin.conf
82020-05-02T21:11:03Z [init] Config file arg: debug="rpc"
92020-05-02T21:11:03Z [init] Config file arg: externalip="95.164.65.194"
102020-05-02T21:11:03Z [init] Config file arg: listenonion="0"
112020-05-02T21:11:03Z [init] Config file arg: logthreadnames="1"
122020-05-02T21:11:03Z [init] Config file arg: par="4"
132020-05-02T21:11:03Z [init] Config file arg: rpcthreads="2"
142020-05-02T21:11:03Z [init] Config file arg: rpcworkqueue="1"
15...
162020-05-02T23:18:12Z [httpworker.0] ThreadRPCServer method=gettxoutsetinfo user=__cookie__
172020-05-02T23:18:12Z [httpworker.0] FlushStateToDisk: write coins cache to disk (337567 coins, 39627kB) started
182020-05-02T23:18:14Z [httpworker.0] FlushStateToDisk: write coins cache to disk (337567 coins, 39627kB) completed (2.05s)
192020-05-02T23:31:50Z [httpworker.1] ThreadRPCServer method=gettxoutsetinfo user=__cookie__
202020-05-02T23:31:51Z [httpworker.1] FlushStateToDisk: write coins cache to disk (3522 coins, 2177kB) started
212020-05-02T23:31:51Z [httpworker.1] FlushStateToDisk: write coins cache to disk (3522 coins, 2177kB) completed (0.02s)
222020-05-02T23:40:36Z [msghand] UpdateTip: new best=0000000000000000000e91c7a666359c09a405e3a5294181100a6aeb7d9d6427 height=628636 version=0x27ffe000 log2_work=91.914443 tx=526304521 date='2020-05-02T23:40:05Z' progress=1.000000 cache=3.2MiB(13600txo) warning='67 of last 100 blocks have unexpected version'
232020-05-02T23:44:47Z [httpworker.0] ThreadRPCServer method=gettxoutsetinfo user=__cookie__
242020-05-02T23:44:48Z [httpworker.0] FlushStateToDisk: write coins cache to disk (14579 coins, 3418kB) started
252020-05-02T23:44:48Z [httpworker.0] FlushStateToDisk: write coins cache to disk (14579 coins, 3418kB) completed (0.09s)
262020-05-02T23:59:13Z [msghand] UpdateTip: new best=000000000000000000034cd30624babe78f29f0944b2e73d784198d4fd1af594 height=628637 version=0x20000000 log2_work=91.914464 tx=526307327 date='2020-05-02T23:54:25Z' progress=0.999998 cache=3.1MiB(12818txo) warning='66 of last 100 blocks have unexpected version'
272020-05-02T23:59:30Z [msghand] New outbound peer connected: version: 70015, blocks=628638, peer=1217 (full-relay)
282020-05-03T00:08:15Z [msghand] UpdateTip: new best=0000000000000000000a9c8ea50980bece5f9168e962c868e2f9d9761d5ddb0c height=628638 version=0x20800000 log2_work=91.914485 tx=526308404 date='2020-05-02T23:54:28Z' progress=0.999995 cache=4.0MiB(21407txo) warning='67 of last 100 blocks have unexpected version'
292020-05-03T00:08:15Z [net] socket sending timeout: 1382s
302020-05-03T00:14:30Z [msghand] UpdateTip: new best=00000000000000000006ceea137d6caebd533d6fd95f8c1da6be15fe01c6d828 height=628639 version=0x20c00000 log2_work=91.914506 tx=526310651 date='2020-05-03T00:08:22Z' progress=0.999998 cache=5.0MiB(30426txo) warning='67 of last 100 blocks have unexpected version'
312020-05-03T00:32:24Z [msghand] UpdateTip: new best=0000000000000000000d0ce69147e7c831a0ea999c5837667a2928960d538cce height=628640 version=0x37ffe000 log2_work=91.914527 tx=526313977 date='2020-05-03T00:31:10Z' progress=1.000000 cache=6.1MiB(41548txo) warning='67 of last 100 blocks have unexpected version'
thread_local is not the root of the performance degradation.
Please see #18538 (comment)
thread_local. And cleans up the build system a bit.
It would make thread-name logging work in the gitian-built binaries which lack
thread_local. And cleans up the build system a bit.
Correct. So keep reviewing.
I am currently syncing a node to debug this issue.
Also, glibc can probably be bumped by one minor version in the next release, which is going to switch to C++17. Then, potentially the same build cleanups can be done?