A bit of a followup from #30840
- In this change the documentation where we refer to the
./configurescript which is now gone and have converted the configure params to use thecmakeequivalent.
A bit of a followup from #30840
./configure script which is now gone and have converted the configure params to use the cmake equivalent.<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--006a51241073e994b41acfe9ec718e94-->
For detailed information about the code coverage, see the test coverage report.
<!--021abf342d371248e50ceaed478a90ca-->
See the guideline for information on the review process.
| Type | Reviewers |
|---|---|
| ACK | maflcko, jonatack, jarolrod, tdb3, pablomartin4btc |
| Stale ACK | davidgumberg |
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.
<!--174a7506f384e20aa4161008e828411d-->
Reviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
It would be nice to fix all remaining instances in one go. See also #30664 (comment) and #30664 (comment)
ACK dd0941cc6de6852c8ee1347c7ccfcc887cd07e4b
nit: since you are there at doc/zmq.md, I think there's a typo close to the end, look for "chain--as".
I think you are still missing:
test/lint/lint-spelling.py:FILES_ARGS = ['git', 'ls-files', '--', ":(exclude)build-aux/m4/", ":(exclude)contrib/seeds/*.txt", ":(exclude)depends/", ":(exclude)doc/release-notes/", ":(exclude)src/qt/locale/", ":(exclude)src/qt/*.qrc", ":(exclude)contrib/guix/patches"]
Also, the pull title should be updated?
22 | @@ -23,9 +23,9 @@ src/bitcoin-node -regtest -printtoconsole -debug=ipc 23 | BITCOIND=bitcoin-node test/functional/test_runner.py 24 | ``` 25 | 26 | -The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option).
A few lines above here. This
cd <BITCOIN_SOURCE_DIRECTORY>
make -C depends NO_QT=1 MULTIPROCESS=1
CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
make
src/bitcoin-node -regtest -printtoconsole -debug=ipc
BITCOIND=bitcoin-node test/functional/test_runner.py
should be replaced with:
cd <BITCOIN_SOURCE_DIRECTORY>
make -C depends NO_QT=1 MULTIPROCESS=1
cmake -B build --toolchain=depends/x86_64-pc-linux-gnu/toolchain.cmake
cmake --build build
build/src/bitcoin-node -regtest -printtoconsole -debug=ipc
BITCOIND=build/src/bitcoin-node build/test/functional/test_runner.py
Although, it also looks like that is currently broken:
BITCOIND=build/src/bitcoin-node build/test/functional/test_runner.py
Temporary test directory at /tmp/test_runner_₿_🏃_20240913_111920
2024-09-13T11:19:20.313000Z TestFramework (INFO): PRNG seed is: 4821438991961925939
2024-09-13T11:19:20.315000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20240913_111920/cache
2024-09-13T11:19:20.316000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 131, in main
self.setup()
File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 301, in setup
self.setup_chain()
File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 392, in setup_chain
self._initialize_chain()
File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 853, in _initialize_chain
self.start_node(CACHE_NODE_ID)
File "/root/ci_scratch/test/functional/test_framework/test_framework.py", line 553, in start_node
node.start(*args, **kwargs)
File "/root/ci_scratch/test/functional/test_framework/test_node.py", line 256, in start
self.process = subprocess.Popen(self.args + extra_args, env=subp_env, stdout=stdout, stderr=stderr, cwd=cwd, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/subprocess.py", line 1026, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib/python3.12/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'build/src/bitcoin-node'
2024-09-13T11:19:20.367000Z TestFramework (INFO): Stopping nodes
2024-09-13T11:19:20.367000Z TestFramework (WARNING): Not cleaning up dir /tmp/test_runner_₿_🏃_20240913_111920/cache
2024-09-13T11:19:20.367000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/test_runner_₿_🏃_20240913_111920/cache/test_framework.log
2024-09-13T11:19:20.367000Z TestFramework (ERROR):
2024-09-13T11:19:20.368000Z TestFramework (ERROR): Hint: Call /root/ci_scratch/test/functional/combine_logs.py '/tmp/test_runner_₿_🏃_20240913_111920/cache' to consolidate all logs
2024-09-13T11:19:20.368000Z TestFramework (ERROR):
2024-09-13T11:19:20.368000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-09-13T11:19:20.368000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-09-13T11:19:20.368000Z TestFramework (ERROR):
Traceback (most recent call last):
File "/root/ci_scratch/build/test/functional/test_runner.py", line 950, in <module>
main()
File "/root/ci_scratch/build/test/functional/test_runner.py", line 571, in main
run_tests(
File "/root/ci_scratch/build/test/functional/test_runner.py", line 624, in run_tests
subprocess.check_output([sys.executable, tests_dir + 'create_cache.py'] + flags + ["--tmpdir=%s/cache" % tmpdir])
File "/usr/lib/python3.12/subprocess.py", line 466, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/bin/python3', '/root/ci_scratch/build/test/functional/create_cache.py', '--cachedir=/root/ci_scratch/build/test/cache', '--configfile=/root/ci_scratch/build/test/functional/../config.ini', '--tmpdir=/tmp/test_runner_₿_🏃_20240913_111920/cache']' returned non-zero exit status 1.
thanks for the feedback!
I updated this to
cd <BITCOIN_SOURCE_DIRECTORY>
make -C depends NO_QT=1 MULTIPROCESS=1
cmake -B build --toolchain=depends/x86_64-pc-linux-gnu/toolchain.cmake
cmake --build build
build/src/bitcoind -regtest -printtoconsole -debug=ipc
BITCOIND=$(pwd)/build/src/bitcoind build/test/functional/test_runner.py
Which does start running the tests please let me know if this looks ok
Wouldn't we want to keep bitcoin-node (part of multiprocess) rather than change to bitcoind? Am I missing something?
Thoughts @ryanofsky ?
re: #30875 (review)
Thanks for the ping! Yes, this should say bitcoin-node not bitcoind to test the multiprocess binary. Either command line will work, but bitcoin-node will make the test more meaningful, especially with changes from #10102 merged, because bitcoind will run tests with the node and wallet in the same process, while bitcoin-node will spawn separate bitcoin-wallet processes
I went ahead and changed it back from bitcoind to bitcoin-node in dc2e223
1062 | @@ -1063,7 +1063,7 @@ bool Chainstate::PreciousBlock(BlockValidationState& state, CBlockIndex* pindex)
1063 |
1064 | - Build and run tests with `-DDEBUG_LOCKORDER` to verify that no potential
1065 | deadlocks are introduced. As of 0.12, this is defined by default when
nit: IMHO, could get rid of the "As of 0.12" part, since we're using CMake now and and 0.12 didn't use it.
thanks! I removed that in 5a85022
22 | @@ -23,9 +23,9 @@ src/bitcoin-node -regtest -printtoconsole -debug=ipc 23 | BITCOIND=bitcoin-node test/functional/test_runner.py 24 | ``` 25 | 26 | -The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option). 27 | +The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `-DWITH_MULTIPROCESS=ON` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option).
nit: "configure script" is an Autotools thing, right?
thanks! I updated this to cmake build instead of configure script in 5a85022
let me know if you'd prefer different wording!
22 | @@ -23,9 +23,9 @@ src/bitcoin-node -regtest -printtoconsole -debug=ipc 23 | BITCOIND=bitcoin-node test/functional/test_runner.py 24 | ``` 25 | 26 | -The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `--enable-multiprocess` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option). 27 | +The configure script will pick up settings and library locations from the depends directory, so there is no need to pass `-DWITH_MULTIPROCESS=ON` as a separate flag when using the depends system (it's controlled by the `MULTIPROCESS=1` option). 28 | 29 | -Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `./configure --enable-multiprocess` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess/blob/master/doc/install.md) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general. 30 | +Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) packages on your system, and just run `cmake -B build -DWITH_MULTIPROCESS=ON` without using the depends system. The configure script will be able to locate the installed packages via [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/). See [Installation](https://github.com/chaincodelabs/libmultiprocess/blob/master/doc/install.md) section of the libmultiprocess readme for install steps. See [build-unix.md](build-unix.md) and [build-osx.md](build-osx.md) for information about installing dependencies in general.
Same here ("configure script")
thanks! I updated this to cmake build instead of configure script in 5a85022
let me know if you'd prefer different wording!
Approach ACK
Thanks for the doc cleanup.
It would be nice to fix all remaining instances in one go. See also #30664 (comment) and #30664 (comment)
There are still some changes I need to address from these comments
Also in test_deterministic_coverage.sh: Run \"./configure --enable-lcov\"
3 | @@ -4,7 +4,7 @@ _This document describes usage of the multiprocess feature. For design informati 4 | 5 | ## Build Option 6 | 7 | -On unix systems, the `--enable-multiprocess` build option can be passed to `./configure` to build new `bitcoin-node`, `bitcoin-wallet`, and `bitcoin-gui` executables alongside existing `bitcoind` and `bitcoin-qt` executables. 8 | +On unix systems, the `-DWITH_MULTIPROCESS=ON` build option can be passed to `cmake -B build` to build new `bitcoin-node`, `bitcoin-wallet`, and `bitcoin-gui` executables alongside existing `bitcoind` and `bitcoin-qt` executables.
On Unix systems, the `-DWITH_MULTIPROCESS=ON` build option can be passed to build the supplemental `bitcoin-node` and `bitcoin-gui` multiprocess executables.
thank you! Updated in 4f72eca
@ryanofsky bitcoin-wallet is built, if enabled, independently of the multiprocess flag (if I'm not confused); should doc/multiprocess.md describe this differently?
Configure summary
=================
Executables:
bitcoind ............................ ON
bitcoin-node (multiprocess) ......... OFF
bitcoin-qt (GUI) .................... ON
bitcoin-gui (GUI, multiprocess) ..... OFF
bitcoin-cli ......................... ON
bitcoin-tx .......................... ON
bitcoin-util ........................ ON
bitcoin-wallet ...................... ON
bitcoin-chainstate (experimental) ... ON
libbitcoinkernel (experimental) ..... ON
@ryanofsky
bitcoin-walletis built, if enabled, independently of the multiprocess flag (if I'm not confused); shoulddoc/multiprocess.mddescribe this differently?
Probably previous description was a little confusing but current version c3d68e11191e036e154916f27f21200d2bf16756 which doesn't mention bitcoin-wallet looks good to me.
This readme was written as part of #10102 before bitcoin-wallet tool existed, so --enable-multiprocess option really did build a new binary that didn't exist before. But now #10102 just adds multiprocess functionality to the existing binary. In the master branch currently though multiproces option has no effect on bitcoin-wallet so it is best not to mention it.
That confirms what I thought when suggesting #30875 (review). Thanks @ryanofsky for the quick reply.
162 | @@ -163,7 +163,7 @@ Note that for `*block` topics, when the block chain tip changes, 163 | a reorganisation may occur and just the tip will be notified. 164 | It is up to the subscriber to retrieve the chain from the last known 165 | block to the new tip. Also note that no notification will occur if the tip 166 | -was in the active chain--as would be the case after calling invalidateblock RPC. 167 | +was in the active chain, as would be the case after calling invalidateblock RPC.
was in the active chain, as would be the case after calling the `invalidateblock` RPC.
thank you! Updated in 4f72eca
98 | @@ -99,7 +99,7 @@ sudo apt-get install qtcreator 99 | #### Setup Qt Creator 100 | 101 | 1. Make sure you've installed all dependencies specified in your systems build instructions 102 | -2. Follow the compile instructions for your system, run `./configure` with the `--enable-debug` flag 103 | +2. Follow the compile instructions for your system, run `cmake -B build` with the `-DCMAKE_BUILD_TYPE=Debug` flag
2. Follow the compile instructions for your system, adding the `-DCMAKE_BUILD_TYPE=Debug` build flag
thank you! Updated in 4f72eca
487 | @@ -488,7 +488,7 @@ def main(): 488 | 489 | if not enable_bitcoind: 490 | print("No functional tests to run.") 491 | - print("Rerun ./configure with --with-daemon and then make") 492 | + print("Rerun cmake -B build with -DBUILD_DAEMON=ON and then make")
print("Re-compile with the -DBUILD_DAEMON=ON build option")
thank you! Updated in 4f72eca
Concept ACK
Since you are here, you could update the translate instructions as well (I've tested them), if @hebasto agrees with the changes:
doc/translation_process.md
@@ -18,8 +18,8 @@ We use automated scripts to help extract translations in both Qt, and non-Qt sou
To automatically regenerate the `bitcoin_en.ts` file, run the following commands:
```sh
-cd src/
-make translate
+cmake -B build -DWITH_BDB=ON -DBUILD_GUI=ON
+cmake --build build --target translate
**Example Qt translation**
doc/translation_strings_policy.md
@@ -97,4 +97,4 @@ The second example reduces the number of pluralized words that translators have
During a string freeze (often before a major release), no translation strings are to be added, modified or removed.
-This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged.
+This can be checked by building `translate` package with `cmake` ([instructions](/doc/translate_process.md)), then verifying that `bitcoin_en.ts` remains unchanged.
Also in
test_deterministic_coverage.sh:Run \"./configure --enable-lcov\"
thank you! Updated in 4f72eca
Since you are here, you could update the translate instructions as well (I've tested them), if @hebasto agrees with the changes:
* `doc/translation_process.md` ```diff @@ -18,8 +18,8 @@ We use automated scripts to help extract translations in both Qt, and non-Qt sou To automatically regenerate the `bitcoin_en.ts` file, run the following commands: ```sh -cd src/ -make translate +cmake -B build -DWITH_BDB=ON -DBUILD_GUI=ON +cmake --build build --target translate **Example Qt translation** ``` * `doc/translation_strings_policy.md` ```diff @@ -97,4 +97,4 @@ The second example reduces the number of pluralized words that translators have During a string freeze (often before a major release), no translation strings are to be added, modified or removed. -This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged. +This can be checked by building `translate` package with `cmake` ([instructions](/doc/translate_process.md)), then verifying that `bitcoin_en.ts` remains unchanged. ```
thank you! Updated in 4f72eca @hebasto if you disagree I can change them back!
From these two comments these are the two that have not been addressed yet in this PR #30664 (comment) #30664 (comment)
test/get_previous_releases.py: './configure {}'.format(config_flags),
contrib/devtools/check-deps.sh:# Output makefile targets, converting library .a paths to libtool .la targets
<!--85328a0da195eb286784d51f73fa0af9-->
🚧 At least one of the CI tasks failed. <sub>Debug: https://github.com/bitcoin/bitcoin/runs/30149215873</sub>
<details><summary>Hints</summary>
Make sure to run all tests locally, according to the documentation.
The failure may happen due to a number of reasons, for example:
Possibly due to a silent merge conflict (the changes in this pull request being incompatible with the current code in the target branch). If so, make sure to rebase on the latest commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the affected test.
An intermittent issue.
Leave a comment here, if you need help tracking down a confusing failure.
</details>
151 | @@ -152,8 +152,7 @@ Most autotools projects can be properly staged using: 152 | 153 | ## Build outputs: 154 | 155 | -In general, the output of a depends package should not contain any libtool
This line can remain: #30902 (review).
428 | @@ -429,9 +429,9 @@ Defining `DEBUG_LOCKCONTENTION` adds a "lock" logging category to the logging 429 | RPC that, when enabled, logs the location and duration of each lock contention 430 | to the `debug.log` file. 431 | 432 | -The `--enable-debug` configure option adds `-DDEBUG_LOCKCONTENTION` to the 433 | +The `-DCMAKE_BUILD_TYPE=Debug` build option adds `-DDEBUG_LOCKCONTENTION` to the 434 | compiler flags. You may also enable it manually for a non-debug build by running 435 | -configure with `-DDEBUG_LOCKCONTENTION` added to your CPPFLAGS, 436 | +build with `-DDEBUG_LOCKCONTENTION` added to your CPPFLAGS,
s/for a non-debug build by running build/by building/
thank you! Addressed this in bc4a929
92 | @@ -97,4 +93,4 @@ The second example reduces the number of pluralized words that translators have 93 | 94 | During a string freeze (often before a major release), no translation strings are to be added, modified or removed. 95 | 96 | -This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged. 97 | +This can be checked by building `translate` package with `cmake` ([instructions](translation_process.md)), then verifying that `bitcoin_en.ts` remains unchanged.
This can be checked by building the `translate` package with `cmake` ([instructions](translation_process.md)), then verifying that `bitcoin_en.ts` remains unchanged.
(nano-nit, "cmake" is written without the backticks in other documents; could be consistent, also regarding referring to it as "cmake" or "CMake", but feel free to ignore)
thank you! Addressed this in bc4a929
I will take a look at the backtick nit, to see if I can find already existing instances of cmake with no backticks
487 | @@ -488,7 +488,7 @@ def main(): 488 | 489 | if not enable_bitcoind: 490 | print("No functional tests to run.") 491 | - print("Rerun ./configure with --with-daemon and then make") 492 | + print("Re-compile with the -DBUILD_DAEMON=ON build option")
print("Re-compile with the -DBUILD_DAEMON=ON build option")
thank you! Addressed this in bc4a929
Concept ACK
Nit: could also fix
--- a/test/util/test_runner.py
+++ b/test/util/test_runner.py
@@ -5,7 +5,7 @@
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test framework for bitcoin utils.
-Runs automatically during `make check`.
+Runs automatically during `ctest --test-dir build/`.
Can also be run manually."""
Concept ACK
Nit: could also fix
--- a/test/util/test_runner.py +++ b/test/util/test_runner.py @@ -5,7 +5,7 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. """Test framework for bitcoin utils. -Runs automatically during `make check`. +Runs automatically during `ctest --test-dir build/`. Can also be run manually."""
thank you! Addressed this in bc4a929
ACK https://github.com/bitcoin/bitcoin/commit/bc4a929cd716cd2b412c70754749d4fda0ca2a10
pico-nit: your branch messes up the 80 column wrapping in a few spots in doc/developer-notes.md. This is not really consistent through the docs anyways, so I don't mind personally.
CI failure looks unrelated (like an old RPC shutdown bug in 0.16.3). https://cirrus-ci.com/task/6654771535282176?logs=ci#L2818
17 | @@ -18,8 +18,8 @@ We use automated scripts to help extract translations in both Qt, and non-Qt sou 18 | 19 | To automatically regenerate the `bitcoin_en.ts` file, run the following commands: 20 | ```sh 21 | -cd src/ 22 | -make translate 23 | +cmake -B build -DWITH_BDB=ON -DBUILD_GUI=ON
This should use the preset dev-mode. Otherwise, deps may be missed.
thanks! Updated in a9964c0
92 | @@ -97,4 +93,4 @@ The second example reduces the number of pluralized words that translators have 93 | 94 | During a string freeze (often before a major release), no translation strings are to be added, modified or removed. 95 | 96 | -This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged. 97 | +This can be checked by building the `translate` package with `cmake` ([instructions](translation_process.md)), then verifying that `bitcoin_en.ts` remains unchanged.
This can be checked by building the `translate` target with `cmake` ([instructions](translation_process.md)), then verifying that `bitcoin_en.ts` remains unchanged.
thanks! Updated in a9964c0
21 | @@ -22,7 +22,7 @@ 22 | #include <utility> 23 | 24 | #ifndef WIN32 25 | -// for posix_fallocate, in configure.ac we check if it is present after this 26 | +// for posix_fallocate, in cmake -B build we check if it is present after this
// for posix_fallocate, in cmake/introspection.cmake we check if it is present after this
thanks! Updated in a9964c0
review ACK bc4a929cd716cd2b412c70754749d4fda0ca2a10
replaced --enable-debug with -DCMAKE_BUILD_TYPE=Debug in developer-notes
replaced --enable-multiprocess with -DWITH_MULTIPROCESS=ON
replaced --disable-zmq with -DWITH_ZMQ=OFF
ACK bc4a929
pico-nit: your branch messes up the 80 column wrapping in a few spots in
doc/developer-notes.md. This is not really consistent through the docs anyways, so I don't mind personally.
thanks for the suggestion! But I think I'll leave those for now since they don't relate to the autotools to cmake update too much and I would prefer to keep the PR more focused on those changes
ACK a9964c04447745435747d9cc557165c43902783b
utACK a9964c04447745435747d9cc557165c43902783b
ACK a9964c04447745435747d9cc557165c43902783b
Tried to find other inconsistencies, but couldn't.
ACK a9964c04447745435747d9cc557165c43902783b
re-ACK a9964c04447745435747d9cc557165c43902783b