build: Fix `make apk` for Android w/ non-default SOURCES

hebasto commented at 12:21 pm on July 26, 2021: member

On master (1488f55fa57a1400a57be837b574183f019c7855) if depends are built with the customized SOURCES_PATH variable, the following make apk fails for Android host (see bitcoin/bitcoin#22522).

Fixes bitcoin/bitcoin#22522.

hebasto added the label Build system on Jul 26, 2021

hebasto added the label Android on Jul 26, 2021

DrahtBot commented at 1:13 pm on July 26, 2021: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#23611 (build: add LTO option to depends (no Qt) by fanquake)
#22552 (build: Improve depends build system robustness by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

hebasto marked this as a draft on Jul 26, 2021

achow101 commented at 6:57 pm on July 26, 2021: member

Seems to work.

hebasto force-pushed on Jul 26, 2021

hebasto marked this as ready for review on Jul 26, 2021

hebasto commented at 7:56 pm on July 26, 2021: member

This PR is based on #22552, and only the last two commits belong to this one.

The dependency on #22552 could be eliminated by replacing $(qt_file_name) with a hardcoded value, but it will definitely bad for maintainability.

icota commented at 8:04 am on July 31, 2021: contributor

I can confirm this fixes my Android build issues. I haven’t yet looked at the code.

MarcoFalke added the label DrahtBot Guix build requested on Aug 23, 2021

DrahtBot removed the label DrahtBot Guix build requested on Aug 25, 2021

DrahtBot added the label Needs rebase on Sep 2, 2021

hebasto force-pushed on Sep 2, 2021

hebasto commented at 8:57 am on September 2, 2021: member

Rebased 886e8048436d0e6f6f0484ccf54bd7ec61fdc2e3 -> 5005c8e912b561b3a572b19bdd9e35ad98b928f2 (pr22555.02 -> pr22555.03) due to the conflict with #22840.

DrahtBot removed the label Needs rebase on Sep 2, 2021

DrahtBot added the label Needs rebase on Nov 13, 2021

hebasto force-pushed on Nov 15, 2021

hebasto commented at 6:27 pm on November 15, 2021: member

Rebased 5005c8e912b561b3a572b19bdd9e35ad98b928f2 -> 57aeb32d888542fba261138ae378ae9e0062c12c (pr22555.03 -> pr22555.04) due to the conflicts with #23473 and #23494.

DrahtBot removed the label Needs rebase on Nov 15, 2021

DrahtBot added the label Needs rebase on Nov 21, 2021

hebasto force-pushed on Nov 21, 2021

hebasto commented at 5:58 pm on November 21, 2021: member

Rebased 57aeb32d888542fba261138ae378ae9e0062c12c -> a5b568ec74528d0ea94b1ee0392d37e338c97155 (pr22555.04 -> pr22555.05) due to the conflict with #23535.

DrahtBot removed the label Needs rebase on Nov 21, 2021

DrahtBot added the label Needs rebase on Nov 25, 2021

hebasto force-pushed on Nov 28, 2021

hebasto commented at 9:10 am on November 28, 2021: member

Rebased a5b568ec74528d0ea94b1ee0392d37e338c97155 -> 03037862fc749685ba2e60c3194d634e16513a54 (pr22555.05 -> pr22555.06) due to the conflict with #23580.

DrahtBot removed the label Needs rebase on Nov 28, 2021

DrahtBot added the label Needs rebase on Nov 30, 2021

hebasto force-pushed on Nov 30, 2021

hebasto commented at 1:36 pm on November 30, 2021: member

Rebased 03037862fc749685ba2e60c3194d634e16513a54 -> df63d6a5f95da9b70992dcea0baa60f18b852667 (pr22555.06 -> pr22555.07) due to the conflict with #23618.

DrahtBot removed the label Needs rebase on Nov 30, 2021

DrahtBot added the label Needs rebase on Dec 3, 2021

hebasto force-pushed on Dec 4, 2021

hebasto commented at 6:05 pm on December 4, 2021: member

Rebased df63d6a5f95da9b70992dcea0baa60f18b852667 -> 5b36a11e7ce731bed8d510ef1428687a0b7a0551 (pr22555.07 -> pr22555.08) due to the conflict with #23489.

DrahtBot removed the label Needs rebase on Dec 4, 2021

bitcoin deleted a comment on Dec 5, 2021

MarcoFalke added the label DrahtBot Guix build requested on Dec 5, 2021

DrahtBot added the label Needs rebase on Dec 9, 2021

fanquake referenced this in commit 65b49f60a4 on Dec 10, 2021

hebasto force-pushed on Dec 10, 2021

hebasto commented at 4:09 pm on December 10, 2021: member

Rebased 5b36a11e7ce731bed8d510ef1428687a0b7a0551 -> 72005867a4ae6e80c361c8c4f3f800b9854c3791 (pr22555.08 -> pr22555.09) due to the conflicts with #23495 and #23673.

DrahtBot removed the label Needs rebase on Dec 10, 2021

sidhujag referenced this in commit 4890778588 on Dec 10, 2021

DrahtBot commented at 6:45 pm on December 14, 2021: contributor

Guix builds

File	commit ac92ab6da58e34993d0641b98eef5b5f55b6cbf9(master)	commit e2337127ba7b5059f2c4e59c975db7cb8a18ea32(master and this pull)
SHA256SUMS.part	`ad612c55e8552e5f...`	`0970538a2e6adcfd...`
*-aarch64-linux-gnu-debug.tar.gz	`dfb9f712e593e855...`	`a3fe0ab31ced0f10...`
*-aarch64-linux-gnu.tar.gz	`81f327b7dccdf9f9...`	`6d2ea22fe3d8422e...`
*-arm-linux-gnueabihf-debug.tar.gz	`5178fa7d71f4cd9c...`	`1bb91053a4f00324...`
*-arm-linux-gnueabihf.tar.gz	`767752a777572495...`	`f052966f1835d847...`
*-osx-unsigned.dmg	`c56ddd5318f66749...`	`56f0690961a9c96d...`
*-osx-unsigned.tar.gz	`18773e80b233ede4...`	`cc06fe5a870f8ff5...`
*-osx64.tar.gz	`5cd526b9b35ab65f...`	`8d9cd56163801d16...`
*-powerpc64-linux-gnu-debug.tar.gz	`1c7a141264ccff9f...`	`e8833c82f409041b...`
*-powerpc64-linux-gnu.tar.gz	`8f89a1546bcc563e...`	`bf47f36027f2b3c0...`
*-powerpc64le-linux-gnu-debug.tar.gz	`546d57375ca436ca...`	`129ba27863024d7a...`
*-powerpc64le-linux-gnu.tar.gz	`f2d44eab7a7facaf...`	`9086cece1d5bab52...`
*-riscv64-linux-gnu-debug.tar.gz	`798b97d2b288d656...`	`d6e31341924439a2...`
*-riscv64-linux-gnu.tar.gz	`98875a06de3c1084...`	`77f46bfa6479294b...`
*-win-unsigned.tar.gz	`47c022024636527f...`	`37d30dd3ce68dee2...`
*-win64-debug.zip	`8444e4a1f462c9e7...`	`cb70a995441d341a...`
*-win64-setup-unsigned.exe	`e34caaf8b5be1a1a...`	`8c5657337459eb73...`
*-win64.zip	`dfade3b4d3688271...`	`f4d690b5b3a152da...`
*-x86_64-linux-gnu-debug.tar.gz	`462ff2a6d604fcf7...`	`227e77664062c9f9...`
*-x86_64-linux-gnu.tar.gz	`11d7ef269ae360d7...`	`0baaad3598da6fd5...`
*.tar.gz	`779113ea1d92a4eb...`	`0dd0babac7cb98f2...`
guix_build.log	`00942c2f5f0e05b1...`	`b49a0176fb0affaf...`
guix_build.log.diff		`9921eed3b3d1ec7d...`

DrahtBot removed the label DrahtBot Guix build requested on Dec 14, 2021

DrahtBot added the label Needs rebase on Dec 15, 2021

RandyMcMillan referenced this in commit 7f64ef8c81 on Dec 23, 2021

hebasto force-pushed on Dec 29, 2021

hebasto commented at 9:08 pm on December 29, 2021: member

Rebased 72005867a4ae6e80c361c8c4f3f800b9854c3791 -> 4a8fec4fde2c57e7be210ccc71a8c185ef035d56 (pr22555.09 -> pr22555.10) due to the conflict with #23744.

DrahtBot removed the label Needs rebase on Dec 29, 2021

DrahtBot added the label Needs rebase on Jan 10, 2022

hebasto force-pushed on Jan 10, 2022

hebasto commented at 1:10 pm on January 10, 2022: member

Rebased 4a8fec4fde2c57e7be210ccc71a8c185ef035d56 -> 580efc56ae17dc8523af2ae18a593ed5cb708c05 (pr22555.10 -> pr22555.11) due to the conflict with #23724.

DrahtBot removed the label Needs rebase on Jan 10, 2022

DrahtBot added the label Needs rebase on Jan 19, 2022

PastaPastaPasta referenced this in commit 46adef2e0b on Apr 7, 2022

PastaPastaPasta referenced this in commit 6e5c27c7a0 on Apr 7, 2022

PastaPastaPasta referenced this in commit ec1f251fda on Apr 7, 2022

PastaPastaPasta referenced this in commit 95204d3fa0 on Apr 11, 2022

hebasto force-pushed on Apr 21, 2022

hebasto commented at 2:22 pm on April 21, 2022: member

Rebased 580efc56ae17dc8523af2ae18a593ed5cb708c05 -> 5f099742f2737c2e44665469a3768cab7b6e74dd (pr22555.11 -> pr22555.12).

hebasto force-pushed on Apr 21, 2022

hebasto commented at 2:36 pm on April 21, 2022: member

Ok, I reworked this PR:

dropped dependency on scary #22552
dropped indentation refactoring commit

Should be a bit easier to review now :)

Friendly ping @achow101 @icota @fanquake

DrahtBot removed the label Needs rebase on Apr 21, 2022

achow101 commented at 8:11 pm on April 21, 2022: member

ACK 7d310694fc7acc6611890e3eaff36631edb86fef

MarcoFalke added the label DrahtBot Guix build requested on Apr 22, 2022

fanquake commented at 8:08 am on April 29, 2022: member

Concept ~0. Adding more code to our depends and build systems, just to accommodate building for android in a “non-default” way is a bit meh. The people using this target with a custom SOURCES_PATH, which I’d assume is not many at all, could also just set QT_BASE_PATH.

DrahtBot added the label Needs rebase on May 5, 2022

build: Fix `make apk` for Android w/ non-default SOURCES_PATH in depends 5d166dfc25

hebasto force-pushed on May 28, 2022

hebasto commented at 12:07 pm on May 28, 2022: member

Rebased 7d310694fc7acc6611890e3eaff36631edb86fef -> 5d166dfc25520906457227b4702621754414e2a3 (pr22555.13 -> pr22555.14) due to the conflict with #24866.

hebasto commented at 12:18 pm on May 28, 2022: member

@fanquake

Adding more code to our depends and build systems…

Also some lines have been removed from src/Makefile.qt.include :)

… just to accommodate building for android in a “non-default” way is a bit meh. The people using this target with a custom SOURCES_PATH, which I’d assume is not many at all…

This PR is not about adding a new feature for a few developers, but about fixing a bug in the existing feature. If consider to accept the point mentioned above, probably #22522 should be closed and the SOURCES_PATH variable should be removed from our depends and build system, no?

… could also just set QT_BASE_PATH.

QT_BASE_PATH=<some-path> make apk won’t fix #22522.

DrahtBot removed the label Needs rebase on May 28, 2022

ghost commented at 12:55 pm on May 30, 2022: none

I followed these steps from doc/build-android.md:

Download Android Studio
SDK Manager -> SDK Tools -> NDK
Use ANDROID_SDK=/home/test/Android/Sdk ANDROID_NDK=/home/test/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/test/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin
Run make && make apk in src/qt which gives same error as shared in #22522

hebasto commented at 7:59 pm on May 30, 2022: member

@1440000bytes

3. Use ANDROID_SDK=/home/test/Android/Sdk ANDROID_NDK=/home/test/Android/Sdk/ndk-bundle make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/test/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin

I didn’t check everything for now, but seeing “NDK (Side be side)” in your screenshot makes me believe that ANDROID_NDK=/home/test/Android/Sdk/ndk-bundle is wrong. It should be something like /home/test/Android/Sdk/ndk/23.1.7779620.

ghost commented at 10:11 pm on May 30, 2022: none

I changed it to ANDROID_NDK=/home/test/Android/Sdk/ndk/24.0.8215888/ but still same error

hebasto commented at 10:36 pm on May 30, 2022: member

@1440000bytes

4. Run make && make apk in src/qt which gives same error as shared in #22522

It seems really strange for me, as in @achow101’s issue "SOURCES_PATH is set to a non-default location", but you do not use the SOURCES_PATH variable at all.

Could you provide your commands and error messages exactly from your terminal please?

DrahtBot commented at 10:25 am on May 31, 2022: contributor

Guix builds

File	commit ba48fcf4a40c5b9888459511fb4233a1b89184cc(master)	commit 59f1f6748481116a01eb700dfdcb27dedfe26cbe(master and this pull)
SHA256SUMS.part	`527e646748fc1d90...`	`3633ba4c946a9af3...`
*-aarch64-linux-gnu-debug.tar.gz	`35e4316bf1d357c4...`	`39e7ab8271962681...`
*-aarch64-linux-gnu.tar.gz	`1f85c099b29ea0dd...`	`90dfc01c299ca33c...`
*-arm-linux-gnueabihf-debug.tar.gz	`d6df7e181bf616c3...`	`dd19e4f2955798ae...`
*-arm-linux-gnueabihf.tar.gz	`c129828b3df2b777...`	`19c00fe0a24f95aa...`
*-arm64-apple-darwin-unsigned.dmg	`d1838dd296848007...`	`d43647aae7f20ddd...`
*-arm64-apple-darwin-unsigned.tar.gz	`fa36fd3da0059b2b...`	`5ed49f7c6c6601b0...`
*-arm64-apple-darwin.tar.gz	`423fff82ffd7ca8e...`	`1920816903e0682f...`
*-powerpc64-linux-gnu-debug.tar.gz	`f4c86c9c0ed36279...`	`31973e854c6f2309...`
*-powerpc64-linux-gnu.tar.gz	`9100b771827546d9...`	`d4c9c26350de4be8...`
*-powerpc64le-linux-gnu-debug.tar.gz	`8c4d736f9a77bde6...`	`22ea80cb86c729e1...`
*-powerpc64le-linux-gnu.tar.gz	`3a9e976703722adf...`	`bef726099a126b48...`
*-riscv64-linux-gnu-debug.tar.gz	`07a4d7b931883408...`	`c5a8df2fc2cccf23...`
*-riscv64-linux-gnu.tar.gz	`61a58feb2c777b96...`	`22755b5baf81da50...`
*-win64-debug.zip	`c860ce15b3921ee5...`	`3af64b885c236bd7...`
*-win64-setup-unsigned.exe	`c6396cd43abd87bb...`	`2226578282ee1930...`
*-win64-unsigned.tar.gz	`500f4a6fac75a565...`	`20e91f114e1573ee...`
*-win64.zip	`9da0e9ab96fadc5e...`	`20b7fd8203e80712...`
*-x86_64-apple-darwin-unsigned.dmg	`426c9a1341c018bb...`	`9dde8fdb0c880d38...`
*-x86_64-apple-darwin-unsigned.tar.gz	`480f25b0387d1e8c...`	`ac5b3295729ac454...`
*-x86_64-apple-darwin.tar.gz	`787812227ae3eb7d...`	`95b8c61721c06027...`
*-x86_64-linux-gnu-debug.tar.gz	`f21aa98511e3600a...`	`00c745c80d2b5675...`
*-x86_64-linux-gnu.tar.gz	`47415f7a5a9eb4c3...`	`5e61c35f86a1badd...`
*.tar.gz	`5b979b5c6941fb92...`	`6380e0a106d85085...`
guix_build.log	`9cf24d56f1eef11b...`	`63239999125f9419...`
guix_build.log.diff		`cb3fd38ea8efb5c6...`

DrahtBot removed the label DrahtBot Guix build requested on May 31, 2022

ghost commented at 5:54 pm on May 31, 2022: none

Could you provide your commands and error messages exactly from your terminal please?

https://pastebin.com/raw/kkduRKvs

hebasto commented at 6:20 pm on May 31, 2022: member

@1440000bytes

Could you provide your commands and error messages exactly from your terminal please?

https://pastebin.com/raw/kkduRKvs

Thank you!

I think you did not build depends, and missed configuration step. It should be like that:

0$ make -C depends HOST=aarch64-linux-android ANDROID_SDK=/home/hebasto/Android/Sdk ANDROID_NDK=/home/hebasto/Android/Sdk/ndk/23.1.7779620 ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/hebasto/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/linux-x86_64/bin
1$ ./autogen.sh
2$ ./configure --quiet CONFIG_SITE=$PWD/depends/aarch64-linux-android/share/config.site --enable-suppress-external-warnings
3$ make clean
4$ cd src/qt
5$ make
6$ make apk

DrahtBot commented at 7:29 pm on June 14, 2022: contributor

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a “draft”.

DrahtBot added the label Needs rebase on Jun 14, 2022

hebasto closed this on Jul 21, 2022

bitcoin locked this on Jul 21, 2023

build: Fix make apk for Android w/ non-default SOURCES_PATH in depends #22555

Conflicts

Guix builds

Guix builds

build: Fix `make apk` for Android w/ non-default SOURCES_PATH in depends #22555