Updated Windows build doc for WSL/Xenial workaround

sipsorcery commented at 8:46 pm on October 2, 2017: member

An update to the Windows build document that provides workarounds for the broken 64 bit mingw32 cross compiler on WSL/Xenial.

This update is an alternative to pull request #11437. While that pull request takes a valid approach by stating building on WSL should be avoided I think it is more useful to give Windows developers a workaround option.

The instructions have been tested on:

Ubuntu 14.04 and 64 bit mingw32 tool chain
Ubuntu 16.04 and 64 bit mingw32 tool chain
Ubuntu 17.04 and 64 bit mingw32 tool chain
Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 32 bit mingw32 tool chain
Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 64 bit mingw32 tool chain

Related items:

Serious incompatibility problems w/ newer mingw-64 on Ubuntu #8653
-fstack-protector-all triggers crashes in mingw-w64 5.3.1 #8732
Windows build appears broken on WSL (buntu okay) #10269
Compilation error for windows target #11437

fanquake added the label Docs and Output on Oct 2, 2017

fanquake added the label Windows on Oct 2, 2017

in doc/build-windows.md:79 in 1777a44306 outdated

75+
76+Ubuntu Trusty 14.04:
77+
78+	sudo apt-get install g++-mingw-w64-x86-64 mingw-w64-x86-64-dev
79+
80+Ubuntu Xenial 16.04 and Windows Subsystem for Linux (*)(**):

meshcollider commented at 2:13 am on October 3, 2017:

Use footnotes ([^1]) rather than * because * is used for formatting. (Same in other places).

in doc/build-windows.md:12 in 1777a44306 outdated

 5@@ -6,8 +6,11 @@ Below are some notes on how to build Bitcoin Core for Windows.
 6 Most developers use cross-compilation from Ubuntu to build executables for
 7 Windows. This is also used to build the release binaries.
 8 
 9-Currently only building on Ubuntu Trusty 14.04 is supported.
10-Other versions are unsupported or known to be broken (e.g. Ubuntu Xenial 16.04).
11+Building on Ubuntu Trusty 14.04 is recommended.
12+It is possible to build on Ubuntu Xenial 16.04. Follow the Windows Subsystem for Linux
13+steps since it uses the same Ubuntu version.
14+Building on Ubuntu Zesty 17.04 has been verified to work using the same stesp as for

meshcollider commented at 2:14 am on October 3, 2017:

stesp -> steps

in doc/build-windows.md:142 in 9030afeac8 outdated

134@@ -107,3 +135,20 @@ as they appear in the release `.zip` archive. This can be done in the following
135 way. This will install to `c:\workspace\bitcoin`, for example:
136 
137     make install DESTDIR=/mnt/c/workspace/bitcoin
138+
139+Footnotes
140+---------
141+
142+[^1] There is currently a bug in the 64 bit mingw-w64 cross compiler packaged for WSL/Ubuntu Xenial 16.04 that

meshcollider commented at 3:39 am on October 3, 2017:

Colon after [^1] :)

in doc/build-windows.md:86 in 9030afeac8 outdated

82+    sudo apt install software-properties-common
83+    sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu zesty universe"
84+    sudo apt update
85+    sudo apt upgrade
86+    sudo wget http://archive.ubuntu.com/ubuntu/pool/main/b/binutils/binutils_2.28-3ubuntu1_amd64.deb
87+    sduo dpkg -i binutils_2.28-3ubuntu1_amd64.deb

meshcollider commented at 3:49 am on October 3, 2017:

Sudo

in doc/build-windows.md:109 in a1e32a0d5a outdated

105@@ -83,10 +106,15 @@ Then build using:
106 
107 To build executables for Windows 32-bit, install the following dependencies:
108 
109-    sudo apt-get install g++-mingw-w64-i686 mingw-w64-i686-dev 
110+    sudo apt install g++-mingw-w64-i686 mingw-w64-i686-dev

meshcollider commented at 4:24 am on October 3, 2017:

The trailing whitespace failure is the end of this line I think. Sorry for so many little reviews :) Remember to squash all your commits at the end

sipsorcery force-pushed on Oct 3, 2017

meshcollider commented at 5:55 am on October 3, 2017: contributor

Gentle utACK https://github.com/bitcoin/bitcoin/commit/5e31c532e2fa1f2261ed0f94739a57cf8d5ee7e9 Will hopefully have some time to test this soon, and compare with #11437 properly

NicolasDorier commented at 5:58 am on October 3, 2017: contributor

Concept ACK. thank you, this will save me so much time! Trying this.

molxyz commented at 6:04 am on October 3, 2017: none

There’s a misspelled word: sduo dpkg -i binutils_2.28-3ubuntu1_amd64.deb

sipsorcery commented at 6:13 am on October 3, 2017: member

@molxyz have you got the latest version? That typo was fixed.

molxyz commented at 7:11 am on October 3, 2017: none

@sipsorcery Ah yes.. sorry I forgot to refresh the page, got it now. This is great, thank you.

NicolasDorier commented at 8:01 am on October 3, 2017: contributor

Still another problem on my WSL, probably unrelated though.

0configure: error: No working boost sleep implementation found.

I think I give up with WSL, keeping my docker :/

sipsorcery commented at 9:33 am on October 3, 2017: member

@NicolasDorier I did see that error once or twice and I believe it was around the same time as I was getting the error about the std::mutex header missing. I haven’t seen it for a while though and I’d estimate I’ve re-installed and built bitcoin on WSL at least half a dozen times in the last week. My suggestion would be to either completely uninstall WSL using lxrun /uninstall /full and then reinstall or at the very least clean the dependencies, including boost, from your bitcoin source tree with git clean -x -f -d.

in doc/build-windows.md:10 in 6e4fa60f33 outdated

 5@@ -6,8 +6,11 @@ Below are some notes on how to build Bitcoin Core for Windows.
 6 Most developers use cross-compilation from Ubuntu to build executables for
 7 Windows. This is also used to build the release binaries.
 8 
 9-Currently only building on Ubuntu Trusty 14.04 is supported.
10-Other versions are unsupported or known to be broken (e.g. Ubuntu Xenial 16.04).
11+Building on Ubuntu Trusty 14.04 is recommended.
12+It is possible to build on Ubuntu Xenial 16.04. Follow the Windows Subsystem for Linux

laanwj commented at 1:57 pm on October 4, 2017:

Should mention early that 16.04 produces incorrect code, so that people don’t get started with it and come to the conclusion later that it’s broken.

sipsorcery commented at 9:09 pm on October 4, 2017:

There is no bad code, provided the mingw32 x-compiler on wsl/xenial is updated as per the instructions in the doc.

in doc/build-windows.md:11 in 6e4fa60f33 outdated

 5@@ -6,8 +6,11 @@ Below are some notes on how to build Bitcoin Core for Windows.
 6 Most developers use cross-compilation from Ubuntu to build executables for
 7 Windows. This is also used to build the release binaries.
 8 
 9-Currently only building on Ubuntu Trusty 14.04 is supported.
10-Other versions are unsupported or known to be broken (e.g. Ubuntu Xenial 16.04).
11+Building on Ubuntu Trusty 14.04 is recommended.
12+It is possible to build on Ubuntu Xenial 16.04. Follow the Windows Subsystem for Linux
13+steps since it uses the same Ubuntu version.

MarcoFalke commented at 7:32 am on October 5, 2017:

This should probably mention that building on WSL currently is by following the building on Xenial notes. (not the other way round)

in doc/build-windows.md:13 in 6e4fa60f33 outdated

10-Other versions are unsupported or known to be broken (e.g. Ubuntu Xenial 16.04).
11+Building on Ubuntu Trusty 14.04 is recommended.
12+It is possible to build on Ubuntu Xenial 16.04. Follow the Windows Subsystem for Linux
13+steps since it uses the same Ubuntu version.
14+Building on Ubuntu Zesty 17.04 has been verified to work using the same steps as for
15+Ubuntu Trusty 14.04.

MarcoFalke commented at 7:32 am on October 5, 2017:

I don’t think this is true, given that there are two different sections. One for trusty, one for zesty.

in doc/build-windows.md:12 in 6e4fa60f33 outdated

 5@@ -6,8 +6,11 @@ Below are some notes on how to build Bitcoin Core for Windows.
 6 Most developers use cross-compilation from Ubuntu to build executables for
 7 Windows. This is also used to build the release binaries.
 8 
 9-Currently only building on Ubuntu Trusty 14.04 is supported.
10-Other versions are unsupported or known to be broken (e.g. Ubuntu Xenial 16.04).
11+Building on Ubuntu Trusty 14.04 is recommended.
12+It is possible to build on Ubuntu Xenial 16.04. Follow the Windows Subsystem for Linux
13+steps since it uses the same Ubuntu version.
14+Building on Ubuntu Zesty 17.04 has been verified to work using the same steps as for

MarcoFalke commented at 7:33 am on October 5, 2017:

I think you can write “17.04 or later”, since 17.10 seems to work as well with the same steps.

in doc/build-windows.md:79 in 6e4fa60f33 outdated

76+Ubuntu Trusty 14.04:
77 
78     sudo apt-get install g++-mingw-w64-x86-64 mingw-w64-x86-64-dev
79 
80-Then build using:
81+Ubuntu Xenial 16.04 and Windows Subsystem for Linux ([^1][^2]):

MarcoFalke commented at 7:35 am on October 5, 2017:

Might want to remove WSL here, since it will no longer be true when the user runs a different ubuntu version than xenial in their WSL.

sipsorcery commented at 8:28 am on October 5, 2017:

Wouldn’t it be better to leave it there and update the doc when WSL gets updated? I suspect the majority of devs reading this doc will be looking for WSL instructions.

in doc/build-windows.md:89 in 6e4fa60f33 outdated

86+    sudo apt upgrade
87+    sudo update-alternatives --config x86_64-w64-mingw32-g++ # Set the default mingw32 g++ compiler option to posix.
88+
89+Ubuntu Zesty 17.04 ([^2]):
90+
91+    sudo apt-get install g++-mingw-w64-x86-64 mingw-w64-x86-64-dev

MarcoFalke commented at 7:37 am on October 5, 2017:

µnit: This installation step is the same for all three version. Might as well mention it only once, mention that no further steps are needed on trusty and that other versions require some workarounds: …

in doc/build-windows.md:140 in 6e4fa60f33 outdated

131@@ -107,3 +132,20 @@ as they appear in the release `.zip` archive. This can be done in the following
132 way. This will install to `c:\workspace\bitcoin`, for example:
133 
134     make install DESTDIR=/mnt/c/workspace/bitcoin
135+
136+Footnotes

MarcoFalke commented at 7:38 am on October 5, 2017:

Maybe ot: For some reason GitHub doesn’t insert links to the footnotes for me.

sipsorcery commented at 9:00 am on October 5, 2017:

Looks like that footnote is not supported on github. According to a comment on this SO answer that format is used on GitLab. I’ll use the alternative suggestion of anchor links.

molxyz commented at 1:35 pm on October 6, 2017:

This is great. I have tested on my WSL Ubuntu Xenial 16.04 several times and it works, but I have a question regarding this part:

sudo update-alternatives –config x86_64-w64-mingw32-g++

There are 2 choices for the alternative x86_64-w64-mingw32-g++ (providing /usr/bin/x86_64-w64-mingw32-g++).

Selection Path Priority Status

0 /usr/bin/x86_64-w64-mingw32-g++-win32 60 auto mode 1 /usr/bin/x86_64-w64-mingw32-g++-posix 30 manual mode 2 /usr/bin/x86_64-w64-mingw32-g++-win32 60 manual mode

The only choice is “1” to get bitcoin compiled successfully, the other choices cause errors in “make”. Is there a reason to leave 3 choices? Sorry, I’m still new to this and the choices made me confused at first until I tried them all.
Thank you for doing this, I’m thrilled to be able to compile bitcoin on WSL again! :D

MarcoFalke commented at 3:19 pm on October 6, 2017:

Note that “0” is the default and the options are 1 (posix) and 2 (win32).

MarcoFalke commented at 7:39 am on October 5, 2017: member

Concept ACK. Some feedback and a nit.

in doc/build-windows.md:86 in 8aebdc1216 outdated

83+
84+    No further steps required
85+
86+Ubuntu Xenial 16.04 and Windows Subsystem for Linux <sup>[1](#footnote1),[2](#footnote2)</sup>:
87+
88+    sudo apt-get install g++-mingw-w64-x86-64 mingw-w64-x86-64-dev software-properties-common

MarcoFalke commented at 11:19 am on October 5, 2017:

µnit: Can remove the “g++-mingw-w64-x86-64 mingw-w64-x86-64-dev”

in doc/build-windows.md:94 in 8aebdc1216 outdated

91+    sudo apt upgrade
92+    sudo update-alternatives --config x86_64-w64-mingw32-g++ # Set the default mingw32 g++ compiler option to posix.
93+
94+Ubuntu Zesty 17.04 <sup>[2](#footnote2)</sup>:
95 
96+    sudo apt-get install g++-mingw-w64-x86-64 mingw-w64-x86-64-dev

MarcoFalke commented at 11:19 am on October 5, 2017:

µnit: Can remove this line.

MarcoFalke commented at 5:03 pm on October 5, 2017: member

Please squash your commits according to https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md#squashing-commits

sipsorcery force-pushed on Oct 5, 2017

laanwj commented at 9:49 am on October 11, 2017: member

Needs rebase for #11437.

sipsorcery commented at 12:05 pm on October 11, 2017: member

#11437 is largely mutually exclusive to this update. It steers devs away from using WSL or Ubuntu 16 due to the non-standard build steps required. This pull request provides those non-standard build steps.

NicolasDorier commented at 1:13 pm on October 13, 2017: contributor

I made a fresh ubuntu WSL 16.04, was unable to install boost, I tried all sort of incantation without success…

 0libboost-all-dev : Depends: libboost-tools-dev but it is not going to be installed
 1                    Depends: libboost-atomic-dev but it is not going to be installed
 2                    Depends: libboost-chrono-dev but it is not going to be installed
 3                    Depends: libboost-context-dev but it is not going to be installed
 4                    Depends: libboost-coroutine-dev but it is not going to be installed
 5                    Depends: libboost-date-time-dev but it is not going to be installed
 6                    Depends: libboost-exception-dev but it is not going to be installed
 7                    Depends: libboost-fiber-dev but it is not going to be installed
 8                    Depends: libboost-graph-dev but it is not going to be installed
 9                    Depends: libboost-graph-parallel-dev but it is not going to be installed
10                    Depends: libboost-iostreams-dev but it is not going to be installed
11                    Depends: libboost-locale-dev but it is not going to be installed
12                    Depends: libboost-log-dev but it is not going to be installed
13                    Depends: libboost-math-dev but it is not going to be installed
14                    Depends: libboost-mpi-dev but it is not going to be installed
15                    Depends: libboost-mpi-python-dev but it is not going to be installed
16                    Depends: libboost-python-dev but it is not going to be installed
17                    Depends: libboost-random-dev but it is not going to be installed
18                    Depends: libboost-regex-dev but it is not going to be installed
19                    Depends: libboost-serialization-dev but it is not going to be installed
20                    Depends: libboost-signals-dev but it is not going to be installed
21                    Depends: libboost-test-dev but it is not going to be installed
22                    Depends: libboost-timer-dev but it is not going to be installed
23                    Depends: libboost-type-erasure-dev but it is not going to be installed
24                    Depends: libboost-wave-dev but it is not going to be installed

Not sure if related with this PR, but the instructions did not seem enough.

sipsorcery commented at 3:26 am on October 14, 2017: member

@NicolasDorier what type of VM, Ubuntu 16 version and platform (x86 or x64)?

I have run the install steps using the latest source form the bitcoin git repo on VMWare Workstation with Ubuntu 16.04.3 Server x64 without any problems and resulting in working Windows executables.

I’m happy to check other permutations if you let me know what your configuration is.

NicolasDorier commented at 7:18 am on October 15, 2017: contributor

This is x64.

0DISTRIB_ID=Ubuntu
1DISTRIB_RELEASE=16.04
2DISTRIB_CODENAME=xenial
3DISTRIB_DESCRIPTION="Ubuntu 16.04.3 LTS"

Problems are on boost… I can’t seems to be able to fetch boost for some reason.

MarcoFalke commented at 8:17 am on October 15, 2017: member

Needs rebase on current master to solve merge conflicts.

sipsorcery commented at 8:25 am on October 15, 2017: member

@MarcoFalke merge conflict is from #11437 which is largely mutually exclusive to this update.

It’s a question of whether to steers devs away from building on WSL/ Ubuntu 16 using non-standard build steps required or only recommend Ubuntu 14 & 17 as per #11437.

MarcoFalke commented at 8:48 am on October 15, 2017: member

@sipsorcery Agree. Though, we can not merge your pull if there are merge conflicts (even if only minor).

I don’t have WSL, so I can’t help with testing and recommendations. How you solve the merge conflict is pretty much up to you. It doesn’t matter if you use -Xtheirs or -Xours or solve it manually …

Updated Windows build doc for WSL/Xenial workarounds. e0fc4a7356

sipsorcery force-pushed on Oct 15, 2017

sipsorcery commented at 11:59 am on October 15, 2017: member

@MarcoFalke no worries, I’ve removed the conflict by pushing my change to the build-windows.md file on top of trunk.

meshcollider commented at 10:10 pm on October 15, 2017: contributor

Just want to point out that your commit isn’t associated with your GitHub account currently, you might have used a different email address? You might want to commit with the same email address so your account gets the credit :)

sipsorcery commented at 6:15 am on October 16, 2017: member

@NicolasDorier is there any reason you’re attempting to install boost packages with apt?

If you only need boost for a bitcoin build the required includes and libraries get installed as part of the build steps that download and compile all the required dependencies.

cd depends make HOST=x86_64-w64-mingw32

NicolasDorier commented at 6:56 am on October 16, 2017: contributor

yes, I need it because I need to run python tests on WSL.

sipa commented at 6:58 am on October 16, 2017: member

The python tests don’t need boost?

NicolasDorier commented at 8:46 am on October 16, 2017: contributor

They don’t but I need to compile bitcoind as a linux binary, which need boost.

sipa commented at 8:50 am on October 16, 2017: member

@NicolasDorier Using the depends build will do all that.

sipsorcery commented at 9:12 am on October 16, 2017: member

On a related note the Windows 10 Fall Creators Update gets released tomorrow (17 Oct 2017) and contains a major update to WSL. The main update is that WSL will now be deployed as a Windows Store App and users will need to choose from different Linux distros; currently Ubuntu (Xenial), openSUSE and SUSE Linux Enterprise Server.

Update: I’ve verified that the Windows bitcoin build on the Fall Creators Update & Ubuntu Windows Store App works successfully. The Ubuntu version hasn’t changed from 16.04.3.

NicolasDorier commented at 9:36 am on October 16, 2017: contributor

my understanding was that

0cd depends
1make HOST=x86_64-w64-mingw32

will just install the deps for making a windows binary. I was unaware it was also adding the libboost dependencies for linux build. Trying that now.

sipa commented at 9:41 am on October 16, 2017: member

make HOST=x86_64-unknown-linux-gnu will do a build of dependencies for Linux build on x86_64.

NicolasDorier commented at 9:44 am on October 16, 2017: contributor

Trying. Maybe this is something to add as exception of Xenial in the doc, since installing package with apt-get is not available?

NicolasDorier commented at 1:16 pm on October 16, 2017: contributor

Ok now

0WARNING: 'automake-1.14' is missing on your system.

So make HOST=x86_64-unknown-linux-gnu does not run, I ran apt-get of autotools.

0autotools-dev is already the newest version (20150820.1).

I think I will provide an easy to build image on docker hub based on trusty and document it. Would, requires Hyper-V for windows (windows 10 Enterprise edition) though.

sipsorcery commented at 2:53 am on October 17, 2017: member

@NicolasDorier the automake error seems to be generated in the protobuf dependency build. The same error was mentioned in #10269. I’m guessing it’s triggered when the apt packages line up in a certain way.

The suggested workaround, which also worked for me in the past, is:

make HOST=x86_64-unknown-linux-gnu AUTOMAKE=:

Edit: Also #11282.

Updated instructions for Windows 10 Fall Creators Update. 7383d77264

NicolasDorier commented at 7:34 am on October 18, 2017: contributor

0autoreconf: running: : --add-missing --copy
1Can't exec ":": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326, <GEN2> line 5.
2autoreconf: failed to run :: No such file or directory
3funcs.mk:242: recipe for target '/mnt/c/Users/NicolasDorier/Documents/Sources/bitcoin-docker/bitcoin/depends/work/build/x86_64-unknown-linux-gnu/libevent/2.1.8-stable-b014fbf5633/.stamp_preprocessed' failed
4make: *** [/mnt/c/Users/NicolasDorier/Documents/Sources/bitcoin-docker/bitcoin/depends/work/build/x86_64-unknown-linux-gnu/libevent/2.1.8-stable-b014fbf5633/.stamp_preprocessed] Error 1

Ha might be because too long path… I don’t know. :(

I think this PR is definitively a way to the right direction, I hope other people can try it, my environment appear to be seriously fucked up.

rebroad commented at 4:54 pm on November 12, 2017: contributor

tested ACK

laanwj commented at 11:48 am on November 13, 2017: member

@rebroad Thanks for testing

laanwj merged this on Nov 13, 2017

laanwj closed this on Nov 13, 2017

laanwj referenced this in commit 7fbf3c638f on Nov 13, 2017

jky7 commented at 6:03 pm on November 13, 2017: none

@NicolasDorier I had the same issue, what ended up working was the following: make HOST=x86_64-w64-mingw32 AUTOCONF=: AUTOHEADER=: AUTOMAKE=: ACLOCAL=:

also I have found that downloading the repo natively from inside WSL was less likely to break things in the build than downloading it in Windows and then working on it from WSL

meshcollider commented at 9:43 am on November 15, 2017: contributor

Post-merge tested ACK on clean install of 64-bit Win10 Home 1709 build 16299.64

NicolasDorier commented at 4:56 am on November 21, 2017: contributor

@josephky ha yes. There are some git config magic to do so git on Windows does not convert LF to CRLF on checkout. And that it convert CRLF to LF on check-in. I did that long time ago, now it works fine.

It was only core.autocrlf=input if I remember…

sipsorcery deleted the branch on Nov 21, 2017

eugene-sy referenced this in commit ce593ae361 on Jan 6, 2018

DrahtBot locked this on Sep 8, 2021

Updated Windows build doc for WSL/Xenial workaround #11438

Selection Path Priority Status