Gitian build issue on Debian host #13991

Following the guide and running $ ./gitian-build.py --setup hebasto 0.17.0rc1

returns an error:

E: No such script: /usr/share/debootstrap/scripts/bionic
Traceback (most recent call last):
  File "./gitian-build.py", line 220, in <module>
    main()
  File "./gitian-build.py", line 203, in main
    setup()
  File "./gitian-build.py", line 39, in setup
    subprocess.check_call(make_image_prog)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/make-base-vm', '--suite', 'bionic', '--arch', 'amd64', '--lxc']' returned non-zero exit status 1

Tested on Debian 8.10 and 9.5.

EDIT: Debian debootstrap package doesn't support bionic.

You can "switch to Ubuntu 18.04" or "install self-compiled lxc that higher than v2.11 and debootstrap higher than 1.0.92 on Debian" to solve this problem. That document needs an update.

@ken2812221

debootstrap higher than 1.0.92 on Debian

There're two debootstrap versions >1.0.92:

• 1.0.108 in sid and buster
• 1.0.100~bpo9+1 in stretch-backports

~Both don't support bionic. Which version of debootstrap do you mean?~

You're right. The site debootstrap (1.0.100~bpo9+1) provides misinformation about the absence of the /usr/share/debootstrap/scripts/bionic.

That document needs an update.

I'm working on it.

I used the stretch-backport debootstrap with lxc 2.0.7 and hit the following issue: https://github.com/devrandom/gitian-builder/issues/196 trying to self compile 2.1.1 lxc to see if that fixes it

@MarcoFalke @ken2812221 Tested on Debian 9.5 with different repos:

gitianuser@debian:~$ apt list -a debootstrap

Listing... Done
debootstrap/unstable,now 1.0.109 all [installed]
debootstrap/testing 1.0.108 all
debootstrap/stretch-backports 1.0.100~bpo9+1 all
debootstrap/stable 1.0.89 all

• 1.0.89 from stretch doesn't work a priori;

• 1.0.100~bpo9+1 from stretch-backports doesn't work with an error while configuring rsyslog package, https://github.com/devrandom/gitian-builder/issues/196;

• 1.0.108 from buster works as expected;

• 1.0.109 from sid works as expected.

same here on debian 9.5 and 8.10 even ubuntu 18.04. how to fix? should i try on debian 10 buster or sid? @hebasto

• log

$ ./gitian-build.py --setup

base-bionic-amd64 already exists, please remove it first
Traceback (most recent call last):
  File "./gitian-build.py", line 262, in <module>
    main()
  File "./gitian-build.py", line 198, in main
    setup()
  File "./gitian-build.py", line 42, in setup
    subprocess.check_call(make_image_prog)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/make-base-vm', '--suite', 'bionic', '--arch', 'amd64', '--lxc']'

@decryp2kanon

same here on debian 9.5 and 8.10 even ubuntu 18.04. how to fix? should i try on debian 10 buster or sid? @hebasto

As mentioned above the version of the debootstrap package really matters. Which version of that package has your Debian system?

@hebasto i tried 1.0.123 but no luck

• debian 10.5

gitianuser3@debian105:~$ apt list -a debootstrap
Listing... Done
debootstrap/now 1.0.123 all [installed,local]
debootstrap/stable 1.0.114 all

• debian 9.5

gitianuser2@debian95:~$ apt list -a debootstrap
Listing... Done
debootstrap/oldstable,now 1.0.89 all [installed]

1.0.108 from buster works as expected; 1.0.109 from sid works as expected.

where and how did you installed this version?

i tried, but still no luck... https://launchpad.net/debian/+source/debootstrap/1.0.109 https://launchpad.net/debian/+source/debootstrap/1.0.108

i still failed on debian 8.10, 9.5 and 10.5 ... but succeed on ubuntu 18.04 thanks! https://gist.github.com/decryp2kanon/67730da9e5f78bfb07f555d106b36686

@decryp2kanon

I've tried to reproduce all steps.

1. VirualBox 6.1.14 r140239 is installed on my host.

2. Clean Debian VM:

$ cat /etc/debian_version 
10.6

3. Installed packages that are listed in gitian-building-setup-gitian-debian.md:

sudo apt install git ruby apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils make ubuntu-archive-keyring curl firewalld

4. Made additional settings:

$ sudo -s
# echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-start" > /etc/sudoers.d/gitian-lxc
# echo "%sudo ALL=NOPASSWD: /usr/bin/lxc-execute" >> /etc/sudoers.d/gitian-lxc
# echo '#!/bin/sh -e' > /etc/rc.local
# echo 'brctl addbr br0' >> /etc/rc.local
# echo 'ip addr add 10.0.3.1/24 broadcast 10.0.3.255 dev br0' >> /etc/rc.local
# echo 'ip link set br0 up' >> /etc/rc.local
# echo 'firewall-cmd --zone=trusted --add-interface=br0' >> /etc/rc.local
# echo 'exit 0' >> /etc/rc.local
# chmod +x /etc/rc.local
# reboot

5. After reboot (tested only Windows build to save time):

$ mkdir gitian
$ cd gitian
$ git clone https://github.com/bitcoin/bitcoin.git
$ cp bitcoin/contrib/gitian-build.py .
$ ./gitian-build.py --setup
$ ./gitian-build.py -b -o w -D -n hebasto 0.20.1
...
Generating report
24f1e8a0e3b86ad82ad9182f32cef05d2332696aadb756d327b5e3bd39a5e19f  bitcoin-0.20.1-win-unsigned.tar.gz
09fc4490938764ac8786f2ee904b06e4ab173320cc1d86d425baa1a60f285e82  bitcoin-0.20.1-win64-debug.zip
1740b9af53db3d50eda34b50eddd3a2e90d7a657d4ab40a4f4aa7ffba52b8ead  bitcoin-0.20.1-win64-setup-unsigned.exe
e59fba67afce011d32b5d723a3a0be12da1b8a34f5d7966e504520c48d64716d  bitcoin-0.20.1-win64.zip
4bbd62fd6acfa5e9864ebf37a24a04bc2dcfe3e3222f056056288d854c53b978  src/bitcoin-0.20.1.tar.gz
93b0bdd06a58f6d17530018a8489322957f0452ec1cb2dd299f777dc01385884  bitcoin-core-win-0.20-res.yml
Done.

same here on debian 9.5 and 8.10 even ubuntu 18.04. how to fix? should i try on debian 10 buster or sid? @hebasto

* log

$ ./gitian-build.py --setup

base-bionic-amd64 already exists, please remove it first
Traceback (most recent call last):
  File "./gitian-build.py", line 262, in <module>
    main()
  File "./gitian-build.py", line 198, in main
    setup()
  File "./gitian-build.py", line 42, in setup
    subprocess.check_call(make_image_prog)
  File "/usr/lib/python3.5/subprocess.py", line 271, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/make-base-vm', '--suite', 'bionic', '--arch', 'amd64', '--lxc']'

Did you try to follow "base-bionic-amd64 already exists, please remove it first" ?

If that doesn't work, you could remove the gitian-builder directory, and make setup from scratch.

For the records, I've tested today to setup debian VM for gitian build and ran across the same error, on debian-8.10.0 and debian-10.9.0:

$ ./gitian-build.py --setup
...
E: No such script: /usr/share/debootstrap/scripts/focal
Traceback (most recent call last):
  File "./gitian-build.py", line 263, in <module>
    main()
  File "./gitian-build.py", line 199, in main
    setup()
  File "./gitian-build.py", line 43, in setup
    subprocess.check_call(make_image_prog)
  File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bin/make-base-vm', '--suite', 'focal', '--arch', 'amd64', '--lxc', '--disksize', '13000']' returned non-zero exit status 1.

i still failed on debian 8.10, 9.5 and 10.5 ... but succeed on ubuntu 18.04 thanks! https://gist.github.com/decryp2kanon/67730da9e5f78bfb07f555d106b36686

I'll try to switch to Ubuntu