Release tarball missing many files from git source #6753

Only in from-git/: .gitattributes
Only in from-git/: .gitignore
Only in from-git/: .travis.yml
Only in from-git/: .tx
Only in from-gitian/: Makefile.in
Only in from-git/: README.md
Only in from-gitian/: aclocal.m4
Only in from-git/: autogen.sh
Only in from-gitian/build-aux: config.guess
Only in from-gitian/build-aux: config.sub
Only in from-gitian/build-aux: depcomp
Only in from-gitian/build-aux: install-sh
Only in from-gitian/build-aux: ltmain.sh
Only in from-gitian/build-aux/m4: libtool.m4
Only in from-gitian/build-aux/m4: ltoptions.m4
Only in from-gitian/build-aux/m4: ltsugar.m4
Only in from-gitian/build-aux/m4: ltversion.m4
Only in from-gitian/build-aux/m4: lt~obsolete.m4
Only in from-gitian/build-aux: missing
Only in from-gitian/: configure
Only in from-git/contrib: README.md
Only in from-git/contrib: bitcoin-qt.pro
Only in from-git/contrib: bitcoind.bash-completion
Only in from-git/contrib: bitrpc
Only in from-git/contrib: debian
Only in from-git/contrib: devtools
Only in from-git/contrib: gitian-descriptors
Only in from-git/contrib: gitian-downloader
Only in from-git/contrib: init
Only in from-git/contrib: linearize
Only in from-git/contrib/macdeploy: LICENSE
Only in from-git/contrib/macdeploy: README.md
Only in from-git/contrib/macdeploy: background.png
Only in from-git/contrib/macdeploy: background.psd
Only in from-git/contrib/macdeploy: background@2x.png
Only in from-git/contrib: qos
Only in from-git/contrib: qt_translations.py
Only in from-git/contrib: seeds
Only in from-git/contrib: spendfrom
Only in from-git/contrib: test-patches
Only in from-git/contrib: testgen
Only in from-git/contrib: tidy_datadir.sh
Only in from-git/contrib: verify-commits
Only in from-git/contrib: verifysfbinaries
Only in from-git/: depends
Only in from-git/doc: Doxyfile
Only in from-git/doc: README_osx.txt
Only in from-git/doc: bitcoin_logo_doxygen.png
Only in from-git/doc: gitian-building
Only in from-git/doc: travis-ci.txt
Only in from-git/share: certs
Only in from-git/share/pixmaps: addressbook16.bmp
Only in from-git/share/pixmaps: addressbook16mask.bmp
Only in from-git/share/pixmaps: addressbook20.bmp
Only in from-git/share/pixmaps: addressbook20mask.bmp
Only in from-git/share/pixmaps: bitcoin-bc.ico
Only in from-git/share/pixmaps: bitcoin128.png
Only in from-git/share/pixmaps: bitcoin128.xpm
Only in from-git/share/pixmaps: bitcoin16.png
Only in from-git/share/pixmaps: bitcoin16.xpm
Only in from-git/share/pixmaps: bitcoin256.png
Only in from-git/share/pixmaps: bitcoin256.xpm
Only in from-git/share/pixmaps: bitcoin32.png
Only in from-git/share/pixmaps: bitcoin32.xpm
Only in from-git/share/pixmaps: bitcoin64.png
Only in from-git/share/pixmaps: bitcoin64.xpm
Only in from-git/share/pixmaps: check.ico
Only in from-git/share/pixmaps: favicon.ico
Only in from-git/share/pixmaps: send16.bmp
Only in from-git/share/pixmaps: send16mask.bmp
Only in from-git/share/pixmaps: send16masknoshadow.bmp
Only in from-git/share/pixmaps: send20.bmp
Only in from-git/share/pixmaps: send20mask.bmp
Only in from-git/share/qt: extract_strings_qt.py
Only in from-git/share/qt: img
Only in from-git/share/qt: make_spinner.py
Only in from-git/share/qt: make_windows_icon.sh
Only in from-git/share/qt: protobuf.pri
Only in from-git/share: ui.rc
Only in from-git/src: .clang-format
Only in from-gitian/src: Makefile.in
Only in from-git/src/config: .empty
Only in from-gitian/src/config: bitcoin-config.h.in
Only in from-git/src/json: LICENSE.txt
Only in from-git/src/json: json_spirit_reader.cpp
Only in from-git/src/json: json_spirit_value.cpp
Only in from-git/src/json: json_spirit_writer.cpp
Only in from-git/src: obj
Only in from-git/src: obj-test
Only in from-git/src/qt: Makefile
Only in from-git/src/qt: bitcoinstrings.cpp
Only in from-git/src/qt/res/movies: makespinner.sh
Only in from-git/src/qt/res: spinner.png
Only in from-git/src/qt/res: src
Only in from-git/src/qt/test: Makefile
Only in from-git/src/secp256k1: .gitignore
Only in from-git/src/secp256k1: .travis.yml
Only in from-gitian/src/secp256k1: Makefile.in
Only in from-git/src/secp256k1: README.md
Only in from-gitian/src/secp256k1: aclocal.m4
Only in from-gitian/src/secp256k1/build-aux: config.guess
Only in from-gitian/src/secp256k1/build-aux: config.sub
Only in from-gitian/src/secp256k1/build-aux: depcomp
Only in from-gitian/src/secp256k1/build-aux: install-sh
Only in from-gitian/src/secp256k1/build-aux: ltmain.sh
Only in from-gitian/src/secp256k1/build-aux/m4: libtool.m4
Only in from-gitian/src/secp256k1/build-aux/m4: ltoptions.m4
Only in from-gitian/src/secp256k1/build-aux/m4: ltsugar.m4
Only in from-gitian/src/secp256k1/build-aux/m4: ltversion.m4
Only in from-gitian/src/secp256k1/build-aux/m4: lt~obsolete.m4
Only in from-gitian/src/secp256k1/build-aux: missing
Only in from-gitian/src/secp256k1: configure
Only in from-git/src/secp256k1: obj
Only in from-git/src/secp256k1/src/java: org
Only in from-git/src/secp256k1/src/java: org_bitcoin_NativeSecp256k1.c
Only in from-gitian/src/secp256k1/src: libsecp256k1-config.h.in
Only in from-git/src/test: Makefile
Only in from-git/src/test: README.md
Only in from-git/src/test/data: README.md
Only in from-git/src/univalue: gen.cpp

AFAIK the only difference should be additional files from autogen.sh...

@theuni Do you know from where this difference comes? Do some of these missing files need to be listed somewhere?

@paveljanik EXTRA_DIST etc in the makefile

Only in from-git/: autogen.sh

Are you sure this is not in from-gitian because we fixed that in #6418?

Hello,

sorry to bump this old issue, but as it is still open I figured why not. @luke-jr I am the guy who answered you earlier on reddit about the tarball here on github.

Many packagers prefer this file because it contains additional things like manpages. Then I noticed that even debian apparently builds its packages from it: (this has the same hash like this)

% sha256sum *.gz
852581d21716b74e0539f7fd2f9c17f5506f1c21c5cec6642b0286823ba97193  bitcoin_0.13.0.orig.tar.gz
852581d21716b74e0539f7fd2f9c17f5506f1c21c5cec6642b0286823ba97193  v0.13.0.tar.gz

In light of the recent warning on bitcoin.org this feels like a very suboptimal situation.

It seems easy to include additional files in the make dist command (which I guess is what gitian uses to create the tarball that gets signed in SHA256SUMS.asc)

The only problem is maybe that the files under contrib/debian are licensed GPL2/GPL3, see: contrib/debian/copyright.

Here: https://github.com/nomnombtc/bitcoin/commit/1f9c6be4960ddaec6f68c4b4983f94f0ae672a85

is something which creates a new variable DIST_CONTRIB in Makefile.am and adds this to EXTRA_DIST. I included a few useful things like contrib/debian subdir (contains manpages, .desktop file), contrib/rpm subdir (contains .spec file to build rpms), and contrib/init (contains things like systemd startup files) and the *.bash-completion files.

The two tarballs should be identical except for the addition of generated build scripts (configure, etc) in the release one. IMO you should just submit that commit as a pull request since it improves the situation.

Okay, here is the PR: #8568

But I agree it would make more sense if they are identical.