Adding Docker/docker-compose files #13542

Ability to build and run bitcoin from Docker. Builds docker image with bitcoin and DB4 4.8.30.NC, starts with docker-compose. Easy to modify startup parameters or add configuration files and scale containers. Image uses CentOS and gosu to start bitcoin as bitcoin user (PID 1).

#10903 was the last time this was suggested. The conclusion then was that this isn't necessarily something we want to maintain here, and would probably be better suited to an external repo.

See: https://github.com/Sjors/docker-bitcoin-core https://github.com/NicolasDorier/docker-bitcoin https://github.com/jamesob/docker-bitcoind

for some examples of such repositories. @stakauskas What's your motivation for adding the Dockerfiles here?

Agree with @fanquake

NACK for the same reasons.

It was first created as an addition to ElementsProject in - https://github.com/ElementsProject/elements/pull/372 , thus appeared here.

Though having Docker images could be beneficial for community wanting to run software in containers and one image would target Windows/Mac/Linux instead of having separate packages for each of them.

I think maintaining several Dockerfile like this as one big advantage: We can use those as part of the CI to be sure bitcoind build correctly on the different OS which should be supported by Bitcoin, as well as running the tests on different distro. (Actually I think this is what we do for now only on ubuntu?) @sipsorcery for example maintain a dockerfile for building on Windows via msvc, and configured CI to run it. This way he can warn you when something break Windows msvc build. Even if you are not fan of windows, I think you will see the advantage to have it for your favorite linux distro instead of relying on outdated documentation and untested dev environment.

It is also way more useful than maintaining a documentation, as anybody can reproduce the build on their environment just by reading the dockerfile related to their OS.

See this as a documentation which can be checked automatically during CI.

It also has the advantage that people can easily distribute their own bitcoin core image with their favorite freshly backed commits.

My repo https://github.com/NicolasDorier/docker-bitcoin is good if you want just an official release, but I totally see the advantage of providing dockerfiles which build from sources.

It is big ACK for me. It is better than trying to maintain a documentation, but serve also other purposes.

I consider these different:

• Dockerfile(s) to build the official image - usually in a separate repository with multiple versions/variants - and usually download the office source code;
• docker-compose to launch 3rd party services to aid development - not relevant here I think.
• docker-compose + multiple Dockerfile to build from local source in different flavours - usually to aid development or CI.

It also has the advantage that people can easily distribute their own bitcoin core image with their favorite freshly backed commits.

This is also possible with external Dockerfile.

I'd compare Dockerfile to RPM spec and docker-compose as a way to manage your containers, e.g systemctl/systemd for services, just portable across different OS versions. While you can run any docker image with 'docker run', it's much easier to start with docker-compose, which allows easier and faster configuration for running containers. In this case testnet/mainnet nodes can be started within one go, scale if needed. Network isolation/ports management provided out of the box, if you wanted 3 nodes, you do not need to change configuration for port mapping, as compose will assign random host -> same port in container, etc...

I don't see what advantage a dockerfile gives us to our existing processes. Note that it is already possible to build any commit from source deterministically by using it's gitian descriptor. Using a dockerfile to create a docker image as opposed to using gitian to create a binary wouldn't be deterministic any more.

Also note that the actual build step is just a single line ./configure && make. Installing the dependencies is what differs per dist and usually the cause of issues. Instead of having a Dockerfile for each distribution, it seems easier to me to just list the dependencies in a readme (and maybe a one-liner for common distributions to install all dependencies)

Note that on travis we already use docker, but I don't see how a Dockerfile would simplify anything there.

If there is a valid use case for a Dockerfile that makes our process easier, I am all in, otherwise I think this will lead to a maintenance burden and nit-picking, because everyone has their own use case and wants the Dockerfile to match that use case.

RPM spec

Sorry for ot, but we had someone contribute an rpm spec file and then it would just sit there unmaintained and now it is unclear what to do with it: #13137

I think we want to avoid doing the same with a Dockerfile.

Installing the dependencies is what differs per dist and usually the cause of issues. Instead of having a Dockerfile for each distribution, it seems easier to me to just list the dependencies in a readme (and maybe a one-liner for common distributions to install all dependencies)

There's no need to have Dockerfile per each distribution, though it's possible and not difficult to maintain. After choosing base image, the whole lifecycle of running(also removing/updating/etc) software is as easy as choosing image:tag, bitcoin:version, instead of downloading package for Windows/Linux/Mac and figuring out deps individually.

Instead of having a Dockerfile for each distribution, it seems easier to me to just list the dependencies in a readme (and maybe a one-liner for common distributions to install all dependencies)

I disagree with this point. This result in outdated documentation, and additional support when somebody complains that it does not work because some outdated dependencies on their system, also it never get tested and too often people break builds without knowing. @sipsorcery very often report Windows msvc breaks as issues and PR. Those could be easily detected if dockerfile were used and built during CI.

As someone new to bitcoin compilation, I found getting a gitian build up and running quite complicated, especially on an OSX machine trying to use virtualization.

I get the motivations around providing Dockerfiles in external projects but that means there is no (obvious) "source of truth" for getting started. If we want more devs poking at bitcoin I think it would be good that there was an official Dockerfile here that just gave you a build that you could play with locally (the intention being development, not production). That way you could test your changes easily. I was about to submit one but then saw this effort and decided to pile on my comment.

I greatly support making it easier for new devs to get a local image running and docker provides exactly that.

A few prior comments may not be entirely familiar with how docker works. We need provide only one file and that file will be universal and will not require any maintenance as it will self-document all the dependencies. A CI build could be used to verify that the container builds and produces the bitcoind binary, which is all that's necessary to make sure nothing is broken as far as deps go. In this regard it's much simpler than an rpm spec and not directly comparable.

Concept ACK, if this helps people to run tests, ci and generally makes it easier to bootstrap a development environment. Though, we shouldn't provide any ways to run this in production, since there are already statically linked, gitian-built binaries available. If you need those releases wrapped into a dockerfile, there is already https://github.com/NicolasDorier/docker-bitcoin/tree/master/core

Concept ACK, if this helps people to run tests, ci and generally makes it easier to bootstrap a development environment.

The initial pull request emerged and divided into two groups dev/prod. Initially it was created as an alternative to running standalone binaries in favor of running it in containerised setup. If it's completely for dev purposes (which is a good start) I'll modify Dockerfile and related files accordingly so it could be more suitable for the purpose.

Going to chime in again to slightly disagree with "we shouldn't provide docker for production", again due to the "source of truth" argument. If you google bitcoin on docker you get a variety of blog posts and repos. It's not that hard to provide a dockerfile and make sure it's working which ensures that it's as easy as possible to run a node. If more nodes is what we want, then providing a production friendly way to spin one up is not a bad idea.

-1 to provide production images here. I think we shouldn't influence deployment.

+1 in providing docker compose services to build and test with diferent toochain and configurations.

+1 in providing official docker images (not sure how to tackle that) elsewhere.

Concept ACK. This is contrib anyway, if it goes unmaintained it can always be simply removed.

@jtimon it can't be unmaintained if included inside CI. Having a docker failing to build is equivalent to having an out of date documentation: the documentation need to be fixed, or the committer need to not break things.

@sipsorcery you might be interested to follow this PR.

Summoning @ruimarinho as he also maintains https://github.com/ruimarinho/docker-bitcoin-core.

I believe providing a dockerfile for production use can be done independently (in a different pull request) of providing a dockerfile for development/testing purposes?

If this pull request is to be used to add a devel dockerfile, I'd suggest prefixing the title with "test" or "qa".

If this pull request is to be used to add a devel dockerfile, I'd suggest prefixing the title with "test" or "qa".

Files in this pull request would match the prod setup. We use it the way it is to run and scale nodes in prod environment. I'd suggest that better way would be to create a separate pull req for development environment as it will differ significantly and close this if there's no need for prod env. Though, I would agree @skwp that official production friendly docker environment would be nice to have. As for someone having resources it's as easy as one command to run x prod/test nodes in the current containerised infrastructure. And I don't see how it would influence deployment as mentioned by @promag apart from the fact that it would make it much easier to deploy and manage.

I mean, should we promote deployments with docker? The way I see, that would be the case if a dockerfile is added.

I mean, should we promote deployments with docker? The way I see, that would be the case if a dockerfile is added.

It wouldn't be a promotion, it would just be an ability for people to choose another variant to running the software. Most of the projects provides official Dockerfiles/docker images in addition to the traditional released binaries. It would also make it easy to have official way to run bitcoin nodes in a container native infrastructures which is quite common approach by todays standards; and possibly attract much more people running nodes, with having official way to do it.

We now also have an external "packaging" repo at https://github.com/bitcoin-core/packaging where things like this may make more sense, just to add to the discussion.

On July 11, 2018 4:07:37 PM UTC, paulius notifications@github.com wrote:

I mean, should we promote deployments with docker? The way I see, that would be the case if a dockerfile is added.

It wouldn't be a promotion, it would just be an ability for people to choose another variant to running the software. Most of the projects provides official Dockerfiles/docker images in addition to the traditional released binaries. It would also make it easy to have official way to run bitcoin nodes in a container native infrastructures which is quite common approach by todays standards. It would possibly attract much more people running nodes, with having official way to do it.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/bitcoin/bitcoin/pull/13542#issuecomment-404225134

@TheBlueMatt Agreed, moving Docker related stuff to bitcoin-core/packaging would make sense.

@TheBlueMatt Agreed, moving Docker related stuff to bitcoin-core/packaging would make sense.

That would solve the "6 people maintaining the same thing independently" problem too. Fine with me. To me the most important part is avoiding that work duplication.

Could you adjust this pull request to be just a patch that adds a link in some doc to https://github.com/bitcoin-core/packaging/blob/master/docker/README.md ?

And everyone who is interested in this should go and review https://github.com/bitcoin-core/packaging/pull/2

I really dislike adding on bitcoin-core/packaging... the whole point is having that run as part of the CI. By moving on separate repo, it makes sure it will never be part of CI because you can't build with the specific commit you pulled via COPY . .

It is quite useless there I think. :(

<!--5d09a71f8925f3f132321140b44b946d-->The last travis run for this pull request was 59 days ago and is thus outdated. To trigger a fresh travis build, this pull request should be closed and re-opened.

As mentioned above, this is now being added to bitcoin-core/packaging/ in https://github.com/bitcoin-core/packaging/pull/2. Closing this PR for now.