Add configuration files for a Docker-based development environment

jamesob commented at 5:31 am on July 22, 2017: member

This PR introduces a number of configuration files which facilitate use of a Docker environment for local development and bitcoind execution. I’ve found this method to be an easy way to quickly setup a local dev environment; the use of a container also allows us to prevent polluting the host system with Bitcoin-specific dependencies.

The base image that I reference in the Dockerfile is maintained by Docker in an “official” image repository. It’s based on Debian 8 “Jessie” and comes with Python 3.6.2 for running the tests.

After obtaining the Docker dependencies (as described in docs/build-docker.md), building and testing bitcoin from scratch can be done in a single command:

0docker-compose run bitcoind test

During the docker image build (docker-compose build bitcoind), system dependencies are installed per docs/build-unix.md (including all non-GUI optional dependencies) but Bitcoin itself isn’t compiled. This is because docker-compose attaches the source tree to the container as a mounted volume after build, which allows changes we make to the source on the host machine to propagate immediately to a running container. The entrypoint script (./bin/docker_entrypoint.sh) is responsible for handling bitcoind compilation as well as providing the test* commands. Clang is installed and used as the default compiler.

You’ll notice that a few files (Dockerfile, bin/, docker-compose.yaml) are introduced to the top-level of the directory structure. I wasn’t eager to do this – I started out in contrib/, but in order for docker-compose to mount the entire source tree, commands have to be executed from the project root.

This is a modest suggestion and I’m happy to help incorporate feedback.

jamesob force-pushed on Jul 22, 2017

in Dockerfile:24 in 71451d41cf outdated

19+WORKDIR /code
20+
21+# Install Berkeley DB 4.8 for use with wallet functionality.
22+ENV BDB_PREFIX /db4
23+RUN mkdir -p $BDB_PREFIX
24+RUN wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz'

emanuelb commented at 6:32 am on July 22, 2017:

it better to use https link with –no-check-certificate option (until download.oracle.com have valid SSL cert)
it better to use -O parameter to specify filename of downloaded file (there was issues in filename auto-detection of wget in the past)
use environment variable for ‘db-4.8.30.NC’ string and use it instead below (can be updated easily in future)

in Dockerfile:27 in 71451d41cf outdated

22+ENV BDB_PREFIX /db4
23+RUN mkdir -p $BDB_PREFIX
24+RUN wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz'
25+RUN echo '12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef  db-4.8.30.NC.tar.gz' \
26+    | sha256sum -c | grep 'db-4.8.30.NC.tar.gz: OK'
27+RUN tar -xzvf db-4.8.30.NC.tar.gz -C $BDB_PREFIX

emanuelb commented at 6:42 am on July 22, 2017:

remove the ‘db-4.8.30.NC.tar.gz’ file after extract operation.
grep command is unneeded.

in Dockerfile:23 in 71451d41cf outdated

18+ENV HOME /code
19+WORKDIR /code
20+
21+# Install Berkeley DB 4.8 for use with wallet functionality.
22+ENV BDB_PREFIX /db4
23+RUN mkdir -p $BDB_PREFIX

emanuelb commented at 6:45 am on July 22, 2017:

use && instead of multiple RUN commands to avoid unneeded layers.

emanuelb changes_requested

emanuelb commented at 6:52 am on July 22, 2017: none

added comments about stuff that better to change in Dockerfile.

fanquake added the label Scripts and tools on Jul 22, 2017

in Dockerfile:28 in 71451d41cf outdated

23+RUN mkdir -p $BDB_PREFIX
24+RUN wget 'http://download.oracle.com/berkeley-db/db-4.8.30.NC.tar.gz'
25+RUN echo '12edc0df75bf9abd7f82f821795bcee50f42cb2e5f76a6a281b85732798364ef  db-4.8.30.NC.tar.gz' \
26+    | sha256sum -c | grep 'db-4.8.30.NC.tar.gz: OK'
27+RUN tar -xzvf db-4.8.30.NC.tar.gz -C $BDB_PREFIX
28+RUN ls ${BDB_PREFIX}

stevendlander commented at 1:26 pm on July 22, 2017:

Unnecessary ls on the BDB directory.

in Dockerfile:14 in 71451d41cf outdated

 9+    build-essential libtool autotools-dev automake pkg-config libssl-dev \
10+    libevent-dev bsdmainutils \
11+    libzmq3 miniupnpc
12+
13+# Install clang for potential use over g++.
14+RUN apt-get install -y clang

stevendlander commented at 1:30 pm on July 22, 2017:

This should also be rolled up with the above apt install block, with the comment above it.

0RUN apt-get update && apt-get install -y \
1    libboost-system-dev \
2    ...
3    miniupnpc \
4    # Install clang for potential use over g++.
5    clang

in doc/build-docker.md:8 in e3248a9f7a outdated

0@@ -0,0 +1,50 @@
1+# Docker Build Instructions and Notes
2+
3+Bitcoind can be easily built on any platform which hosts a 
4+[Docker](https://en.wikipedia.org/wiki/Docker_(software)) daemon. This is
5+likely the lowest-effort way to get a bitcoin development environment on your
6+machine.
7+
8+The Docker development enviornment does not facilitate development of the

stevendlander commented at 4:32 pm on July 22, 2017:

nit: environment

jamesob force-pushed on Jul 22, 2017

jamesob commented at 4:58 pm on July 22, 2017: member

@emanuelb @stevendlander thanks for the feedback. I’ve incorporated that as well as a few changes to make running bitcoind itself more straightforward.

Add configuration files for a Docker-based development environment

Thanks to @emanuelb @stevendlander for feedback.

c86b77fc44

jamesob force-pushed on Jul 22, 2017

greenaddress commented at 11:12 pm on July 23, 2017: contributor

I didn’t review and for now neutral on the concept but why jessie and not stretch?

greenaddress commented at 11:14 pm on July 23, 2017: contributor

Also rather than having lots of && which make it very unreadable why not using COPY for a bash script file and then RUN on it? the script would be reusable to setup dependency on Jessie/Stretch non docker too.

promag commented at 11:34 pm on July 23, 2017: member

I understand why you want this but a Dockerfile in the root usually is used to build the service container. The built image doesn’t even have bitcoind. Also, docker-compose.yml has 1 service only, with the intent to avoid calling docker with those parameters.

The usage here is for development, so I would rename Dockerfile to Dockerfile.dev (or something), remove docker-compose.yml (less dependencies) and update the instructions.

However I dunno if this is something worth to maintain here, or something you could put in another repo.

I can review if this receives concept ACK.

jamesob commented at 11:57 pm on July 23, 2017: member

Those are all good points. I think closing this PR and creating a personal repo is probably the best route.

Does anyone think it would be worth filing a separate PR to put the db-4 installation script into contrib/ or elsewhere?

jamesob closed this on Jul 23, 2017

Sjors commented at 7:25 pm on May 6, 2018: member

@jamesob did you ever end up updating your local repo?

I created yet another Dockerfile: https://github.com/Sjors/docker-bitcoin-core

It borrows a lot of code from @NicolasDorier’s project, but it builds from source. It lets you build any tag or commit (only tried it for the latest master though). Only support modern BDB.

It also lets you attach a separate volume for use with -blocksdir (which is why I needed a non-tagged build).

I tend to agree that there are too many ways to do this to warrant an “official” Docker file, unless perhaps if they end being used to replace Gitian, or maybe for Travis.

MarcoFalke locked this on Sep 8, 2021

Add configuration files for a Docker-based development environment #10903