Docs: How can we can revamp the Bitcoin Core dev docs, encourage contributions and avoid bikeshedding? #20132

Reviewing Docs PRs (very low risk)

My personal view is that there are high risk PRs (consensus code), medium risk PRs (non consensus code, GUI?), low risk PRs (tests) and very low risk PRs (docs) in terms of introducing bugs, vulnerabilities and so a review process that makes sense for consensus code doesn't make much sense for docs.

We should encourage use of simple ACKs and NACKs (Concept ACK, Approach ACK don't make much sense in a doc scenario) and doc nits can generally be addressed in a follow up PR if the nit commenter cares that much about the nit.

Docs and certainly nits should not take up valuable review time from long term contributors and maintainers.

We certainly don't want inaccuracies or poor quality docs introduced to the Core repo. So an ACK/NACK review is still important. And if a reviewer feels strongly enough that a certain doc PR is a deterioration we absolutely need a NACK. But we also want to avoid circles of conflicting nits because reviewers feel that they need to review in the same meticulous way as they review consensus code. I observe people generally want to add nits rather than a simple ACK because it feels as if they are being helpful.

The various sources of Bitcoin Core developer documentation

• bitcoin/bitcoin/doc is the primary place for completed developer documentation. I propose creating a /developer directory here. The developer notes is very long now and should in my opinion be split up into separate docs within this new /developer directory. (NACKed)

• bitcoin-core/bitcoin-devwiki is used for drafting and general adhoc stuff that doesn't fit anywhere else. This is not a general documentation resource. Once something is finalized here it should be brought into this repo bitcoin/bitcoin/doc.

• bitcoin-core/docs is only used for Gitian build files currently. This is not a general documentation resource.

• btcinformation.org (@harding and @achow101 are contributors) contain external developer resources that can be linked to if/when required. When developer documentation doesn't fit into bitcoin/bitcoin/doc a good alternative is there.

• bitcoincore.org was historically used as an information source for certain topics (RBF and SegWit). However, today it is just used to provide binaries of Core releases, release notes and isn't used to provide additional information.

• Bitcoin StackExchange is a good place for asking questions that you don't think are answered in the primary docs. After receiving an answer there please consider opening a PR to improve the docs in this repo.

I don't think we should be afraid of introducing concepts and links to other resources, related PRs, related PR review club sessions, transcripts of relevant presentations in docs like multiprocess.md. For example, that doc hasn't been updated since July 2019 (no criticism intended but this may be due to docs being lesser priority, the headache of nit reviews, or merely the confusion of which location certain information should go.)

Policy on linking to external URLs from docs

Ideally we only want to link to resources that will still be there in 5-10 years. Bitcoin StackExchange, @kanzure transcript site, PRs etc have stood test of time in terms of persistent links and I am confident that Bitcoin Core PR review club, Bitcoin Optech, Blockchain Commons will too.

If we start splitting up longer docs we can potentially draw in docs like some of @jonatack docs that I regularly direct people to rather than the Bitcoin Core docs. Or this blog post from @luke-jr on how to securely install Bitcoin Core. Or some of the related dev content on Stack Exchange such as this on multiprocess.

These are personal views and I welcome thoughts and feedback (NACKs welcome).

I'm going to use this issue to start collecting lists of docs that need adding, improving (for now)

New user docs needed

• Backups and restore (https://github.com/bitcoin/bitcoin/issues/20149)
• Fee estimations, fee settings (https://github.com/bitcoin/bitcoin/issues/20087)
• How to securely install Bitcoin based on this by @luke-jr

Existing user docs that need improving

New dev docs needed

• Debugging based on this by @fjahr
• Functional test framework README based on this by @glozow (there is a functional test README but not a functional test framework README)

Existing dev docs that need improving

• Split up developer notes into separate files (NACK'ed)
• Move most of functional test docs from the test README to the functional test README (See #19013 (comment))

• Functional test README based on this by @glozow

I wasn't sure what would be appropriate for the README but I'll open a PR :)

I do wonder if it should be a new doc under a new developer directory. It probably should. Don't want a really long functional test README....

If you do open a PR before I set up that new directory, I'm thinking bitcoin/doc/developer/functional-test-framework.md etc

Ok I will wait for you instead then, actually :salute: and feel free to use my personal doc

Is there anything wrong with the existing readmes? https://github.com/bitcoin/bitcoin/tree/master/test#running-tests-locally

Having two in different places will be more confusing than one. Also, moving the readme will break any (internal) and external links.

Definitely not moving the READMEs. Just resist making them much longer and instead link to other standalone files.

I think I'll ditch the idea of a developer directory. Just add more files to the doc directory.

* Split up developer notes into separate files

Just one opinion, but I'm not in favor of breaking doc/developer-notes.md into more files. It's handy to be able to look in that one file.

There is also doc/productivity.md that, along with CONTRIBUTING.md and doc/developer-notes.md, represent a sort of getting-started trinity. I find that the fewer files needed to be read to initially get started, the better. For now I suppose that includes those three, plus the unit test and functional test README files and the build doc for your OS.

If we start splitting up longer docs we can potentially draw in docs like some of @jonatack docs that I regularly direct people to rather than the Bitcoin Core docs. Or this blog post from @luke-jr on how to securely install Bitcoin Core. Or some of the related dev content on Stack Exchange such as this on multiprocess.

Thanks. I maintain and update it fairly regularly and am not sure the maintainers would want that additional update and review burden here. The content is also fairly subjective, so I'm not suggesting to move it into the repo. I don't know, though.

Did you read doc/developer-notes.md from beginning to end when you were geting started? :) Haha making edits or keeping that doc up to date seems like a nightmare to me. I avoid it like the plague.

Good to know though. Any NACKs and it ain't happening.

Having docs on personal websites, personal gists and Medium seems suboptimal to me. Especially if they are superior to the docs in this repo.

Hehe :) I search in developer-notes.md pretty often; it's the repo doc I consult the most frequently.

Some more discussion on the docs on #23154. @MarcoFalke:

While this might be fine to merge, I disagree with the sentiment that this should be merged because it "is just documentation and nothing can go wrong". Anything can go wrong if the documentation is wrong. For example, loss of privacy with the wrong settings or loss of funds with the wrong wallet handling documentation.

Also, I am wondering about the audience of this doc.

Our current doc/ folder is a mess, mixing dev docs with user docs, without any structure. Surely having documentation is important, but I suspect that unstructured documentation is potentially worse than no documentation at all. @ryanofsky:

I am wondering who is the audience of this document? [...]

I think the concerns here go a little beyond scope of this PR. Some of the files in the doc/ folder are clearly targeted at developers and not helpful to users like https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md. Some files like https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md and the release notes are mostly useful for end users, and definitely intended to be read by them. I think ideally someone would go through the doc folder, keeping the files that are intended to be read by users but moving the other ones that are intended for developers/testers/contributors somewhere else like doc/devel,doc/internal, doc/contrib, doc/design.

This document is a basically a design document (other examples: golang, chromium) intended to by read by developers. It should be useful not just to people reviewing the PR, but also anybody going forward who want to understand how this feature which touches different parts of the source code is implemented and what some of design considerations are.

At the top of this file, there is a small amount of information that could be useful to users, but it is brief and basically just linking to RPCs and a script. If bitcoin had a user manual like many other projects do, this section could be copied there. But bitcoin doesn't have a user manual so, 🤷

If it is a developer, it might be better to put the documentation in the source code. Otherwise it will be hard for developers to find it and it will more easily get out of date.

I think this could be addressed by organizing the doc directory a little better and linking to the document from the source code so it is easier to find.

If the target audience is a user, I am wondering why internal implementation details are mentioned and why they are relevant to the user.

Users are not target audience here, AFAICT.

If the target audience is a reviewer of your pull request, I am wondering why this needs a doc at all in the source tree. Wouldn't a pull request description or so be a better place?

I think this is a false dichotomy. If I am reviewing a pull request that includes end-user release notes, those release notes are usually more helpful to me for understanding behavior of the PR than any of the weedsy implementation details in the PR description or other parts of the diff.

PR descriptions do tend to be better places than design documents for certain information, like describing details or drawbacks of previous code, but design documents are useful both for current developers reviewing changes, and for future developers want to understand how a feature works. It is not an either/or.

Closing for now, but feel free to continue discussion or reopen