Reorganize Bitcoin Core directory structure for easier database management #12145

issue Ulmo opened this issue on January 11, 2018
  1. Ulmo commented at 10:27 AM on January 11, 2018: none

    <!-- This issue tracker is only for technical issues related to Bitcoin Core. General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com. For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/. If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! -->

    <!-- Describe the issue -->

    I suspect that the subdirectory blocks/index is accessed more frequently and does better on a fast local disk even if the rest of the blocks subdirectory is on a slower disk. The symlinks required for that are a bit obscene from a maintenance perspective, so I'd like you to reorganize the subdirectories of Bitcoin to unnest the more frequently accessed files directories from the less frequently accessed larger things, i.e., either move blocks/index into index (e.g., ~/.bitcoin/index rather than ~/.bitcoin/blocks/index), or move blocks/*.* into a new subdirectory blocks/data and you can leave blocks/index where it is.

    <!--- What behavior did you expect? -->

    Less difficulty managing apportionment of large subdirectories and frequently accessed files onto different speed disks

    <!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->

    A canonical root-based symlink was required, which can offer difficulties in maintenance in the future.

    <!--- How reliably can you reproduce the issue, what are the steps to do so? -->

    Oh gosh. Must you with the extra unnecessary questions?

    <!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->

    git as of January 11, 2018. (git has no manual, so I have no idea which version control system version number.)

    <!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->

    Presumably platform independent, but Mac.

    <!-- Any extra information that might be useful in the debugging process. -->

    <!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

    P.S., did you know * is only able to be typed in this Github post with the text \*?

  2. jsarenik commented at 1:07 PM on January 12, 2018: none
    $ git --version
    git version 2.15.0
    

    As far as I know and just proved on Linux with ext4 filesystem (but should be the same on Mac I guess), there is no overhead accessing a particular file in a directory full of thousands of other files. It takes a while to list such a big directory, but for accessing a particular file which name is known (like index) there is no need to list the contents of the directory.

    I hope that helps.

    $ mkdir dirs files; seq 999999 | while read a; do mkdir dirs/$a; touch files/$a; done
    $ time ls files/ > /dev/null
    real	0m 2.71s
    user	0m 1.31s
    sys	0m 0.59s
    lomidrevo:/mnt/yocto/test$ time sh -c "echo test > files/test"
    real	0m 0.03s
    user	0m 0.00s
    sys	0m 0.00s
    lomidrevo:/mnt/yocto/test$ time cat files/test
    test
    real	0m 0.03s
    user	0m 0.00s
    sys	0m 0.00s
    
  3. Willtech commented at 1:27 AM on January 14, 2018: contributor

    @Ulmo If you are interested in git manuals, you can try Git From The Bottom Up and Pro Git. Enjoy!

  4. fanquake closed this on Jan 22, 2018

  5. DrahtBot locked this on Sep 8, 2021

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2026-04-29 09:15 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me