Cannot sync with limited RAM and Swap #24700

<!-- This issue tracker is only for technical issues related to Bitcoin Core. General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com. For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/. If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! Any report, issue or feature request related to the GUI should be reported at https://github.com/bitcoin-core/gui/issues/ -->

<!-- Describe the issue -->

While syncing headers with bitcoin-qt my device runs out of memory. I have a computer with only 1GB of RAM and no swap; while syncing headers it runs out of memory. Approximately, 600,000 headers is where it runs out.

<!--- What behavior did you expect? -->

I expected to be able to sync all the headers and then start downloading/verifying the blockchain.

<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->

While syncing the headers my device runs out of memory.

<!--- How reliably can you reproduce the issue, what are the steps to do so? -->

I can reproduce this issue every time. Simply, start the bitcoin-qt application and wait until it runs out of memory at approximately 600,000 headers.

<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->

debian package 22.0-1

<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->

Orange Pi One http://www.orangepi.org/orangepione/ CPU: H3 Quad-core Cortex-A7 H.265/HEVC 4K Drive: 256GB microSD

<!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->

<!-- Any extra information that might be useful in the debugging process. -->

<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

This issue is probably caused because all the headers are stored in RAM.

Eventually this will become a problem with common lower end computers even if they have swap when the index becomes too large.

It is an important issue to consider so that cheap nodes can be run in third world countries to ensure good decentralization of the network.

Do you have a wallet or indexes enabled? What are the settings? On my system bitcoin-qt uses about 400 MB with default settings.

@MarcoFalke Default settings, no wallet. The blockchain has not synced at all yet, it is just syncing the headers.

I would suggest looking at sudo dmesg | less output. Search for oom-kill, before that there will be full process list at that point, with information about memory consumption for each process. But it's likely a lot of RAM used by GUI stuff (as bitcoin-qt was mentioned). For such memory constrained devices, no-GUI command-line only would be better.

Did you follow the steps in https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-memory.md ?

The default settings are geared toward normal consumer PCs. If you run on especially low-end hardware you might need to tweak the settings.

The symptoms sound similar to #24760

@laanwj I have tried the reduce memory options with no success. @MarcoFalke It may be similar, but there are a lot of details that I am unfamiliar with.

@BebeSparkelSparkel is this still an issue for you with 24.0.1?

I just tried to replicate with a Lubuntu VM using 1GB RAM and the headers all synced fine a number of times in a row. I set my machine up as close as I thought I could to yours with:

$ sudo swapoff -a
$ grep MemTotal /proc/meminfo
MemTotal:			992260 kB
$ wget https://bitcoincore.org/bin/bitcoin-core-24.0.1/bitcoin-24.0.1-x86_64-linux-gnu.tar.gz
$ tar -xvzf bitcoin-core-24.0.1
$ ./bitcoin-core-24.0.1/bin/bitcoind

Typical debug.log

Perhaps this got inadvertently fixed by #25717 ?

To rule out that this was caused by bitcoin-qt, @BebeSparkelSparkel could also try to run with just bitcoind?

Ah good catch, I missed he was running bitcoin-qt (and foolishly assumed on a low powered device like that it would be bitcoind)... In the meantime I can re-test with bitcoin-qt.

Update: bitcoin-qt has passed headers sync and entered into IBD on the same 1GB machine with swap disabled.

I think therefore unless @BebeSparkelSparkel confirms this is still an issue for him on his hardware we can close this one out soon.

Also, @BebeSparkelSparkel was running the debian package, which is compiled differently. And on arm64, not x86.

But yeah closing for now. Please let us know if this is still an issue or if this should be re-opened.

If you still have the arm machine running, it would be good to re-check with the binaries from https://bitcoincore.org/en/download/

I will try again when I get the chance and let you know.