Cannot sync with limited RAM and Swap

BebeSparkelSparkel commented at 4:04 am on March 29, 2022: none

While syncing headers with bitcoin-qt my device runs out of memory. I have a computer with only 1GB of RAM and no swap; while syncing headers it runs out of memory. Approximately, 600,000 headers is where it runs out.

I expected to be able to sync all the headers and then start downloading/verifying the blockchain.

While syncing the headers my device runs out of memory.

I can reproduce this issue every time. Simply, start the bitcoin-qt application and wait until it runs out of memory at approximately 600,000 headers.

debian package 22.0-1

Orange Pi One http://www.orangepi.org/orangepione/ CPU: H3 Quad-core Cortex-A7 H.265/HEVC 4K Drive: 256GB microSD

This issue is probably caused because all the headers are stored in RAM.

Eventually this will become a problem with common lower end computers even if they have swap when the index becomes too large.

It is an important issue to consider so that cheap nodes can be run in third world countries to ensure good decentralization of the network.

fanquake added the label Resource usage on Mar 29, 2022

bitcoin deleted a comment on Mar 29, 2022

maflcko commented at 7:45 am on March 29, 2022: member

Do you have a wallet or indexes enabled? What are the settings? On my system bitcoin-qt uses about 400 MB with default settings.

maflcko added the label Questions and Help on Mar 29, 2022

BebeSparkelSparkel commented at 3:55 am on March 31, 2022: none

@MarcoFalke Default settings, no wallet. The blockchain has not synced at all yet, it is just syncing the headers.

kristapsk commented at 4:09 am on March 31, 2022: contributor

I would suggest looking at sudo dmesg | less output. Search for oom-kill, before that there will be full process list at that point, with information about memory consumption for each process. But it’s likely a lot of RAM used by GUI stuff (as bitcoin-qt was mentioned). For such memory constrained devices, no-GUI command-line only would be better.

laanwj commented at 11:10 am on April 6, 2022: member

Did you follow the steps in https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-memory.md ?

The default settings are geared toward normal consumer PCs. If you run on especially low-end hardware you might need to tweak the settings.

maflcko commented at 11:29 am on April 6, 2022: member

The symptoms sound similar to #24760

bitcoin deleted a comment on Apr 9, 2022

bitcoin deleted a comment on Apr 27, 2022

BebeSparkelSparkel commented at 3:55 am on May 1, 2022: none

@laanwj I have tried the reduce memory options with no success. @MarcoFalke It may be similar, but there are a lot of details that I am unfamiliar with.

willcl-ark commented at 4:13 pm on March 24, 2023: member

@BebeSparkelSparkel is this still an issue for you with 24.0.1?

I just tried to replicate with a Lubuntu VM using 1GB RAM and the headers all synced fine a number of times in a row. I set my machine up as close as I thought I could to yours with:

0$ sudo swapoff -a
1$ grep MemTotal /proc/meminfo
2MemTotal:			992260 kB
3$ wget https://bitcoincore.org/bin/bitcoin-core-24.0.1/bitcoin-24.0.1-x86_64-linux-gnu.tar.gz
4$ tar -xvzf bitcoin-core-24.0.1
5$ ./bitcoin-core-24.0.1/bin/bitcoind

Typical debug.log

Perhaps this got inadvertently fixed by #25717 ?

maflcko commented at 4:41 pm on March 24, 2023: member

To rule out that this was caused by bitcoin-qt, @BebeSparkelSparkel could also try to run with just bitcoind?

willcl-ark commented at 4:43 pm on March 24, 2023: member

Ah good catch, I missed he was running bitcoin-qt (and foolishly assumed on a low powered device like that it would be bitcoind)… In the meantime I can re-test with bitcoin-qt.

willcl-ark commented at 4:51 pm on March 24, 2023: member

Update: bitcoin-qt has passed headers sync and entered into IBD on the same 1GB machine with swap disabled.

I think therefore unless @BebeSparkelSparkel confirms this is still an issue for him on his hardware we can close this one out soon.

maflcko commented at 4:58 pm on March 24, 2023: member

Also, @BebeSparkelSparkel was running the debian package, which is compiled differently. And on arm64, not x86.

But yeah closing for now. Please let us know if this is still an issue or if this should be re-opened.

If you still have the arm machine running, it would be good to re-check with the binaries from https://bitcoincore.org/en/download/

maflcko closed this on Mar 24, 2023

BebeSparkelSparkel commented at 3:59 am on May 23, 2023: none

I will try again when I get the chance and let you know.

bitcoin locked this on May 22, 2024

Cannot sync with limited RAM and Swap #24700