Bitcoin seems to leak dynamic memory big time #6833

After starting bitcoin as a daemon it takes about 1 GB of memory from the 3 GB my VPS has available. But some 3 hours later is has occupied an additional 1 GB and this continues creeping down until only hardly anything has been left available. Five hours after start: only 59MB left of initial free 2 GB.

Environment: VPS with 64 bit hardware and 32 bit Debian, compiler gcc 4.9.3 Recent libboost and libevent. No GUI as it runs on a Virtual Private Server. Includes one wallet. Is limited to 50 external connections. Home compiled bitcoin-master (12/10/2015, v 119900, protocol v 70011). Have also tried the current 32 bit binary release. Same effect. Is current in having the database with blocks (379012 15/10/2015).

Have seen reports in this sense before but it was discarded as an effect of getting the block database current. But that cannot be it. Are others seeing this and what can be done?

This is likely due to the recent high volume of transactions filling the memory pool.

The recently released 0.11.1 versio changes the default relay limit to temporarily combat this. You can also change this in the config file or command line.

For 0.12, a more thorough solution that limits the size of the mempool is being worked on.

You can check the mempool with getmempoolinfo.

If you don't want to upgrade (for whatever reason), AFAIK -minrelaytxfee=0.00005 as a CLI or config option should put you in line with #6793 as @sipa mentioned.

edit: bitcoin, not satoshis

thanks for your reactions and tips. My setting was 'minrelaytxfee = 0.00001'. Will study the effect of increasing that and report. Result: with setting of 5500 Satoshi for relay tx fee the memory still goes to next to nothing. it has only 22 MB left now. So this relay issue may be one of the problems but cannot discard memory leaks in the software yet. Looking via 'top' to the memory use per pid after restarting bitcoin: at first very high CPU usage but after a minute or so that reduces to typically 2-5 % of CPU.

After appr. 1 minute memory usage by Bitcoin is 7% After 3 hours it is 35% and all that is available. After 7 hours 'top' reports 43% memory for bitcoin.

Output of 'getmempoolinfo' after 7 hours: Key Value size 3118 bytes 7224457 usage 12790064

Any ideas what else to try to make Bitcoin behave memory-wise?

@wwortel probably best just to data dump (graphs are good too) of memory usage over time to show case if this is an actual issue, and maybe compare with the different options.

@dcousens: Tnx. The results show that disabling the wallet only delays the occupation of all available memory by a little. In both cases, with and without wallet, in the end only a few tens of k of RAM memory are left free. It takes between 1.5 and 2 hours for bitcoind to have confiscated appr. 1 GB of RAM.

It looks like the graphs above don't agree with your initial statement?

After starting bitcoin as a daemon it takes about 1 GB of memory from the 3 GB my VPS has available. But some 3 hours later is has occupied an additional 1 GB and this continues creeping down until only hardly anything has been left available.

3 hours later, only 1GB is still in use (~30%)?

It depends on the amount of memory becoming available after having stopped bitcoind. That was more the first time than later times. As the VPS is live for other processes cannot do a reboot all the time. First time it started from ~2 GB given as free. Or is there a command to make any unused memory available again without a reboot? Doubt that it makes any difference. 1 or 2 GB, bitcoind will eat it all. Setting the min. tx relay fee to 5.0 E-5 has not stopped this memory hunger. With the 1 GB taken by bitcoind in the graphs above there is nothing left for other programmes. What I find strange is that when I run 'top' and sort for memory use %MEM that, apart from the thirty-some percent taken by bitcoind, no other processes claim the rest. A few 0.1 and 0,2 % for things like httpd but nothing that amounts up to the other 70%.

Doubt that it makes any difference. 1, 2, or three GB, bitcoind will eat it all.

How are you setting these caps? Can you show the data for the 3 GB case? I'm not seeing this behaviour on my instance at all.

Have just rebooted the VPS and it reports 2.9 GB free memory now with everything (httpd, bind9, sendmail, stunnel, saslauth) running except bitcoind. That 2.9 value is stable. W.r.t. the caps: did not set anything. 3 GB is what the VPS is provided with by the company that rents it to me. But it seems that running bitcoind, apart from taking all that memory, also creates a situation in which not all memory that bitcoind is reported to be using is released afterwards. It becomes like orphan memory. Will start bitcoind again and the %MEM will be logged every minute.

Have restarted bitcoind without the start-stop-daemon option --background and am watching the messages it gives out. There are continuously messages of the kind: receive version message: /BitCoinJ:0.12-SNAPSHOT/Satoshi:0.11.0/: version 70001, blocks=379431, us=127.0.0.1:8333, peer=308 and sometimes things like: receive version message: /bitcoin-seeder:0.01/: version 60000, blocks=230000, us=(my-external-ip):8333, peer=323 2015-10-18 10:53:37 receive version message: /bitnodes.21.co:0.1/: version 70002, blocks=379431, us=(my-external-ip):8333, peer=324 After ~40 minutes it has already passed peer=400. Is this normal behaviour or is something running away?

did you modify the dbcache parameter?

@pstratem: no, it is the default and not set in .conf

But after rebooting the VPS now the %MEM has stabilized at 21.7 %. This level was reached after 2.5 hours and the following hour has not changed. Free memory is reported low but that is because 2G of 3G are used as buffer/cache. Will now experiment with stopping and restarting bitcoind to see if it leaves availability of memory problems behind.

As I cannot reproduce the original problem (after closing bitcoind all cache can be released and reused) I close this issue. It must have been an anomaly in the VPS.