libdb_cxx-4.8 leaks? #5380

issue jonasschnelli openend this issue on November 26, 2014
  1. jonasschnelli commented at 3:46 pm on November 26, 2014: contributor

    I just stumbled over some (many) mem leaks from libdb_cxx-4.8. Local build of current master. Tested on OSX 10.10.

    Does someone has a valgrind setup ready and can test this on a non-OSX system?

    bildschirmfoto 2014-11-26 um 16 44 36

  2. laanwj commented at 9:29 am on December 1, 2014: member
    So are these just one-time leaks or do they accumulate as the application is running?
  3. laanwj added the label Bug on Dec 1, 2014
  4. jonasschnelli commented at 11:30 am on December 1, 2014: contributor
    I wasn’t looking very deep into this. But it seams that the it’s leaking more and more… I will re-test, re-analyze it soon. I also tried in valgrind in a Ubuntu VM… horrible performance, but also detected some things. Need to create a better setup and will re-analyze soon.
  5. jonasschnelli commented at 8:09 am on December 5, 2014: contributor

    Just did a valgrind session on ubuntu. LevelDB or the bitcoind usage of it seams to produce multiple leaks (not to underestimate!). I assume bitcoind produces some serious mem leaks.

    Here is a session dump: this is a -regtest only session with a empty regtest dir with no further block creations. No nodes connected. http://paste.ubuntu.com/9379050/

  6. laanwj commented at 8:40 am on December 5, 2014: member
    Neither of those in the pasted log seems related to leveldb. Just BerkeleyDB and the wallet. What sizes are we talking about here? Do they grow over time? Leaking some one-time setup structures is unclean, but hardly a critical issue.
  7. jonasschnelli commented at 8:45 am on December 5, 2014: contributor

    Sorry. The pastbin is somehow useless. Here is a valgrind output of a very short testnet session with downloading some 100 blocks (see below). It seams to be not a huge problem on ubuntu.

    The definitive lost come from the wallet db (bdb) and might be fixed without huge effort. The still reachable blocks (which grow largely over time) are probably coming out of leveldb. But could be because of leveldb’s internals and might be okay.

    0==20446== LEAK SUMMARY:
1==20446==    definitely lost: 96 bytes in 1 blocks
2==20446==    indirectly lost: 0 bytes in 0 blocks
3==20446==      possibly lost: 152 bytes in 1 blocks
4==20446==    still reachable: 1,708 bytes in 11 blocks
5==20446==         suppressed: 0 bytes in 0 blocks
6==20446== Reachable blocks (those to which a pointer was found) are not shown.
7==20446== To see them, rerun with: --leak-check=full --show-leak-kinds=all
  8. jonasschnelli commented at 9:04 am on December 5, 2014: contributor
    Sorry. Mix it up! LevelDB is clean. It’s just the BDB Wallet who is leaking. I try to go behind it and fix those.
  9. paveljanik commented at 4:50 pm on January 7, 2015: contributor

    After a few minutes of running bitcoind on testnet, with wallet (-disablewallet -> no leaks):

     0xxx $ leaks bitcoind | grep "Call stack" | sort | uniq -c | sort -rn
 180895   Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:791 | CDBEnv::Open(boost::filesystem::path const&) db.cpp:94 | DbEnv::open(char const*, unsigned int, int) | __env_open | __env_attach_regions | __lock_open | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 2  44    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1103 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CWalletDB::LoadWallet(CWallet*) walletdb.cpp:635 | CDB::ReadAtCursor(Dbc*, CDataStream&, CDataStream&, unsigned int) .db.h:244 | Dbc::get(Dbt*, Dbt*, unsigned int) | __dbc_get_pp | __dbc_iget | __bamc_get | __bamc_next | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 3   7    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:150 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_common | __db_vrfy_putpageinfo | __db_put | __dbc_put | __dbc_iput | __bamc_put | __bam_split | __db_new | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 4   4    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:150 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_open | __db_new_file | __bam_new_file | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 5   3    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:150 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_open | __env_setup | __env_mpool | __memp_fopen | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 6   2    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:150 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_vrfy_pgset | __db_open | __db_new_file | __bam_new_file | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 7   2    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:150 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __bam_vrfy_structure | __bam_vrfy_subtree | __bam_vrfy_subtree | __db_vrfy_pgset_inc | __db_put | __dbc_put | __dbc_iput | __bamc_put | __bam_split | __db_new | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 8   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:791 | CDBEnv::Open(boost::filesystem::path const&) db.cpp:94 | DbEnv::open(char const*, unsigned int, int) | __env_open | __env_attach_regions | __memp_open | __memp_init | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 9   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1103 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CWalletDB::LoadWallet(CWallet*) walletdb.cpp:635 | CDB::ReadAtCursor(Dbc*, CDataStream&, CDataStream&, unsigned int) .db.h:244 | Dbc::get(Dbt*, Dbt*, unsigned int) | __dbc_get_pp | __dbc_iget | __db_goff | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
10xxx $

    I have tried bitcoind with db-6.1.19.NC and --with-incompatible-bdb and it is much better (the first type of leaks is resolved, but two new “invented”):

     0  49    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CWalletDB::LoadWallet(CWallet*) walletdb.cpp:635 | CDB::ReadAtCursor(Dbc*, CDataStream&, CDataStream&, unsigned int) .db.h:244 | Dbc::get(Dbt*, Dbt*, unsigned int) | __dbc_get_pp | __dbc_iget | __bamc_get | __bamc_next | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 1   7    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_common | __db_vrfy_putpageinfo | __db_put | __dbc_put | __dbc_iput | __bamc_put | __bam_split | __db_new | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 2   4    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_open | __bam_new_file | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 3   4    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CWalletDB::LoadWallet(CWallet*) walletdb.cpp:635 | CDB::ReadAtCursor(Dbc*, CDataStream&, CDataStream&, unsigned int) .db.h:244 | Dbc::get(Dbt*, Dbt*, unsigned int) | __dbc_get_pp | __dbc_iget | __db_goff | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 4   3    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_vrfy_pgset | __db_open | __env_setup | __env_mpool | __memp_fopen | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 5   3    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_open | __env_setup | __env_mpool | __memp_fopen | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 6   2    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __db_vrfy_dbinfo_create | __db_vrfy_pgset | __db_open | __bam_new_file | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 7   2    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:820 | CDBEnv::Verify(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, bool (*)(CDBEnv&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)) db.cpp:151 | Db::verify(char const*, char const*, std::__1::basic_ostream<char, std::__1::char_traits<char> >*, unsigned int) | __db_verify_internal | __db_verify | __bam_vrfy_structure | __bam_vrfy_subtree | __bam_vrfy_subtree | __db_vrfy_pgset_inc | __db_put | __dbc_put | __dbc_iput | __bamc_put | __bam_split | __db_new | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 8   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:791 | CDBEnv::Open(boost::filesystem::path const&) db.cpp:94 | DbEnv::open(char const*, unsigned int, int) | __env_open | __env_attach_regions | __memp_open | __memp_init | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
 9   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CWalletDB::LoadWallet(CWallet*) walletdb.cpp:635 | CDB::ReadAtCursor(Dbc*, CDataStream&, CDataStream&, unsigned int) .db.h:244 | Dbc::get(Dbt*, Dbt*, unsigned int) | __dbc_get_pp | __dbc_iget | __bamc_get | __bamc_search | __bam_search | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
10   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CDB::CDB(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*) db.cpp:266 | bool CDB::Exists<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) .db.h:211 | Db::exists(DbTxn*, Dbt*, unsigned int) | __db_exists | __db_get_pp | __db_get | __dbc_iget | __bamc_get | __bamc_search | __bam_search | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
11   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CDB::CDB(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*) db.cpp:266 | bool CDB::Exists<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) .db.h:211 | Db::exists(DbTxn*, Dbt*, unsigned int) | __db_exists | __db_get_pp | __db_get | __dbc_iget | __bamc_get | __bamc_search | __bam_search | __bam_get_root | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
12   1    Call stack: [thread 0x7fff771b0300]: | start | main bitcoind.cpp:185 | AppInit(int, char**) bitcoind.cpp:148 | AppInit2(boost::thread_group&) init.cpp:1104 | CWallet::LoadWallet(bool&) wallet.cpp:1652 | CDB::CDB(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*) db.cpp:251 | Db::open(DbTxn*, char const*, char const*, DBTYPE, unsigned int, int) | __db_open_pp | __db_open | __bam_read_root | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
13   1    Call stack: [thread 0x11492e000]: | thread_start | _pthread_body | _pthread_body | boost::(anonymous namespace)::thread_proxy(void*) | ThreadFlushWalletDB(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) walletdb.cpp:840 | DbEnv::lsn_reset(char const*, unsigned int) | __env_lsn_reset_pp | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc 
14   1    Call stack: [thread 0x11492e000]: | thread_start | _pthread_body | _pthread_body | boost::(anonymous namespace)::thread_proxy(void*) | ThreadFlushWalletDB(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) walletdb.cpp:840 | DbEnv::lsn_reset(char const*, unsigned int) | __env_lsn_reset_pp | __db_open | __bam_read_root | __memp_fget | __memp_alloc | __env_alloc | __os_malloc | malloc | malloc_zone_malloc

    10s snapshots:

    0Process 63781: 81027 leaks for 9409888 total leaked bytes.
1Process 63781: 81027 leaks for 9409888 total leaked bytes.
2Process 63781: 81027 leaks for 9409888 total leaked bytes.

    10MB constantly.

    As all leaks are coming from libdb, I’d suggest closing this issue.

  10. jonasschnelli commented at 6:47 pm on January 7, 2015: contributor

    I think as long as bitcoin core produces leaks (even because of leaking subsystems) we should consider keeping this issue open. My debuggings lead me to possible forgotten database closing (but didn’t had a deep look)

    But for sure: it’s low priority.

  11. laanwj added the label Resource usage on Feb 16, 2016
  12. MarcoFalke commented at 12:35 pm on May 9, 2020: member
    I don’t think there is anything we can do to prevent leaks in the unmaintained bdb other than moving away from it. See #18916
  13. MarcoFalke closed this on May 9, 2020
  14. DrahtBot locked this on Feb 15, 2022


jonasschnelli laanwj paveljanik MarcoFalke

Labels
Bug Resource usage

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-10-04 22:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me