Creating too many wallets exhausts file descriptor limit and leads to crash #27732

issue fjahr openend this issue on May 23, 2023
  1. fjahr commented at 11:38 pm on May 23, 2023: contributor

    Is there an existing issue for this?

    • I have searched the existing issues

    Current behaviour

    This behavior was first noted by @EthanHeilman here: #11368 (comment) but it was conflated with the RPC issue there while I now think this is a separate issue.

    When a lot of wallets are created in quick succession, sqlite will exhaust all the available file descriptors and the node crashes the next time it tries to create file for unrelated reasons (blk, rev etc.).

    Expected behaviour

    The node shouldn’t crash.

    Steps to reproduce

    0for i in $(seq 1 100); do src/bitcoin-cli createwallet "fd-wallet$i"; done

    Relevant log output

    https://0bin.net/paste/swIJVCaU#CuCCWv7cC4ZscxjCEiauqEhB1icPiMHkYo+D5dC8mSr

    How did you obtain Bitcoin Core

    Compiled from source

    What version of Bitcoin Core are you using?

    master

    Operating system and version

    macOS

    Machine specifications

    No response

  2. maflcko added the label Wallet on May 24, 2023
  3. dimitaracev commented at 6:53 pm on June 13, 2023: contributor
    I can take this if still available.
  4. fjahr commented at 9:57 pm on June 13, 2023: contributor

    I can take this if still available.

    Feel free to investigate, i am not working on this currently.

  5. dimitaracev commented at 2:09 pm on June 14, 2023: contributor
    Unfortunately I was not able to reproduce this after multiple tests. Do you maybe have a more specific scenario where the issue would occur?
  6. fjahr commented at 2:48 pm on June 14, 2023: contributor

    Unfortunately I was not able to reproduce this after multiple tests. Do you maybe have a more specific scenario where the issue would occur?

    I don’t really have any more specifics to give. This was the latest master at the time, compiled only with standard flags and no other wallets present etc. I reproduced the issue 4-5 times consistently with script given above. What OS are you on? If you are not on macOS maybe you have to lower the limit of allowed file descriptors.

  7. dimitaracev commented at 2:53 pm on June 14, 2023: contributor
    I’m on an ArchLinux installation, and I see that the number of allowed file descriptors is pretty high which is probably why I can’t reproduce it. Will lower them and try again, thanks.
  8. maflcko commented at 3:28 pm on June 14, 2023: member

    Relevant log output … 0bin

    If you upload logs, it could help to either put them in the comment (maybe via <details><summary></summary></details>) or by uploading it to the same site that hosts the comment. Otherwise the external site is just another thing that can go down or is unreachable, which is the case here for me at least.

  9. fjahr commented at 3:33 pm on June 14, 2023: contributor

    If you upload logs, it could help to either put them in the comment (maybe via <details><summary></summary></details>) or by uploading it to the same site that hosts the comment. Otherwise the external site is just another thing that can go down or is unreachable, which is the case here for me at least.

    GitHub didn’t allow me to paste the logs in a comment because they were too big…

  10. fjahr commented at 3:34 pm on June 14, 2023: contributor
    Trying with attached file… wallet_exhaust.txt
  11. fjahr commented at 3:35 pm on June 14, 2023: contributor
    Ok, seems to have worked. But when I created the issue it failed.
  12. maflcko commented at 3:37 pm on June 14, 2023: member

    Thanks, seems like the relevant short excerpt is:

     02023-05-23T22:32:53Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/FJ/Library/Application Support/Bitcoin/testnet3/wallets/fd-wallet99/wallet.dat-journal) - Too many open files
 12023-05-23T22:32:53Z SQLite Error. Code: 14. Message: statement aborts at 1: [BEGIN EXCLUSIVE TRANSACTION] unable to open database file
 22023-05-23T22:32:53Z Using SQLite Version 3.39.5
 32023-05-23T22:32:53Z Using wallet /Users/FJ/Library/Application Support/Bitcoin/testnet3/wallets/fd-wallet100
 42023-05-23T22:32:53Z SQLite Error. Code: 14. Message: cannot open file at line 46922 of [554764a6e7]
 52023-05-23T22:32:53Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/FJ/Library/Application Support/Bitcoin/testnet3/wallets/fd-wallet100/wallet.dat-journal) - Too many open files
 62023-05-23T22:32:53Z SQLite Error. Code: 14. Message: statement aborts at 1: [BEGIN EXCLUSIVE TRANSACTION] unable to open database file
 72023-05-23T22:33:38Z UpdateTip: new best=00000000607ba1efbd078e5a305b832ae14a6c444b5aa9b557db4982cca74aed height=1148054 version=0x20000000 log2_work=69.251056 tx=14198802 date='2017-06-24T09:42:34Z' progress=0.216198 cache=45.4MiB(394627txo)
 82023-05-23T22:33:38Z UpdateTip: new best=00000000b7e5ec444dfcf9cff739783eb0550c692ff1c66cfe37af6374fbaad6 height=1148055 version=0x20000000 log2_work=69.251056 tx=14199082 date='2017-06-24T10:02:36Z' progress=0.216202 cache=50.5MiB(414075txo)
 92023-05-23T22:33:39Z UpdateTip: new best=0000000013869d488791f4961dcf1b1f93f050b0a787616359b6355a70701e86 height=1148056 version=0x20000000 log2_work=69.251056 tx=14199702 date='2017-06-24T10:29:53Z' progress=0.216212 cache=50.8MiB(414298txo)
102023-05-23T22:33:39Z New outbound peer connected: version: 70015, blocks=2435012, peer=6 (outbound-full-relay)
112023-05-23T22:33:39Z UpdateTip: new best=0000000086afcb0260af2c4f609535db645a0f765484d389834c0309030eabcb height=1148057 version=0x20000000 log2_work=69.251056 tx=14200140 date='2017-06-24T10:52:00Z' progress=0.216219 cache=50.8MiB(414672txo)
122023-05-23T22:33:39Z UpdateTip: new best=000000009be200143cc507115d9a57504241f80536b0e9fa0a304fc9f13a7f14 height=1148058 version=0x00000004 log2_work=69.251056 tx=14200141 date='2017-06-24T11:15:47Z' progress=0.216219 cache=50.8MiB(414674txo)
132023-05-23T22:33:39Z New outbound peer connected: version: 70016, blocks=2435012, peer=5 (outbound-full-relay)
142023-05-23T22:33:39Z UpdateTip: new best=000000000e0e1b8cc908edb67e8a9329a256b162ab45a8bbfe83a9773fe32bbd height=1148059 version=0x20000000 log2_work=69.251056 tx=14200255 date='2017-06-24T11:39:07Z' progress=0.216220 cache=50.8MiB(414513txo)
152023-05-23T22:34:26Z UpdateTip: new best=000000003aae175dee028b660af7116bf8fa66ad2934845e961b9839a18821d8 height=1148060 version=0x20000000 log2_work=69.251056 tx=14200492 date='2017-06-24T12:00:07Z' progress=0.216224 cache=50.8MiB(414378txo)
162023-05-23T22:34:26Z UpdateTip: new best=00000000f7bd2dbd40619d3a20755603f010012af19f5aafbbab46ae374160db height=1148061 version=0x20000000 log2_work=69.251056 tx=14201162 date='2017-06-24T12:20:08Z' progress=0.216234 cache=50.8MiB(414715txo)
172023-05-23T22:34:26Z UpdateTip: new best=0000000003110e54d36f4113fc94f46a4463e5da18eb356e4cc0d67bb2fa79c5 height=1148062 version=0x20000000 log2_work=69.251056 tx=14201706 date='2017-06-24T12:57:58Z' progress=0.216242 cache=50.8MiB(414666txo)
182023-05-23T22:34:26Z UpdateTip: new best=000000003b71aa64a460b2f7ebe40a959ed3e9c2fe02af8df4a99fda44efe906 height=1148063 version=0x20000000 log2_work=69.251056 tx=14202041 date='2017-06-24T13:19:45Z' progress=0.216247 cache=50.8MiB(414737txo)
192023-05-23T22:34:26Z New outbound peer connected: version: 70016, blocks=2435012, peer=7 (outbound-full-relay)
202023-05-23T22:34:26Z New outbound peer connected: version: 70016, blocks=2435012, peer=8 (outbound-full-relay)
212023-05-23T22:34:26Z Synchronizing blockheaders, height: 2435012 (~100.00%)
222023-05-23T22:34:26Z UpdateTip: new best=00000000a41344a056a034e823dd4108d6186f9d61449354c29d617f5222bad3 height=1148064 version=0x20000000 log2_work=69.251056 tx=14202449 date='2017-06-24T13:56:46Z' progress=0.216254 cache=50.8MiB(414869txo)
232023-05-23T22:34:26Z UpdateTip: new best=00000000823daf27e85da59baa4c6a6b90d98283d45f425c84351592dc7655df height=1148065 version=0x20000000 log2_work=69.251056 tx=14202845 date='2017-06-24T14:20:57Z' progress=0.216260 cache=52.3MiB(428380txo)
242023-05-23T22:34:51Z Unable to open file /Users/FJ/Library/Application Support/Bitcoin/testnet3/blocks/rev00076.dat
252023-05-23T22:34:51Z ERROR: UndoWriteToDisk: OpenUndoFile failed
262023-05-23T22:34:51Z *** Failed to write undo data
272023-05-23T22:34:51Z Error: A fatal internal error occurred, see debug.log for details
282023-05-23T22:34:51Z tor: Thread interrupt
292023-05-23T22:34:51Z addcon thread exit
302023-05-23T22:34:51Z Shutdown: In progress...
312023-05-23T22:34:51Z opencon thread exit
322023-05-23T22:34:51Z torcontrol thread exit
332023-05-23T22:34:51Z ERROR: ConnectTip: ConnectBlock 00000000c101ef80a98b3e6e11d925578c02512f37731838a911591e7e64b858 failed, Failed to write undo data
342023-05-23T22:34:51Z ERROR: ProcessNewBlock: ActivateBestChain failed (Failed to write undo data)
352023-05-23T22:34:51Z msghand thread exit
362023-05-23T22:34:51Z net thread exit
372023-05-23T22:34:51Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat started
382023-05-23T22:34:51Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat completed (0.00s)
392023-05-23T22:34:52Z scheduler thread exit
402023-05-23T22:34:52Z Writing 0 unbroadcast transactions to disk.
412023-05-23T22:34:52Z Dumped mempool: 4.75e-05s to copy, 0.00576754s to dump
  13. dimitaracev commented at 7:15 pm on June 14, 2023: contributor

    Unfortunately I was not able to reproduce this after multiple tests. Do you maybe have a more specific scenario where the issue would occur?

    I don’t really have any more specifics to give. This was the latest master at the time, compiled only with standard flags and no other wallets present etc. I reproduced the issue 4-5 times consistently with script given above. What OS are you on? If you are not on macOS maybe you have to lower the limit of allowed file descriptors.

    Okay, I was able to reproduce it when lowering the file descriptors and will look further into it. Interesting enough, in my case it doesn’t crash the node but the file descriptors seem to be locked, and can’t load the already created wallets.

     02023-06-14T19:22:21Z [fd-wallet1] Releasing wallet
 12023-06-14T19:22:22Z Using SQLite Version 3.40.1
 22023-06-14T19:22:22Z Using wallet /home/dimitaracev/.bitcoin/regtest/wallets/fd-wallet1
 32023-06-14T19:22:22Z init message: Loading wallet…
 42023-06-14T19:22:22Z [fd-wallet1] Wallet file version = 10500, last client version = 259900
 52023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = 5e306353541637e7a8a5544e5b257ba8f1d3c488f2e9fd7cbf8b050742e3ec8d, type = legacy, internal = false
 62023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = 46059f1ecfbe2666420840110eb03da53c829a2a1296bb63c64d5daf562d070b, type = p2sh-segwit, internal = false
 72023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = e60db0e290b20b41cccd1c62b47120159af0e747b5829eb611118c6f053df157, type = bech32, internal = false
 82023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = f706629e5c33dfe527c2f5ec6e2c0f0d36f3cc75f6e56b711554198e9120d2ca, type = bech32m, internal = false
 92023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = 6c17557bdb8f060982bd9478d20c33989fec840f5fca6104f5c9b77ae2000425, type = legacy, internal = true
102023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = 6120ab62678492d7f653409175de0ac0bfbab67f7501b2387533477c8a14ee17, type = p2sh-segwit, internal = true
112023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = c9bb633eaf2982beb1711e977e8e679374160a898cf392a886f81ac41829c5e7, type = bech32, internal = true
122023-06-14T19:22:22Z [fd-wallet1] Setting spkMan to active: id = 15400e0ed8a94b5e6b245b991e78dc2f467873cea42ee508d4a3563bdb6456f1, type = bech32m, internal = true
132023-06-14T19:22:22Z [fd-wallet1] Keys: 8 plaintext, 0 encrypted, 0 w/ metadata, 8 total. Unknown wallet records: 0
142023-06-14T19:22:22Z SQLite Error. Code: 14. Message: cannot open file at line 42907 of [df5c253c0b]
152023-06-14T19:22:22Z SQLite Error. Code: 14. Message: os_unix.c:42907: (24) open(/home/dimitaracev/.bitcoin/regtest/wallets/fd-wallet1/wallet.dat-journal) - 
162023-06-14T19:22:22Z SQLite Error. Code: 14. Message: statement aborts at 14: [INSERT or REPLACE into main values(?, ?)] unable to open database file
172023-06-14T19:22:22Z WriteKey: Unable to execute statement: unable to open database file
182023-06-14T19:22:22Z [fd-wallet1] Releasing wallet
192023-06-14T19:27:36Z CreateNewBlock(): block weight: 892 txs: 0 fees: 0 sigops 400
202023-06-14T19:27:36Z Saw new header hash=1786ef9d4d7625c83d48e6e706c2d2cd53cb73f2023ad369aa9200a5269bf87f height=405
212023-06-14T19:27:36Z UpdateTip: new best=1786ef9d4d7625c83d48e6e706c2d2cd53cb73f2023ad369aa9200a5269bf87f height=405 version=0x30000000 log2_work=9.665336 tx=406 date='2023-06-14T19:27:36Z' progress=1.000000 cache=0.3MiB(304txo)
  14. dimitaracev commented at 8:10 pm on June 14, 2023: contributor
    After a deeper look, I think the crashing may be related to how macOS handles file descriptors when you try to use more than the limit. Bitcoin Core loads the wallet immediately after you create it and it keeps that file descriptor open for easy access. You can try running bitcoin-cli listwallets and you should see all of the wallets created up until the point where you go over the file descriptor limit.
  15. fjahr commented at 11:41 am on June 18, 2023: contributor

    After a deeper look, I think the crashing may be related to how macOS handles file descriptors when you try to use more than the limit.

    It’s not how macOS handles it, any OS will cause bitcoind to crash if you use more fd than the limit. It’s just that among all the OS macOS has the lowest standard limit and so such issue are usually observed there first.

    Bitcoin Core loads the wallet immediately after you create it and it keeps that file descriptor open for easy access. You can try running bitcoin-cli listwallets and you should see all of the wallets created up until the point where you go over the file descriptor limit.

    Yes. But this doesn’t change that bitcoind can crash and ideally it shouldn’t, which is why I opened this issue.

  16. bitcoin deleted a comment on Jun 18, 2023
  17. maflcko added the label macOS on Jun 21, 2023
  18. sr-gi commented at 2:43 pm on June 22, 2023: member

    After a deeper look, I think the crashing may be related to how macOS handles file descriptors when you try to use more than the limit.

    It’s not how macOS handles it, any OS will cause bitcoind to crash if you use more fd than the limit. It’s just that among all the OS macOS has the lowest standard limit and so such issue are usually observed there first.

    Bitcoin Core loads the wallet immediately after you create it and it keeps that file descriptor open for easy access. You can try running bitcoin-cli listwallets and you should see all of the wallets created up until the point where you go over the file descriptor limit.

    Yes. But this doesn’t change that bitcoind can crash and ideally it shouldn’t, which is why I opened this issue.

    I’m able to trigger this on macOS with a slightly bigger for loop (the default fd limit for macOS is 256, so anything around that ballpark should trigger). However, looks like this is not making bitcoind crash anymore, at least not for me.

     0[...]
 12023-06-22T14:38:41Z [rpc] ThreadRPCServer method=createwallet user=user
 22023-06-22T14:38:41Z Using SQLite Version 3.39.5
 32023-06-22T14:38:41Z Using wallet /Users/sergi/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet132
 42023-06-22T14:38:41Z SQLite Error. Code: 14. Message: cannot open file at line 46922 of [554764a6e7]
 52023-06-22T14:38:41Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/sergi/.bitcoin/regtest/wallets/fd-wallet132/wallet.dat-journal) - Too many open files
 62023-06-22T14:38:41Z SQLite Error. Code: 14. Message: statement aborts at 1: [BEGIN EXCLUSIVE TRANSACTION] unable to open database file
 72023-06-22T14:38:41Z [http] Received a POST request for / from 127.0.0.1:49976
 82023-06-22T14:38:41Z [rpc] ThreadRPCServer method=createwallet user=user
 92023-06-22T14:38:41Z Using SQLite Version 3.39.5
102023-06-22T14:38:41Z Using wallet /Users/sergi/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet133
112023-06-22T14:38:41Z SQLite Error. Code: 14. Message: cannot open file at line 46922 of [554764a6e7]
122023-06-22T14:38:41Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/sergi/.bitcoin/regtest/wallets/fd-wallet133/wallet.dat-journal) - Too many open files
132023-06-22T14:38:41Z SQLite Error. Code: 14. Message: statement aborts at 1: [BEGIN EXCLUSIVE TRANSACTION] unable to open database file
142023-06-22T14:38:41Z [http] Received a POST request for / from 127.0.0.1:49977
15[...]
  19. willcl-ark commented at 11:09 am on September 15, 2023: member

    I’m able to trigger this on macOS with a slightly bigger for loop (the default fd limit for macOS is 256, so anything around that ballpark should trigger). However, looks like this is not making bitcoind crash anymore, at least not for me.

    I found similar behaviour on master@ f608a409f7591b4f5cf170898bee58b9d9dcf1b6 My system encountered errors after adding only 131 wallets, but did not crash and was able to shutdown cleanly too:

      0
  1<snip>
  2
  32023-09-15T11:02:35Z Using wallet /Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet131
  42023-09-15T11:02:35Z SQLite Error. Code: 14. Message: cannot open file at line 42184 of [554764a6e7]
  52023-09-15T11:02:35Z SQLite Error. Code: 14. Message: os_unix.c:42184: (24) openDirectory(/Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet131) - Too many open files
  62023-09-15T11:02:36Z init message: Loading wallet…
  72023-09-15T11:02:36Z [fd-wallet131] Wallet file version = 10500, last client version = 259900
  82023-09-15T11:02:36Z [fd-wallet131] Legacy Wallet Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total.
  92023-09-15T11:02:36Z [fd-wallet131] Descriptors: 0, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total.
 102023-09-15T11:02:36Z [fd-wallet131] Setting minversion to 169900
 112023-09-15T11:02:36Z [fd-wallet131] Setting spkMan to active: id = 5113ed2b9223576f1d1c3fe357d6ebe2af35519f034f609fb4c4369d2960dbab, type = legacy, internal = false
 122023-09-15T11:02:36Z [fd-wallet131] Setting spkMan to active: id = 61b14d4f338e78d4a290e717ed231adcade3b905e4850ff9350ba86c7540a940, type = p2sh-segwit, internal = false
 132023-09-15T11:02:37Z [fd-wallet131] Setting spkMan to active: id = 2c567a4fc58bbe1d167a615d6b0fbef0b3e53bef086c4b82af4bdff6d0f2a971, type = bech32, internal = false
 142023-09-15T11:02:37Z [fd-wallet131] Setting spkMan to active: id = 5907eac043ec550b3956bc2cd7be24d29e19f1e7a1ec4916808e25a64190dd5c, type = bech32m, internal = false
 152023-09-15T11:02:37Z [fd-wallet131] Setting spkMan to active: id = 2b7830bb39af4881c58d7d541112651266613283244c7e0452f96336b286c309, type = legacy, internal = true
 162023-09-15T11:02:37Z [fd-wallet131] Setting spkMan to active: id = 9c10d5b227a5f5c69638cda034ab663636cdca420d536952243fdfbfadb4a242, type = p2sh-segwit, internal = true
 172023-09-15T11:02:38Z [fd-wallet131] Setting spkMan to active: id = 92a633ac0a53837f27c3c579a45c4ee2b7ec2395fea19967f0341c301325e9c6, type = bech32, internal = true
 182023-09-15T11:02:38Z [fd-wallet131] Setting spkMan to active: id = 008c71a0d87a54001383a03f4e64188020f512150a25c7826ccf82d03c084b78, type = bech32m, internal = true
 192023-09-15T11:02:38Z [fd-wallet131] Wallet completed loading in            2619ms
 202023-09-15T11:02:38Z [fd-wallet131] setKeyPool.size() = 8000
 212023-09-15T11:02:38Z [fd-wallet131] mapWallet.size() = 0
 222023-09-15T11:02:38Z [fd-wallet131] m_address_book.size() = 0
 232023-09-15T11:02:38Z Using SQLite Version 3.39.5
 242023-09-15T11:02:38Z Using wallet /Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet132
 252023-09-15T11:02:38Z SQLite Error. Code: 14. Message: cannot open file at line 46922 of [554764a6e7]
 262023-09-15T11:02:38Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet132/wallet.dat) - Too many open files
 272023-09-15T11:02:38Z Using SQLite Version 3.39.5
 282023-09-15T11:02:38Z Using wallet /Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet133
 292023-09-15T11:02:38Z SQLite Error. Code: 14. Message: cannot open file at line 46922 of [554764a6e7]
 302023-09-15T11:02:38Z SQLite Error. Code: 14. Message: os_unix.c:46922: (24) open(/Users/will/Library/Application Support/Bitcoin/regtest/wallets/fd-wallet133/wallet.dat) - Too many open files
 31
 32<snip>
 33
 34^C2023-09-15T11:02:48Z tor: Thread interrupt
 352023-09-15T11:02:48Z opencon thread exit
 362023-09-15T11:02:48Z addcon thread exit
 372023-09-15T11:02:48Z torcontrol thread exit
 382023-09-15T11:02:48Z Shutdown: In progress...
 392023-09-15T11:02:48Z net thread exit
 402023-09-15T11:02:48Z msghand thread exit
 412023-09-15T11:02:48Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat started
 422023-09-15T11:02:48Z DumpAnchors: Flush 0 outbound block-relay-only peer addresses to anchors.dat completed (0.00s)
 432023-09-15T11:02:48Z scheduler thread exit
 442023-09-15T11:02:48Z Writing 0 unbroadcast transactions to disk.
 452023-09-15T11:02:48Z Dumped mempool: 0.00206033s to copy, 0.00876938s to dump
 462023-09-15T11:02:48Z Flushed fee estimates to fee_estimates.dat.
 472023-09-15T11:02:55Z [fd-wallet131] Releasing wallet
 482023-09-15T11:02:55Z [fd-wallet130] Releasing wallet
 492023-09-15T11:02:55Z [fd-wallet129] Releasing wallet
 502023-09-15T11:02:55Z [fd-wallet128] Releasing wallet
 512023-09-15T11:02:55Z [fd-wallet127] Releasing wallet
 522023-09-15T11:02:55Z [fd-wallet126] Releasing wallet
 532023-09-15T11:02:55Z [fd-wallet125] Releasing wallet
 542023-09-15T11:02:55Z [fd-wallet124] Releasing wallet
 552023-09-15T11:02:55Z [fd-wallet123] Releasing wallet
 562023-09-15T11:02:55Z [fd-wallet122] Releasing wallet
 572023-09-15T11:02:55Z [fd-wallet121] Releasing wallet
 582023-09-15T11:02:55Z [fd-wallet120] Releasing wallet
 592023-09-15T11:02:55Z [fd-wallet119] Releasing wallet
 602023-09-15T11:02:55Z [fd-wallet118] Releasing wallet
 612023-09-15T11:02:55Z [fd-wallet117] Releasing wallet
 622023-09-15T11:02:55Z [fd-wallet116] Releasing wallet
 632023-09-15T11:02:55Z [fd-wallet115] Releasing wallet
 642023-09-15T11:02:55Z [fd-wallet114] Releasing wallet
 652023-09-15T11:02:55Z [fd-wallet113] Releasing wallet
 662023-09-15T11:02:55Z [fd-wallet112] Releasing wallet
 672023-09-15T11:02:55Z [fd-wallet111] Releasing wallet
 682023-09-15T11:02:55Z [fd-wallet110] Releasing wallet
 692023-09-15T11:02:55Z [fd-wallet109] Releasing wallet
 702023-09-15T11:02:55Z [fd-wallet108] Releasing wallet
 712023-09-15T11:02:55Z [fd-wallet107] Releasing wallet
 722023-09-15T11:02:55Z [fd-wallet106] Releasing wallet
 732023-09-15T11:02:55Z [fd-wallet105] Releasing wallet
 742023-09-15T11:02:55Z [fd-wallet104] Releasing wallet
 752023-09-15T11:02:55Z [fd-wallet103] Releasing wallet
 762023-09-15T11:02:55Z [fd-wallet102] Releasing wallet
 772023-09-15T11:02:55Z [fd-wallet101] Releasing wallet
 782023-09-15T11:02:55Z [fd-wallet100] Releasing wallet
 792023-09-15T11:02:55Z [fd-wallet99] Releasing wallet
 802023-09-15T11:02:55Z [fd-wallet98] Releasing wallet
 812023-09-15T11:02:55Z [fd-wallet97] Releasing wallet
 822023-09-15T11:02:55Z [fd-wallet96] Releasing wallet
 832023-09-15T11:02:55Z [fd-wallet95] Releasing wallet
 842023-09-15T11:02:55Z [fd-wallet94] Releasing wallet
 852023-09-15T11:02:55Z [fd-wallet93] Releasing wallet
 862023-09-15T11:02:55Z [fd-wallet92] Releasing wallet
 872023-09-15T11:02:55Z [fd-wallet91] Releasing wallet
 882023-09-15T11:02:55Z [fd-wallet90] Releasing wallet
 892023-09-15T11:02:55Z [fd-wallet89] Releasing wallet
 902023-09-15T11:02:55Z [fd-wallet88] Releasing wallet
 912023-09-15T11:02:55Z [fd-wallet87] Releasing wallet
 922023-09-15T11:02:55Z [fd-wallet86] Releasing wallet
 932023-09-15T11:02:56Z [fd-wallet85] Releasing wallet
 942023-09-15T11:02:56Z [fd-wallet84] Releasing wallet
 952023-09-15T11:02:56Z [fd-wallet83] Releasing wallet
 962023-09-15T11:02:56Z [fd-wallet82] Releasing wallet
 972023-09-15T11:02:56Z [fd-wallet81] Releasing wallet
 982023-09-15T11:02:56Z [fd-wallet80] Releasing wallet
 992023-09-15T11:02:56Z [fd-wallet79] Releasing wallet
1002023-09-15T11:02:56Z [fd-wallet78] Releasing wallet
1012023-09-15T11:02:56Z [fd-wallet77] Releasing wallet
1022023-09-15T11:02:56Z [fd-wallet76] Releasing wallet
1032023-09-15T11:02:56Z [fd-wallet75] Releasing wallet
1042023-09-15T11:02:56Z [fd-wallet74] Releasing wallet
1052023-09-15T11:02:56Z [fd-wallet73] Releasing wallet
1062023-09-15T11:02:56Z [fd-wallet72] Releasing wallet
1072023-09-15T11:02:56Z [fd-wallet71] Releasing wallet
1082023-09-15T11:02:56Z [fd-wallet70] Releasing wallet
1092023-09-15T11:02:56Z [fd-wallet69] Releasing wallet
1102023-09-15T11:02:56Z [fd-wallet68] Releasing wallet
1112023-09-15T11:02:56Z [fd-wallet67] Releasing wallet
1122023-09-15T11:02:56Z [fd-wallet66] Releasing wallet
1132023-09-15T11:02:56Z [fd-wallet65] Releasing wallet
1142023-09-15T11:02:56Z [fd-wallet64] Releasing wallet
1152023-09-15T11:02:56Z [fd-wallet63] Releasing wallet
1162023-09-15T11:02:56Z [fd-wallet62] Releasing wallet
1172023-09-15T11:02:56Z [fd-wallet61] Releasing wallet
1182023-09-15T11:02:56Z [fd-wallet60] Releasing wallet
1192023-09-15T11:02:56Z [fd-wallet59] Releasing wallet
1202023-09-15T11:02:56Z [fd-wallet58] Releasing wallet
1212023-09-15T11:02:56Z [fd-wallet57] Releasing wallet
1222023-09-15T11:02:56Z [fd-wallet56] Releasing wallet
1232023-09-15T11:02:56Z [fd-wallet55] Releasing wallet
1242023-09-15T11:02:56Z [fd-wallet54] Releasing wallet
1252023-09-15T11:02:56Z [fd-wallet53] Releasing wallet
1262023-09-15T11:02:56Z [fd-wallet52] Releasing wallet
1272023-09-15T11:02:56Z [fd-wallet51] Releasing wallet
1282023-09-15T11:02:56Z [fd-wallet50] Releasing wallet
1292023-09-15T11:02:56Z [fd-wallet49] Releasing wallet
1302023-09-15T11:02:56Z [fd-wallet48] Releasing wallet
1312023-09-15T11:02:56Z [fd-wallet47] Releasing wallet
1322023-09-15T11:02:56Z [fd-wallet46] Releasing wallet
1332023-09-15T11:02:56Z [fd-wallet45] Releasing wallet
1342023-09-15T11:02:56Z [fd-wallet44] Releasing wallet
1352023-09-15T11:02:56Z [fd-wallet43] Releasing wallet
1362023-09-15T11:02:56Z [fd-wallet42] Releasing wallet
1372023-09-15T11:02:56Z [fd-wallet41] Releasing wallet
1382023-09-15T11:02:56Z [fd-wallet40] Releasing wallet
1392023-09-15T11:02:56Z [fd-wallet39] Releasing wallet
1402023-09-15T11:02:56Z [fd-wallet38] Releasing wallet
1412023-09-15T11:02:56Z [fd-wallet37] Releasing wallet
1422023-09-15T11:02:56Z [fd-wallet36] Releasing wallet
1432023-09-15T11:02:56Z [fd-wallet35] Releasing wallet
1442023-09-15T11:02:56Z [fd-wallet34] Releasing wallet
1452023-09-15T11:02:56Z [fd-wallet33] Releasing wallet
1462023-09-15T11:02:56Z [fd-wallet32] Releasing wallet
1472023-09-15T11:02:56Z [fd-wallet31] Releasing wallet
1482023-09-15T11:02:56Z [fd-wallet30] Releasing wallet
1492023-09-15T11:02:56Z [fd-wallet29] Releasing wallet
1502023-09-15T11:02:56Z [fd-wallet28] Releasing wallet
1512023-09-15T11:02:56Z [fd-wallet27] Releasing wallet
1522023-09-15T11:02:56Z [fd-wallet26] Releasing wallet
1532023-09-15T11:02:56Z [fd-wallet25] Releasing wallet
1542023-09-15T11:02:56Z [fd-wallet24] Releasing wallet
1552023-09-15T11:02:56Z [fd-wallet23] Releasing wallet
1562023-09-15T11:02:56Z [fd-wallet22] Releasing wallet
1572023-09-15T11:02:56Z [fd-wallet21] Releasing wallet
1582023-09-15T11:02:56Z [fd-wallet20] Releasing wallet
1592023-09-15T11:02:56Z [fd-wallet19] Releasing wallet
1602023-09-15T11:02:56Z [fd-wallet18] Releasing wallet
1612023-09-15T11:02:56Z [fd-wallet17] Releasing wallet
1622023-09-15T11:02:56Z [fd-wallet16] Releasing wallet
1632023-09-15T11:02:56Z [fd-wallet15] Releasing wallet
1642023-09-15T11:02:56Z [fd-wallet14] Releasing wallet
1652023-09-15T11:02:56Z [fd-wallet13] Releasing wallet
1662023-09-15T11:02:56Z [fd-wallet12] Releasing wallet
1672023-09-15T11:02:56Z [fd-wallet11] Releasing wallet
1682023-09-15T11:02:56Z [fd-wallet10] Releasing wallet
1692023-09-15T11:02:56Z [fd-wallet9] Releasing wallet
1702023-09-15T11:02:56Z [fd-wallet8] Releasing wallet
1712023-09-15T11:02:56Z [fd-wallet7] Releasing wallet
1722023-09-15T11:02:56Z [fd-wallet6] Releasing wallet
1732023-09-15T11:02:56Z [fd-wallet5] Releasing wallet
1742023-09-15T11:02:56Z [fd-wallet4] Releasing wallet
1752023-09-15T11:02:56Z [fd-wallet3] Releasing wallet
1762023-09-15T11:02:56Z [fd-wallet2] Releasing wallet
1772023-09-15T11:02:56Z [fd-wallet1] Releasing wallet
1782023-09-15T11:02:56Z Shutdown: done
  20. edilmedeiros commented at 10:14 pm on August 2, 2024: contributor

    I’m still cleaning up a script that made me hit this limit, will share it later, but it’s not much different from what’s in the issue description (creating many wallets in a loop). I’m setting up a signet for an online program with prefunded wallets.

    I’m on macos Sonoma 14.5.

    Can check the limits with.

    0$ ulimit -a
1-t: cpu time (seconds)              unlimited
2-f: file size (blocks)              unlimited
3-d: data seg size (kbytes)          unlimited
4-s: stack size (kbytes)             8176
5-c: core file size (blocks)         0
6-v: address space (kbytes)          unlimited
7-l: locked-in-memory size (kbytes)  unlimited
8-u: processes                       2666
9-n: file descriptors                256

    Increasing the file descriptors limit for the current shell was simple as (did not required sudo).

     0$ ulimit -n 8192
 1
 2$ ulimit -a
 3-t: cpu time (seconds)              unlimited
 4-f: file size (blocks)              unlimited
 5-d: data seg size (kbytes)          unlimited
 6-s: stack size (kbytes)             8176
 7-c: core file size (blocks)         0
 8-v: address space (kbytes)          unlimited
 9-l: locked-in-memory size (kbytes)  unlimited
10-u: processes                       2666
11-n: file descriptors                8192

    It does the job for the current shell. Apparently you can’t make it unlimited, but can use a large number.

    Doesn’t fix bitcoind breaking, but works as a workaround if you can estimate how many files descriptors you’re going to need.

  21. edilmedeiros commented at 2:49 pm on October 26, 2024: contributor

    I confirm this is still happening on v28 using this script (based on the work of @pinheadmz) to create 300 wallets.

    ulimit -n 8192 (or higher, doesn’t require sudo) is a functioning workaround.


fjahr dimitaracev maflcko sr-gi willcl-ark edilmedeiros

Labels
Wallet macOS

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-30 15:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me