MacOS: importdescriptors RPC call very slow with release and source-built binaries, but fast for homebrew binaries #25724

issue casey openend this issue on July 27, 2022
  1. casey commented at 9:30 pm on July 27, 2022: contributor

    I’m working on a project on MacOS that makes RPC calls using the Rust bitcoincore-rpc crate, against the bitcoind prebuilt binary available here.

    Some RPC calls succeed, but some fail with this error:

    0Error: Rpc(JsonRpc(Transport(SocketError(Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" }))))

    After much debugging, we found that when we used the version of bitcoind provided by homebrew, the exact same sequence of RPC calls would succeed. We were able to reproduce this on both an x86 mac and an ARM mac.

    We haven’t been able to figure out what the disparity is, but looking at the homebrew bitcoind formula defiinition might provide some clues.

    In particular:

    0  def install
1    system "./autogen.sh"
2    system "./configure", *std_configure_args,
3                          "--disable-dependency-tracking",
4                          "--disable-silent-rules",
5                          "--with-boost-libdir=#{Formula["boost@1.76"].opt_lib}"
6    system "make", "install"
7    pkgshare.install "share/rpcauth"
8  end

    It looks like homebrew is passing additional arguments to configure, std_configure_args, and is using a version of boost built by homebrew. I tried using brew install -s boost@1.76 bitcoind, to see if this also produced a binary that worked, and it did. The logs from that build are available here.

    I’ve run into similar problems like this one a few times before, i.e., RPC calls to bitcoind failing on MacOS due to inscrutable I/O errors. One time, I was able to fix it by increasing the open file limit using ulimit -n BIG_NUMBER, but that doesn’t seem to work here.

    My best guess is that the homebrew poeople have gotten pretty good at configuring packages on MacOS, and they’re passing some flag, or using a patched version of boost, or linking it differently, such that the binary they produce isn’t affected by this issue.

  2. casey added the label Bug on Jul 27, 2022
  3. casey commented at 9:31 pm on July 27, 2022: contributor
    I’m trying to track down the specific RPC call which is failing, but it’s in a dependency, bdk::Wallet::sync, so it’s a bit tricky to isolate.
  4. casey commented at 9:36 pm on July 27, 2022: contributor

    It looks like the failing call is importdescriptors with a bunch of descriptors:

    0[2022-07-27T21:34:09Z DEBUG bdk::blockchain::rpc] importing 100 script_pubkeys (some maybe already imported)
1[2022-07-27T21:34:09Z DEBUG bitcoincore_rpc] JSON-RPC request: importdescriptors [[{"desc":"raw(00145846369f3d6ba366d6f5a903fb5cf4dca3763c0e)#k9wh6v62","timestamp":"now"},{"desc":"raw(001420800aabf13f3a4c4ce3ce4c66cecf1d17f21a6e)#6m0hlfh4","timestamp":"now"},{"desc":"raw(0014c6bf9715e06d73ebf9b3b02d5cc48d24d8bbabc1)#wyavh36r","timestamp":"now"},{"desc":"raw(00141ba7807b3f46af113beaea5c698428ce7138cd8a)#jctdsups","timestamp":"now"},{"desc":"raw(00140c1bd27f10fff01b36ddf3c1febaa1acff19b080)#9s6nc3pk","timestamp":"now"},{"desc":"raw(00141226e31987e4bc2e63c0ee12908f675e40464b20)#9pp7qm39","timestamp":"now"},{"desc":"raw(0014f73f149f7503960a5e849c6ee7a8a8c336f631cb)#qtkxv9fc","timestamp":"now"},{"desc":"raw(0014c8ccb4d81ffc769fc5fdd8d7eed69b0e0cae5749)#hn39qayv","timestamp":"now"},{"desc":"raw(001498565aead2d67a22a6021d55210f2a917fc22169)#6ar3vwsx","timestamp":"now"},{"desc":"raw(001403013248ac0cd9eabe176cad162cda2a19f771e1)#4m47mukd","timestamp":"now"},{"desc":"raw(00147de17826fab4e7572755ad8ddbe40ce61f63a699)#03v5x8vf","timestamp":"now"},{"desc":"raw(00146402f6560278b5c87cf1173f17fd971e3f0e41a1)#0pjd3l95","timestamp":"now"},{"desc":"raw(0014fc7707f8913a5ed05e9c95f79993fa63fc95fb91)#fdxpnn5d","timestamp":"now"},{"desc":"raw(00149118bb0f57eea8ba2a8b63c3dd4886ec39b6f559)#v3e0s6wa","timestamp":"now"},{"desc":"raw(001424f59b558b0094690d208845d2d4aec7eab05917)#qp3f02gu","timestamp":"now"},{"desc":"raw(001407cfbc97a867f3adc6e74a86a12ab50a6c96dfdc)#nacutv73","timestamp":"now"},{"desc":"raw(0014ad0e3b494957a6556d48d201dcd08a56a1337e71)#lxs5utkw","timestamp":"now"},{"desc":"raw(0014a44766df4820e741cf3e69de06604377803b1d72)#xlduxz49","timestamp":"now"},{"desc":"raw(0014ab3ce65379c43bd52a31245b66482ccad72c56e6)#7m30zf3n","timestamp":"now"},{"desc":"raw(0014014c4f73107497cbb8a28e95d8558726bcf353b0)#5hn4u62g","timestamp":"now"},{"desc":"raw(0014e20f4b7479cd94d63dad0af7d422ab318502edc9)#ankgt3u9","timestamp":"now"},{"desc":"raw(0014a4ad33e67d990ce5dbeda8fc033338e878175bff)#kz76eymj","timestamp":"now"},{"desc":"raw(00148c59654a57e921c61b36ec55b9f4973809ba3aa7)#4a0gg950","timestamp":"now"},{"desc":"raw(00141622125f724ae21931c8ad3fcf6d2251a5ff3101)#pa9v83g6","timestamp":"now"},{"desc":"raw(001492888404b62de9dce757beda671f56622979b277)#0w3ev6uv","timestamp":"now"},{"desc":"raw(0014e55f7a7a04ccbb3997ac43f9565f7000cc601021)#axc47qx8","timestamp":"now"},{"desc":"raw(0014f12cb2e9b420c40ff5e0815b1fdc491f30696e82)#pkp43cmf","timestamp":"now"},{"desc":"raw(0014306057cd7518cd37cec725cbb9057c169913dc0e)#0mv0nk0l","timestamp":"now"},{"desc":"raw(0014bbbfe144b731a96530b0ce2644482e2e439842bf)#42a244xg","timestamp":"now"},{"desc":"raw(0014d142df70072bb78d6422e1f5bcd0144a514d1a30)#7d8xpujn","timestamp":"now"},{"desc":"raw(00141fdf3bb05c1b6d780e9054c89bd619609bf1f82d)#y0x67vfq","timestamp":"now"},{"desc":"raw(00148939d2d80b6591267c59af6b1614ef28e8505c0d)#w453gc92","timestamp":"now"},{"desc":"raw(001429b662406f03efe9c20d9227c463aa0c37613768)#mmdtm3vj","timestamp":"now"},{"desc":"raw(0014339611d35e9e2789fd1c2c4c59744cd528c30448)#5cm5cu7x","timestamp":"now"},{"desc":"raw(00148db3e38ada7f7da7b853a85564063a34fdbb7bde)#6zvgvvrg","timestamp":"now"},{"desc":"raw(001493a50be6af43b13ce318c49a5d4088a446348d86)#fxtearpa","timestamp":"now"},{"desc":"raw(0014c325b3bceb1a06526ed375d6af7e21a3b18d3c4f)#x8kut84a","timestamp":"now"},{"desc":"raw(0014a86d53c183affc9f41119f9a8a12f1c23ea3f22a)#p307pvnt","timestamp":"now"},{"desc":"raw(0014c780b4dd93a2bbdbcda22e1b7e26db56ca4538a0)#juna8rk7","timestamp":"now"},{"desc":"raw(0014e4ed854a807959a7ee787e0f21fd749b23cf7971)#4es3xghl","timestamp":"now"},{"desc":"raw(0014b75df751f7407cb7adbd281d9f34a34dd5f26b3a)#8hm2usj7","timestamp":"now"},{"desc":"raw(0014d5ebb3be03f2ad8810fd4e246a60c1a437f27962)#rymmus3n","timestamp":"now"},{"desc":"raw(001411bbabfb5e10d17e6d29fe6870ac48b9ca560234)#6k8wxlzf","timestamp":"now"},{"desc":"raw(0014cd701716b718e1cbd474f2f7685c4c7939b18359)#v9rhxfjx","timestamp":"now"},{"desc":"raw(0014ebf9b6888c54d99b5df846749e397dee0c18f54d)#c79pex2s","timestamp":"now"},{"desc":"raw(0014d7bea15cd50f69ea154fc906004d1b22eda94eec)#0nj2uyj4","timestamp":"now"},{"desc":"raw(001489102bee375f6579cbc781267dfe499d0015e85d)#hputch5y","timestamp":"now"},{"desc":"raw(0014b79ce1bf47da6786ac1e4abbb53a19a691e2d9bf)#x30zcf0g","timestamp":"now"},{"desc":"raw(00144747c2e3c4a4298919490b62c483c5ae044e54d9)#wme4nj20","timestamp":"now"},{"desc":"raw(001465bdadb594af71cc17cbab558f9795d12c7cab7f)#nkz9msgr","timestamp":"now"},{"desc":"raw(001405aa550fd71c6c78144464106e0e2c6698c1da63)#8462trry","timestamp":"now"},{"desc":"raw(00145d67a29230c89cc8312afbe06eedb4252d043ef4)#yqevjyff","timestamp":"now"},{"desc":"raw(00140c7eb4ec81793be62515d8683c8ddb6b16151e8e)#pj934vfx","timestamp":"now"},{"desc":"raw(0014d7f23a11d8539844fc8478e82f19e91f5ccae342)#8w3r73pj","timestamp":"now"},{"desc":"raw(0014360c671c07af4527a56d83b06648dfab0975a13f)#j6darl3v","timestamp":"now"},{"desc":"raw(00145d61e5900db40a49346d809e8f4335d68705fff5)#wl06dsvk","timestamp":"now"},{"desc":"raw(00144b2a163a76f392d975efef958444f6aaac9ddec8)#3rt3rl27","timestamp":"now"},{"desc":"raw(00147beeca8b2390e7c708d3ba83e05d7e34fc055a8e)#z2uxn97h","timestamp":"now"},{"desc":"raw(001408a53ef1e2a9ddf2f7b7a4eb574d2b390a2b1c99)#qrjrpl6u","timestamp":"now"},{"desc":"raw(0014d55cdd2b295c2e3387838fd8b025923edc88ee63)#crf74usx","timestamp":"now"},{"desc":"raw(0014a38c98d4ba62bb9df9db62a1a8bb58eb2a07a07b)#e2pmfkqm","timestamp":"now"},{"desc":"raw(0014dd90129a2ed473d5c96f2eb58dca81c01369542e)#y6hsstz4","timestamp":"now"},{"desc":"raw(0014af7e5643830997210c65f4370eaf5a7dc764d45e)#m7r3gq88","timestamp":"now"},{"desc":"raw(00149cdf1765b03e1cf5145945394aeddc1a0e6693ac)#mhxdece7","timestamp":"now"},{"desc":"raw(00148d2e1bac1c70c98e60c34a0e7cea3bca4653f304)#qwpq5fur","timestamp":"now"},{"desc":"raw(001459755bf4b210e00fa403a0c556e11b842310abde)#4pyt2tu8","timestamp":"now"},{"desc":"raw(0014b449a14ad2951df4723a5db67fa3ed6da564c09b)#m0v3vj8d","timestamp":"now"},{"desc":"raw(0014317ddfe4d602826759215e1abea701b9903e486f)#4k8w7762","timestamp":"now"},{"desc":"raw(00146cc862530d795fe077586bdbb33f511ac7cb01ff)#8eprpvnv","timestamp":"now"},{"desc":"raw(0014ae91241bb67aba596d1ca6cf6e715b73668625be)#rnhyfshy","timestamp":"now"},{"desc":"raw(00146c9fa5cc60fc595d839521f43f17a4c6360e2de2)#434dv46f","timestamp":"now"},{"desc":"raw(0014b089ceabb72af3a98ef4a05f6c456e9f24c671b2)#d4lcthch","timestamp":"now"},{"desc":"raw(0014771ad9bdde222dd69b4941c648409fbe346aa313)#e4fs2a7y","timestamp":"now"},{"desc":"raw(0014a8498b2097f007f2ab78c6921041b6d8b8f330de)#dd7pyhz6","timestamp":"now"},{"desc":"raw(00146faf2f0528e266cb1d06b356cc031d455bd3dc63)#9uxar8u3","timestamp":"now"},{"desc":"raw(0014f9907afae2c3f8be3bce12dca19be2867245f88d)#7y6gnm25","timestamp":"now"},{"desc":"raw(001431aacee7ab7794a8766723536c62f05b7f63fc84)#0cf3ks45","timestamp":"now"},{"desc":"raw(0014c06f7953519dbfd1e25ff18d001be590401ecba6)#lrnv7qcq","timestamp":"now"},{"desc":"raw(0014434089931c3b52d2def67f62eac99d23058a9f09)#cg0nslav","timestamp":"now"},{"desc":"raw(0014bf4d1f68349dd65990d6338ac61ca946e98f8cc5)#tx8gf6l3","timestamp":"now"},{"desc":"raw(001417a66b36201d81e59f48939c67755358164dec66)#4x3h7lv2","timestamp":"now"},{"desc":"raw(0014064bcbe8d6208bb58e2147a5bbcd97dbdcd298a4)#qd9yj6zk","timestamp":"now"},{"desc":"raw(0014f2dc57444e57098f00c57060f97da9cc8fadac3c)#f5uwe54v","timestamp":"now"},{"desc":"raw(00142fb4648a340c06a2f2f9742d2e7832e8f347a2a2)#q6fx2u8j","timestamp":"now"},{"desc":"raw(0014564a50bddba40328fd6bea63ea434f6e47269333)#reh8v5v0","timestamp":"now"},{"desc":"raw(001401f256832bf51c8690c921412ee0d89b8dcc5d8d)#l7tyefep","timestamp":"now"},{"desc":"raw(0014ad3377bab25e7ce5de2d243c2af1a1a744c5ba25)#n8hylkg4","timestamp":"now"},{"desc":"raw(00146cde66581cc43d654da4d139660fce31dbd27821)#58pr9vpr","timestamp":"now"},{"desc":"raw(0014d39f838ee2bd8734fb90354d2b15a1c1eb8900ac)#s526lyam","timestamp":"now"},{"desc":"raw(001444fafb985f591ccfe2b9990e0e75783bb0bde701)#cpmq7gwp","timestamp":"now"},{"desc":"raw(00145d75454b0038bc70f12306484544038558140938)#90h7e468","timestamp":"now"},{"desc":"raw(00147a01fddfdee428515cf4c0ac5ac609fd0724620f)#9cejp0ux","timestamp":"now"},{"desc":"raw(001406eb1337f2a3daf860b7ac8b9b46c10d0f70010e)#mk362vyg","timestamp":"now"},{"desc":"raw(0014d6622202a9ddbd298e0abdaebcdd1ebf13f43dea)#8j4fmku8","timestamp":"now"},{"desc":"raw(00148b4e1f7cb1819dd6f3386d3bd3b49f82ec69bbd5)#gap3ycw7","timestamp":"now"},{"desc":"raw(001462b2fe372c6ca2a39bb453b17bae10310bcd02bb)#ndftc243","timestamp":"now"},{"desc":"raw(0014b9abb44d5504daff04903f5dcfdbd6bb1613d586)#fu2d8tt2","timestamp":"now"},{"desc":"raw(0014762aa4cd9edc41765dff328b3662ac7c424ac771)#mh2d5gj8","timestamp":"now"},{"desc":"raw(00144426d291d198ccd0df68521758afe8cff08743a8)#c40nwl9x","timestamp":"now"},{"desc":"raw(00140b970dd18df6eaaa790b0050b6929587c47f58ea)#wndmanqc","timestamp":"now"}]]
2[2022-07-27T21:34:24Z DEBUG bitcoincore_rpc] JSON-RPC failed parsing reply of importdescriptors: JsonRpc(Transport(SocketError(Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" })))
  5. casey commented at 6:01 am on July 28, 2022: contributor
    Just to add another data point, I built bitcoin core manually from source, i.e., not using homebrew, and I ran into the same error, so it seems like homebrew is definitely configuring the binary differently.
  6. 0xB10C commented at 3:54 pm on July 28, 2022: contributor
    does the Bitcoin Core debug.log (with -debug=rpc -debug=libevent -debug=walletdb) tell you anything?
  7. maflcko commented at 5:04 pm on July 28, 2022: member
    Do the functional tests pass? (./test/functional/test_runner)
  8. fanquake commented at 11:32 am on July 29, 2022: member

    We haven’t been able to figure out what the disparity is, but looking at the homebrew bitcoind formula defiinition might provide some clues.

    The only thing in the formula definition that I think would be interesting is the version of Boost being used (1.76.0). All the other configure flags are pretty standard, and shouldn’t drastically change the binary being produced:

    0    system "./configure", *std_configure_args,
1                          "--disable-dependency-tracking",
2                          "--disable-silent-rules",
3                          "--with-boost-libdir=#{Formula["boost@1.76"].opt_lib}"

    std_configure_args expands to: "--disable-debug", "--disable-dependency-tracking", "--prefix=#{prefix}", "--libdir=#{lib}" so it looks like there’s a duplicate --disable-dependency-tracking. I’ve opened a PR to remove it.

  9. fanquake added the label macOS on Jul 30, 2022
  10. fanquake commented at 12:36 pm on July 30, 2022: member
    I’ve also opened a PR to update the formula to use Boost 1.79.0: https://github.com/Homebrew/homebrew-core/pull/106923, as the pinning to 1.76.0 seems like a leftover from when 22.0 wouldn’t compile with Boost 1.78.0.
  11. fanquake commented at 3:15 pm on July 30, 2022: member
    @casey can you retry you tests (after updating) now that the homebrew bitcoind is using boost 1.79.0.
  12. casey commented at 8:03 am on August 6, 2022: contributor

    does the Bitcoin Core debug.log (with -debug=rpc -debug=libevent -debug=walletdb) tell you anything?

    Nope, nothing interesting gets printed.

    Do the functional tests pass? (./test/functional/test_runner)

    All the functional tests pass.

    @casey can you retry you tests (after updating) now that the homebrew bitcoind is using boost 1.79.0.

    I tried after updating, and bitcoind/homebrew is still passing, and bitcoind/built-from-source is still failing.

    I created this repo, which contains a makefile with recipes pass and fail that, on my machine, pass and fail respectively. Pass installs bitcoin from homebrew, and fail builds it from source.

    I’ve been trying to reproduce with a GitHub Actions workflow, but I haven’t been successful. Both make pass and make fail pass on CI. I have an M1 Mac, but I’ve reproduced it on a friend’s x86 Mac, so it doesn’t seem to be related either to the architecture, or to some quirk of my machine.

    This issue on bitcoincore-rpc is pretty much identical. I didn’t think it was the same because in my case, the difference is between two bitcoind binaries, and not two bitcoincore-rpc crate versions.

    I wish I could run bitcoind under strace, but I’m on a version of MacOS that doesn’t allow running dtrace (macos’s strace equivalent) unless I reboot into recovery mode and run scary commands that disable some kind of system integrity protection.

  13. casey commented at 10:03 pm on August 12, 2022: contributor

    I’ve been continuing to debug this, and the reason that I’m hitting this with a release binary and not with the homebrew binary, is because rust-bitcoincore-rpc has a 15 second timeout, and for some reason, the importdescriptors call is taking ~5s with the homebrew binary, but ~32s with the release binary.

    This makes me think that it has to do with the weird system integrity stuff that Apple’s been adding that slows everything done.

    The theory is: Homebrew does some special signing magic to the binaries it produces that allows them to run without some slow security check. The binaries built from source, or downloaded from bitcoincore.org do not have this signing magic, which makes them run slower. Usually this isn’t a problem, but importdescriptors is either especially slow, or does something that requires a bunch of security checks, so it runs much slower, hits the RPC library timeout of 15 seconds, which gives the above error.

  14. casey renamed this:
    RPC call on MacOS failing with bitcoind binary from bitcoincore.org but working with binary from homebrew
    MacOS: `importdescriptors` RPC call very slow with release and source-built binaries, but fast for homebrew binaries
    on Aug 12, 2022
  15. casey commented at 10:09 pm on August 12, 2022: contributor

    The failing call is:

      0# create a wallet with privatekeys disabled
  1bitcoin-cli createwallet test true
  2# import a bunch of descriptors
  3bitcoin-cli import importdescriptors \
  4'[
  5  {"desc":"raw(00145846369f3d6ba366d6f5a903fb5cf4dca3763c0e)#k9wh6v62","timestamp":"now"},
  6  {"desc":"raw(001420800aabf13f3a4c4ce3ce4c66cecf1d17f21a6e)#6m0hlfh4","timestamp":"now"},
  7  {"desc":"raw(0014c6bf9715e06d73ebf9b3b02d5cc48d24d8bbabc1)#wyavh36r","timestamp":"now"},
  8  {"desc":"raw(00141ba7807b3f46af113beaea5c698428ce7138cd8a)#jctdsups","timestamp":"now"},
  9  {"desc":"raw(00140c1bd27f10fff01b36ddf3c1febaa1acff19b080)#9s6nc3pk","timestamp":"now"},
 10  {"desc":"raw(00141226e31987e4bc2e63c0ee12908f675e40464b20)#9pp7qm39","timestamp":"now"},
 11  {"desc":"raw(0014f73f149f7503960a5e849c6ee7a8a8c336f631cb)#qtkxv9fc","timestamp":"now"},
 12  {"desc":"raw(0014c8ccb4d81ffc769fc5fdd8d7eed69b0e0cae5749)#hn39qayv","timestamp":"now"},
 13  {"desc":"raw(001498565aead2d67a22a6021d55210f2a917fc22169)#6ar3vwsx","timestamp":"now"},
 14  {"desc":"raw(001403013248ac0cd9eabe176cad162cda2a19f771e1)#4m47mukd","timestamp":"now"},
 15  {"desc":"raw(00147de17826fab4e7572755ad8ddbe40ce61f63a699)#03v5x8vf","timestamp":"now"},
 16  {"desc":"raw(00146402f6560278b5c87cf1173f17fd971e3f0e41a1)#0pjd3l95","timestamp":"now"},
 17  {"desc":"raw(0014fc7707f8913a5ed05e9c95f79993fa63fc95fb91)#fdxpnn5d","timestamp":"now"},
 18  {"desc":"raw(00149118bb0f57eea8ba2a8b63c3dd4886ec39b6f559)#v3e0s6wa","timestamp":"now"},
 19  {"desc":"raw(001424f59b558b0094690d208845d2d4aec7eab05917)#qp3f02gu","timestamp":"now"},
 20  {"desc":"raw(001407cfbc97a867f3adc6e74a86a12ab50a6c96dfdc)#nacutv73","timestamp":"now"},
 21  {"desc":"raw(0014ad0e3b494957a6556d48d201dcd08a56a1337e71)#lxs5utkw","timestamp":"now"},
 22  {"desc":"raw(0014a44766df4820e741cf3e69de06604377803b1d72)#xlduxz49","timestamp":"now"},
 23  {"desc":"raw(0014ab3ce65379c43bd52a31245b66482ccad72c56e6)#7m30zf3n","timestamp":"now"},
 24  {"desc":"raw(0014014c4f73107497cbb8a28e95d8558726bcf353b0)#5hn4u62g","timestamp":"now"},
 25  {"desc":"raw(0014e20f4b7479cd94d63dad0af7d422ab318502edc9)#ankgt3u9","timestamp":"now"},
 26  {"desc":"raw(0014a4ad33e67d990ce5dbeda8fc033338e878175bff)#kz76eymj","timestamp":"now"},
 27  {"desc":"raw(00148c59654a57e921c61b36ec55b9f4973809ba3aa7)#4a0gg950","timestamp":"now"},
 28  {"desc":"raw(00141622125f724ae21931c8ad3fcf6d2251a5ff3101)#pa9v83g6","timestamp":"now"},
 29  {"desc":"raw(001492888404b62de9dce757beda671f56622979b277)#0w3ev6uv","timestamp":"now"},
 30  {"desc":"raw(0014e55f7a7a04ccbb3997ac43f9565f7000cc601021)#axc47qx8","timestamp":"now"},
 31  {"desc":"raw(0014f12cb2e9b420c40ff5e0815b1fdc491f30696e82)#pkp43cmf","timestamp":"now"},
 32  {"desc":"raw(0014306057cd7518cd37cec725cbb9057c169913dc0e)#0mv0nk0l","timestamp":"now"},
 33  {"desc":"raw(0014bbbfe144b731a96530b0ce2644482e2e439842bf)#42a244xg","timestamp":"now"},
 34  {"desc":"raw(0014d142df70072bb78d6422e1f5bcd0144a514d1a30)#7d8xpujn","timestamp":"now"},
 35  {"desc":"raw(00141fdf3bb05c1b6d780e9054c89bd619609bf1f82d)#y0x67vfq","timestamp":"now"},
 36  {"desc":"raw(00148939d2d80b6591267c59af6b1614ef28e8505c0d)#w453gc92","timestamp":"now"},
 37  {"desc":"raw(001429b662406f03efe9c20d9227c463aa0c37613768)#mmdtm3vj","timestamp":"now"},
 38  {"desc":"raw(0014339611d35e9e2789fd1c2c4c59744cd528c30448)#5cm5cu7x","timestamp":"now"},
 39  {"desc":"raw(00148db3e38ada7f7da7b853a85564063a34fdbb7bde)#6zvgvvrg","timestamp":"now"},
 40  {"desc":"raw(001493a50be6af43b13ce318c49a5d4088a446348d86)#fxtearpa","timestamp":"now"},
 41  {"desc":"raw(0014c325b3bceb1a06526ed375d6af7e21a3b18d3c4f)#x8kut84a","timestamp":"now"},
 42  {"desc":"raw(0014a86d53c183affc9f41119f9a8a12f1c23ea3f22a)#p307pvnt","timestamp":"now"},
 43  {"desc":"raw(0014c780b4dd93a2bbdbcda22e1b7e26db56ca4538a0)#juna8rk7","timestamp":"now"},
 44  {"desc":"raw(0014e4ed854a807959a7ee787e0f21fd749b23cf7971)#4es3xghl","timestamp":"now"},
 45  {"desc":"raw(0014b75df751f7407cb7adbd281d9f34a34dd5f26b3a)#8hm2usj7","timestamp":"now"},
 46  {"desc":"raw(0014d5ebb3be03f2ad8810fd4e246a60c1a437f27962)#rymmus3n","timestamp":"now"},
 47  {"desc":"raw(001411bbabfb5e10d17e6d29fe6870ac48b9ca560234)#6k8wxlzf","timestamp":"now"},
 48  {"desc":"raw(0014cd701716b718e1cbd474f2f7685c4c7939b18359)#v9rhxfjx","timestamp":"now"},
 49  {"desc":"raw(0014ebf9b6888c54d99b5df846749e397dee0c18f54d)#c79pex2s","timestamp":"now"},
 50  {"desc":"raw(0014d7bea15cd50f69ea154fc906004d1b22eda94eec)#0nj2uyj4","timestamp":"now"},
 51  {"desc":"raw(001489102bee375f6579cbc781267dfe499d0015e85d)#hputch5y","timestamp":"now"},
 52  {"desc":"raw(0014b79ce1bf47da6786ac1e4abbb53a19a691e2d9bf)#x30zcf0g","timestamp":"now"},
 53  {"desc":"raw(00144747c2e3c4a4298919490b62c483c5ae044e54d9)#wme4nj20","timestamp":"now"},
 54  {"desc":"raw(001465bdadb594af71cc17cbab558f9795d12c7cab7f)#nkz9msgr","timestamp":"now"},
 55  {"desc":"raw(001405aa550fd71c6c78144464106e0e2c6698c1da63)#8462trry","timestamp":"now"},
 56  {"desc":"raw(00145d67a29230c89cc8312afbe06eedb4252d043ef4)#yqevjyff","timestamp":"now"},
 57  {"desc":"raw(00140c7eb4ec81793be62515d8683c8ddb6b16151e8e)#pj934vfx","timestamp":"now"},
 58  {"desc":"raw(0014d7f23a11d8539844fc8478e82f19e91f5ccae342)#8w3r73pj","timestamp":"now"},
 59  {"desc":"raw(0014360c671c07af4527a56d83b06648dfab0975a13f)#j6darl3v","timestamp":"now"},
 60  {"desc":"raw(00145d61e5900db40a49346d809e8f4335d68705fff5)#wl06dsvk","timestamp":"now"},
 61  {"desc":"raw(00144b2a163a76f392d975efef958444f6aaac9ddec8)#3rt3rl27","timestamp":"now"},
 62  {"desc":"raw(00147beeca8b2390e7c708d3ba83e05d7e34fc055a8e)#z2uxn97h","timestamp":"now"},
 63  {"desc":"raw(001408a53ef1e2a9ddf2f7b7a4eb574d2b390a2b1c99)#qrjrpl6u","timestamp":"now"},
 64  {"desc":"raw(0014d55cdd2b295c2e3387838fd8b025923edc88ee63)#crf74usx","timestamp":"now"},
 65  {"desc":"raw(0014a38c98d4ba62bb9df9db62a1a8bb58eb2a07a07b)#e2pmfkqm","timestamp":"now"},
 66  {"desc":"raw(0014dd90129a2ed473d5c96f2eb58dca81c01369542e)#y6hsstz4","timestamp":"now"},
 67  {"desc":"raw(0014af7e5643830997210c65f4370eaf5a7dc764d45e)#m7r3gq88","timestamp":"now"},
 68  {"desc":"raw(00149cdf1765b03e1cf5145945394aeddc1a0e6693ac)#mhxdece7","timestamp":"now"},
 69  {"desc":"raw(00148d2e1bac1c70c98e60c34a0e7cea3bca4653f304)#qwpq5fur","timestamp":"now"},
 70  {"desc":"raw(001459755bf4b210e00fa403a0c556e11b842310abde)#4pyt2tu8","timestamp":"now"},
 71  {"desc":"raw(0014b449a14ad2951df4723a5db67fa3ed6da564c09b)#m0v3vj8d","timestamp":"now"},
 72  {"desc":"raw(0014317ddfe4d602826759215e1abea701b9903e486f)#4k8w7762","timestamp":"now"},
 73  {"desc":"raw(00146cc862530d795fe077586bdbb33f511ac7cb01ff)#8eprpvnv","timestamp":"now"},
 74  {"desc":"raw(0014ae91241bb67aba596d1ca6cf6e715b73668625be)#rnhyfshy","timestamp":"now"},
 75  {"desc":"raw(00146c9fa5cc60fc595d839521f43f17a4c6360e2de2)#434dv46f","timestamp":"now"},
 76  {"desc":"raw(0014b089ceabb72af3a98ef4a05f6c456e9f24c671b2)#d4lcthch","timestamp":"now"},
 77  {"desc":"raw(0014771ad9bdde222dd69b4941c648409fbe346aa313)#e4fs2a7y","timestamp":"now"},
 78  {"desc":"raw(0014a8498b2097f007f2ab78c6921041b6d8b8f330de)#dd7pyhz6","timestamp":"now"},
 79  {"desc":"raw(00146faf2f0528e266cb1d06b356cc031d455bd3dc63)#9uxar8u3","timestamp":"now"},
 80  {"desc":"raw(0014f9907afae2c3f8be3bce12dca19be2867245f88d)#7y6gnm25","timestamp":"now"},
 81  {"desc":"raw(001431aacee7ab7794a8766723536c62f05b7f63fc84)#0cf3ks45","timestamp":"now"},
 82  {"desc":"raw(0014c06f7953519dbfd1e25ff18d001be590401ecba6)#lrnv7qcq","timestamp":"now"},
 83  {"desc":"raw(0014434089931c3b52d2def67f62eac99d23058a9f09)#cg0nslav","timestamp":"now"},
 84  {"desc":"raw(0014bf4d1f68349dd65990d6338ac61ca946e98f8cc5)#tx8gf6l3","timestamp":"now"},
 85  {"desc":"raw(001417a66b36201d81e59f48939c67755358164dec66)#4x3h7lv2","timestamp":"now"},
 86  {"desc":"raw(0014064bcbe8d6208bb58e2147a5bbcd97dbdcd298a4)#qd9yj6zk","timestamp":"now"},
 87  {"desc":"raw(0014f2dc57444e57098f00c57060f97da9cc8fadac3c)#f5uwe54v","timestamp":"now"},
 88  {"desc":"raw(00142fb4648a340c06a2f2f9742d2e7832e8f347a2a2)#q6fx2u8j","timestamp":"now"},
 89  {"desc":"raw(0014564a50bddba40328fd6bea63ea434f6e47269333)#reh8v5v0","timestamp":"now"},
 90  {"desc":"raw(001401f256832bf51c8690c921412ee0d89b8dcc5d8d)#l7tyefep","timestamp":"now"},
 91  {"desc":"raw(0014ad3377bab25e7ce5de2d243c2af1a1a744c5ba25)#n8hylkg4","timestamp":"now"},
 92  {"desc":"raw(00146cde66581cc43d654da4d139660fce31dbd27821)#58pr9vpr","timestamp":"now"},
 93  {"desc":"raw(0014d39f838ee2bd8734fb90354d2b15a1c1eb8900ac)#s526lyam","timestamp":"now"},
 94  {"desc":"raw(001444fafb985f591ccfe2b9990e0e75783bb0bde701)#cpmq7gwp","timestamp":"now"},
 95  {"desc":"raw(00145d75454b0038bc70f12306484544038558140938)#90h7e468","timestamp":"now"},
 96  {"desc":"raw(00147a01fddfdee428515cf4c0ac5ac609fd0724620f)#9cejp0ux","timestamp":"now"},
 97  {"desc":"raw(001406eb1337f2a3daf860b7ac8b9b46c10d0f70010e)#mk362vyg","timestamp":"now"},
 98  {"desc":"raw(0014d6622202a9ddbd298e0abdaebcdd1ebf13f43dea)#8j4fmku8","timestamp":"now"},
 99  {"desc":"raw(00148b4e1f7cb1819dd6f3386d3bd3b49f82ec69bbd5)#gap3ycw7","timestamp":"now"},
100  {"desc":"raw(001462b2fe372c6ca2a39bb453b17bae10310bcd02bb)#ndftc243","timestamp":"now"},
101  {"desc":"raw(0014b9abb44d5504daff04903f5dcfdbd6bb1613d586)#fu2d8tt2","timestamp":"now"},
102  {"desc":"raw(0014762aa4cd9edc41765dff328b3662ac7c424ac771)#mh2d5gj8","timestamp":"now"},
103  {"desc":"raw(00144426d291d198ccd0df68521758afe8cff08743a8)#c40nwl9x","timestamp":"now"},
104  {"desc":"raw(00140b970dd18df6eaaa790b0050b6929587c47f58ea)#wndmanqc","timestamp":"now"}
105]'
  16. hebasto commented at 6:48 am on August 13, 2022: member

    the importdescriptors call is taking ~5s with the homebrew binary, but ~32s with the release binary.

    Could this depend on a folder where binary resides?

  17. casey commented at 3:35 am on August 15, 2022: contributor

    Could this depend on a folder where binary resides?

    Interesting idea! It doesn’t look like it though. I copied my source-built bitcoind and bitcoin-cli to /opt/homebrew/bin, and importdescriptors still took 30 seconds.

  18. achow101 commented at 0:36 am on August 20, 2022: member

    The issue is with sqlite.

    I replicated this issue and found that my self compile was using sqlite provided by homebrew, while the homebrew build uses the system’s sqlite. When I reconfigured the build to use the system sqlite, the import went as fast as the homebrew build. The releases have this issue because we build sqlite in our depends and statically link against it.

    It seems that there is some difference between the macOS provided sqlite and sqlite built by everyone else. IIRC Apple uses sqlite a lot so perhaps they have modified the sqlite they ship to perform faster and those changes are not upstreamed?

  19. fanquake referenced this in commit d216d714aa on Sep 2, 2022
  20. fanquake commented at 2:05 pm on September 2, 2022: member
    I’ve opened #25985 to resolve the build from source half of this issue, and have a few changes to make to our sqlite build in depends that might offer some minor performance improvement (although for all platforms).
  21. achow101 referenced this in commit a52ff619a4 on Oct 17, 2022
  22. fanquake referenced this in commit 419bdc534f on Oct 18, 2022
  23. adam2k referenced this in commit 680fdd0d34 on Oct 19, 2022
  24. fanquake referenced this in commit bf42d7de56 on Oct 20, 2022
  25. sidhujag referenced this in commit ab43cfcabe on Oct 23, 2022
  26. achow101 commented at 11:14 pm on October 26, 2022: member
    #25985 was merged.
  27. achow101 closed this on Oct 26, 2022
  28. satsie referenced this in commit 9154340368 on Oct 31, 2022
  29. vertiond referenced this in commit 773a5ca963 on Nov 3, 2022
  30. janus referenced this in commit 4f0139e8f3 on Jan 20, 2023
  31. bitcoin locked this on Oct 26, 2023


casey 0xB10C maflcko fanquake hebasto achow101

Labels
Bug macOS

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-29 01:12 UTC
This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me