initial walletpassphrase slow and freezing #12423

When running walletpassphrase on a large wallet.dat (ie about 300,000 segwit P2SH addresses) the command sometimes takes a very long time, and if command is run multiple times at once (while running slow) it causes bitcoin core to freeze (stops syncing and requires kill -9 to stop/restart)

During the locked up walletpassphrase commands being run the following messages start to appear in the debug.log:

018-01-24 01:01:04 socket sending timeout: 1201s

Note that this problem is intermittent, so sometimes the command runs instantly, other times it takes a few minutes, and other times it completely freezes the node.

I think this can be closed as actually the issue appears to have been insufficient disk IO capacity on the nodes virtual machine.

I have this issue a lot as well and disk I/O capacity would be no issue on this machine.

@Crypto2 mind sharing system, bitcoind version and wallet size?

Xeon E5-1660 v3 @ 3.00GHz w/64GB of RAM and Intel NVMe storage, Bitcoin 0.18.0 (but been happening at least through the whole 0.17 series as well), wallet size ~5GB. dbcache=16384 in config file.

The machine is dedicated just to running bitcoind, nothing else on it except Linux itself.

Bitcoin-Core is currently not optimized for wallets that large. During walletpassphrase, each key gets a AES256CBC decryption (in your case 300'000) and all the decrypted keys are kept in memory. During the time of that decryption process, the main lock is held which can lead to disconnects on the p2p side.

Resurrecting and older one here but has anyone thought about having walletpassphrase decrypt the master key then just decrypt the keys as needed on demand versus decrypting them all at once? (odds are you won't end up using but a small percentage of the privkeys anyway on larger wallets.)

@Crypto2 That's what happens. I don't know why decrypting a larger wallet would be slower.

@Crypto2 That's what happens. I don't know why decrypting a larger wallet would be slower.

The first time the wallet is unlocked, every key is decrypted although the decrypted keys are not stored. After that, any subsequent unlocks will only decrypt one key to determine which encryption key to use.

Yeah it's that 1st time I'm talking about. That's the killer; takes 30+ minutes lol. Maybe should add a way to skip the decrypt every key part.

Maybe should add a way to skip the decrypt every key part.

I believe the full key decryption is being done in order to check for corruption. Maybe an equally valid way would be to have a checksum/hash of the encrypted data? I'll investigate.

This is in the 0.18.0 codebase, I see they have moved it to wallet.cpp in master, but as a quick fix I did this: (default off for safety)

@@ -184,6 +184,7 @@ bool CCryptoKeyStore::Unlock(const CKeyingMaterial& vMasterKeyIn, bool accept_no

         bool keyPass = mapCryptedKeys.empty(); // Always pass when there are no encrypted keys
         bool keyFail = false;
+        bool quickUnlock = (fDecryptionThoroughlyChecked || gArgs.GetBoolArg("-quickwalletdecrypt", false));
         CryptedKeyMap::const_iterator mi = mapCryptedKeys.begin();
         for (; mi != mapCryptedKeys.end(); ++mi)
         {
@@ -196,7 +197,7 @@ bool CCryptoKeyStore::Unlock(const CKeyingMaterial& vMasterKeyIn, bool accept_no
                 break;
             }
             keyPass = true;
-            if (fDecryptionThoroughlyChecked)
+            if (quickUnlock)
                 break;
         }
         if (keyPass && keyFail)

Checksum/hash would be better for sure though.

@achow101 In native descriptor wallets this won't be a problem anymore, right? As there'll just be one private key for the master key?

@achow101 In native descriptor wallets this won't be a problem anymore, right? As there'll just be one private key for the master key?

Yes

Really weird this wasn't fixed in 0.19.0+