Password for encrypted wallets on all pw protected functions (such as sendtoaddress, dumpprivkey, etc) #23620

Is your feature request related to a problem? Please describe.

<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->

I want this to be VERY clear, this is not from an experience of being hacked (well, yet anyway, knock on wood). This is completely theoretical based on my experience creating a webapp that interacts with bitcoind via RPC calls and thus it got me to thinking about how the hotwallet could be hacked, how to prevent it, etc.

Unlocking the wallet for X amount of time is a MASSIVE security flaw imo. Let's say you have 10 'withdraw' processes running and they all have to 'unlock' the wallet to processes their sends. Thats fine if they finish quickly (this isnt always the case btw). But what if you have 100s or 1000s or something just causes it to go slowly and you have to increase the unlock period for some reason? Then what you've got is a wallet that is essentially in an "unlocked" stated almost constantly. This is a really bad idea. It would be better to ask for the pw on specific functions instead of a global scope.

Use case: Server 1: Apache webserver, maybe has other services on it like mysql, ssh, etc, something that's vulnerable to be hacked Server 2: Hotwallet server, has absolutely nothing on it other than bitcoind in RPC mode, it's buttoned down as much as possible short of actually unplugging it from the router.

Server 1 has RPC creds on it so that some application (like a web app) can interact with a BTC wallet. It does NOT have the BTC wallet encryption pw on it however. So it's only means of sending out BTC or creating transactions is when Server 2 checks a queue that resides on server 1......... Months go by and one day the machine is HACKED. The hacker then monitors the server and notices that they can make RPC calls to the wallet (oh crap). But the wallet is encrypted so we're good right?! HA. WRONG. They have access to the queue which seems to take a long time to empty. Then they notice that the wallet is being unlocked consistently to empty the queue...... It this point it's game over for mr wallet.

Server 2 Describe the solution you'd like

<!-- A clear and concise description of what you want to happen. -->

I'd like to see at minimum sendtoaddress, sendmany, sendrawtransaction, send, and createrawtransaction refuse to function if a pw on an encrypted wallet is detected. Essentially anything that isnt a Read (except for something like dumping keys, obviously that should be as well).

Describe alternatives you've considered

<!-- A clear and concise description of any alternative solutions or features you've considered. -->

There are alternatives but I'm not interested in ANY of them as they are overly convoluted, difficult to setup, and just not a good user experience as a developer at all, all the while solving no security problems at all, at some point on some device (whether connected to other devices or not) that unlock period is going to come into play and when it does, this vulnerability rears it's head. I realize I can setup a 'read only' wallet (and I have), but had bitcoind not had this issue, I would not have had to do this for my purposes. I can see other use cases involving cell phones as well.

Additional context

<!-- Add any other context or screenshots about the feature request here. -->

I don't understand your solution. The send functions already don't work if there is a passphrase on the wallet (i.e. it is encrypted) and the wallet is locked. Are you suggesting that they simply don't function at all if the wallet is encrypted? That would make the wallet useless.

I would be possible to add a passphrase argument to all of the functions that need the wallet to be unlocked that would obviate the need for constant walletpassphrase and walletlock calls, but that would be a convenience rather than providing any additional security.

No, what you dont understand is that the wallet has to be "unlocked for X seconds" which leaves it open for OTHER processes other than the one making the send to make it's OWN send (IE: hacker emptying your wallet for example). Time locked mechanisms are in general a sign of something being developed incorrectly (although not always, IE: Bitcoin's mining algorithm to slow things down is a good example of when to use a time lock security method). The bitcoind's walletpassphrase RPC call is one such animal. It should be eliminated and replaced with pw arguments on functions that need to be behind a pw in an encrypted scenario. Maybe I wasnt clear about that part. 1 Ring to Rule them all is not such a great security plan.

Wallet encryption is for protecting the on-disk wallet file. For protecting RPC users, use RPC passwords.

RPC passwords are not for securing a wallet and have no business being in this discussion. The RPC pw creds are on the vulnerable server making them irrelevant to what the problem is (time locking security method on the WALLET, not on a network protocol which is RPC). All RPC server's (mine included) have u/p's on them but that doesnt mean a wallet is secure.

I'd like to see at minimum sendtoaddress, sendmany, sendrawtransaction, send, and createrawtransaction refuse to function if a pw on an encrypted wallet is detected. Essentially anything that isnt a Read (except for something like dumping keys, obviously that should be as well).

Sorry I don't understand this. what else would be required apart from walletpassphrase and rpcpassword ? Can you share some example or demo in which these two don't work?

And if its really an issue which can be considered vulnerability, please send email to https://github.com/bitcoin/bitcoin/blob/master/SECURITY.md

Maybe I'm not explaining this right. Do you know what a time lock security method is and why they are used? walletpassphrase unlocks your wallet for X amount of time and if you dont lock it again afterwards OR your process simply takes to long, some other process can come along and hijack a now unlocked wallet. There is a time and place for time locked security measures, however accessing a wallet is not one of them.

Maybe this analogy would be better. Go unlock your phone and put it on the table in the middle of a busy prison. How long do you think it will be before someone picks it up and starts poking around on it to see what they can find? Here we have the same problem. Unlocking a wallet for X amount of time. Each function should instead ask for the pw rather than having 1 function that unlocks everything for X milliseconds.

I do consider this a security vulnerability but it's a DESIGNED security vulnerability. Thus I think it belongs in here as a feature, not a security bug. I have not heard a single good reason why a time lock mechanism was used so far. Take coinomi for example, does their wallet unlock for X amount of time? No. It asks for a pw on every function that should be protected. Why do you think that is?

Again, the wallet encryption password is for encrypting the wallet file on disk; it is for preventing the private keys being exposed to disk in plaintext form, which may matter if the wallet is going to be stored/copied to untrusted media. It is not for authenticating RPC clients. Access to the wallet from RPC is controlled through RPC passwords. If an RPC client has valid credentials, it has the right to transact.

It's not for "encrypting" the wallet, that's only PART of what adding a password does. It also controls access to RPC functions (sendtoaddress for example). If the RPC call doesnt first unlock the wallet, it cant make a send. The RPC password is only for access to the RPC server, it's not for unlocking the wallet. The wallet server itself has to make RPC calls to interact with the wallet just the same as a Webserver. You cant use the same U/P for both operations. Even if you did, the problem still remains, RPC has no way to lock SPECIFIC functions. That's not it's job. Thats the job of the wallet itself (thus why each RPC function should ask for the wallet's encryption pw, not the RPC pw, they are not the same nor used for the same purpose).

RPC from the wallet server itself is trusted (to a certain point) as it has to be. RPC from a webserver is NOT trusted and should never be.

It also controls access to RPC functions (sendtoaddress for example).

Well, no. That's not what it is for, sorry. You wanting to have that function does not make it so.

RPC has no way to lock SPECIFIC functions. That's not it's job.

I'd say that's exactly the function of RPC passwords, only it's not implemented. What it sounds like to me is that you're asking for a way to prevent certain RPC clients from having restrictions on their access (e.g. no ability to send coins, or being restricted to certain wallets). There have been pull requests and ideas around that in the past, though none currently actively being developed.

There is no point in this function being carried by the encryption password, as that password would be shared across all RPC clients.

They arent being developed because it's pretty clear to me that no one here understand Time Locking security methods (which is really scary considering the forum we're talking in) and when they should and should not be used. You dont time UNlock a wallet. Regardless of how it's being accessed whether it be for an RPC call or putting your phone down on the table in the middle of a busy prison. Period. End of story. I guess I'll just have to look at using another wallet option other than core when the time comes. For now I'm limping along with Read-Only wallets which are utterly terrible for Hotwallet applications.

Anyway, thanks for the discussion. I guess it's better than having gone ignored. Didnt fix anything but maybe some day someone who understands the problem like I do will be able to explain it better. Time locking/unlocking doesnt belong on wallet applications. I thought that was just common sense. But I guessed wrong.

Am I right is describing the setup you're envisioning as: you have two RPC clients, talking to the same bitcoind RPC server, and both interacting with the same wallet, only one of them is supposed to have watch-only access, while the other also is supposed to have the ability to send coins?

Would it be an option to have two separate bitcoind processes, and have an actual watch-only copy of the wallet in the one which isn't supposed to have send permissions (without private keys entirely)? That way you don't have to expose the private key material at all to any system connected to the one which is supposed to have watch-only access.

If I understand correctly, the attack is that the wallet is unlocked for some time during which both legitimate and malicious RPCs could be received by bitcoind which require the wallet to be unlocked, and thus an attacker could send funds from the wallet while it is unlocked. And thus the proposed solution is to limit the scope of the passphrase to each RPC request that needs it by having a parameter for the passphrase rather than having the passphrase provided in a separate RPC.

But in order for this to be a problem, an attacker would first have to acquire RPC credentials. Because RPC traffic is unencrypted, one way an attacker could acquire those credentials is via a man-in-the-middle attack and intercept the request. But if they are able to do that, then they can also intercept the contents of the RPC request which would include the encryption password, regardless of whether it is provided in the RPC or as a separate walletpassphrase call. So the proposed solution does not prevent this form of attack.

Another way the attacker could acquire RPC credentials is by compromising the machine sending the RPCs. Then they may be able to either intercept the traffic as above, or find the file(s) containing the RPC credentials and read them. That machine would also have to store the wallet passphrase somewhere, so if the attacker can get the RPC credentials, then it stands to reason they can get the wallet passphrase as well. In this scenario, it once again does not matter how the passphrase is provided to bitcoind, whether that's walletpassphrase or in the specific RPC.

Theoretically, an attacker could acquire RPC credentials without acquiring the wallet passphrase, in which case the proposed solution would actually be a benefit. However I am unable to think of an actual scenario where that could happen.

Overall, the credentials that really matter here are the RPC credentials, and not the wallet passphrase.

I don't think the proposed solution would necessarily be bad to implement, but the benefits it actually provides is not clear. Because of the way that passphrases are handled in the wallet currently, implementing the proposed solution would require significant refactoring and overhaul in the way that wallet passphrases are handled. This would be fairly invasive and cause conflicts with a lot of other wallet work going on. So there needs to be a practical benefit for doing that, and not just a hypothetical one.

The benefit is to get rid of the Time unlock/lock methodology on a wallet application. No wallet should work like that. To me, that would be worth 100s of man hours to refactor what needs to be refactored because having a wallet vulnerable for any amount of time simply because a "lock" function failed to work (which WILL happen when dealing with RPC IE: network hiccup) or because some process required the wallet to be unlocked for a long period of time is simply unacceptable given that this is a financial based application. I was rather floored when I discovered a time mechanism on the daemon to begin with. At first I thought I was just missing something, but after enough digging, it became clear, I wasnt missing anything, it really was a security issue. The best thing I can equate this to is how wallets on phones work. Would you trust a wallet that unlocked your phone and wallet for X amount of time and expected you to "re-lock" it every time or wait for X amount of time to pass? I know I sure wouldnt. I'd want it to ask for a pw on every single command that needs it. None of my phones nor phone wallets are setup to "time out" in order to relock (although it's an option). It should be noted that implementing this doesnt necessarily mean that the walletpassphrase has to go away. It can still exist for those that want to brave the waters and use it (just like the millions of people walking around with phones that have lockscreens bypassed for 10, 20, even 60 mins, ACK!). But I'd be on the "get rid of it" side of the fence.

"Theoretically, an attacker could acquire RPC credentials without acquiring the wallet passphrase, in which case the proposed solution would actually be a benefit. However I am unable to think of an actual scenario where that could happen."

You just described every webserver setup in existence (IE: How many of your servers still run apache and mysql on the same server? Same goes for apache and bitcoind, dont run them on the same server lol). This would happen in any scenario where a webserver is compromised, but the wallet server is not. And since webserver is compromised, so are the RPC creds because the RPC creds HAVE to reside on the webserver. If that's not a valid scenario, nothing is :D The difference is that the Encryption PW only resides on the Bitcoind server, NOT the apache server. So the bitcoind server is free to make sends since it has the encryption pw available to it in a file somewhere (again, remember this is a HOTwallet setup, it's expected to have at least some exposure but that doesnt mean it has to be completely wide open with a time unlock/lock mechanism exposing itself to RPC abuse), where as the Apache server never does (unless of course, you're a hacker looking to exploit RPC which is the case I explained in the initial post). Ever. It simply puts them in a queue for the bitcoind server to pick up via some script (in my case a PHP/Mysql cron job).

RPC has no way to lock SPECIFIC functions. That's not it's job. Thats the job of the wallet itself (thus why each RPC function should ask for the wallet's encryption pw, not the RPC pw, they are not the same nor used for the same purpose).

It is the RPC's job to restrict what functions are or are not allowed, or at least it should be (obviously that is not implemented). The wallet passphrase is intended only for encrypting the wallet; the fact that some RPCs are unavailable when the wallet is locked is a side effect of the encryption, not that the lack of a passphrase is making those RPCs unavailable. It's more-so the wallet informing the RPC caller that it is unable to perform the function rather than the RPC server saying that the function is disallowed.

The benefit is to get rid of the Time unlock/lock methodology on a wallet application. No wallet should work like that. To me, that would be worth 100s of man hours to refactor what needs to be refactored

IMO there is no tangible benefit for doing that over a RPC permissions solution. To me, it is not worth the effort, but if it is to you, feel free to implement it yourself or pay someone to do so - this is an open source project after all. An existing pattern being "bad" is not a reason to change it; a reason to change it is that it causes actual observed problems.

The best thing I can equate this to is how wallets on phones work. Would you trust a wallet that unlocked your phone and wallet for X amount of time and expected you to "re-lock" it every time or wait for X amount of time to pass?

Is that not how most things work? I don't use mobile wallets, but even for the more security sensitive things on my phone, they stay "unlocked" until after I close the app by pressing the home button. Most users expect to have distinct sessions where they enter their passphrase once at the beginning of the session, do a bunch of actions, and then sign out. Even with my phone itself, it locks after some minutes of inactivity, or I have to lock it manually by pressing the lock button. Generally, having to enter a passphrase for every action is bad UX. IIRC the few times I have experimented with mobile wallets were largely the same.

You just described every webserver setup in existence (IE: How many of your servers still run apache and mysql on the same server? Same goes for apache and bitcoind, dont run them on the same server lol). This would happen in any scenario where a webserver is compromised, but the wallet server is not. And since webserver is compromised, so are the RPC creds because the RPC creds HAVE to reside on the webserver.If that's not a valid scenario, nothing is :D The difference is that the Encryption PW only resides on the Bitcoind server, NOT the apache server. So the bitcoind server is free to make sends since it has the encryption pw available to it in a file somewhere (again, remember this is a HOTwallet setup, it's expected to have at least some exposure but that doesnt mean it has to be completely wide open with a time unlock/lock mechanism exposing itself to RPC abuse), where as the Apache server never does (unless of course, you're a hacker looking to exploit RPC which is the case I explained in the initial post). Ever.

Ah, I see.

As noted by others, a more general solution is to have restrict RPCs based on the RPC credentials. This would also protect unencrypted wallets as well.

It simply puts them in a queue for the bitcoind server to pick up via some script (in my case a PHP/Mysql cron job).

If you're doing that, how does having the wallet passphrase per RPC help? The attacker who has compromised your web server can insert their own send commands into your queue.

Is that not how most things work? I don't use mobile wallets, but even for the more security sensitive things on my phone, they stay "unlocked" until after I close the app by pressing the home button.

ACK!!! NO! That is NOT how most things work! It's literally an option that you're not disabling! You should absolutely NEVER own a phone that is Time Lock/Unlocked, nor should you be using a wallet that has the same feature enabled.

I use Android, when I unlock the phone, I use it. Then I turn the screen off. There is an option to bypass the lock screen if I then immediately turn on the screen again. Its an OPTION. It's not baked in default operation. All my phones lock whenever the screen is OFF regardless of time length. Ok, so thats the phone example. Let's skip to a real world WALLET example:

Coinomi, a great little wallet that millions use (even though it's closed source, I wish it werent). When you open it, it lets you view whatever you want (just like a read-only bitcoind wallet). But if you try anything that common sense dictates should require a pw (such as sending coins, showing seed phrase, etc), it PROMPTS for a pw. Once you enter the pw, the wallet REMAINS LOCKED. It doesnt unlock for X amount of time. The end result is that only the calling function had access to the password protected area, no other function or process could have used it. For example, if you send 2 coins out at the EXACT same time, giving 1 a pw, and the other no pw, the one without a pw would FAIL. But in Bitcoind, only ONE process needs to provide the pw in order for BOTH calls to work. That's a problem.

The "time" piece of this methodology we're discussing both in bitcoind and in other wallets/phones is for CONVIENENCE ONLY. It' actually LOWERS the security of your device when they are setup with time lock/unlocks.

If you're doing that, how does having the wallet passphrase per RPC help? The attacker who has compromised your web server can insert their own send commands into your queue.

Currently with the time locking, the attacker who has access to ONLY RPC via the webserver can wait for the wallet server to start processing it's 'withdraws' or more likely, just flood it with send requests until one works. One or more of them is bound to work because the wallet is going to be in "unlocked" state at some point because the Wallet server is being forced to "unlock" it for X amount of time. But if this time locking mechanism was removed, that type of attack would never work. It would just forever be declined since the attacker doesnt know nor have access to the wallet's encryption pw. One of the MAIN reasons for separating ANY service from a web server is specifically to combat this type of situation, where creds can be safely stored away from the webserver. Bitcoind is absolutely no exception to that rule. It's not always for performance gains (although that is another great reason for separation).

PS: can you msg my mom real quick with your phone? Then go get your coat real quick? You can just leave your phone on the table too. Thanks. LOL. Turn off that option, you dont need it :)

I use Android, when I unlock the phone, I use it. Then I turn the screen off. There is an option to bypass the lock screen if I then immediately turn on the screen again. Its an OPTION. It's not baked in default operation. All my phones lock whenever the screen is OFF regardless of time length.

The locking on screen off is what I was describing, not the remaining unlocked behavior. I had interpreted your analgy to be entering a passphrase before every single action, e.g. before opening an app on your phone.

Currently with the time locking, the attacker who has access to ONLY RPC via the webserver can wait for the wallet server to start processing it's 'withdraws' or more likely, just flood it with send requests until one works.

Sure, but you stated that the web server was compromised, and in this scenario it sounds like the web server can initiate sends without the wallet's passphrase because it places RPCs. If that is the case, then the passphrase is not useful.

Regardless, being able to disallow RPCs based on the RPC credentials would achieve the same thing.

Fun fact, the GUI does exactly what you describe. The wallet is unlocked when necessary, and locked when signing is done. IIRC this is done as a convenience, not for security. I'm not sure whether other processes could access the wallet while it is unlocked via the GUI as I believe the GUI thread holds a lock on the wallet the entire time. An implementation for the RPC could probably be done in a similar way, but IMO that would be rather fragile (easy to mess up) so it wouldn't really provide the security guarantees that you seek.

Sure, but you stated that the web server was compromised, and in this scenario it sounds like the web server can initiate sends without the wallet's passphrase because it places RPCs. If that is the case, then the passphrase is not useful.

That is NOT the case. Notice how I said "Flooding". The reason for the flooding is specifically because the comprimised server does NOT have the encryption pw in order to make it's own sends. It has to wait for some other process/server/person to unlock the wallet and then it can piggy back on THEIR pw entry, it never makes it's own pw call. The passphrase method I'm trying to describe would to stop the "flooding" scenario from ever working in the first place. Have you ever been to a gated community or building and then walked in after someone else used their passcard to get in? Thats the scenario we're hitting here.

BTW, I can tell you that the GUI suffers from the SAME PROBLEM. If you make a send transaction and then something else on the device has been spamming RPC calls to do a send..... (assuming RPC is on of course, it's off by default I think, unless u've been tricked into enabling it) Guess what's going to happen. By by wallet balance :( Time locking is just so bad in most scenarios. There are very very few scenarios where time locking security is the correct method.

I dread the day when I see this on the security mailing list in response to some trojan that does exactly this type of spamming: DO NOT unlock your wallets until XYZ patch has been put into place as your coins may be stolen.

The passphrase method I'm trying to describe would to stop the "flooding" scenario from ever working in the first place. Have you ever been to a gated community or building and then walked in after someone else used their passcard to get in? Thats the scenario we're hitting here.

Tailgating? hmm I have seen and tried this in offices but never online. Interesting concept TBH which could be used in attacks as I already searched few things. Can be considered any attack with some social engineering IMO.

Hmm, yes tailgating could be a good description of it. Like I said, I'm not the greatest at describing things but I do tend to think I do an ok job at it :P But seem to be failing pretty miserably here trying to describe the problem. So ya, an RPC user can "tailgate" on someone else's unlock process, regardless of where the call came from, whether it was via RPC or the GUI client. There are already many known trojans that do screen captures of seed phrases on phones, I wouldnt doubt there theres probably one that spams RPC calls as well and it would be difficult to know how the coins went missing since both attacks would show the same exact symptoms, unauthorized transaction. However I think this type of attack will be more prevalent on hotwallets (such as exchanges, ecommerce sites, etc) where RPC is likely to be enabled.

For myself I know I wont be a victim of this type of attack since I was able to identify it myself while developing an app thats communicating with bitcoind via RPC and have switched to using read-only wallets on the webservers themselves (terrible waste of resources, but its what I have to work with atm until this time lock issue goes away). But for others who dont think like I do, they may not even realize that this is a vulnerability. And if I can see this, I know someone smarter and meaner than me has seen and used this.

For myself I know I wont be a victim of this type of attack since I was able to identify it myself while developing an app thats communicating with bitcoind via RPC. But for others who dont think like I do, they may not even realize that this is a vulnerbility.

I don't know what is your first language. Take your time and draft a good post in your language with pictures which can always be translated by others or algos. Maybe it could help other devs or included in RPC docs security section in this repository.

Can you explain why this is not fixed by using -rpcwhitelist for the apache/mysql webserver? If there is no way for the server to call the RPC method in the first place, it shouldn't matter whether the wallet is unlocked or not.

Side note: You can think of walletpassphrase working a bit like sudo.

1. It's nothing like sudo, sudo is actually secure, walletpassphrase is NOT secure. Sudo doesnt suffer from other users hijacking your pw session or other users being able to spam sudo hoping to get in when you enter a pw for your own sudo session. Sudo is a shaky analogy since sudo is actually safe to use. walletpassphrase works like a community guarded gate. It's very easily circumvented simply by following someone else in. As prayank23 so elegantly put it, tailgating. walletpassphrase is vulnerable to other processes tailgating their way in. There is no tailgating problem with sudo even though it DOES have a timed lock/unlock feature. However, you're not the brightest on the planet if you actually enable that feature (it's on by default in ubuntu, it's always the first thing I TURN OFF) unless you are glued to your chair or trust everyone in the room when you're not at the keyboard ;) However, you dont have to worry about someone else logging in with their account and hijacking your open time unlocked sudo session like you do with walletpassphrase.

2. webserver not being in the rpcwhitelist would make the webserver unable to call ANY RPC functions at all. That's pointless. A webserver must be able to communicate with a hotwallet. Otherwise, what's the point of RPC at all, at that point, might as well not even have any RPC functionality if it cant be used in very valid and practical scenarios.

webserver not being in the rpcwhitelist would make the webserver unable to call ANY RPC functions at all. That's pointless.

That was not my suggestion. I suggested to create a list of allowed RPCs for the webserver.

Ok, if it's possible to whitelist specific RPC calls and not have that whitelist file on the webserver, that could be a workable solution. How would you go about creating such a whitelist, I'm not aware of any such capability in Bitcoind. I thought you were talking about whitelisting the server itself via firewall rules of iptables or something.

See bitcoin-core.daemon --help | grep -C19 rpcwhite for the format.

Ok cool, now we're gettin some where :D This whitelist idea isnt perfect but it's a good enough solution for what I would need. I would totally be satisfied with locking down which RPC functions the webserver can use. It certainly doesnt need to be using them all! lol. Thanks for the tip on the whitelist, I didnt think to look at help on bitcoind itself, I've been using bitcoin-cli help menus.

This whitelist idea isnt perfect but it's a good enough solution for what I would need.

Good to hear your immediate issue is fixed.

If this feature request here is still relevant, I suggest opening a new issue, summarizing the discussion in a few technical sentences (not using screaming uppercase, etc). Otherwise, it will be time consuming to read through 28 comments to understand this feature request in the future.

Closing for now per #23620 (comment)