Java Authenticator broken for 0.12 RPC server on Linux #7462

jlopp commented at 7:03 PM on February 3, 2016: contributor

I'm in the process of testing 0.12 in our dev environment for one of our java-based services. This service makes RPC requests to bitcoind, but when I upgrade from 0.11 to 0.12 every request now gets a 401 "unauthorized" response.

The RPC server is clearly still functional because I can query it manually via cURL. However, breaking Java's Authenticator may be problematic - it is the recommended way to authenticate according to the wiki: https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)

One strange thing to note is that usage of Java Authenticator still works on OS X with Core 0.12. This appears to be a platform-specific issue. I'm not sure about Windows.

I was able to implement a workaround by setting the authentication in the HTTP request manually as shown in http://www.avajava.com/tutorials/lessons/how-do-i-connect-to-a-url-using-basic-authentication.html

sdaftuar commented at 8:01 PM on February 3, 2016: member

I'm just guessing here, but if this is related to RPC over SSL, then I think support for that was dropped in 0.12, see #5677. (No idea why this wouldn't affect OS X as well though, so maybe this is a bad guess.)

Only other thing I can think of is if this is somehow related to the new cookie file stuff, which would allow you to start up bitcoind without an rpcuser/password in the config (a departure from older versions), though in that case I'd expect curl to not work either unless you were using the credentials from the .cookie file.

EDIT: another guess -- perhaps you're encountering some platform-specific libevent bug?

jlopp commented at 8:29 PM on February 3, 2016: contributor

It's definitely not SSL - never used SSL for RPC. Both of the bitcoind instances I'm testing on have an rpcuser and password set, so it shouldn't be using the cookie.

laanwj commented at 7:41 AM on February 4, 2016: member

To diagnose it would be useful to have an intercept using tcpdump/wireshark of the HTTP session (both the request line and the headers, back and forth).

laanwj added the label RPC on Feb 4, 2016

jlopp commented at 7:44 PM on February 4, 2016: contributor

I'm not familiar with the tools so let me know if I need to change anything to get better info. Here's the output I get when running 0.12rc2:

19:26:15.615855 POST / HTTP/1.1
Content-Type: application/json-rpc
Cache-Control: no-cache, no-store
Accept: application/json-rpc
Pragma: no-cache
User-Agent: Java/1.8.0_72
Host: 127.0.0.1:18332
Connection: keep-alive
Content-Length: 34

{"method":"getrawmempool","id":""}

19:26:15.615877 POST / HTTP/1.1
Content-Type: application/json-rpc
Cache-Control: no-cache, no-store
Accept: application/json-rpc
Pragma: no-cache
User-Agent: Java/1.8.0_72
Host: 127.0.0.1:18332
Connection: keep-alive
Content-Length: 45

{"method":"estimatefee","id":"","params":[1]}

19:26:15.616042 HTTP/1.1 401 Unauthorized
Date: Thu, 04 Feb 2016 19:26:15 GMT
Content-Length: 0
Content-Type: text/html; charset=ISO-8859-1

19:26:15.616082 HTTP/1.1 401 Unauthorized
Date: Thu, 04 Feb 2016 19:26:15 GMT
Content-Length: 0
Content-Type: text/html; charset=ISO-8859-1

On the other hand, here's the output I see when I run my service against 0.11.2:

19:31:40.206717 POST / HTTP/1.1
Content-Type: application/json-rpc
Cache-Control: no-cache, no-store
Accept: application/json-rpc
Pragma: no-cache
User-Agent: Java/1.8.0_72
Host: 127.0.0.1:18332
Connection: keep-alive
Content-Length: 45

{"method":"estimatefee","id":"","params":[1]}

19:31:40.206868 HTTP/1.0 401 Authorization Required
Date: Thu, 04 Feb 2016 19:31:40 +0000
Server: bitcoin-json-rpc/v0.11.2
WWW-Authenticate: Basic realm="jsonrpc"
Content-Type: text/html
Content-Length: 296

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/1999/REC-html401-19991224/loose.dtd">
<HTML>
<HEAD>
<TITLE>Error</TITLE>
<META HTTP-EQUIV='Content-Type' CONTENT='text/html; charset=ISO-8859-1'>
</HEAD>
<BODY><H1>401 Unauthorized.</H1></BODY>
</HTML>

19:31:40.210566 POST / HTTP/1.1
Content-Type: application/json-rpc
Cache-Control: no-cache, no-store
Accept: application/json-rpc
Pragma: no-cache
User-Agent: Java/1.8.0_72
Host: 127.0.0.1:18332
Connection: keep-alive
Content-Length: 45
Authorization: Basic Yml0Y29pbnJwYzphc2Rm

{"method":"estimatefee","id":"","params":[1]}

19:31:40.210753 HTTP/1.1 200 OK
Date: Thu, 04 Feb 2016 19:31:40 +0000
Connection: keep-alive
Content-Length: 44
Content-Type: application/json
Server: bitcoin-json-rpc/v0.11.2

{"result":-1.00000000,"error":null,"id":""}

19:31:40.231422 POST / HTTP/1.1
Content-Type: application/json-rpc
Cache-Control: no-cache, no-store
Accept: application/json-rpc
Pragma: no-cache
User-Agent: Java/1.8.0_72
Host: 127.0.0.1:18332
Connection: keep-alive
Authorization: Basic Yml0Y29pbnJwYzphc2Rm
Content-Length: 45

{"method":"estimatefee","id":"","params":[2]}

19:31:40.231624 HTTP/1.1 200 OK
Date: Thu, 04 Feb 2016 19:31:40 +0000
Connection: keep-alive
Content-Length: 43
Content-Type: application/json
Server: bitcoin-json-rpc/v0.11.2

{"result":0.00022187,"error":null,"id":""}

So, offhand there is a clear difference in the response from bitcoind; it seems to be no longer returning the HTML. I'm not completely sure that this is what causes Java's Authenticator class to break, but it sounds like a distinct possibility. It /appears/ that the way Java's Authenticator class works is that it makes an initial unauthenticated request, then upon parsing a 401 response it sets the Authorization header on future requests. Still not sure why this would be a problem on Linux but not OS X.

However, this does explain why I can upgrade my node from 0.11.2 to 0.12 while the service is still running and the RPC requests continue to work - it's only after restarting my service that the calls fail.

Edit:@laanwj cleaned up the post a bit to manually demux the TCP streams; the transport-layer details make it hard to see the forest for the trees. I think tcpdump has an option to show TCP stream content instead of raw packets, but this will do.

laanwj commented at 9:37 AM on February 5, 2016: member

I think this is the key WWW-Authenticate: Basic realm="jsonrpc". No such header in the new response.

laanwj referenced this in commit f20ae0e336 on Feb 5, 2016

laanwj referenced this in commit 1e493fcbfc on Feb 5, 2016

laanwj referenced this in commit 5bac0a943a on Feb 5, 2016

laanwj commented at 9:53 AM on February 5, 2016: member

@jlopp Can you test if #7472 solves your problem?

jlopp commented at 6:05 PM on February 5, 2016: contributor

:+1: Confirmed fixed; thanks!

laanwj referenced this in commit 7c06fbd8f5 on Feb 8, 2016

laanwj closed this on Feb 9, 2016

laanwj referenced this in commit b2f2b85ad5 on Feb 9, 2016

GamerSg referenced this in commit 47f1bc79e6 on Feb 27, 2016

MarcoFalke locked this on Sep 8, 2021

lyricidal referenced this in commit 1207f666df on Mar 27, 2023