Simple gzip compression can be very useful when you're dealing with a lot of getblocktemplate or getrawmempool requests. It decreases GBT latency and allows you to save a lot of bandwidth.
Just for example, I have few instances of bitcoind which are being used as template sources for eloipool. Each getblocktemplate response has a size of ~2MB. Being gzipped it shrinks down to ~800KB, which is a fairly reasonable size. :)
@CryptoManiac why not just put a reverse proxy like nginx in front of the RPC interface?
You really shouldn't use RPC over a network...
@jameshilliard Well, this also may be a viable solution. Unfortunately, some people are limited to Windows operating system. To handle connections, the NT version of nginx is currently using only select(), so it's very slow. :(
@luke-jr Why not? If you have a walletless daemon, which listening at the private interface, then I'd say that nothing bad could happen.
However, I understand the main idea. Closing for now.
Just for example, I have few instances of bitcoind which are being used as template sources for eloipool. Each getblocktemplate response has a size of ~2MB. Being gzipped it shrinks down to ~800KB, which is a fairly reasonable size. :)
This does increase the latency by bit, though, so on faster networks wouldn't want to use this.
In any case I agree that this belongs externally. RPC is in the first place for local communication, for doing anything like compression, encryption put it behind a tunnel or nginx instance.
@laanwj It depends on your TCP/IP stack settings. In FreeBSD, for example, sometimes it's recommended to set net.inet.tcp.recvspace parameter to 8k. With such receive buffer size you can get a significant improvement even for a local instance of bitcoind, speed up looks like a linear function of compression ratio.
If you want compress data, why not reverse-proxy over apache and enable mod_deflate?
@jonasschnelli Because sometimes it doesn't help. Nginx needs to get the data from the upstream, so it can be affected by tcp/ip settings. Unfortunately, bitcoin doesn't support RPC over unix domain sockets for now.
Unfortunately, bitcoin doesn't support RPC over unix domain sockets for now.
Now that would be a welcome addition. But apparently it needs changes to evhttp. There's a similar ticket open for Transmission, which also uses evhttp.
I just realized that it can be implemented with the current evhttp just fine - do the listening manually, set the socket to non-blocking, then pass the acceptfd to evhttp_accept_socket_with_handle, similarly as I did here for cloudabi: https://github.com/laanwj/bitcoin/blob/2017_03_cabi_fs/src/httpserver.cpp#L360.
@laanwj Nice one, thanks.