Add ZMQ support. Report blocks and transactions via ZMQ. #5303

 301 | @@ -298,7 +302,7 @@ std::string HelpMessage(HelpMessageMode mode)
 302 |      strUsage += "  -zapwallettxes=<mode>  " + _("Delete all wallet transactions and only recover those parts of the blockchain through -rescan on startup") + "\n";
 303 |      strUsage += "                         " + _("(1 = keep tx meta data e.g. account owner and payment request information, 2 = drop tx meta data)") + "\n";
 304 |  #endif
 305 | -
 306 | +    strUsage += "  -zmqpub=<endpoint>     " + _("Publish blocks and transactions on ZMQ port 'endpoint'") + "\n";

Diapolo commented at 12:32 PM on November 19, 2014:

Is there a default you could list here?

 922 | @@ -919,6 +923,10 @@ bool AppInit2(boost::thread_group& threadGroup)
 923 |      BOOST_FOREACH(string strDest, mapMultiArgs["-seednode"])
 924 |          AddOneShot(strDest);
 925 |  
 926 | +    if (mapArgs.count("-zmqpub"))
 927 | +      ZMQInitialize(mapArgs["-zmqpub"]);
 928 | +
 929 | +

Diapolo commented at 12:32 PM on November 19, 2014:

Nit: Unneeded space and indentation.

   0 | @@ -0,0 +1,25 @@
   1 | +// Copyright (c) 2009-2010 Satoshi Nakamoto
   2 | +// Copyright (c) 2014 The Bitcoin developers
   3 | +// Distributed under the MIT/X11 software license, see the accompanying
   4 | +// file COPYING or http://www.opensource.org/licenses/mit-license.php.
   5 | +#ifndef ZMQPORTS_H

Diapolo commented at 12:33 PM on November 19, 2014:

Better: BITCOIN_ZMQPORTS_H here and for the end comment.

   0 | @@ -0,0 +1,175 @@
   1 | +// Copyright (c) 2009-2010 Satoshi Nakamoto

Diapolo commented at 12:35 PM on November 19, 2014:

That file has the wrong intentation all over the file and the wrong license.

 143 | @@ -144,6 +144,12 @@ AC_ARG_ENABLE([glibc-back-compat],
 144 |    [use_glibc_compat=$enableval],
 145 |    [use_glibc_compat=no])
 146 |  
 147 | +AC_ARG_ENABLE([zmq],
 148 | +  [AC_HELP_STRING([--disable-zmq],
 149 | +  [Disable ZMQ notifications])],
 150 | +  [use_zmq=no],

luke-jr commented at 1:05 PM on December 5, 2014:

This should use $enableval, or it will disable it with --enable-zmq

Minor nit: prefer --disable-zeromq as the name.

 765 | +if test "x$use_zmq" == "xyes"; then
 766 | +  AC_MSG_RESULT([yes])
 767 | +  PKG_CHECK_MODULES([ZMQ],[libzmq],
 768 | +    [AC_DEFINE([ENABLE_ZMQ],[1],[Define to 1 to enable ZMQ functions])],
 769 | +    [AC_DEFINE([ENABLE_ZMQ],[0],[Define to 1 to enable ZMQ functions])
 770 | +     AC_MSG_WARN([libzmq not found, disabling])

luke-jr commented at 1:05 PM on December 5, 2014:

Better to error if --enable-zmq is explicitly passed.

  51 | +  zmqPubSocket = zmq_socket(zmqContext, ZMQ_PUB);
  52 | +  if (!zmqPubSocket) {
  53 | +    zmqError("Unable to open ZMQ pub socket");
  54 | +    return;
  55 | +  }
  56 | +

amrali commented at 10:56 PM on March 11, 2015:

I think ZMQ_SNDHWM should be set here to configurable value (at least through configure.ac) this provides the option to the user on when to start dropping messages. the default value of zero means a queue as large as the available memory which is dangerous because networks could be slow or the production of messages is faster than the network I/O output -- a precursor to a memory overflow crash.

ZMQ_LINGER is another important socket property to set here, the default value of -1 (infinite linger period on socket shutdown when there are queued messages to be sent) will prevent the application from shutting down gracefully. A configurable value (or a reasonable one) should be set to prevent this scenario. The linger period starts after a call to zmq_close on a socket or zmq_ctx_term on a context, once it expires all queued messages will be dropped.

 143 | +      zmq_setsockopt(zmqPubSocket, ZMQ_LINGER, &linger, sizeof(linger));
 144 | +      zmq_close(zmqPubSocket);
 145 | +      zmqPubSocket = 0;
 146 | +    }
 147 | +
 148 | +    zmq_ctx_destroy(zmqContext);

amrali commented at 11:02 PM on March 11, 2015:

I think zmq_ctx_term should be used here instead if ZMQ_LINGER is to be set to some reasonable value to allow sending off what is possibly the last few queued messages (see my comment above the zmq_bind call). Not sure if this is desired or important to you.