Issue with testing GBT/STM/GWK bfgminer-to-bitcoind communication #5785

The content below makes more sense when referencing the diagram at https://bitcoin.org/en/developer-guide#solo-mining.

Currently running mainNet and testNet bitcoind v0.9.4 instances on an OS X 10.10 laptop and an OS X 10.9 MacMini, for a total of four blockchains. Also running bfgminer 5.0.0 (http://luke.dashjr.org/programs/bitcoin/files/bfgminer/5.0.0/bfgminer-5.0.0.zip) on both computers. Was able to successfully get bfgminer to communicate with bitcoind instances (mainNet and testNet) on the localhost using GetBlockTemplate & Stratum & GetWork.

However, I can't get bfgminer on one computer to communicate to a bitcoind instance on the other computer. Yet the bitcoind instances on the laptop and the MacMini communicate just fine, no known firewall issues. (The dedicated MacMini is effectively being used a proxy/relay for the laptop that frequently connects and disconnects <by having "connect=192.168.222.50" within the laptop's bitcoin.conf file>, for quick blockchain synchronizations.)

For the MacMini proxy, the testnet config contain an "rpcallowip=192.168.222.0/24" statement which should allow for communication between the two computers on the private 192.168.222 Class C network. Pointing the bfgminer on the laptop (having a USB-based ASIC device attached) to the Macmini fails for all three mining protocols. I initially thought it was a bfgminer issue, but now I believe it is a bitcoind issue because when I point my Antminer C1 to the bitcoind it also never connects.

Is there something I'm not doing right? Is there some other secret sauce I need to know about?

FYSA - I'm only tweaking working bfgminer JSON instances to point a miner in a different direction. Been doing this for a while.

bitcoind only supports GBT (not stratum nor getwork). What URI are you using with BFGMiner, and what error(s) do you get?

It's a pleasure to receive a response from the bfgminer guru...

I'm aware the documentation emphasizes that only GBT is supported. However, when the miner and bitcoind were on the same host. The bfgminer provided me text feedback indicating that getwork and stratum were being applied (up top) with the JSON bfgconfig files being configured accordingly.

Sticking to the GBT theme that you are driving, here is what the corresponding JSON conf files looks like.

1 { 2 "pools" : [ 3 { 4 "url" : "http://192.168.222.90:18332", 5 "user" : "1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa", 6 "pass" : "NOT-TO-BE-SHARED", 7 "pool-priority" : "0", 8 "generate-to" : "1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa", 9 "coinbase-addr" : "1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa", 10 "coinbase-sig" : "blacknight@bellsouth.net" 11 } 12 ] 13 , 14 "no-stratum" : true, 15 "no-getwork" : true, 16 "kernel-path" : "/Users/Skaht/Projects/BFGminer/bin", 17 "config" : "/Users/Skaht/Projects/BFGminer/etc/BFG-testnet-GBT-Solo-remote.conf", 18 "debuglog" : true, 19 "algo" : "fastauto", 20 "api-listen" : true, 21 "api-mcast-port" : "4030", 22 "api-port" : "4030", 23 "expiry" : "120", 24 "expiry-lp" : "3600", 25 "failover-switch-delay" : "300", 26 "log" : "5", 27 "no-pool-disable" : true, 28 "no-client-reconnect" : true, 29 "queue" : "20", 30 "quiet-work-updates" : true, 31 "quiet-work-update" : true, 32 "scan-time" : "60", 33 "show-processors" : true, 34 "show-procs" : true, 35 "skip-security-checks" : "0", 36 "submit-stale" : true, 37 "temp-hysteresis" : "3", 38 "verbose" : true, 39 "shares" : 0, 40 "scan" : [ 41 "/dev/cu.usbserial-00001014" 42 ], 43 "set-device" : [ 44 "OCL:binary=no" 45 ], 46 "api-allow" : "W:127.0.0.1,192.168.222/24" 47 }

Give me a few minutes to recreate the conditions so I can cut-n-paste the errors.

VR/Scott

Here is the cut-n-paste with a few privacy redactions resulting from a very slow firing up of the miner after issuing "bfgminer -c BFG-testNet-GBT-Solo-remote.conf":

[2015-02-10 21:06:17] Press any key to exit, or BFGMiner will try again in 15s. [2015-02-10 21:06:33] No servers were found that could be used to get work from. [2015-02-10 21:06:33] Please check the details from the list below of the servers you have input [2015-02-10 21:06:33] Most likely you have input the wrong URL, forgotten to add a port, or have not set up workers [2015-02-10 21:06:33] Pool: 0 URL: http://192.168.222.90:18332 User: 1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa Password: NOT-TO-BE-SHARED [2015-02-10 21:06:33] Press any key to exit, or BFGMiner will try again in 15s. [2015-02-10 21:06:48] Testing pool http://192.168.222.90:18332 [2015-02-10 21:06:48] HTTP request failed: The requested URL returned error: 403 Forbidden [2015-02-10 21:06:48] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-10 21:06:48] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-10 21:06:48] Closing socket for stratum pool 0 [2015-02-10 21:06:48] Pool 0 slow/down or URL or credentials invalid [2015-02-10 21:06:49] No servers were found that could be used to get work from. [2015-02-10 21:06:49] Please check the details from the list below of the servers you have input [2015-02-10 21:06:49] Most likely you have input the wrong URL, forgotten to add a port, or have not set up workers [2015-02-10 21:06:49] Pool: 0 URL: http://192.168.222.90:18332 User: 1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa Password: NOT-TO-BE-SHARED [2015-02-10 21:06:49] Press any key to exit, or BFGMiner will try again in 15s. [2015-02-10 21:07:05] No servers were found that could be used to get work from. [2015-02-10 21:07:05] Please check the details from the list below of the servers you have input [2015-02-10 21:07:05] Most likely you have input the wrong URL, forgotten to add a port, or have not set up workers [2015-02-10 21:07:05] Pool: 0 URL: http://192.168.222.90:18332 User: 1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa Password: NOT-TO-BE-SHARED [2015-02-10 21:07:05] Press any key to exit, or BFGMiner will try again in 15s. [2015-02-10 21:07:20] Testing pool http://192.168.222.90:18332 [2015-02-10 21:07:20] HTTP request failed: The requested URL returned error: 403 Forbidden [2015-02-10 21:07:20] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-10 21:07:20] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-10 21:07:20] Closing socket for stratum pool 0 [2015-02-10 21:07:20] Pool 0 slow/down or URL or credentials invalid [2015-02-10 21:07:21] No servers were found that could be used to get work from. [2015-02-10 21:07:21] Please check the details from the list below of the servers you have input [2015-02-10 21:07:21] Most likely you have input the wrong URL, forgotten to add a port, or have not set up workers [2015-02-10 21:07:21] Pool: 0 URL: http://192.168.222.90:18332 User: 1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa Password: NOT-TO-BE-SHARED [2015-02-10 21:07:21] Press any key to exit, or BFGMiner will try again in 15s.

The receiving bitcoind conf file contains the following pertinent lines:

rpcuser=1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa rpcpassword=NOT-TO-BE-SHARED txindex=1 server=1 rpcallowip=localhost/24 rpcallowip=192.168.222.0/24 testnet=1 port=18333 rpcport=18332 keypool=100 alertnotify=echo %s | mail -s "Bitcoin Alert" blacknight@bellsouth.net

I'm certain the credentials username and password are correct.

Here the working result from using the BFG-testNet-GBT-Solo-local.conf file:

bfgminer version 5.0.0-unknown - Started: [2015-02-10 21:33:28] - [ 0 days 00:01:06] [M]anage devices [P]ool management [S]ettings [D]isplay options [H]elp [Q]uit Pool 0: 127.0.0.1 Diff:4.61M - GBT LU:[21:33:27] User:1Devn8jxj1fULfu6DbgoP7sEyu11DXYzXa Block: ...63473d3aa20a7103 Diff:4.61M (33.03T) Started: [21:33:27] I:? ST:29 F:0 NB:1 AS:0 BW:[514/ 9 B/s] E:0.00 BS:287

1/3 50.0C | 9.46/ 9.38/ 8.34Gh/s | A:0 R:0+0(none) HW:2/1.6%

BFL 0a: 51.0C | 0.0/ 0.0/ 0.0 h/s | A:0 R:0+0(none) HW:0/none BFL 0b: 51.0C | 3.72/ 3.64/ 3.44Gh/s | A:0 R:0+0(none) HW:0/none BFL 0c: 51.0C | 0.0/ 0.0/ 0.0 h/s | A:0 R:0+0(none) HW:0/none BFL 0d: 51.0C | 1.74/ 1.69/ 1.82Gh/s | A:0 R:0+0(none) HW:1/3.4% BFL 0e: 51.0C | 0.0/ 0.0/ 0.0 h/s | A:0 R:0+0(none) HW:0/none BFL 0f: 51.0C | 0.0/ 0.0/ 0.0 h/s | A:0 R:0+0(none) HW:0/none BFL 0g: 51.0C | 0.0/ 0.0/ 0.0 h/s | A:0 R:0+0(none) HW:0/none

BFL 0h: 51.0C | 4.23/ 4.16/ 2.86Gh/s | A:0 R:0+0(none) HW:1/2.2%

[2015-02-10 21:33:27] Loaded configuration file /Users/Scott/Projects/BFGminer/etc/BFG-testNet-GBT-Solo-local.conf [2015-02-10 21:33:27] Probing for an alive pool [2015-02-10 21:33:27] Testing pool http://127.0.0.1:18332 [2015-02-10 21:33:27] Network difficulty changed to 4.61M (33.03T) [2015-02-10 21:33:27] New block: ...63473d3aa20a7103 diff 4.61M (33.03T) [2015-02-10 21:33:27] Selected getblocktemplate protocol for pool 0 [2015-02-10 21:33:27] Pool 0 http://127.0.0.1:18332 alive [2015-02-10 21:33:27] No suitable long-poll found for http://127.0.0.1:18332 [2015-02-10 21:33:28] API running in IP access mode on port 4030 [2015-02-10 21:33:29] BFL 0: Opened /dev/cu.usbserial-00001014

The GBT request (your baby) is using the HTTP longpoll mechanism, so I don't think there is a request issue. It doesn't look like I can configure my Antminer C1 to speak anything other than stratum.

You probably shouldn't have the "config" option in the config files... that would cause a recursive include. Not sure it's the problem in this case, though..

"No suitable long-poll found" means it is not using longpolling - which is expected for v0.9.4. I don't see why you think strautm or getwork are being used - the UI you pasted clearly shows only GBT is... bitcoind does not support anything but GBT, so stratum-only miners will simply not work.

Now to your main problem... the BFGMiner config has "url" : "http://192.168.1.90:18332", which is an IP outside your allowed subnet of 192.168.222.0/24. Assuming the 192.168.1.90 network is also a /24 (as is standardly the case), the source IP from this will also be outside 192.168.222.0/24, which is not allowed to use the RPC server thus gets a 403 Forbidden response.

Busted!!! My manual IP address "redaction" did not change 192.168.1.90 to be 192.168.222.90. I can cut-n-paste the IP address from the config file and ping the pasted address. The url IP address is not the issue. (I also currently have bitcoind processes on the same computers talking to one another. Both the local bitcoind and the remote bitcoind daemons are using the same user ID and passwords.)

I deleted the "config" file option to see if that makes a difference. No difference in behaviors for either the broken BFG-testnet-GBT-Solo-remote.conf configuration file or the functioning BFG-testNet-GBT-Solo-local.conf configuration file that executes properly.

Concerning stratum and getwork, I have different bfgminer config files the exercises stratum and getwork capabilities. For example, my BFG-testNet-GWK-Solo-local.conf file (not mentioned in the thread above) contains to force getwork to be used:

"no-stratum" : true, "no-gbt" : true,

As you already know, bfgminer provides visual feedback as to what mining protocol is being used. (Granted I like to use of GBT because it provides individual miners autonomy as to what transactions they want to mine as opposed to stratum where all miners are mining what the pool says...)

While I have your attention:

1. What git "tag" from https://github.com/luke-jr/bfgminer.git does http://luke.dashjr.org/programs/bitcoin/files/bfgminer/5.0.0/bfgminer-5.0.0.zip map to?

2. Is bfgminer now supporting out of tree builds since the following configure flags, associated with cross compiling, are listed in the associated "bfgminer-5.0.0/configure"? System types: --build=BUILD configure for building on BUILD [guessed] --host=HOST cross-compile to build programs to run on HOST [BUILD] --target=TARGET configure for building compilers for TARGET [HOST]

3. If this continues to be a bfgminer discussion is it more appropriate to be moved to https://github.com/luke-jr/bfgminer/issues?

Thanks for your assistance.

FYSA - I like mining at http://eligius.st:-)

Right now, this seems more likely a bitcoind issue.

I forgot you're using an older(kindof) version - maybe try using 192.168.1.* rather than 192.168.1.0/24 syntax? I'm not sure when that changed.

Tried numerous combinations (blue in the face) for the bitcoind core daemon's config file for rpcallowip. Its not the issue either. Still getting the following message from bfgminer for a remote machine internal to my private network on the same subnet running bitcoind.

[2015-02-11 18:28:46] Loaded configuration file BFG-testNet-GBT-Solo-remote.conf [2015-02-11 18:28:51] Probing for an alive pool [2015-02-11 18:28:51] Testing pool http://192.168.1.222:18332 [2015-02-11 18:28:51] HTTP request failed: The requested URL returned error: 403 Forbidden [2015-02-11 18:28:51] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-11 18:28:51] JSON decode failed(1): '[' or '{' expected near 'HTTP' [2015-02-11 18:28:51] Closing socket for stratum pool 0 [2015-02-11 18:28:51] Pool 0 slow/down or URL or credentials invalid

I'm also beginning to think this is a bitcoin core issue...

It was interesting testing feedback from the bfgminer that bitcoin core v0.9.4 did support appear to support all three mining protocols (i.e. GBT, STM, GWK) provided both the miner and bitcoin core were on the same computer.

If I could only get GBT to function across a network for bitcoin core, I would be happy. From experimenting/testing, I'm also starting to notice a few other quirks with v0.9.4. For example, the difficulty factor is alway unity. However, when I probe bfgminer with an RPC C library I wrote a year ago with a "coin" command, I do receive the proper difficulty factor for the testNet. (Will post the library on GitHub when understand the power of "git" to make my own repository.)

I compiled an Obelisk node a few weeks back, really https://github.com/libbitcoin/libbitcoin-server. Very little documentation exist for using it. Think I'll switch gears back to that track...

Thanks for your assistance:-)

Luke is right, 0.9 does indeed use the wildcard format instead of real subnet/CIDR notation. You must use the funky wildcard notation, eg, rpcallowip=192.168.1.*

Is this still a problem with more recent versions?