tb
#12314
0❯ bitcoin-cli --rpcuser=x --rpcpassword=y -regtest getnewaddress "" "bech32"
1bcrt1q6rhpng9evdsfnn833a4f4vej0asu6dk5srld6x
By BIP173 Segwit address format bitcoind should return a tb prefixed address for testnet, and by developer examples, regtest is a blockchain with the same basic rules as testnet.
bitcoind should return a bech32 address with the same prefix as testnet.
0.16.0rc1
bcrt. Not everything with regtest follows that of testnet, nor does it need to.
Not everything with regtest follows that of testnet, nor does it need to.
Agree on that. By reading the developer examples and the BIP173 i was lead to expecting a tb prefixed address.
Yes, this is intentional. I don’t see a reason why regtest should produce addresses that can be confused with testnet.
Thanks for confirming 👍
bcrt prefix in BIP 173 though, neither does it mention regtest mode.
I do agree that regression testing should primarily be done within one repository, but it does make sense to have a similar structure for integration testing between implementations. Next to the official test network, I think it makes sense to have some standards on a single regression test network to facilitate cross-implementation testing.
I realize that standards in Bitcoin currently mostly go hand in hand with Core development, but with Lightning the process has been way more diverse, creating a need for some form of standard integration testing.
So, while the integration efforts could specify a custom regtest chain, it would be way easier if different implementations could already provide a compatible one.
We tend to change the regtest parameter whenever it makes our testing or maintaining easier, so even if it is unlikely that the bech32 prefix will be changed, I am slightly -0 on adding it to the BIP.
Also see #8994, which makes it easier to launch regtest like chains and thus impossible to documents all of them in the BIP.
Yes, this is intentional. I don’t see a reason why regtest should produce addresses that can be confused with testnet.
Reasons:
There is still not too late to get rid of bcrt. It’s not a consensus change (since regtests are in consensus with themselves anyway), and I doubt anyone would miss it.
I think part of the issue here is the question whether regtest is an internal Bitcoin Core thing, or a framework for testing infrastructure on top.
Historically at least, regtest was purely an internal thing for Bitcoin Core’s own tests. There is no standardization, and it has undergone rule changes several times whenever it was convenient.
On the other hand, testnet was explicitly designed for applications to test integration with, is dealt with in BIPs, though I can see why it may be hard to use in some cases.
I don’t know what the solution is. I’m opposed to adding regtest specifications to BIPs in general, as this would make it harder to adapt to changes that need testing in Core itself. We could consider changing the regtest prefix to something else; I don’t think it’s a strong reduction in test capability.
Overall, the expectation of using regtest externally may bite us again in other situations. Maybe we need even more networks? (signet comes to mind, though global networks aren’t what you always need either).
@sipa I guess I can use this opportunity to share some thought of nature of Bitcoin “nets” from my experiences at Bitgo (thought are mine though). Please excuse if it is a bit scattered and random.
On the IDs, hashes, addresses, etc. level there’s no reason to make any distinctions between testnet, regtest, simnet or whatever other non-mainnets. Conceptually the are just two types of networks: real one, and not real one. Mostly for the end user not to ever get confused. If the software needs to know if it is running against testnet vs regtest vs simnet, it can carry that information out of band - that’s not a problem. It doesn’t have to be embedded in every single address. Quite the opposite: having to potentially come up with additional logic in every single piece of code handling Bitcoin, to support every type of “net” that we could come up with is just a hindrance.
There’s mainnet - the real deal. Everybody understands mainnet.
Then there’s a testnet. What’s not-obvious is that testnet is not a test network for companies integrating Bitcoin. It’s a test network for end users of Bitcoin. It’s global, shared and behaves almost exactly like a mainnet, so users can do testing of the whole ecosystem, send funds between different companies and so on. From the perspective of a company that integrates with Bitcoin, testnet is actually what I call a “prod-net” - production network, exposed to end users. When testnet systems are having problems, users report normal support cases to get it fixed, just like a mainnet prod systems.
Testnet is also not really suitable for Bitcoin integration testing. It is just too slow and does not give enough control. Any practical yet extensive test suite needs to have an ability to control mining and what exactly goes on the block. Regtest is perfect for that. I don’t know if it was intended for it or not, but in practice it works great.
Regtest node is not a mock, fake or whatever. It’s a real Bitcoin Core node. From an outside perspective of a blockchain data consumer it looks very much like a normal testnet node. But you can drive it manually like a mock. You can setup a test instance of your whole system, swap a node for a regtest node, drive it with rpc commands in your test suite and see if what comes in and out is what you expect. You don’t have to wait for blocks being mined, and you can simulate stuff like reorgs. Consensus with anyone else irrelevant, because there’s no way to coordinate anyway.
For anyone in a similar situation: I eventually figured out that it is probably easier to patch bitcoin core node to just not use this custom prefix, than try to introduce and maintain all the code to support it.
Indeed the fix is simple: https://github.com/dpc/bitcoin/commit/99e63500f767f5dbf966280e9b2a27fb4aecb213 , though from what I can tell it will break a lot of existing tests that expect bcrt.
I asked around and I still haven’t seen a single practical problem that different prefixes would solve, so I hope the whole idea can still be revised, especially that it is not baked into any consensus or even the BIP itself.
I don’t think the Bitcoin Core test suite cares what prefix is used for the regtest chain, so I don’t mind changing it to tb. There shouldn’t be any downsides and if it helps testing of 3rd party applications, why not?
If someone wants to pick this up, a good start would be to cherry-pick https://github.com/dpc/bitcoin/commit/99e63500f767f5dbf966280e9b2a27fb4aecb213 and then fixup the test from there.
I just hope there’s an agreement on the reasoning here. I just point out that there is a substantial cost in making every downstream consumer of Bitcoin Core RPC deal specifically with minor details of distinct “testnets”. If there’s a good practical reason to have such a distinction that have benefits out-weighting the costs, than maybe there are alternative approaches like making it configurable, etc. But if no one has any use for it, than it seems simplest to just get rid of it.
I have also not played with signet or any other potential testnet-like modes/setups, so if someone is familiar with them - it might be worthwhile to rethink if there are any cases like that there. Ideally, if a downstream consumer decides to e.g. switch from testnet to signet for a better user-experience, it would be great if that did not require any alterations.
@dpc I don’t think there is any reason why having a separate prefix for regtest is advantageous to us, but at the time it was introduced, it was also assumed to be of no cost. I had no idea people were using regtest for testing other things than Bitcoin Core itself.
As I said, if there’s a good reason (like you seem to have), changing it to ’tb’ is fine by me.
However, I am worried about encouraging people using regtest for external testing in general. We can’t guarantee that regtest won’t undergo radical changes as our test needs change. This makes me wonder if we shouldn’t define a standard “external regtest” network instead, with an intention if it being as close as possible to testnet (or mainnet?) as possible while remaining usable for testing.
PS: why ’tb’ and not ‘bc’? Regtest is closer to mainnet than to testnet in many aspects.
We can’t guarantee that regtest won’t undergo radical changes as our test needs change. This makes me wonder if we shouldn’t define a standard “external regtest” network instead
At least from our perspective, anything like this works. We can pass different flags, change config, etc. That would be a local change, and not introduce any work downstream of the rpc interface.
I could imagine, that in the future there could be a mode even more tailored to external integration testing, separate from regtest, but as things are right now it’s not really neccessary. I realize that the reward schedule is different on regtest, and probably things like difficulty, but these are not very important - a downstream service just takes the data returned by Bitcoin Core at its face value, and is not in the business of second guessing the consensus rules. If Bitcoin Core says the coinbase tx had a 25 BTC coinbase output at height 50, then that’s what must have really happened.
PS: why ’tb’ and not ‘bc’? Regtest is closer to mainnet than to testnet in many aspects.
Seems that testing against “testnet” is the most natural. If someone has any sort of service that needs additional testing APIs etc. they will turn such things on in the configs of their non-production env. which would typically be a regtest/testnet.
This makes me wonder if we shouldn’t define a standard “external regtest” network instead
That indeed sounds like a good idea, as regtest is currently used to do any kind of tests and not only Bitcoin Core tests.
Whenever to use tb or bc, I think it’s important to stick to the same configuration on all types of addresses for regtest and not just bech32, since in current implementation legacy regtest and testnet are exactly the same (even bitcoinrpc says one is valid in the other network), which is not the case for bech32 (and additionals checks needs to be made for properly testing)
PS: why ’tb’ and not ‘bc’? Regtest is closer to mainnet than to testnet in many aspects.
We just really need to distinguish mainnet from testing funds, and regtest is not mainnet. :) Regnet is expedient to test e.g. behaviour around confirmations, channel open and closes, and making indexers safe through chainforks. I’d actually bet that a ton of other Bitcoin projects use regtest for some controlled environment end-to-end tests.
However, I am worried about encouraging people using regtest for external testing in general. We can’t guarantee that regtest won’t undergo radical changes as our test needs change. This makes me wonder if we shouldn’t define a standard “external regtest” network instead, with an intention if it being as close as possible to testnet (or mainnet?) as possible while remaining usable for testing.
We include regtest “hardforks” and other breaking changes in the release notes (e.g. https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.0.md#testing-changes or https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.19.0.1.md#tests), so I think regtest is already meant to be also used externally.
It’s not that rust-bitcoin can’t work with the same prefix for multiple networks; it can: https://github.com/rust-bitcoin/rust-bitcoin/pull/291/files#diff-7769ce7612a45cca417dce18a5aa6a50R388
The problem is that when rust-bitcoin (or, I assume any other library) parses address, it needs somehow to distinguish whether tb comes from testnet, regtest or signet (like here: https://github.com/rust-bitcoin/rust-bitcoin/pull/291/files#diff-7769ce7612a45cca417dce18a5aa6a50R417) - and it’s not clear how to do that.
it needs somehow to distinguish whether tb comes from testnet, regtest or signet
Why?
It’s very surprising to me that you at any point in time would want to decode an address without knowing what network it’s for.
I also don’t see why that isn’t a problem already - P2PKH and P2SH addresses for regtest are the same as for testnet.
Just to confirm, at p2pkh and p2sh addresses for signet are identical to testnet? Otherwise, we have a problem with bolt11 invoices and fallback addresses:
0 assert b11['fallbacks'][0]['type'] == 'P2PKH'
1> assert b11['fallbacks'][0]['addr'] == 'mk2QpYatsKicvFVuTAQLBryyccRXMUaGHP'
2E AssertionError: assert 'sNkrc4kvorCt...cnzRg2jLt6saY' == 'mk2QpYatsKic...ryyccRXMUaGHP'
3E - mk2QpYatsKicvFVuTAQLBryyccRXMUaGHP
4E + sNkrc4kvorCtNKT7o34nycnzRg2jLt6saY
But signet support in c-lightning was added while signet was still in flux, so maybe it doesn’t use different address versions any more?
tb and regtest addresses start bcrt. Energy is probably better spent ensuring community documentation and tutorials accurately reflect this.
I have to say that this singe issue can be a massive productivity drain for people trying to build testable infrastructure on top of bitcoin core.
I understand some internal tests might break, but how much work are we talking about here? I’d be willing to contribute in fixing those in my spare time if it means that we get rid of the bcrt prefix forever.
I don’t get the rationale that it potentially makes more sense today than it did three years ago.
I think there was software out there (citation needed) that treated bech32 address prefixes as a unique identifier for the whole chain. Now that there are two chains with the same prefix (and affected software likely removed that incorrect assumption), there is one less reason to not do the change, thus it makes more sense.
I have to say that this singe issue can be a massive productivity drain for people trying to build testable infrastructure on top of bitcoin core. I understand some internal tests might break, but how much work are we talking about here? I’d be willing to contribute in fixing those in my spare time if it means that we get rid of the bcrt prefix forever.
I think it is quite the oppposite, most (if not all) bitcoin libraries today support the bcrt prefix. There is no benefit to changing it to tb, you are only destroying information that otherwise would be in the address (if it is regtest vs testnet)
Information that, as has been stated repeatedly before is not only irrelevant, but can easily be communicated out of band. And I don’t know what libraries you’re using but bitcoinjs-lib is giving me trouble with that.
Anyways, I don’t want to over-extend an already extended thread, but keeping something that is clearly giving multiple devs more trouble than it is solving because in some hypothetical edge-case it might be useful doesn’t sound like a good reason.
Information that, as has been stated repeatedly before is not only irrelevant, but can easily be communicated out of band. And I don’t know what libraries you’re using but bitcoinjs-lib is giving me trouble with that.
Anyways, I don’t want to over-extend an already extended thread, but keeping something that is clearly giving multiple devs more trouble than it is solving because in some hypothetical edge-case it might be useful doesn’t sound like a good reason.
Looks like bitcoinjs-lib has support for bcrt
bcrt.
