RFC: Rust code integration

theuni commented at 6:16 pm on April 11, 2019: member

This is work from Jeremy Rubin integrated into our buildsystem. It allows for rust code to be used directly inside of Bitcoin Core. In its current form, it demonstrates calling a single hello_world function from rust code.

This is not intended to be merged as-is, but instead to serve as a reference for anyone who might be interested in trying out some rust code inside of Bitcoin Core. I have no idea what works. I have lots of questions about debugging, threading, etc. But instead of trying to hack and document how things work, we thought it’d be fun for everyone to be able to poke at it and scratch our heads together :). If something interesting comes out of it, a discussion about merging can happen then.

It is surprisingly functional. The rust tools are impeccable. I would’ve thought this would be a project that would take months/years, but the rust devs have done such a good job that mostly everything already just works. The gitian descriptors have been modified to actually incorporate working rust code. All of our currently supported platforms seem to already work (even macOS cross!) with the exception of 32bit windows and ancient linux distros. The specific issues are documented in the gitian descriptors. For now, Gitian needs net access and a larger disk volume, but those are both very temporary issues

Why rust? I don’t know. Maybe not. But I think it’s a fair assumption to say that Rust code will eventually end up in Bitcoin Core as the result of adding a new dependency. Adoption is happing quickly. So, I think it prudent to look ahead and not only be prepared, but actively help usher it in. Already I have a few things that I would like to work on and upstream to rust-lang to make our integration nicer, and I assume that more will be uncovered as it actually gets used.

Anybody have any fun ideas?

Adds a rust library to bitcoin to demonstrate linking and building rust code.

The demonstration library compiles a rust hello world example and auto-generates
a header which can be included in C++ code.

Co-Authored-By: Jeremy Rubin <j@rubin.io>
Co-Authored-By: Cory Fields <cory-nospam-@coryfields.com>

e7f900ed0d

DO NOT MERGE. Add rust examples to gitian descriptors 0bb8b2e997

luke-jr commented at 6:18 pm on April 11, 2019: member

NACK Rust. There’s no reasonable way to bootstrap a Rust compiler without using trusted third party binaries.

theuni commented at 6:21 pm on April 11, 2019: member

@luke-jr Objection noted.

DrahtBot added the label Build system on Apr 11, 2019

DrahtBot added the label Scripts and tools on Apr 11, 2019

MarcoFalke added the label Needs gitian build on Apr 11, 2019

MarcoFalke commented at 6:53 pm on April 11, 2019: member

@luke-jr So this is insufficient: https://www.gnu.org/software/guix/blog/2018/bootstrapping-rust/ ?

DrahtBot commented at 6:55 pm on April 11, 2019: member

DrahtBot added the label Needs rebase on Apr 11, 2019

theuni commented at 9:12 pm on April 11, 2019: member

@MarcoFalke As noted in the summary, for now, gitian requires net access to download the dependencies. That will be fixed before merge.

Also, It requires a toolchain tarball that hasn’t been trimmed, so it’s very large. That means that the size of the gitian disk needs to be increased:

0$ dd if=/dev/zero of=base-bionic-amd64 bs=1k seek=20480k count=1

Afaik that won’t affect builds in any other way, but be sure to make a backup to be safe.

The toolchain package can be created by running this script: https://github.com/bitcoin/bitcoin/pull/15798/files#diff-8acecf4a86a4f206bddfcf142d451a05

laanwj commented at 9:31 am on April 12, 2019: member

Thanks for working on this. I think this is great work, and I think it’s good to be able to bridge the gap to the bitcoin-rust world.

I also understand the concerns with merging this right now, so I think it makes sense to maintain this as a separate branch for now. I’m willing to do this.

daira commented at 10:30 am on April 12, 2019: contributor

FYI, Zcash makes extensive use of Rust code. You’re welcome (of course, since it’s MIT-licensed) to look at and reuse our build support for that. It’s fully deterministic and statically linked, and all of the transitively depended-on Rust crates, as well as the toolchain, are pinned by hash. Support was initially added in https://github.com/zcash/zcash/pull/2183 , but you would also need some of this list of merged PRs mentioning Rust.

fanquake added the label Brainstorming on Apr 12, 2019

fanquake commented at 1:53 pm on April 12, 2019: member

Why rust? I don’t know. Maybe not.

🚀 I used a brew installed rust & cargo. Worked out of the box. Keen to have a play around.

0make[1]: Nothing to be done for `all'.
1make[1]: Nothing to be done for `all-am'.
2
3src/bitcoind
4Hello World!
52019-04-12T13:10:35Z Bitcoin Core version v0.18.99.0-0bb8b2e99 (release build)

There was some CRust related discussion on IRC here (line 401).

MarcoFalke removed the label Needs gitian build on May 2, 2019

fanquake commented at 10:50 am on June 15, 2019: member

If anyone’s interested, I’ll be maintaining a rebased version of this branch (with a couple simple commits on top) here: https://github.com/fanquake/bitcoin/tree/theuni-with-rust-example-working-rebased, and branching off of that to play with more Rust integration. I’m using a brew installed rust and cargo.

fanquake commented at 2:35 am on August 24, 2019: member

My branch has been re-rebased and I’ve added a commit to update cbindgen to 0.9.0.

hebasto commented at 8:39 pm on August 29, 2019: member

@theuni

That means that the size of the gitian disk needs to be increased:
0$ dd if=/dev/zero of=base-bionic-amd64 bs=1k seek=20480k count=1

There is a new --disksize option (https://github.com/devrandom/gitian-builder/pull/222).

elichai commented at 2:30 pm on August 31, 2019: contributor

I personally like cbindgen, but I still think that generated code should be tracked in the version control(aka git) so we can easily notice changes done via cbindgen.

laanwj added this to the milestone 0.20.0 on Sep 9, 2019

laanwj commented at 2:41 pm on September 9, 2019: member

What would be nice for messing around with this is a way to log (e.g. call LogPrintf and/or LogPrint) from rust code.

Edit: ~~will give this a shot~~ Feel free to take over the top commit from here: https://github.com/laanwj/bitcoin/tree/2019_09_rust

0   log!("hello from rust {}\n", domain_str);
1   log_cat!(LogFlags::NET, "(cat) hello from rust {}\n", domain_str);

fanquake commented at 9:02 pm on October 9, 2019: member

Closing this in favour of #17090. Please engage in “Rust inside Bitcoin Core” discussions there instead.

fanquake closed this on Oct 9, 2019

laanwj removed the label Needs rebase on Oct 24, 2019

DrahtBot locked this on Dec 16, 2021

RFC: Rust code integration #15798