This supersedes #1215.
fill_random was added in 2022 (#748), and its implementation hasn't changed since, so after a few years, it's time for an overhaul. Since fill_random is meant to be copied into other code, it should be a good template. With this PR:
- It works for arbitrary sizes. Currently, it doesn't on any platform:
getentropy()(macOS, OpenBSD) fails for requests larger than 256 bytes,getrandom()(Linux, FreeBSD) may return fewer bytes than requested, which is treated as failure, andBCryptGenRandom()(Windows) takes aULONGlength. - It aborts on failure instead of returning an error (second commit). Callers can't do anything sensible on failure, and an error return in code that gets copied invites mishandling (e.g., ignoring the return value and using an unfilled buffer). Rust's std and BoringSSL don't return an error either: Rust's std panics, and BoringSSL aborts. Nonetheless,
fill_randomkeeps returningint(now always1plus unreachablereturn 0afterabort()), and the examples keep checking it. People may copyfill_randomwithout the examples, or theexampleswithout fill_random, so both should be defensive on their own. - It no longer recommends falling back to
/dev/urandomon Linux (third commit). A fallback matters only for Linux < 3.17, and simply reading from/dev/urandomisn't a correct fallback anyway because it doesn't wait for the entropy pool to be initialized. - It handles a request for 0 bytes consistently on all platforms by returning early (fourth commit). This is an edge case on which implementations may differ, e.g., NetBSD's
arc4random_buf()passes the buffer pointer tomemcpy()even for size 0, which is undefined behavior if the pointer is NULL. As a result,datacan be NULL ifsizeis 0, consistent with the library's API (e.g., themsgargument ofsecp256k1_schnorrsig_verify). - It supports NetBSD.
Overview of randomness sources (after the PR)
- Linux:
getrandom()in a loop, retrying onEINTR - macOS and other Apple platforms:
CCRandomGenerateBytes() - FreeBSD, NetBSD, OpenBSD:
arc4random_buf(), which works for arbitrary sizes and never returns an error. The OpenBSDgetentropyman page recommends it for ordinary random data. - Windows:
BCryptGenRandom()in a loop overULONG-sized chunks
Side note: Since bitcoin/bitcoin#27699, Core detects getrandom() via <sys/random.h> (CMake check), which FreeBSD (>= 12) and NetBSD (>= 10) provide as well. So these presumably take the getrandom() branch, not the sysctl() branch whose comment says "FreeBSD, NetBSD and similar" (last touched in bitcoin/bitcoin#18359). @fanquake, is this intended?
<details> <summary>
Detailed rationale and background
</summary>
Why not arc4random_buf() on Linux
On Linux, arc4random_buf() is not an option: it's not available in musl or glibc < 2.36, and glibc doesn't declare it under -std=c89 unless _DEFAULT_SOURCE is defined.
Why CCRandomGenerateBytes() on Apple platforms
On Apple platforms, arc4random_buf() and CCRandomGenerateBytes() draw from the same generator (since macOS 10.15), and Apple's getentropy man page recommends both for ordinary random data. But Apple's arc4random_buf() ignores the return value of the underlying generator, so a failure would silently leave the buffer unfilled, whereas CCRandomGenerateBytes() reports it. This should also make fill_random usable on iOS and other non-macOS Apple platforms, where getentropy() isn't public API (sys/random.h exists only in the macOS SDK), but I haven't tested this.
What does Rust's std do?
Apart from Rust's /dev/urandom fallback for Linux < 3.17, this ends up doing the same as Rust's std, which (like a C program) always links libc, with one deliberate exception: On Windows, Rust's std uses ProcessPrng(), which always succeeds and takes a SIZE_T length. However, the Windows SDK provides neither a declaration nor an import library for it, so it has to be loaded at runtime via LoadLibrary() and GetProcAddress() (as BoringSSL and Chromium do). That's too much ceremony for an example, so this PR keeps BCryptGenRandom().
What does Bitcoin Core do?
Where the requirements overlap, Bitcoin Core's GetOSRand() makes the same choices as this PR: it uses getrandom() on Linux, BCryptGenRandom() on Windows and arc4random_buf() on OpenBSD, and it aborts on failure. It differs elsewhere because it solves a different problem: it always requests exactly 32 bytes, which Core mixes into its own RNG together with other entropy sources. So it can rely on per-call limits (it calls getrandom() only once, relying on the guarantee for requests of up to 256 bytes, and it uses getentropy() on macOS). Moreover, it selects the API via build-time feature detection (HAVE_GETRANDOM, HAVE_GETENTROPY_RAND, HAVE_SYSCTL_ARND) with a fallback to /dev/urandom, which a header that people copy can't do.
Minimum OS versions
This doesn't raise any minimum OS version: CCRandomGenerateBytes() has been available since macOS 10.10, and arc4random_buf() since FreeBSD 8.0, NetBSD 6.0 and OpenBSD 4.4, i.e., before the getentropy()/getrandom() versions the current code requires (macOS 10.12, FreeBSD 12.0, OpenBSD 5.6). Earlier versions of these BSDs implemented arc4random with RC4, but it was replaced in FreeBSD 12.0, NetBSD 7.0 and OpenBSD 5.5. On all three BSDs, <stdlib.h> declares arc4random_buf() under -std=c89 without feature-test macros.
</details>