util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD #9821

These are available in sandboxes without access to files or to devices. Also they are considered safer and more straightforward to use than /dev/urandom as reading from a file has quite a few edge cases:

• Linux: getrandom(buf, buflen, 0). getrandom(2) was introduced in version 3.17 of the Linux kernel.
• OpenBSD: getentropy(buf, buflen). The getentropy(2) function appeared in OpenBSD 5.6.
• FreeBSD and NetBSD: sysctl(KERN_ARND). Not sure when this was added but it has existed for quite a while.

Alternatives:

• Linux has sysctl CTL_KERN / KERN_RANDOM / RANDOM_UUID which gives 16 bytes of randomness. This may be available on older kernels, however sysctl is deprecated on Linux and even removed in some distros so we shouldn't use it.

Add tests for GetOSRand():

• Test that no error happens (otherwise RandFailure() which aborts)
• Test that all 32 bytes are overwritten (initialize with zeros, try multiple times)

Discussion:

• When to use these? Currently they are always used when available. Another option would be to use them only when /dev/urandom is not available. But this would mean these code paths receive less testing, and I'm not sure there is any reason to prefer /dev/urandom.

Closes: #9676. I've tested this on all three OS-es.

Concept ACK. I think this should use a runtime test case at init, not just a boost testcase-- code may be built on a system where it works but run elsewhere and this is too important to mess up. :)

I'm OK with adding this to the sanity checks, but in all fairness we don't check that for /dev/urandom either, we used to not even have a boost testcase for this code path. Also a failure while reading randomness will already always trigger a fatal abort.

I think getrandom if available should be preferred over using /dev/urandom. While a local root access can both replace /dev/urandom with something else or intercept getrandom calls, I expect the latter to require more invasive/detectable changes.

~Two questions:~

• ~What if you're using a 2.25 or later glibc (which has the getrandom function), but your Linux kernel is pre-3.17 (which does not have the syscall)? I have a hard time fnding information about this in the glibc documentation. Either it simulates it by reading /dev/urandom instead, or it fails. If the actual behavior is to fail, we should probably fall back to reading /dev/urandom at runtime rather than at compile time.~
• ~Given that our release binaries should probably not be required to work only on post-2.25 glibc, with this patch we can't compile release binaries against a post-2.25 glibc...~

Nevermind, it seems you're using the syscall directly.

but in all fairness we don't check that for /dev/urandom either,

Yes but we don't have an issue there where there are older kernels that don't have the functionality... (Also, I would be of the view that we should have a runtime test currently too-- but I didn't want to bloat up the original PR that added this.)

Concept ACK. utACK on the build changes in particular.

Nevermind, it seems you're using the syscall directly.

Well it still makes sense. On Linux, syscalls that are not implemented will predictably return an error w/ errno ENOSYS. We want to support older kernels (3.16 isn't that old - 2014) so need to handle this and fall back to /dev/urandom. Currently it will crash with a "randomness error" on those platforms [but only if compiled with kernel headers that did have the syscall]. Edit: added

~@laanwj It seems that Debian Jessie (stable) ships with Linux kernel 3.16, right before 3.17 which introduced getrandom(). That means that a binary compiled on a system with a >=3.17 kernel won't run on Jessie anymore, at all. I think we should prefer trying to call getrandom(), but if it fails, fall back to reading /dev/urandom.~

It seems I'm consistently commenting on old states of this PR.

utACK 7e6dcd9995b99e894b8017f09016c405b066ca36

It seems none of the new methods are available on OSX (or at least in the build environment in Travis), but a new method can be added later.

It seems none of the new methods are available on OSX (or at least in the build environment in Travis), but a new method can be added later.

It may well be that OSX has no way to read kernel randomness besides /dev/urandom. We checked and their own frameworks use that, and nothing else. But yes someone with more knowledge about OSX could take a look at that later.

Libressl has getentropy emulation for a number of different OS's that may be useful as a reference.

Libressl has getentropy emulation for a number of different OS's that may be useful as a reference.

Thanks. That source confirms that there is no other way besides using /dev/urandom in OSX.

Thanks. That source confirms that there is no other way besides using /dev/urandom in OSX.

While digging around I found that OSX 10.12+ does have getentropy, however it seems to be defined in sys/random.h instead of unistd.h as in OpenBSD:

__OSX_AVAILABLE(10.12) __IOS_AVAILABLE(10.0) __TVOS_AVAILABLE(10.0) __WATCHOS_AVAILABLE(3.0)
int getentropy(void* buffer, size_t size);

Example:

#include <stdlib.h>
#include <unistd.h>
#include <sys/random.h>

int main()
{
    unsigned char buf[32];
    if (getentropy(buf, sizeof(buf)) < 0) {
        return 1;
    }
    write(STDOUT_FILENO, buf, sizeof(buf));
    return 0;
}

I'm not sure how to make autoconf detect something that may be in different headers.