Using a ramdisk on OSX sped up the test suite by about 5x (using default jobs=4) on my M1 macbook pro running macOS Monterey 12.3.1. This PR adds the relevant OSX commands following the Linux directions.
Default:
8204 s (accumulated)
Runtime: 2104 s
following commands from the PR:
1606 s (accumulated)
Runtime: 421 s
ramdisk + jobs=32:
2090 s (accumulated)
Runtime: 85 s
<!--e57a25ab6845829454e8d69fc972939a-->
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.
<!--021abf342d371248e50ceaed478a90ca-->
See the guideline for information on the review process.
| Type | Reviewers |
|---|---|
| ACK | brunoerg, jonatack, willcl-ark |
| Concept ACK | RandyMcMillan, kristapsk |
| Stale ACK | Sjors, stickies-v |
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.
138 | @@ -137,6 +139,27 @@ Once finished with the tests and the disk, and to free the ram, simply unmount t 139 | sudo umount /mnt/tmp 140 | ``` 141 | 142 | +**OSX**
Unsure, but I thought "macOS" was the naming nowadays.
**macOS**
138 | @@ -137,6 +139,27 @@ Once finished with the tests and the disk, and to free the ram, simply unmount t 139 | sudo umount /mnt/tmp 140 | ``` 141 | 142 | +**OSX** 143 | + 144 | +To create a 4GB ramdisk named "ramdisk" on OSX at `/Volumes/ramdisk/`:
To create a 4 GiB RAM disk named "ramdisk" at `/Volumes/ramdisk/`:
146 | +```bash 147 | +diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608) 148 | +``` 149 | + 150 | +Configure the size at the end of the command, in terms of block size 151 | +(`8388608 / 2048 = 4096` or 4 GB). To run the tests using the ramdisk:
(`8388608 / 2048 = 4096` or 4 GiB). To run the tests using the RAM disk:
113 | @@ -114,6 +114,8 @@ options. Run `test/functional/test_runner.py -h` to see them all. 114 | If you have available RAM on your system you can create a ramdisk to use as the `cache` and `tmp` directories for the functional tests in order to speed them up. 115 | Speed-up amount varies on each system (and according to your ram speed and other variables), but a 2-3x speed-up is not uncommon. 116 | 117 | +**Linux** 118 | + 119 | To create a 4GB ramdisk on Linux at `/mnt/tmp/`:
To create a 4 GiB RAM disk at `/mnt/tmp/`:
Thanks for the review! All nits addressed throughout the entire section at ab6f73a1f65bcfd59fea07f3067312a757dba2f8
utACK 62ac3b4
That looks pretty much like what I use on macOS.
138 | @@ -137,6 +139,27 @@ Once finished with the tests and the disk, and to free the ram, simply unmount t 139 | sudo umount /mnt/tmp 140 | ``` 141 | 142 | +**OSX** 143 | + 144 | +To create a 4GB ramdisk named "ramdisk" on OSX at `/Volumes/ramdisk/`: 145 | + 146 | +```bash 147 | +diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608)
I think this could "feel safer" for people to run if it were split into two commands (even though it's using a subshell)?
You need to do this again at every reboot, so it's nice to have a single incantation in your bash history.
OK, that makes sense too :)
agree with Sjors
OK, leaving this as one-liner for now
doubt:
To create a 4 GiB RAM disk named "ramdisk" at `/Volumes/ramdisk/`:
Does it create a RAM disk named "ramdisk" at /Volumes/, resulting in /Volumes/ramdisk/ or it creates a RAM disk named "ramdisk" into /Volumes/ramdisk/?
Concept ACK
Concept ACK
Tested these steps on my macOS machine and it worked as expected:
➜ bitcoin-core-dev git:(master) ✗ diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608)
Started erase on disk5
Unmounting disk
Erasing
Initialized /dev/rdisk5 as a 4 GB case-insensitive HFS Plus volume
Mounting disk
Finished erase on disk5 (ramdisk)
➜ bitcoin-core-dev git:(master) ✗ ./test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp
Temporary test directory at /Volumes/ramdisk/tmp/test_runner_₿_🏃_20230221_120656
Running Unit Tests for Test Framework Modules
..........
----------------------------------------------------------------------
Ran 10 tests in 0.595s
OK
1/260 - feature_taproot.py passed, Duration: 31 s
...
➜ bitcoin-core-dev git:(master) ✗ umount /Volumes/ramdisk
Concept ACK
146 | +```bash 147 | +diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608) 148 | +``` 149 | + 150 | +Configure the size at the end of the command, in terms of block size 151 | +(`8388608 / 2048 = 4096` or 4 GiB). To run the tests using the RAM disk:
nit: I think the formula can be expressed a bit more useable like this:
(`4096 MiB * 2048 blocks/MiB = 8388608 blocks` for 4GiB). To run the tests using the RAM disk:
sure, updated
ACK ab6f73a1f65bcfd59fea07f3067312a757dba2f8
I use this command on my M1 too.
re-ACK cb7be3a
ACK cb7be3a237c7e5a3d09f24259be015557dc72bca
Tested on my Macbook, worked as expected and speeded up the test suite noticeably.
145 | + 146 | +```bash 147 | +diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608) 148 | +``` 149 | + 150 | +Configure the size at the end of the command, in terms of block size
Am I understanding this correctly?
Configure the size, expressed as the number of blocks, at the end of the command
Also, if you retouch, s/ram/RAM/ in this line:
Once finished with the tests and the disk, and to free the ram, simply unmount the disk:
great, both updated
utACK cb7be3a237c7e5a3d09f24259be015557dc72bca
utACK 2f84ad7b9e62dd710940c2f265b65973b94864d7
ACK 2f84ad7b9e62dd710940c2f265b65973b94864d7
M1 Max, macOS 13.2
$ diskutil erasevolume HFS+ ramdisk $(hdiutil attach -nomount ram://8388608)
Started erase on disk6
Unmounting disk
Erasing
Initialized /dev/rdisk6 as a 4 GB case-insensitive HFS Plus volume
Mounting disk
Finished erase on disk6 (ramdisk)
$ diskutil info disk6 | grep -e Size
Disk Size: 4.3 GB (4294967296 Bytes) (exactly 8388608 512-Byte-Units)
Device Block Size: 512 Bytes
Allocation Block Size: 4096 Bytes
$ test/functional/test_runner.py --cachedir=/Volumes/ramdisk/cache --tmpdir=/Volumes/ramdisk/tmp
Temporary test directory at /Volumes/ramdisk/tmp/test_runner_₿_🏃_20230222_105652
..........
ALL | ✓ Passed | 2702 s (accumulated)
Runtime: 292 s
$ umount /Volumes/ramdisk
ACK 2f84ad7b9e62dd710940c2f265b65973b94864d7