ci: Clone entire bitcoin-core/qa-assets repo only when run fuzzing #23938

pull hebasto wants to merge 1 commits into bitcoin:master from hebasto:220102-assets changing 1 files +8 −4

hebasto commented at 10:22 AM on January 2, 2022: member

This PR speeds up CI tasks that run unit tests but do not run fuzzing.

On my machine:

$ time git clone --depth=1 https://github.com/bitcoin-core/qa-assets
Cloning into 'qa-assets'...
remote: Enumerating objects: 289750, done.
remote: Counting objects: 100% (289750/289750), done.
remote: Compressing objects: 100% (207687/207687), done.
remote: Total 289750 (delta 16863), reused 275449 (delta 12092), pack-reused 0
Receiving objects: 100% (289750/289750), 1.39 GiB | 4.79 MiB/s, done.
Resolving deltas: 100% (16863/16863), done.
Updating files: 100% (294515/294515), done.

real	7m43,417s
user	2m39,771s
sys	0m43,272s

DrahtBot added the label Tests on Jan 2, 2022

MarcoFalke commented at 5:51 PM on January 2, 2022: member

Once we drop bionic, this can be replaced by:

git clone --depth 1 --filter=blob:none --sparse https://github.com/bitcoin-core/qa-assets
cd qa-assets/
git sparse-checkout set unit_test_data

in ci/test/04_install.sh:96 in 2db1fe3dcd outdated
```
  92 |    if [ ! -d "${DIR_QA_ASSETS}" ]; then
  93 |      DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets "${DIR_QA_ASSETS}"
  94 |    fi
  95 |  
  96 |    export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
  97 |    export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
```
MarcoFalke commented at 12:50 PM on January 6, 2022:

Is this still needed?

hebasto commented at 2:16 PM on January 6, 2022:

Yes, in case both "$RUN_FUZZ_TESTS" = "true" and "$RUN_UNIT_TESTS" = "true".

MarcoFalke commented at 2:20 PM on January 6, 2022:

It is not possible to run both.

Fuzz tests can only run with libfuzzer (right now), in which case unit tests don't work and vice-versa

hebasto commented at 2:28 PM on January 6, 2022:

Thanks! Updated.
ci: Clone entire bitcoin-core/qa-assets repo only when run fuzzing 0b7c55f157
hebasto force-pushed on Jan 6, 2022
hebasto commented at 2:28 PM on January 6, 2022: member
Updated 2db1fe3dcd9fc5b2a95f6e5e6ed63d8625b16d6d -> 0b7c55f157514e1845934fff3f9c66e862c526ca (pr23938.01 -> pr23938.02, diff):
- addressed @MarcoFalke's comment
MarcoFalke commented at 2:29 PM on January 6, 2022: member

cr ACK 0b7c55f157514e1845934fff3f9c66e862c526ca
MarcoFalke merged this on Jan 6, 2022
MarcoFalke closed this on Jan 6, 2022
sidhujag referenced this in commit cb428e1770 on Jan 6, 2022
hebasto deleted the branch on Jan 6, 2022
DrahtBot locked this on Jan 6, 2023

Contributors

hebasto

MarcoFalke

Labels

Tests