Update secp256k1 subtree to libsecp256k1 version 0.2.0

sipa commented at 4:50 am on December 13, 2022: member

Now that libsecp256k1 has a release (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-December/021271.html), update the subtree to match it.

The changes themselves are not very impactful for Bitcoin Core, but include:

It’s no longer needed to specify whether contexts are for signing or verification or both (all contexts support everything), so make use of that in this PR.
Verification operations can use the static context now, removing the need for some infrastructure in pubkey.cpp to make sure a context exists.
Most modules are now enabled by default, so we can drop explicit enabling for them.
CI improvements (in particular, MSVC and more recent MacOS)
Introduction of an internal int128 type, which has no effect for GCC/Clang builds, but enables 128-bit multiplication in MSVC, giving a ~20% speedup there (but still slower than GCC/Clang).
Release process changes (process documentation, changelog, …).

Squashed 'src/secp256k1/' changes from 44c2452fd3..21ffe4b22a

21ffe4b22a Merge bitcoin-core/secp256k1#1055: Prepare initial release
e025ccdf74 release: prepare for initial release 0.2.0
6d1784a2e2 build: add missing files to EXTRA_DIST
8c949f56da Merge bitcoin-core/secp256k1#1173: Don't use compute credits for now
13bf1b6b32 changelog: make order of change types match keepachangelog.com
b1f992a552 doc: improve release process
7e5b22684f Don't use compute credits for now
ad39e2dc41 build: change package version to 0.1.0-dev
5c789dcd73 Merge bitcoin-core/secp256k1#1168: Replace deprecated context flags with NONE in benchmarks and tests
d6dc0f4ae3 tests: Switch to NONE contexts in module tests
0c8a5caddd tests: Switch to NONE contexts in tests.c
86540e9e1f tests: add test for deprecated flags and rm them from run_context
caa0ad631e group: add gej_eq_var
37ba744f5b tests: Switch to NONE contexts in exhaustive and ctime tests
8d7a9a8eda benchmarks: Switch to NONE contexts
90618e9263 doc: move CHANGELOG from doc/ to root directory
e3f84777eb Merge bitcoin-core/secp256k1#1126: API cleanup with respect to contexts
4386a2306c examples: Switch to NONE contexts
7289b51d31 docs: Use doxygen style if and only if comment is user-facing
e7d0185c90 docs: Get rid of "initialized for signing" terminology
06126364ad docs: Tidy and improve docs about contexts and randomization
e02d6862bd selftest: Expose in public API
e383fbfa66 selftest: Rename internal function to make name available for API
d2c6d48de3 tests: Use new name of static context
53796d2b24 contexts: Rename static context
72fedf8a6c docs: Improve docs for static context
316ac7625a contexts: Deprecate all context flags except SECP256K1_CONTEXT_NONE
477f02c4de Merge bitcoin-core/secp256k1#1165: gitignore: Add *.sage.py files autogenerated by sage [skip ci]
092be61c5e gitignore: Add *.sage.py files autogenerated by sage
1a553ee8be docs: Change signature "validation" to "verification"
ee7341fbac docs: Never require a verification context
751c4354d5 Merge bitcoin-core/secp256k1#1152: Update macOS image for CI
2286f80902 Merge bitcoin-core/secp256k1#993: Enable non-experimental modules by default
e40fd277b7 Merge bitcoin-core/secp256k1#1156: Followups to int128_struct arithmetic
99bd335599 Make int128 overflow test use secp256k1_[ui]128_mul
a8494b02bf Use compute credits for macOS jobs
3afce0af7c Avoid signed overflow in MSVC AMR64 secp256k1_mul128
c0ae48c995 Update macOS image for CI
9b5f589d30 Heuristically decide whether to use int128_struct
63ff064d2f int128: Add test override for testing __(u)mulh on MSVC X64
f2b7e88768 Add int128 randomized tests
6138d73be4 Merge bitcoin-core/secp256k1#1155: Add MSan CI jobs
ddf2b2910e Merge bitcoin-core/secp256k1#1000: Synthetic int128 type.
86e3b38a4a Merge bitcoin-core/secp256k1#1149: Remove usage of CHECK from non-test file
00a42b91b3 Add MSan CI job
44916ae915 Merge bitcoin-core/secp256k1#1147: ci: print env to allow reproducing the job outside of CI
c2ee9175e9 Merge bitcoin-core/secp256k1#1146: ci: prevent "-v/--version: not found" irrelevant error
e13fae487e Merge bitcoin-core/secp256k1#1150: ci: always cat test_env.log
a340d9500a ci: add int128_struct tests
dceaa1f579 int128: Tidy #includes of int128.h and int128_impl.h
2914bccbc0 Simulated int128 type.
6a965b6b98 Remove usage of CHECK from non-test file
5c9f1a5c37 ci: always cat all logs_snippets
49ae843592 ci: mostly prevent "-v/--version: not found" irrelevant error
4e54c03153 ci: print env to allow reproducing the job outside of CI
a43e982bca Merge bitcoin-core/secp256k1#1144: Cleanup `.gitignore` file
f5039cb66c Cleanup `.gitignore` file
798727ae1e Revert "Add test logs to gitignore"
41e8704b48 build: Enable some modules by default
694ce8fb2d Merge bitcoin-core/secp256k1#1131: readme: Misc improvements
88b00897e7 readme: Fix line break
78f5296da4 readme: Sell "no runtime dependencies"
ef48f088ad readme: Add IRC channel
9f8a13dc8e Merge bitcoin-core/secp256k1#1128: configure: Remove pkgconfig macros again (reintroduced by mismerge)
cabe085bb4 configure: Remove pkgconfig macros again (reintroduced by mismerge)
3efeb9da21 Merge bitcoin-core/secp256k1#1121: config: Set preprocessor defaults for ECMULT_* config values
6a873cc4a9 Merge bitcoin-core/secp256k1#1122: tests: Randomize the context with probability 15/16 instead of 1/4
17065f48ae tests: Randomize the context with probability 15/16 instead of 1/4
c27ae45144 config: Remove basic-config.h
da6514a04a config: Introduce DEBUG_CONFIG macro for debug output of config
63a3565e97 Merge bitcoin-core/secp256k1#1120: ecmult_gen: Skip RNG when creating blinding if no seed is available
d0cf55e13a config: Set preprocessor defaults for ECMULT_* config values
55f8bc99dc ecmult_gen: Improve comments about projective blinding
7a86955800 ecmult_gen: Simplify code (no observable change)
4cc0b1b669 ecmult_gen: Skip RNG when creating blinding if no seed is available
af65d30cc8 Merge bitcoin-core/secp256k1#1116: build: Fix #include "..." paths to get rid of further -I arguments
40a3473a9d build: Fix #include "..." paths to get rid of further -I arguments
43756da819 Merge bitcoin-core/secp256k1#1115: Fix sepc256k1 -> secp256k1 typo in group.h
069aba8125 Fix sepc256k1 -> secp256k1 typo in group.h
accadc94df Merge bitcoin-core/secp256k1#1114: `_scratch_destroy`: move `VERIFY_CHECK` after invalid scrach space check
cd47033335 Merge bitcoin-core/secp256k1#1084: ci: Add MSVC builds
1827c9bf2b scratch_destroy: move VERIFY_CHECK after invalid scrach space check
49e2acd927 configure: Improve rationale for WERROR_CFLAGS
8dc4b03341 ci: Add a C++ job that compiles the public headers without -fpermissive
51f296a46c ci: Run persistent wineserver to speed up wine
3fb3269c22 ci: Add 32-bit MinGW64 build
9efc2e5221 ci: Add MSVC builds
2be6ba0fed configure: Convince autotools to work with MSVC's archiver lib.exe
bd81f4140a schnorrsig bench: Suppress a stupid warning in MSVC
09f3d71c51 configure: Add a few CFLAGS for MSVC
3b4f3d0d46 build: Reject C++ compilers in the preprocessor
1cc0941414 configure: Don't abort if the compiler does not define __STDC__
cca8cbbac8 configure: Output message when checking for valgrind
1a6be5745f bench: Make benchmarks compile on MSVC

git-subtree-dir: src/secp256k1
git-subtree-split: 21ffe4b22a9683cf24ae0763359e401d1284cc7a

9d47e7b71b

Update secp256k1 subtree to upstream libsecp256k1 version 0.2.0 3d8a6ae283

DrahtBot commented at 4:50 am on December 13, 2022: contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	Sjors, achow101, jonasnick
Concept ACK	real-or-random

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#25465 (build: remove boost library detection by fanquake)
#24742 (build: prune Boost headers in depends by fanquake)
#23561 (BIP324: Handshake prerequisites by dhruv)
#23432 (BIP324: CPubKey encode/decode to elligator-swift by dhruv)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

sipa force-pushed on Dec 13, 2022

maflcko added the label DrahtBot Guix build requested on Dec 13, 2022

in configure.ac:2015 in 21482af8e8 outdated

2011@@ -2012,7 +2012,7 @@ LIBS_TEMP="$LIBS"
2012 unset LIBS
2013 LIBS="$LIBS_TEMP"
2014 
2015-ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --enable-module-schnorrsig"

Sjors commented at 4:53 pm on December 13, 2022:

Should be then explicitly disable some others? https://github.com/bitcoin-core/secp256k1/commit/41e8704b484652cf5bbb2b7ecc27feedc3cf0ae1

--disable-module-ecdh and --disable-module-extrakeys? Or were these already enabled indirectly?

real-or-random commented at 7:13 pm on December 13, 2022:

schnorrsig depends on extrakeys, so extrakeys is automatically enabled already now.

ecdh is a prerequisite for https://github.com/bitcoin/bitcoin/pull/23561/ (see https://github.com/bitcoin/bitcoin/pull/23561/commits/240fd59a4817df1b3375c03d03dc0cbc9550530d) but I’m not sure if this is a reason to enable it right now.

sipa commented at 7:51 pm on December 13, 2022:

I’ve removed the ECDH module.

in src/pubkey.cpp:391 in ad327e1513 outdated

387@@ -388,7 +388,7 @@ ECCVerifyHandle::ECCVerifyHandle()
388 {
389     if (refcount == 0) {
390         assert(secp256k1_context_verify == nullptr);
391-        secp256k1_context_verify = secp256k1_context_create(SECP256K1_CONTEXT_VERIFY);
392+        secp256k1_context_verify = secp256k1_context_create(SECP256K1_CONTEXT_NONE);

real-or-random commented at 7:18 pm on December 13, 2022:

We could equally use the static context here if we want to get rid of all the refcounting (but that’s a somewhat larger change).

sipa commented at 7:52 pm on December 13, 2022:

Done. That’s a much bigger cleanup.

real-or-random commented at 7:18 pm on December 13, 2022: contributor

Concept ACK

DrahtBot removed the label DrahtBot Guix build requested on Dec 13, 2022

sipa force-pushed on Dec 13, 2022

in build_msvc/libsecp256k1/libsecp256k1.vcxproj:17 in b8fb14e8ba outdated

13@@ -14,7 +14,7 @@
14   </ItemGroup>
15   <ItemDefinitionGroup>
16     <ClCompile>
17-      <PreprocessorDefinitions>ENABLE_MODULE_ECDH;ENABLE_MODULE_RECOVERY;ENABLE_MODULE_EXTRAKEYS;ENABLE_MODULE_SCHNORRSIG;%(PreprocessorDefinitions)</PreprocessorDefinitions>

Sjors commented at 7:54 pm on December 13, 2022:

Do you need to explicitly disable it here too?

sipa commented at 7:57 pm on December 13, 2022:

No, this bypasses the build system and passes the C defines directly. Those only have an ENABLE_ form; the default is off for anything that isn’t defined.

instagibbs commented at 8:16 pm on January 12, 2023:

was ecdh enabled for any reason?

sipa commented at 8:47 pm on January 12, 2023:

Bitcoin Core doesn’t use the libsecp256k1 ecdh module, so it should be disabled. I don’t know why it was enabled here before.

real-or-random commented at 7:59 pm on December 13, 2022: contributor

The third commit says “Invoke secp256k1_selftest() during initialization.” but that happens in a different commit.

edit: by the way, if we can guarantee that the secp256k1_context_create(SECP256k1_CONTEXT_NONE) in key.cpp runs before the functions in pubkey.cpp, we could in theory drop the secp256k1_selftest() call. But yeah, I don’t think it’s a good idea: it creates more coupling between the files and is easy to overlook in the future.

Adapt to libsecp256k1 API changes

* Use SECP256K1_CONTEXT_NONE when creating signing context, as
  SECP256K1_CONTEXT_SIGN is deprecated and unnecessary.
* Use secp256k1_static_context where applicable.

4462cb0498

Remove explicit enabling of default modules 3bfca788b0

Add secp256k1_selftest call 2022917223

sipa force-pushed on Dec 13, 2022

sipa commented at 8:08 pm on December 13, 2022: member

The third commit says “Invoke secp256k1_selftest() during initialization.” but that happens in a different commit.

Fixed.

bitcoin deleted a comment on Dec 13, 2022

Sjors commented at 11:38 pm on December 13, 2022: member

I compiled 202291722300b86f36e97de7960d40a32544c2d1 on a Linux and an Intel mac, compiled and successfully ran the tests.

maflcko added the label DrahtBot Guix build requested on Dec 14, 2022

DrahtBot commented at 5:55 pm on December 14, 2022: contributor

Guix builds

File	commit 678889e6c6231cf461de59eefe6fb8eb07468848(master)	commit 2ea8f89c28a7f80ea90ef6fede2ea53c86ef086e(master and this pull)
SHA256SUMS.part	`49aa40ef9b94f7ce...`	`c1dcb3f424a2e99e...`
*-aarch64-linux-gnu-debug.tar.gz	`a93340960b19c90a...`	`ecb64fc6bb49c24c...`
*-aarch64-linux-gnu.tar.gz	`f7d9b398bec8ed3a...`	`95b5e24ba267da65...`
*-arm-linux-gnueabihf-debug.tar.gz	`9c3063929beb36e8...`	`bc9dc022946e5b7b...`
*-arm-linux-gnueabihf.tar.gz	`b31ec6a01a8f646f...`	`cc7b7c95f52d3857...`
*-arm64-apple-darwin-unsigned.dmg	`976f869f3a17af74...`	`b6673b7e07af41a3...`
*-arm64-apple-darwin-unsigned.tar.gz	`9e53f2d2d725869c...`	`ee142515d5288599...`
*-arm64-apple-darwin.tar.gz	`617e7f24db299fcf...`	`f76a8be090f066ab...`
*-powerpc64-linux-gnu-debug.tar.gz	`ece049f36398a61f...`	`60b7dc310cbdb278...`
*-powerpc64-linux-gnu.tar.gz	`4c41c03295d5e1bd...`	`e66b1f029e13246d...`
*-powerpc64le-linux-gnu-debug.tar.gz	`73a48e1cc36cc074...`	`2f0cc090e1ae4fda...`
*-powerpc64le-linux-gnu.tar.gz	`ae2e7655bff5f833...`	`368799c69d54f90b...`
*-riscv64-linux-gnu-debug.tar.gz	`317cfaf9eb278e16...`	`7c504186bf78f53f...`
*-riscv64-linux-gnu.tar.gz	`8e2930c98099f8c8...`	`919e76089337f778...`
*-win64-debug.zip	`c1b38de5f2d91cd0...`	`66e037b52a2db085...`
*-win64-setup-unsigned.exe	`7b02d4e5905a5550...`	`8c337e4132de70b0...`
*-win64-unsigned.tar.gz	`09c5874855fa03ab...`	`8058ba5ae25780db...`
*-win64.zip	`158479e2e0987a7c...`	`9cbde8f09ea4e362...`
*-x86_64-apple-darwin-unsigned.dmg	`01a9abefed526c22...`	`4389e6a2f5ed6e76...`
*-x86_64-apple-darwin-unsigned.tar.gz	`89b5c04e3459f654...`	`fb2836ae8977f816...`
*-x86_64-apple-darwin.tar.gz	`86407596a5a3c987...`	`c8909ae985e2f8a3...`
*-x86_64-linux-gnu-debug.tar.gz	`1ece469c5ef915d0...`	`df8c44df07e83a72...`
*-x86_64-linux-gnu.tar.gz	`97d5d292434cae7f...`	`f1529b27556d12ae...`
*.tar.gz	`b4583cf6bbc78e88...`	`ac85ac56cf0e2855...`
guix_build.log	`e45f417c7e72378d...`	`1ccf618669d93e84...`
guix_build.log.diff		`a4370a200e853dfe...`

DrahtBot removed the label DrahtBot Guix build requested on Dec 14, 2022

sipa commented at 5:02 am on January 12, 2023: member

Anything left to do here?

maflcko added this to the milestone 25.0 on Jan 12, 2023

fanquake commented at 9:03 am on January 12, 2023: member

cc @apoelstra @jonasnick

Sjors commented at 12:18 pm on January 12, 2023: member

For other reviewers, including myself, this should be useful for testing a subtree update: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#git-subtree-checksh

Sjors commented at 2:40 pm on January 12, 2023: member

ACK 202291722300b86f36e97de7960d40a32544c2d1, but 4462cb04986d77eddcfc6e8f75e04dc278a8147a could use more eyes on it.

instagibbs commented at 8:22 pm on January 12, 2023: member

commit message for 4462cb04986d77eddcfc6e8f75e04dc278a8147a

misnames secp256k1_context_static as secp256k1_static_context

achow101 commented at 8:56 pm on January 12, 2023: member

ACK 202291722300b86f36e97de7960d40a32544c2d1

jonasnick commented at 9:26 pm on January 12, 2023: contributor

utACK 202291722300b86f36e97de7960d40a32544c2d1

fanquake merged this on Jan 13, 2023

fanquake closed this on Jan 13, 2023

sidhujag referenced this in commit 4576368fab on Jan 13, 2023

kwvg referenced this in commit 4829833e2d on Sep 8, 2023

kwvg referenced this in commit 3dc1594206 on Sep 9, 2023

kwvg referenced this in commit 396e0a0f98 on Sep 9, 2023

kwvg referenced this in commit 80d49d59eb on Sep 24, 2023

kwvg referenced this in commit d753cb3ce4 on Sep 28, 2023

ogabrielides referenced this in commit c3971c9ca0 on Nov 15, 2023

UdjinM6 referenced this in commit 8ed892acfe on Nov 15, 2023

UdjinM6 referenced this in commit 76efcb4e5e on Nov 16, 2023

UdjinM6 referenced this in commit e8443f5237 on Nov 20, 2023

PastaPastaPasta referenced this in commit a3f29982ad on Nov 21, 2023

PastaPastaPasta referenced this in commit 39ce20cddc on Nov 21, 2023

bitcoin locked this on Jan 13, 2024

Update secp256k1 subtree to libsecp256k1 version 0.2.0 #26691

Reviews

Conflicts

Guix builds