This PR speeds up descriptor parsing, for basically descriptors that use xpub/xpriv, through some targeted improvements (I've been working on fuzz targets for wallet stuff for a while. I've noticed that some of them are slow so I started to work on understanding why and improve what can be improved. The finding came from it):
Avoid redundant base58 decoding when parsing extended keys (this is the major improvement)
ParsePubkeyInner unconditionally called both DecodeExtKey() and DecodeExtPubKey(), each running a full DecodeBase58Check (two SHA256 passes over ~111 bytes). A new DecodeExtKeyOrPubKey() function performs a single base58 decode and then checks the prefix, halving the cryptographic work per key. The interfaces of DecodeSecret, DecodeExtKey, and DecodeExtPubKey are also updated to accept string_view instead of const std::string&, eliminating unnecessary string copies in the descriptor parser and allowing DecodeBase58/DecodeBase58Check to use pointer-range iteration instead of strlen.
Replace O(n) charset lookup with precomputed table
DescriptorChecksum previously called std::string::find() for every character in the descriptor string, doing up to 96 comparisons per character. This replaces it with a 256-entry precomputed lookup table for O(1) lookups. The benefit scales with descriptor length.
Parse() with the following descriptor:
tr(xpub6CGCzpvrZVssNNXuP1Awcz8GoakYyJHra2WHASXfCzsD9szbQFoT1pLMdQopyqaSqXUprH7fkk8NG2pUbQa5HpNzpWmEsdNZDHxXvSLPTrn/*,{{multi_a(20,xpub6CGCzpvrZVssNNXuP1Awcz8GoakYyJHra2WHASXfCzsD9szbQFoT1pLMdQopyqaSqXUprH7fkk8NG2pUbQa5HpNzpWmEsdNZDHxXvSLPTrn/*,xpub6C5MW95EE8hAmxm96LdAFYkweaAktQoojCf2MwJmNL9Fi6oUPeBnpFdEzhs9zHwr4HQEeF1r1yAUR8MqDJqk4YPteh75rNoCKCtud7shDDk/*,xpub6DNEUJP1zLAZ1P6gYRba9FyxuppmkrPmSKz3FLM7msRwn2LEr4LnKfAnNoSoAbnbW2orTfsitdfWTxLbJCZ6LizqV7cFUavSAjWzNjqdfCP/*,xpub6Cm5sAgCZQvYtnatYPdFuLhsD9VtgY3Rtt48TrdgqXm4DttXfM6nFmFTko8KcyGhPswTvPcTzfpMUvqV3F9tgLPvzfzoKwjmU668vBmGXEX/*,xpub6DN66R61Luv95HLvznR2ZWsbYuGBoK9LDR6XiNm1pvp5zRJ2vFhLQ4PBe2pDn59sKmRnEFbdLtBVZFN3QAujcGZ5PomGCvrnisxkP2fcgHg/*,xpub6D2GtAtYodmjwEoAzorKUta44NDvAB4JjafYXxsQ5hTWvCAK55nnkyNHfdjrsx3umfgGe1nnHut13mLuFqqi2fs7VVyv6rht76ddrDGhrTY/*,xpub6Bfi8jsDMuu2XHQ1TGRB6ehjdHKKmTwRFQYU6QcgHoZswqbzKsXs9Qd87nAxevjZwuQ37jaVnQoPF6wY34BJoeLxXkEYMpKz2CBHLcEmJVF/*,xpub6CYiNdQesStiBwZkWjPBxwxxf6oPMH9meBMujCwBX74X8nCpMF9zRZUbDrU9raQah1LQcnzLQrHC6Kr6LNCmsBpaMkYD3U6FyhHZfTHQGUB/*,xpub6CddszJHLa3VisZ6ufU4hiFcGsuuPHo7hYa24o3kPuPfAbMED4ajuDa6svuqxxBxYgUQhN8LK7MtTMetquNo7YsWhiV8hX2gBfjszJVXgkG/*,xpub6DAM262AxC4Um3rxBiP5FoA7SJJG3a7MjaJkEyXWyMA6iNuyqWueKZHSSNjKEWnn8Wi3PUQNTV1Z3a59C2Qaidha88GTWtAMhJBAWBBvuTN/*,xpub6ByZcsLsr8xTCJmYxvLpLrsFzuHNXRLR4xMyB9ZFLek1hDVh7zba2xAVTU8M6eonFFAAPYpTA76bDuopF5zyBkZqFc3rtJ3ziH1BUXkjC6j/*,xpub6Bx2LErujvR1noXrk7RHQB9htiDcPM5qYuV7Jg1nZh5jWeK8pYB4DnYQ73EX9sB9M6rnNsF8LDPUejHiXrBx2JJQSdVQ9i9FfEzJLuAnzEd/*,xpub6DSBog1uLngjBWVDUpLTQ19jGGKovgc3iTd1ijL4fjk9sgNqrnG5kGLjUWncay2jbqdgt6yrbYSHhfwsTMp7i8pUD1FMCQuFyqdsZpgUNH8/*,xpub6C53ov8LpuGp5MM4ghqUFJcdTnXJCRHyXaz1RVCys2uQErpdzUWHtm8Bsg8NJyaqfhGWvm9WPSE7jMneCRw1xs1CTajpToXyRqSCc41JiuY/*,xpub6CscWPNXdEybbJVvx7fJTrfHyTkKZtb97qcpQ9fH4k4xtm4nkckSpvy86heQsdyoF28CugwLHg5kLDwpGoPQAYXQ3RG9cJB2Na1FdeN1wt5/*,xpub6Cmrivt6w3kVAxw7NmcKSY19gSLmC2tVwx46cdigr9d1UEnttJCGZJ7NQu1MkugpZgF9rQ3j3NKUiZ2MoaC2QoZUfi4nwxbpH39SQZaBHut/*,xpub6D1Ey5JAYnBxFE8i2V2SUsEpJt7r3ykJKcShcpT2xN9CXtVQy5dKeYiZ9D1bdnkPrmgEajrSuA5nWqf94s6u7FwvXy3cU4BXUMgAt9C8wGT/*,xpub6CwJ2vicnoSJNFDzhonucHNVHDwyb1vyiu2VF6zmjQrzGb4qoAma1i2QJ7qRNe1qmpvvWwzSuQDgMnCECFQhZaPKuPFjcMo33r2xenBauxf/*,xpub6CiLNz8mFB4fB51x6gMNu44Up9PyVhq7fSxQb8UzYQermvMShwWyqzQe99Uiyxpfy1ow42xUGpPGBHf8ErxDbQojBf7Uy7hKFgYk8KpgMm2/*,xpub6DQ2qVMZ2o2QGGpYDsw5hpAcygNVEycAkkCib2Lexwm8gYWMCSecZ256AbAV93Rgze6h8yEoTcqUnvnz7L69WUqp7cXd85X8HoeKWjPHRRZ/*),multi_a(20,xpub6CGCzpvrZVssNNXuP1Awcz8GoakYyJHra2WHASXfCzsD9szbQFoT1pLMdQopyqaSqXUprH7fkk8NG2pUbQa5HpNzpWmEsdNZDHxXvSLPTrn/*,xpub6C5MW95EE8hAmxm96LdAFYkweaAktQoojCf2MwJmNL9Fi6oUPeBnpFdEzhs9zHwr4HQEeF1r1yAUR8MqDJqk4YPteh75rNoCKCtud7shDDk/*,xpub6DNEUJP1zLAZ1P6gYRba9FyxuppmkrPmSKz3FLM7msRwn2LEr4LnKfAnNoSoAbnbW2orTfsitdfWTxLbJCZ6LizqV7cFUavSAjWzNjqdfCP/*,xpub6Cm5sAgCZQvYtnatYPdFuLhsD9VtgY3Rtt48TrdgqXm4DttXfM6nFmFTko8KcyGhPswTvPcTzfpMUvqV3F9tgLPvzfzoKwjmU668vBmGXEX/*,xpub6DN66R61Luv95HLvznR2ZWsbYuGBoK9LDR6XiNm1pvp5zRJ2vFhLQ4PBe2pDn59sKmRnEFbdLtBVZFN3QAujcGZ5PomGCvrnisxkP2fcgHg/*,xpub6D2GtAtYodmjwEoAzorKUta44NDvAB4JjafYXxsQ5hTWvCAK55nnkyNHfdjrsx3umfgGe1nnHut13mLuFqqi2fs7VVyv6rht76ddrDGhrTY/*,xpub6Bfi8jsDMuu2XHQ1TGRB6ehjdHKKmTwRFQYU6QcgHoZswqbzKsXs9Qd87nAxevjZwuQ37jaVnQoPF6wY34BJoeLxXkEYMpKz2CBHLcEmJVF/*,xpub6CYiNdQesStiBwZkWjPBxwxxf6oPMH9meBMujCwBX74X8nCpMF9zRZUbDrU9raQah1LQcnzLQrHC6Kr6LNCmsBpaMkYD3U6FyhHZfTHQGUB/*,xpub6CddszJHLa3VisZ6ufU4hiFcGsuuPHo7hYa24o3kPuPfAbMED4ajuDa6svuqxxBxYgUQhN8LK7MtTMetquNo7YsWhiV8hX2gBfjszJVXgkG/*,xpub6DAM262AxC4Um3rxBiP5FoA7SJJG3a7MjaJkEyXWyMA6iNuyqWueKZHSSNjKEWnn8Wi3PUQNTV1Z3a59C2Qaidha88GTWtAMhJBAWBBvuTN/*,xpub6ByZcsLsr8xTCJmYxvLpLrsFzuHNXRLR4xMyB9ZFLek1hDVh7zba2xAVTU8M6eonFFAAPYpTA76bDuopF5zyBkZqFc3rtJ3ziH1BUXkjC6j/*,xpub6Bx2LErujvR1noXrk7RHQB9htiDcPM5qYuV7Jg1nZh5jWeK8pYB4DnYQ73EX9sB9M6rnNsF8LDPUejHiXrBx2JJQSdVQ9i9FfEzJLuAnzEd/*,xpub6DSBog1uLngjBWVDUpLTQ19jGGKovgc3iTd1ijL4fjk9sgNqrnG5kGLjUWncay2jbqdgt6yrbYSHhfwsTMp7i8pUD1FMCQuFyqdsZpgUNH8/*,xpub6C53ov8LpuGp5MM4ghqUFJcdTnXJCRHyXaz1RVCys2uQErpdzUWHtm8Bsg8NJyaqfhGWvm9WPSE7jMneCRw1xs1CTajpToXyRqSCc41JiuY/*,xpub6CscWPNXdEybbJVvx7fJTrfHyTkKZtb97qcpQ9fH4k4xtm4nkckSpvy86heQsdyoF28CugwLHg5kLDwpGoPQAYXQ3RG9cJB2Na1FdeN1wt5/*,xpub6Cmrivt6w3kVAxw7NmcKSY19gSLmC2tVwx46cdigr9d1UEnttJCGZJ7NQu1MkugpZgF9rQ3j3NKUiZ2MoaC2QoZUfi4nwxbpH39SQZaBHut/*,xpub6D1Ey5JAYnBxFE8i2V2SUsEpJt7r3ykJKcShcpT2xN9CXtVQy5dKeYiZ9D1bdnkPrmgEajrSuA5nWqf94s6u7FwvXy3cU4BXUMgAt9C8wGT/*,xpub6CwJ2vicnoSJNFDzhonucHNVHDwyb1vyiu2VF6zmjQrzGb4qoAma1i2QJ7qRNe1qmpvvWwzSuQDgMnCECFQhZaPKuPFjcMo33r2xenBauxf/*,xpub6CiLNz8mFB4fB51x6gMNu44Up9PyVhq7fSxQb8UzYQermvMShwWyqzQe99Uiyxpfy1ow42xUGpPGBHf8ErxDbQojBf7Uy7hKFgYk8KpgMm2/*,xpub6DQ2qVMZ2o2QGGpYDsw5hpAcygNVEycAkkCib2Lexwm8gYWMCSecZ256AbAV93Rgze6h8yEoTcqUnvnz7L69WUqp7cXd85X8HoeKWjPHRRZ/*)},{multi_a(20,xpub6CGCzpvrZVssNNXuP1Awcz8GoakYyJHra2WHASXfCzsD9szbQFoT1pLMdQopyqaSqXUprH7fkk8NG2pUbQa5HpNzpWmEsdNZDHxXvSLPTrn/*,xpub6C5MW95EE8hAmxm96LdAFYkweaAktQoojCf2MwJmNL9Fi6oUPeBnpFdEzhs9zHwr4HQEeF1r1yAUR8MqDJqk4YPteh75rNoCKCtud7shDDk/*,xpub6DNEUJP1zLAZ1P6gYRba9FyxuppmkrPmSKz3FLM7msRwn2LEr4LnKfAnNoSoAbnbW2orTfsitdfWTxLbJCZ6LizqV7cFUavSAjWzNjqdfCP/*,xpub6Cm5sAgCZQvYtnatYPdFuLhsD9VtgY3Rtt48TrdgqXm4DttXfM6nFmFTko8KcyGhPswTvPcTzfpMUvqV3F9tgLPvzfzoKwjmU668vBmGXEX/*,xpub6DN66R61Luv95HLvznR2ZWsbYuGBoK9LDR6XiNm1pvp5zRJ2vFhLQ4PBe2pDn59sKmRnEFbdLtBVZFN3QAujcGZ5PomGCvrnisxkP2fcgHg/*,xpub6D2GtAtYodmjwEoAzorKUta44NDvAB4JjafYXxsQ5hTWvCAK55nnkyNHfdjrsx3umfgGe1nnHut13mLuFqqi2fs7VVyv6rht76ddrDGhrTY/*,xpub6Bfi8jsDMuu2XHQ1TGRB6ehjdHKKmTwRFQYU6QcgHoZswqbzKsXs9Qd87nAxevjZwuQ37jaVnQoPF6wY34BJoeLxXkEYMpKz2CBHLcEmJVF/*,xpub6CYiNdQesStiBwZkWjPBxwxxf6oPMH9meBMujCwBX74X8nCpMF9zRZUbDrU9raQah1LQcnzLQrHC6Kr6LNCmsBpaMkYD3U6FyhHZfTHQGUB/*,xpub6CddszJHLa3VisZ6ufU4hiFcGsuuPHo7hYa24o3kPuPfAbMED4ajuDa6svuqxxBxYgUQhN8LK7MtTMetquNo7YsWhiV8hX2gBfjszJVXgkG/*,xpub6DAM262AxC4Um3rxBiP5FoA7SJJG3a7MjaJkEyXWyMA6iNuyqWueKZHSSNjKEWnn8Wi3PUQNTV1Z3a59C2Qaidha88GTWtAMhJBAWBBvuTN/*,xpub6ByZcsLsr8xTCJmYxvLpLrsFzuHNXRLR4xMyB9ZFLek1hDVh7zba2xAVTU8M6eonFFAAPYpTA76bDuopF5zyBkZqFc3rtJ3ziH1BUXkjC6j/*,xpub6Bx2LErujvR1noXrk7RHQB9htiDcPM5qYuV7Jg1nZh5jWeK8pYB4DnYQ73EX9sB9M6rnNsF8LDPUejHiXrBx2JJQSdVQ9i9FfEzJLuAnzEd/*,xpub6DSBog1uLngjBWVDUpLTQ19jGGKovgc3iTd1ijL4fjk9sgNqrnG5kGLjUWncay2jbqdgt6yrbYSHhfwsTMp7i8pUD1FMCQuFyqdsZpgUNH8/*,xpub6C53ov8LpuGp5MM4ghqUFJcdTnXJCRHyXaz1RVCys2uQErpdzUWHtm8Bsg8NJyaqfhGWvm9WPSE7jMneCRw1xs1CTajpToXyRqSCc41JiuY/*,xpub6CscWPNXdEybbJVvx7fJTrfHyTkKZtb97qcpQ9fH4k4xtm4nkckSpvy86heQsdyoF28CugwLHg5kLDwpGoPQAYXQ3RG9cJB2Na1FdeN1wt5/*,xpub6Cmrivt6w3kVAxw7NmcKSY19gSLmC2tVwx46cdigr9d1UEnttJCGZJ7NQu1MkugpZgF9rQ3j3NKUiZ2MoaC2QoZUfi4nwxbpH39SQZaBHut/*,xpub6D1Ey5JAYnBxFE8i2V2SUsEpJt7r3ykJKcShcpT2xN9CXtVQy5dKeYiZ9D1bdnkPrmgEajrSuA5nWqf94s6u7FwvXy3cU4BXUMgAt9C8wGT/*,xpub6CwJ2vicnoSJNFDzhonucHNVHDwyb1vyiu2VF6zmjQrzGb4qoAma1i2QJ7qRNe1qmpvvWwzSuQDgMnCECFQhZaPKuPFjcMo33r2xenBauxf/*,xpub6CiLNz8mFB4fB51x6gMNu44Up9PyVhq7fSxQb8UzYQermvMShwWyqzQe99Uiyxpfy1ow42xUGpPGBHf8ErxDbQojBf7Uy7hKFgYk8KpgMm2/*,xpub6DQ2qVMZ2o2QGGpYDsw5hpAcygNVEycAkkCib2Lexwm8gYWMCSecZ256AbAV93Rgze6h8yEoTcqUnvnz7L69WUqp7cXd85X8HoeKWjPHRRZ/*),multi_a(20,xpub6CGCzpvrZVssNNXuP1Awcz8GoakYyJHra2WHASXfCzsD9szbQFoT1pLMdQopyqaSqXUprH7fkk8NG2pUbQa5HpNzpWmEsdNZDHxXvSLPTrn/*,xpub6C5MW95EE8hAmxm96LdAFYkweaAktQoojCf2MwJmNL9Fi6oUPeBnpFdEzhs9zHwr4HQEeF1r1yAUR8MqDJqk4YPteh75rNoCKCtud7shDDk/*,xpub6DNEUJP1zLAZ1P6gYRba9FyxuppmkrPmSKz3FLM7msRwn2LEr4LnKfAnNoSoAbnbW2orTfsitdfWTxLbJCZ6LizqV7cFUavSAjWzNjqdfCP/*,xpub6Cm5sAgCZQvYtnatYPdFuLhsD9VtgY3Rtt48TrdgqXm4DttXfM6nFmFTko8KcyGhPswTvPcTzfpMUvqV3F9tgLPvzfzoKwjmU668vBmGXEX/*,xpub6DN66R61Luv95HLvznR2ZWsbYuGBoK9LDR6XiNm1pvp5zRJ2vFhLQ4PBe2pDn59sKmRnEFbdLtBVZFN3QAujcGZ5PomGCvrnisxkP2fcgHg/*,xpub6D2GtAtYodmjwEoAzorKUta44NDvAB4JjafYXxsQ5hTWvCAK55nnkyNHfdjrsx3umfgGe1nnHut13mLuFqqi2fs7VVyv6rht76ddrDGhrTY/*,xpub6Bfi8jsDMuu2XHQ1TGRB6ehjdHKKmTwRFQYU6QcgHoZswqbzKsXs9Qd87nAxevjZwuQ37jaVnQoPF6wY34BJoeLxXkEYMpKz2CBHLcEmJVF/*,xpub6CYiNdQesStiBwZkWjPBxwxxf6oPMH9meBMujCwBX74X8nCpMF9zRZUbDrU9raQah1LQcnzLQrHC6Kr6LNCmsBpaMkYD3U6FyhHZfTHQGUB/*,xpub6CddszJHLa3VisZ6ufU4hiFcGsuuPHo7hYa24o3kPuPfAbMED4ajuDa6svuqxxBxYgUQhN8LK7MtTMetquNo7YsWhiV8hX2gBfjszJVXgkG/*,xpub6DAM262AxC4Um3rxBiP5FoA7SJJG3a7MjaJkEyXWyMA6iNuyqWueKZHSSNjKEWnn8Wi3PUQNTV1Z3a59C2Qaidha88GTWtAMhJBAWBBvuTN/*,xpub6ByZcsLsr8xTCJmYxvLpLrsFzuHNXRLR4xMyB9ZFLek1hDVh7zba2xAVTU8M6eonFFAAPYpTA76bDuopF5zyBkZqFc3rtJ3ziH1BUXkjC6j/*,xpub6Bx2LErujvR1noXrk7RHQB9htiDcPM5qYuV7Jg1nZh5jWeK8pYB4DnYQ73EX9sB9M6rnNsF8LDPUejHiXrBx2JJQSdVQ9i9FfEzJLuAnzEd/*,xpub6DSBog1uLngjBWVDUpLTQ19jGGKovgc3iTd1ijL4fjk9sgNqrnG5kGLjUWncay2jbqdgt6yrbYSHhfwsTMp7i8pUD1FMCQuFyqdsZpgUNH8/*,xpub6C53ov8LpuGp5MM4ghqUFJcdTnXJCRHyXaz1RVCys2uQErpdzUWHtm8Bsg8NJyaqfhGWvm9WPSE7jMneCRw1xs1CTajpToXyRqSCc41JiuY/*,xpub6CscWPNXdEybbJVvx7fJTrfHyTkKZtb97qcpQ9fH4k4xtm4nkckSpvy86heQsdyoF28CugwLHg5kLDwpGoPQAYXQ3RG9cJB2Na1FdeN1wt5/*,xpub6Cmrivt6w3kVAxw7NmcKSY19gSLmC2tVwx46cdigr9d1UEnttJCGZJ7NQu1MkugpZgF9rQ3j3NKUiZ2MoaC2QoZUfi4nwxbpH39SQZaBHut/*,xpub6D1Ey5JAYnBxFE8i2V2SUsEpJt7r3ykJKcShcpT2xN9CXtVQy5dKeYiZ9D1bdnkPrmgEajrSuA5nWqf94s6u7FwvXy3cU4BXUMgAt9C8wGT/*,xpub6CwJ2vicnoSJNFDzhonucHNVHDwyb1vyiu2VF6zmjQrzGb4qoAma1i2QJ7qRNe1qmpvvWwzSuQDgMnCECFQhZaPKuPFjcMo33r2xenBauxf/*,xpub6CiLNz8mFB4fB51x6gMNu44Up9PyVhq7fSxQb8UzYQermvMShwWyqzQe99Uiyxpfy1ow42xUGpPGBHf8ErxDbQojBf7Uy7hKFgYk8KpgMm2/*,xpub6DQ2qVMZ2o2QGGpYDsw5hpAcygNVEycAkkCib2Lexwm8gYWMCSecZ256AbAV93Rgze6h8yEoTcqUnvnz7L69WUqp7cXd85X8HoeKWjPHRRZ/*)}})
This PR:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 90,253.50 | 11,079.90 | 0.2% | 0.01 | `ParseDescriptor`
Master:
| ns/op | op/s | err% | total | benchmark
|--------------------:|--------------------:|--------:|----------:|:----------
| 127,812.50 | 7,823.96 | 0.5% | 0.01 | `ParseDescriptor`
I didn't put the benchmark itself here but if reviewers think worth, I can do it.
Joint work with claude 😅