Except for two rpcauth and blockfilterindex fixes, this PR does not change any behavior outside of tests. It is just supposed to enforce internal consistency and prevent bugs by ensuring that list arguments are always retrieved with GetArgs() and non-list arguments are always retrieved with GetArg(). Followup PRs could use the ALLOW_LIST flags for better documentation and error checking in the future. For example, #17493 builds on this to disallow conflicting config values.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.
Conflicts
Reviewers, this pull request conflicts with the following ones:
#27491 (refactor: Move chain constants to the util library by TheCharlatan)
#27480 (doc: Improve documentation of rpcallowip by willcl-ark)
#27446 (Allow configuring target block time for a signet by benthecarman)
#27302 (init: Error if ignored bitcoin.conf file is found by ryanofsky)
#27231 (Fix logging RPC and -debugexclude with 0/none values, add test coverage, improve docs by jonatack)
#27114 (p2p: Allow whitelisting outgoing connections by brunoerg)
#26966 (index: blockfilter initial sync speedup, parallelize process by furszy)
#26938 ([WIP] p2p: asmap, avoid inbound connections from a specific AS by brunoerg)
#24170 (p2p, rpc: Manual block-relay-only connections with addnode by mzumsande)
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.
DrahtBot added the label
Needs rebase
on Dec 19, 2019
ryanofsky force-pushed
on Dec 20, 2019
DrahtBot removed the label
Needs rebase
on Dec 20, 2019
promag
commented at 10:10 pm on December 22, 2019:
member
I don’t think it makes sense to ever include any of the ALLOW_ANY types in forbid (if you don’t want a string, then in reality you probably require a boolean eg), so having:
0 constexpr unsigned int ALLOW_ANY_EQUIV = ALLOW_BOOL | ALLOW_INT | ALLOW_STRING;
12 assert((forbid & ALLOW_ANY_EQUIV) ==0);
3if (*flags & ALLOW_ANY) require &=~ALLOW_ANY_EQUIV;
45..67return (*flags & ALLOW_ANY) ==0; // since ignore_types is gone now
might make sense?
ryanofsky
commented at 8:23 pm on January 9, 2020:
In commit “refactor: Always enforce ALLOW_LIST in CheckArgFlags” (4f3e5e9988280ac96b7979c352da218bbf939ed0):
I see your point that CheckArgFlags is never called forbidding bool/int/string flags, so no need to update forbid and probably no reason to even want to, so I could make a simplification along those lines. I think I will at least drop the forbid &= ~ignore_types line.
in
src/test/util_tests.cpp:344
in
4f3e5e9988outdated
(with a section as well to check overrides/repeats etc work correctly) and then try setting up all the args as ALLOW_ANY and check for appropriate errors, then all the args as ALLOW_BOOL and check for appropriate results/errors, and all the args as ALLOW_LIST and check for appropriate errors?
(Might be nicer to separate out the test infrastructure changes first, so that it’s easy to see all the changes to test results in the commit that changes the functionality, but YMMV)
ryanofsky
commented at 8:23 pm on January 9, 2020:
In commit “refactor: Always enforce ALLOW_LIST in CheckArgFlags” (4f3e5e9988280ac96b7979c352da218bbf939ed0)
I think it might be better to restructure the tests than have these special functions that tweak the flags.
I’m not too inclined to want to restructure the tests, though I’d be happy to review a PR that did. I also wouldn’t want to do it in this PR because I’d like the test changes here to reflect only reflect behavior that’s changing, and not have other differences.
in
src/httprpc.cpp:254
in
4bd1c3b160outdated
259@@ -260,7 +260,7 @@ static bool InitRPCAuthentication()
260 LogPrintf("Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcauth for rpcauth auth generation.\n");
261 strRPCUserColonPass = gArgs.GetArg("-rpcuser", "") + ":" + gArgs.GetArg("-rpcpassword", "");
262 }
263- if (gArgs.GetArg("-rpcauth","") != "")
264+ if (!gArgs.GetArgs("-rpcauth").empty())
ryanofsky
commented at 8:23 pm on January 9, 2020:
In commit “refactor: Fix more ALLOW_LIST arguments” (4bd1c3b1606021dff2b39a1927781b3c53428bbe)
gArgs.IsArgSet("-rpcauth") ?
That would do the wrong thing if the argument is negated. IsArgSet is generally broken and misused for list settings (and a lot of non-list settings), see #17783
ajtowns
commented at 5:05 am on January 9, 2020:
contributor
Concept ACK, shame all these checks are happening at runtime rather than compile time though
ryanofsky
commented at 9:10 pm on January 9, 2020:
contributor
Thanks for the review!
shame all these checks are happening at runtime rather than compile time though
Agree, but I think adding checks and types like this PR and #17783 are doing is really the hard part. Switching enforcement from runtime to compile time after constraints are in already place would be easier by comparison
ryanofsky force-pushed
on Jan 24, 2020
ryanofsky
commented at 9:23 pm on January 24, 2020:
contributor
DrahtBot added the label
Needs rebase
on Jan 30, 2020
ryanofsky force-pushed
on Sep 2, 2020
DrahtBot removed the label
Needs rebase
on Sep 2, 2020
DrahtBot added the label
Needs rebase
on Sep 15, 2020
ryanofsky force-pushed
on Sep 25, 2020
DrahtBot removed the label
Needs rebase
on Sep 25, 2020
ryanofsky force-pushed
on Sep 28, 2020
DrahtBot added the label
Needs rebase
on Sep 29, 2020
ryanofsky force-pushed
on Oct 2, 2020
DrahtBot removed the label
Needs rebase
on Oct 2, 2020
DrahtBot added the label
Needs rebase
on Oct 29, 2020
ryanofsky force-pushed
on Apr 10, 2021
DrahtBot removed the label
Needs rebase
on Apr 10, 2021
ryanofsky force-pushed
on Apr 11, 2021
DrahtBot added the label
Needs rebase
on Apr 15, 2021
ryanofsky force-pushed
on Jun 2, 2021
DrahtBot removed the label
Needs rebase
on Jun 2, 2021
DrahtBot added the label
Needs rebase
on Aug 22, 2021
ryanofsky force-pushed
on Aug 24, 2021
ryanofsky force-pushed
on Dec 30, 2021
DrahtBot removed the label
Needs rebase
on Dec 30, 2021
DrahtBot added the label
Needs rebase
on Feb 9, 2022
uvhw referenced this in commit
47d44ccc3e
on Feb 14, 2022
ryanofsky force-pushed
on Sep 22, 2022
DrahtBot removed the label
Needs rebase
on Sep 22, 2022
ryanofsky force-pushed
on Sep 24, 2022
DrahtBot added the label
Needs rebase
on Nov 15, 2022
ryanofsky force-pushed
on Nov 29, 2022
DrahtBot removed the label
Needs rebase
on Nov 29, 2022
DrahtBot added the label
Needs rebase
on Jan 11, 2023
fanquake marked this as a draft
on Feb 8, 2023
fanquake
commented at 11:24 am on February 8, 2023:
member
Moved this to draft for now, given it’s based on another PR, which itself is a draft.
ryanofsky force-pushed
on Feb 14, 2023
DrahtBot removed the label
Needs rebase
on Feb 15, 2023
DrahtBot added the label
Needs rebase
on Mar 20, 2023
Add ArgsManager flags to parse and validate settings on startup
This commit does not change current application behavior. It adds new
ALLOW_BOOL, ALLOW_INT, ALLOW_STRING, and ALLOW_LIST flags which are available
to be used by new settings, without affecting parsing of any existing settings.
The new flags validate settings earlier on startup and provide more detailed
error messages to users.
The new flags also provide stricter error checking. For example, a double
negated option like -nosetting=0 is treated like an error instead of true, and
an unrecognized bool value like -setting=true is treated like an error instead
of false. The new flags also trigger errors if a non-list setting is specified
multiple times in the same section of a configuration file, instead of silently
ignoring later values.
The new flags also provide type information that allows ArgsManager
GetSettings() and GetSettingsList() methods to return typed integer and boolean
values instead of unparsed strings.
The changes have no effect on current application behavior because the new
flags are only used in unit tests. The existing ALLOW_ANY checks in the
CheckValueTest unit test confirm that no current validation or parsing behavior
is changing.
68347d6950
Update ArgManager GetArg helper methods to work better with ALLOW flags
Update GetArg, GetArgs, GetBoolArg, and GetIntArg helper methods to work
conveniently with ALLOW_BOOL, ALLOW_INT, and ALLOW_STRING flags. This commit
does not change application behavior in any way because these flags are new
and currently only used in unit tests.
The GetArg methods are convenience wrappers around the GetSetting method. The
GetSetting method returns the originally parsed settings values in their
declared bool/int/string types, while the GetArg wrappers provide extra
type-coercion and default-value fallback features as additional conveniences
for callers.
This commit makes two changes to GetArg, GetArgs, GetBoolArg, and GetIntArg
helper methods when BOOL/INT/STRING flags are used:
1. GetArg methods will now raise errors if they are called with inconsistent
flags. For example, GetArgs will raise a logic_error if it is called on a
non-LIST setting, GetIntArg will raise a logic_error if it is called
on a non-INT setting.
2. GetArg methods will now avoid various type coersion footguns when they are
called on new BOOL/INT/STRING settings. Existing ALLOW_ANY settings are
unaffected. For example, negated settings will return "" empty strings
instead of "0" strings (in the past the "0" strings caused strangeness like
"-nowallet" options creating wallet files named "0"). The new behaviors are
fully specified and checked by the `CheckValueTest` unit test.
The ergonomics of the GetArg helper methods are subjective and the behaviors
they implement can be nitpicked and debated endlessly. But behavior of these
helper methods does not dictate application behavior, and they be bypassed by
calling GetSetting and GetSettingList methods instead. If it's necessary,
behavior of these helper methods can also be changed again in the future.
The changes have no effect on current application behavior because the new
flags are only used in unit tests. The `setting_args` unit test and ALLOW_ANY
checks in the `CheckValueTest` unit test are unchanged and confirm that
`GetArg` methods behave exactly the same (returning the same values and
throwing the same exceptions) for ALLOW_ANY flags before and after this change.
scripted-diff: Add ALLOW_LIST flag to arguments retrieved with GetArgs
This change has no effect on behavior, and is basically just a documentation
change at this point. The ALLOW_LIST flag is currently ignored unless
ALLOW_BOOL, ALLOW_INT, or ALLOW_STRING flags are also present, and these flags
are not used yet.
-BEGIN VERIFY SCRIPT-
for f in `git grep -n 'GetArgs(' | grep -v _tests | sed -n 's/.*GetArgs("\([^"]\+\)".*/\1/p' | sort -u`; do
git grep -l -- "$f" | xargs sed -i "/AddArg(\"$f[=\"]/ s/ArgsManager::ALLOW_ANY/& | ArgsManager::ALLOW_LIST/g"
done
-END VERIFY SCRIPT-
76d30ef525
refactor: Fix more ALLOW_LIST arguments
- Remove ALLOW_LIST flag from bitcoin-wallet -wallet and -debug arguments. They
are list arguments for bitcoind, but single arguments for bitcoin-wallet.
- Add ALLOW_LIST flag to -includeconf arg (missed by scripted diff since it's
not accessed through GetArgs)
- Add ALLOW_LIST flag to -debug, -loglevel, -whitebind, and -whitelist args
(missed by scripted diff due to line breaks in AddArgs calls)
- Add ALLOW_LIST flag to -zmq args (missed by scripted diff due to programmatic
GetArgs calls)
This change has no effect on behavior, and is basically just a documentation
change at this point. The ALLOW_LIST flag is currently ignored unless
ALLOW_BOOL, ALLOW_INT, or ALLOW_STRING flags are also present, and these flags
are not used yet.
49e25e1fa7
Always reject empty -rpcauth="" arguments
Previous behavior was nonsensical:
- If an empty -rpcauth="" argument was specified as the last command
line argument, it would cause all other -rpcauth arguments to be
ignored.
- If an empty -rpcauth="" argument was specified on the command line
followed by any nonempty -rpcauth argument, it would cause an error.
- If an empty "rpcauth=" line was specified after non-empty rpcauth line
it would cause an error.
- If an empty "rpcauth=" line in a config file was the entry in the
config file it would cause all rpcauth entries to be ignored, unless
the last command line argument was a nonempty -rpcauth argument, in
which case it would be ignored.
New behavior is simple:
- If an empty "rpcauth=" config line or -rpcauth="" command line
argument is used it will cause an error
Previous behavior was inconsistent: if -blockfilterindex or
-blockfilterindex="" arguments were specified they would normally enable all
block filter indexes, but could also trigger "Unknown -blockfilterindex value"
errors if followed by later -blockfilterindex arguments.
It was confusing that the same -blockfilterindex options could sometime trigger
errors and sometimes not depending on option position. It was also confusing
that an empty -blockfilterindex="" setting could enable all indexes even though
indexes are disabled by default.
New behavior is more straightforward:
- -blockfilterindex and -blockfilterindex=1 always enable indexes
- -noblockfilterindex and -blockfilterindex=0 always disable indexes
- -blockfilterindex="" is always an unknown value error
The meaning of these options no longer changes based on option position.
d97b30d33c
refactor: Always enforce ALLOW_LIST in CheckArgFlags
Prevent GetArg() from being called on ALLOW_LIST arguments, and GetArgs() from
being called on non-list arguments.
This checking was previously skipped unless typed INT/BOOL/STRING flags were
present, but now it's always done.
This change has no effect on external behavior. It is just supposed to enforce
internal consistency and prevent bugs caused by using the wrong GetArg method
to retrieve settings.
7e70f985ea
ryanofsky force-pushed
on May 3, 2023
DrahtBot removed the label
Needs rebase
on May 3, 2023
DrahtBot
commented at 4:37 pm on May 9, 2023:
contributor
🐙 This pull request conflicts with the target branch and needs rebase.
DrahtBot added the label
Needs rebase
on May 9, 2023
DrahtBot
commented at 2:00 am on August 7, 2023:
contributor
There hasn’t been much activity lately and the patch still needs rebase. What is the status here?
Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
DrahtBot
commented at 1:27 am on November 5, 2023:
contributor
There hasn’t been much activity lately and the patch still needs rebase. What is the status here?
Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
DrahtBot
commented at 1:27 am on February 4, 2024:
contributor
⌛ There hasn’t been much activity lately and the patch still needs rebase. What is the status here?
Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
DrahtBot
commented at 1:26 am on May 3, 2024:
contributor
⌛ There hasn’t been much activity lately and the patch still needs rebase. What is the status here?
Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
Is it no longer relevant? ➡️ Please close.
Did the author lose interest or time to work on this? ➡️ Please close it and mark it ‘Up for grabs’ with the label, so that it can be picked up in the future.
This is a metadata mirror of the GitHub repository
bitcoin/bitcoin.
This site is not affiliated with GitHub.
Content is generated from a GitHub metadata backup.
generated: 2024-07-01 10:13 UTC
This site is hosted by @0xB10C More mirrored repositories can be found on mirror.b10c.me
