doc: Fix gen-manpages, rewrite in Python #24263

pull laanwj wants to merge 2 commits into bitcoin:master from laanwj:2022-02-gen-manpages changing 4 files +77 −56
  1. laanwj commented at 3:36 pm on February 4, 2022: member

    Rewrite the manual page generation script in Python.

    This:

    • solves ‘-’ stripping issue (fixes #22681)
    • makes that a copyright footer is generated correctly again

    Also change the release process to swap gen-manpages and update RC steps, so that the pages will have the correct rc and/or final version.

  2. laanwj added the label Docs on Feb 4, 2022
  3. laanwj added this to the milestone 23.0 on Feb 4, 2022
  4. laanwj force-pushed on Feb 4, 2022
  5. laanwj commented at 3:54 pm on February 4, 2022: member

    Does anyone understand this mypy output?

     0contrib/devtools/gen-manpages.py:26: error: No overload variant of "run" matches argument types "List[str]", "bool", "bool", "bool"  [call-overload]
     1contrib/devtools/gen-manpages.py:26: note: Possible overload variants:
     2contrib/devtools/gen-manpages.py:26: note:     def run(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., stdin: Union[None, int, IO[Any]] = ..., stdout: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., env: Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]], None] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, check: bool = ..., encoding: str, errors: Optional[str] = ..., input: Optional[str] = ..., timeout: Optional[float] = ...) -> CompletedProcess[str]
     3contrib/devtools/gen-manpages.py:26: note:     def run(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., stdin: Union[None, int, IO[Any]] = ..., stdout: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., env: Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]], None] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, check: bool = ..., encoding: Optional[str] = ..., errors: str, input: Optional[str] = ..., timeout: Optional[float] = ...) -> CompletedProcess[str]
     4contrib/devtools/gen-manpages.py:26: note:     <3 more similar overloads not shown, out of 5 total overloads>
     5contrib/devtools/gen-manpages.py:31: error: Argument 1 to "join" has incompatible type "Optional[str]"; expected "Union[str, PathLike[str]]"  [arg-type]
     6contrib/devtools/gen-manpages.py:38: error: Argument 1 to "join" has incompatible type "Optional[str]"; expected "Union[str, PathLike[str]]"  [arg-type]
     7contrib/devtools/gen-manpages.py:40: error: No overload variant of "run" matches argument types "List[str]", "bool", "bool"  [call-overload]
     8contrib/devtools/gen-manpages.py:40: note: Possible overload variants:
     9contrib/devtools/gen-manpages.py:40: note:     def run(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., stdin: Union[None, int, IO[Any]] = ..., stdout: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., env: Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]], None] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, check: bool = ..., encoding: str, errors: Optional[str] = ..., input: Optional[str] = ..., timeout: Optional[float] = ...) -> CompletedProcess[str]
    10contrib/devtools/gen-manpages.py:40: note:     def run(args: Union[Union[bytes, str], Sequence[Union[str, bytes, PathLike[str], PathLike[bytes]]]], bufsize: int = ..., executable: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., stdin: Union[None, int, IO[Any]] = ..., stdout: Union[None, int, IO[Any]] = ..., stderr: Union[None, int, IO[Any]] = ..., preexec_fn: Optional[Callable[[], Any]] = ..., close_fds: bool = ..., shell: bool = ..., cwd: Union[str, bytes, PathLike[str], PathLike[bytes], None] = ..., env: Union[Mapping[bytes, Union[str, bytes, PathLike[str], PathLike[bytes]]], Mapping[str, Union[str, bytes, PathLike[str], PathLike[bytes]]], None] = ..., universal_newlines: bool = ..., startupinfo: Any = ..., creationflags: int = ..., restore_signals: bool = ..., start_new_session: bool = ..., pass_fds: Any = ..., *, check: bool = ..., encoding: Optional[str] = ..., errors: str, input: Optional[str] = ..., timeout: Optional[float] = ...) -> CompletedProcess[str]
    11contrib/devtools/gen-manpages.py:40: note:     <3 more similar overloads not shown, out of 5 total overloads>
    

    I can’t reproduce it locally with mypy 0.761 and python 3.8.10. I don’t think I’m passing anything weird to subprocess.run or path.join.

    Edit: also tried with mypy 0.931. No dice. Edit.2: FIgured it out, I think. capture_output is new in Python 3.7 so too recent.

  6. doc: Fix gen-manpages, rewrite in Python
    Rewrite the manual page generation script in Python.
    
    This:
    - Solves '-' stripping issue (fixes #22681)
    - Makes that copyright footer is generated again
    42c202893b
  7. doc: Swap gen-manpages and update RC steps in release process 87f54060ff
  8. laanwj force-pushed on Feb 4, 2022
  9. DrahtBot commented at 8:50 pm on February 4, 2022: member

    The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

    Conflicts

    Reviewers, this pull request conflicts with the following ones:

    • #22235 (script: add script to generate example bitcoin.conf by josibake)

    If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

  10. ghost commented at 9:14 pm on February 4, 2022: none
    Concept ACK
  11. laanwj commented at 8:58 am on February 5, 2022: member

    Something that is weird, but outside scope of this particular PR to fix. is that the copyright message is emitted inconsistently between binaries:

    binary -help -version
    bitcoind yes no
    bitcoin-cli no no
    bitcoin-tx no no
    bitcoin-wallet no no
    bitcoin-util no no
    bitcoin-qt no yes

    I don’t have a strong opinion on whether it should be shown with -version or -help, but it probably should be either, and consistent between binaries.

  12. laanwj commented at 9:52 am on February 14, 2022: member
    @dongcarl Can I get a concept ACK here at least? It should exactly do what you propose in #22681 (comment)
  13. in contrib/devtools/gen-manpages.py:45 in 87f54060ff
    40+        r = subprocess.run([abspath, '--version'], stdout=subprocess.PIPE, universal_newlines=True)
    41+    except IOError:
    42+        print(f'{abspath} not found or not an executable', file=sys.stderr)
    43+        sys.exit(1)
    44+    # take first line (which must contain version)
    45+    verstr = r.stdout.split('\n')[0]
    


    dongcarl commented at 6:27 pm on February 14, 2022:

    Nit

    0    verstr = r.stdout.splitlines()[0]
    

    fanquake commented at 9:01 pm on February 21, 2022:
    done in #24409.
  14. in contrib/devtools/gen-manpages.py:40 in 87f54060ff
    35+copyright = None
    36+versions = []
    37+for relpath in BINARIES:
    38+    abspath = os.path.join(builddir, relpath)
    39+    try:
    40+        r = subprocess.run([abspath, '--version'], stdout=subprocess.PIPE, universal_newlines=True)
    


    dongcarl commented at 6:28 pm on February 14, 2022:
    Any reason not to do check=True here?

    fanquake commented at 8:54 pm on February 21, 2022:
    The script will fail because bitcoin-wallet returns non-0.

    laanwj commented at 8:48 am on February 22, 2022:
    Yes, I initially did check but as @fanquake says. I’m not sure why bitcoin-wallet does that actually.

    fanquake commented at 10:22 am on February 23, 2022:
    Addresses in #24428.
  15. dongcarl approved
  16. dongcarl commented at 6:31 pm on February 14, 2022: member

    Code Review ACK 87f54060ffffdb56c97594efdca378bace5323df

    Had to remind myself of the situation but everything looks good, just a few nits that are 100% ignore-able.

  17. theStack commented at 7:16 pm on February 20, 2022: member
    Concept ACK
  18. gruve-p commented at 9:49 pm on February 20, 2022: contributor
    Concept ACK
  19. fanquake approved
  20. fanquake commented at 2:17 pm on February 21, 2022: member
    ACK 87f54060ffffdb56c97594efdca378bace5323df - tested generating and opening the man pages locally, but didn’t run through the release process. Will propose some changes to address consolidating the help / version output.
  21. fanquake merged this on Feb 21, 2022
  22. fanquake closed this on Feb 21, 2022

  23. fanquake referenced this in commit cb8bea5e37 on Feb 21, 2022
  24. fanquake referenced this in commit 0ed0d5dcb2 on Feb 22, 2022
  25. fanquake referenced this in commit 5a89bed410 on Feb 22, 2022
  26. sidhujag referenced this in commit 090a56e493 on Feb 22, 2022
  27. MarcoFalke referenced this in commit 3c565302aa on Feb 23, 2022
  28. fanquake referenced this in commit 93c8b7a4ad on Feb 23, 2022
  29. fanquake referenced this in commit 394fdcc9d6 on Feb 23, 2022
  30. fanquake referenced this in commit 05a0be9546 on Feb 23, 2022
  31. fanquake referenced this in commit dd532ee9c4 on Feb 23, 2022
  32. sidhujag referenced this in commit 4ac249e754 on Feb 23, 2022
  33. MarcoFalke referenced this in commit d76a423809 on Sep 20, 2022
  34. DrahtBot locked this on Feb 23, 2023

github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-12-23 03:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me