tools for analyzing, updating and adding copyright headers in source files #8674

isle2983 commented at 2:08 AM on September 7, 2016: contributor

Three scripts in contrib/devtools/ are introduced:

copyright_header_report.py

outputs a report of copyright claims in the source files
helps to identify files that are missing copyright
helps to identify files that were added that have a new copyright holder beyond the current known set
helps to find typos in the copyright header

copyright_header_update.py

replaces fix-copyright-headers.py
does file editing in native python rather than an external call to perl

copyright_header_insert.py

inserts a properly-formatted header with proper dates into a given source file

Altogether, I hope they make the task of managing the copyright headers a bit easier.

fanquake added the label Docs and Output on Sep 7, 2016

fanquake commented at 2:18 AM on September 7, 2016: member

I'm not sure that maintaining copyright headers is such an issue that it warrants splitting one tool into three. Interested in other opinions.

MarcoFalke commented at 5:30 PM on September 7, 2016: member

Agree with @fanquake: I'd prefer to have the functionality (report and update) in a single script.

Also, I don't think copyright_header_insert.py is required. (If someone forgets to add the header, they will forget to run the script as well)

isle2983 force-pushed on Sep 8, 2016

isle2983 commented at 1:31 AM on September 8, 2016: contributor

I have condensed the three files into one and made them subcommands:

$ ./copyright_header_report.py <base_directory> [verbose]
$ ./copyright_header_update.py <base_directory>
$ ./copyright_header_report.py <file>

becomes:

$ ./copyright_header.py report <base_directory> [verbose]
$ ./copyright_header.py update <base_directory>
$ ./copyright_header.py insert <file>

running with no subcommand to copyright_header.py lists a usage string with the subcommands. Running a subcommand without arguments displays the subcommands usage string.

If we are against the 'insert' functionality, I will remove it, however I think it is useful for a few things not mentioned:

It defines the exact format of a proper header
It gets the date right
It can be a building block of a strict CI system that rejects bad/missing headers and directs you to "please run./copyright_header.py insert <your file>" to fix the problem.

fanquake commented at 3:15 AM on September 9, 2016: member

You'll also need to update the contrib docs here.

[devtools] script support for managing source file copyright headers

Three subcommands to this script:

1) ./copyright_header.py report

Examines git-tracked files with extensions that match:

INCLUDE = ['*.h', '*.cpp', '*.cc', '*.c', '*.py']

Helps to:

-> Identify source files without copyright
-> Identify source files added with something other than "The Bitcoin Core
developers" holder so we can be sure it is appropriate
-> Identify unintentional typos in the copyright line

2) ./copyright_header.py update

Replaces fix-copyright-headers.py. It does file editing in native python
rather than subprocessing out to perl as was the case with
fix-copyright-headers.py. It also shares code with the 'report' functions.

3) ./copyright_header.py insert

Inserts a copyright header into a source file with the proper format and
dates.

159597a2b8

isle2983 force-pushed on Sep 10, 2016

isle2983 commented at 9:07 PM on September 10, 2016: contributor

Thanks @fanquake. 159597a2b8967c8bc8106d98705d68932500b58a includes an update to README.md to document the new script and subcommands. Also, I tweaked some grammar and fixed some spelling in the usage strings inside copyright_header.py for clarity and made sure they matched README.md in content.

Rendered README.md

isle2983 renamed this:
~~tools for analyzing, updating and adding copyright headers in souce files~~
tools for analyzing, updating and adding copyright headers in source files
on Sep 10, 2016

laanwj commented at 3:37 PM on September 13, 2016: member

Concept ACK 159597a

in contrib/devtools/copyright_header.py:None in 159597a2b8

  36 | +    # python init:
  37 | +    '*__init__.py',
  38 | +]
  39 | +EXCLUDE_COMPILED = re.compile('|'.join([fnmatch.translate(m) for m in EXCLUDE]))
  40 | +
  41 | +INCLUDE = ['*.h', '*.cpp', '*.cc', '*.c', '*.py']

luke-jr commented at 10:23 PM on September 21, 2016:

Probably should include *.sh, *.am, *.m4, and *.include

Might make more sense as an exclusion list...

MarcoFalke commented at 10:36 PM on September 21, 2016:

We don't use cc

laanwj commented at 1:45 AM on September 22, 2016:

Leveldb does, though, but yes it makes no sense to include that here.

laanwj merged this on Nov 2, 2016

laanwj closed this on Nov 2, 2016

laanwj referenced this in commit bae81b83d6 on Nov 2, 2016

rebroad commented at 2:17 AM on December 21, 2016: contributor

Who should run this and when?

isle2983 commented at 2:54 AM on December 21, 2016: contributor

Good question.

I have a working branch that hooks it into TravisCI to fail branches that are missing copyright headers - forcing the submitter to either add them or to manually exclude the file from the check. I plan on submitting the PR when I get the chance to polish it a bit more

If the overall feeling is that this kind of thing is helpful and not a pain for day-to-day work, there might be utility in doing similar scripts for TravisCI to gradually ratchet in enforcement of basic whitespace rules and/or eventually clang-format and/or pylint style checks.

Also, the years in the headers need to be upgraded every so often to make sure they are current. December 31st might be a good day to run the script and post the PR.

laanwj commented at 8:45 AM on December 21, 2016: member

Who should run this and when?

The best time to update a copyright header is when you touch the file
The second-best is a single mass update at the end of the year of all the files that were touched in that year (that's what the scripting is for).

codablock referenced this in commit bb823614d6 on Sep 19, 2017

sickpig referenced this in commit e99001b239 on Jan 10, 2018

sickpig referenced this in commit 3a885692f6 on Jan 10, 2018

gandrewstone referenced this in commit 350c39cbef on Jan 11, 2018

codablock referenced this in commit 0b93a58947 on Jan 13, 2018

andvgal referenced this in commit 8cccc624a9 on Jan 6, 2019

MarcoFalke locked this on Sep 8, 2021