RFC: “Insufficient review” tag for closed PRs #29839

issue ryanofsky openend this issue on April 9, 2024
  1. ryanofsky commented at 9:37 pm on April 9, 2024: contributor

    During PR triage today, it seems like we closed or considered closing a number of PRs purely due to lack of review and interest in reviewing, not due to any problems with the PRs themselves.

    I think it’s reasonable to close PRs like these, because we have a severe lack of reviewer bandwidth, so having lots of low priority PRs open can make it harder for higher priority PRs to receive the attention they deserve.

    However, I think it would be good to distinguish between PRs that are good but closed because they provide relatively minor benefits, and PRs that were closed because they have problems or need more work.

    Suggestion: I want to suggest closing these PRs with an “Insufficient review” tag and message like “This PR has not attracted enough review while it’s been open, and is being closed to focus limited reviewer attention on PRs that seem like a higher priority to other project contributors. If there is more interest in reviewing this, it can be reopened.”

    I think an “Insufficient review” tag would be useful to know at a glance which closed PRs seem like good changes and don’t have known problems, and to be able to quantify how many potentially good PRs are getting left behind due to review shortage. I also think if the closing message describes the problem as a review shortage, not a problem with the PR itself, it might be less discouraging to contributors, especially new contributors. And it might even encourage more review.

    For reference, the current closing message for these PRs is “The PR didn’t seem to attract much attention in the past. Also, the issue seems not important enough right now to keep it sitting around idle in the list of open PRs. Closing due to lack of interest.”

  2. maflcko commented at 9:10 am on April 11, 2024: member

    I agree that it should be added, but I think there is an issue that not everyone is happy to NACK a pull request, so instead they’ll wait for the pull request to go stale and be closed “naturally” to avoid having to spend time to NACK it.

    Just something to watch out for when applying the label liberally, but otherwise sgtm.

  3. laanwj added the label Brainstorming on Apr 14, 2024
  4. edilmedeiros commented at 3:14 am on May 9, 2024: contributor

    I like the idea.

    Another idea would be to have a/the bot to automatically tag the PR with insufficient review if it gets stalled for a few weeks, before closing it, both to facilitate closing them for lack of activity but also to try to attract review effort for it.

    Since we are brainstorming how to get more attention to review, what about also having a tag for PRs related to projects that the core developers defined as priority for the current release cycle (priority)?

    This will tend to attract more review effort overall, even for those more complex changes. It’s not required for experienced reviewers since they can easily identify those from context. On the other hand, it might indicate to newbies like me what is most needed or what to pick if I would like to contribute to stuff that’s in the fringes.

    Obvious downside is increased bureaucratic effort on repo maintenance.

  5. maflcko commented at 3:59 pm on May 13, 2024: member

    Another idea would be to have a/the bot to automatically tag the PR with insufficient review if it gets stalled for a few weeks, before closing it, both to facilitate closing them for lack of activity but also to try to attract review effort for it.

    Not sure if this can be done automatically, because evaluating whether review is sufficient or not may involve a broader understanding. For example, the existing review may have been sufficient, but a minor change was pushed and no one yet did the trivial re-review. Also, the PR may have sufficient review, but was simply forgotten by maintainers. Finally, the PR may be waiting on the author to address feedback, instead of waiting for review.

    Maybe the message and duration for inactive_stale_ (https://github.com/maflcko/DrahtBot/blob/6ecd75ffd60675be57ce794687769e3298150560/stale/config.yml#L14-L19) could be adjusted to include the new label and wording proposed here, but I think going lower than 90 days in the bot may be problematic, because some PRs are difficult and are expected to take longer.


github-metadata-mirror

This is a metadata mirror of the GitHub repository bitcoin/bitcoin. This site is not affiliated with GitHub. Content is generated from a GitHub metadata backup.
generated: 2024-09-28 22:12 UTC

This site is hosted by @0xB10C
More mirrored repositories can be found on mirror.b10c.me