Comment on stale issues #109

Nyholm · 2020-11-07T13:12:45Z

This PR adds a PingStaleIssuesCommand. It will look for old inactive issues and start a process with them.

Bot will make a comment to encourage activity and add label "Staled".
Bot will make a comment to inform the issue will be closed
Bot will close the issue.
The process can be interrupted with anyone making a comment on the issue or the "Keep open" label is added.

The exact times between the steps described above is defined as constants in PingStaleIssuesCommand.

Nyholm · 2020-11-07T13:29:54Z

This cannot run on a github action like #103 because it needs to create a scheduled task.

wouterj · 2020-11-07T13:53:30Z

I usually don't like auto-closing staled issues: Contrary to staled PRs, there is nothing I as issue owner can do to "unstall" this issue. E.g. I don't know every open source library enough to be able to fix issues I find, so I take the time to open an issue instead. If it's not considered important enough to fix, a maintainer should close it (with a friendly message). If it wasn't closed, it means there is some relevancy of the issue (it just doesn't have enough priority). If it's closed after a year, the issue is there still, it's just hidden.

E.g. some real case: symfony/symfony-docs#4258 This feature is indeed undocumented and useful enough to document. We just haven't yet found the correct example or place in the documentation to fix this issue. That doesn't, imho, mean we should close it.

Another real case I discovered yesterday is for instance owasp-modsecurity/ModSecurity-nginx#198 (comment) , where a bot autoclosed this issue 2 times already. The PR is just blocked by another PR, there is no reason to close it and "hide" this missing feature.

Nyholm · 2020-11-07T14:07:24Z

It is indeed tricky. But if an issue has not gotten any comments or activity in over a year, it is not likely it will get fixed soon anyways. You can just make a comment to say whatever and the issue will be open for another [time period].

I agree with your use case. You as a maintainer can set "Keep open" label on this issue.

Another real case I discovered yesterday

I agree with this too. That is why we don't auto close PRs.

Currently I added a time period for 3 months. Which means, that if there are no activity for 3 months, then we add this comment. We could change this time period to any other value if we think it is too long or too short.

ogizanagi · 2020-11-07T14:20:46Z

I wonder if we really have any benefits re-developing this here, in Carsonbot?
We mentioned https://github.com/actions/stale some weeks ago about this. Wouldn't it fit?

Nyholm · 2020-11-07T14:24:17Z

Yeah, this is the same as stalebot. But the logic to "redevelop" is way less than 50 lines of code. And we already have a bot =)

Nyholm · 2020-11-08T17:35:59Z

This PR is now "done" and ready for review

jderusse · 2020-11-14T16:24:16Z

I personally don't like auto-closed PR, In my experience, when working with k8s repo, the issues are full of comments "issue stale", "re-open"... It's a lot of noise and VERY frustrating for the user that open the PR and for the user that had the exact same issue.

Instead of adding a label to by-pass the bot and keeping the PR open, what's about closing the PR when we don't want to fix it?.

wouterj · 2020-11-14T16:53:14Z

Regarding @jderusse's description: what about changing it from opt-out to opt-in? E.g. have a label that effectively says "hmm, not sure about this one", which will result in auto close within 2 weeks if there is no response.

In the docs, I'm using the Waiting feedback for this process manually already.

jderusse · 2020-11-14T16:55:49Z

yeah, a waiting feedback on "not reproductible" issues would make more sense to me

Nyholm · 2020-11-14T18:14:51Z

Thank you for the reviews. Note that this is not about PRs. This is about closing issues only.

Here is an example timeline:

Day 1: Issue open.
Day 366: There is no acitivty, Bot makes a comment
Day 370: User makes a comment
Day 736: There is no activity, Bot makes another comment
Day 750: Bot closes the issue

I don't think there is a lot of noise... I also don't see no point in having an issue open that there is no activity around... Sure, there are exceptions, that is why we have the "Keep open" label.

Nyholm · 2020-11-14T20:01:37Z

After some discussions with @wouterj, we agree that it is important to consider what we say to the authors. We know they spent time creating this issue.

After x days of delay: Message 1, should encourage an answer. Maybe the users will close the issue themselves?
After y days of delay: Message 2, comment and say that we are closing the issue.

Should we make this even more fancy?

After x days of delay: Message 1,
After y days of delay: Message 2, comment and say that we are about to close the issue.
After z days of delay: Close the issue.

I need some help improving the content in StaleIssueCommentGenerator.

wouterj

After the discussion with @Nyholm my doubts have somewhat been taken away. The goal here is not to close as many issues as possible, the goal here is to close issues that aren't relevant anymore (e.g. they have been resolved or were no longer needed).

I think the current wording better conveys this message. 👍 I would suggest to add something like "There hasn't been much activity here recently" or the like, to indicate the "trigger" for this message.

src/Api/Issue/GithubIssueApi.php

src/Service/StaleIssueCommentGenerator.php

Nyholm · 2020-11-15T08:41:49Z

Thank you. I've updated the PR accordingly.

Nyholm · 2020-11-18T18:44:49Z

src/Api/Issue/GithubIssueApi.php

@@ -44,6 +44,14 @@ public function open(Repository $repository, string $title, string $body, array
        }
    }

+    public function lastCommentWasMadeByBot(Repository $repository, $number): bool
+    {
+        $allComments = $this->issueCommentApi->all($repository->getVendor(), $repository->getName(), $number);


Since this is not using pagination. We will only fetch 100 comments. That means that we will never auto close issues with more that 100 comments.

Im fine with that atm.

wouterj · 2020-11-21T11:39:32Z

Something I just realized: 300 issues in the code repository and +/- 90 issues in the docs repository are currently within this filter. We should probably organize a small core team sprint day to check these issues before the first deploy closes them.

The advantage of this bot is that it tries to engage the issue creators in determining whether the issue needs to stay open. We would loose that advantage for the biggest set of issues (the first set), if we have to check them all by hand. Maybe we should make the first close window longer (e.g. 4 weeks), in order to give the user the normal 2 weeks to respond and then have 2 weeks for the core team to review the issues were nobody responded? What do you think? After this first round, we can revert the window back to 2 weeks.

Nyholm · 2020-11-21T13:20:52Z

I've added a second message.

So, Day 1: Make a comment to encourage and action by the user
Day: 15: Do a ping and inform that we will close the issue if we dont hear anything
Day 30: Close the issue.

nicolas-grekas · 2020-12-09T14:04:18Z

What about making this a bit more "manual", eg when an issue has the status "waiting feedback", then we autoclose on 1 month? Applying the label should be manual. I think all issues deserve a comment from someone in the community, no need for the bot to say hello.

Nyholm · 2020-12-09T14:07:14Z

I agree that we could update this in the future. We can also add more rules etc.
But that is not a blocker from merging this now. The current implementation is the most relaxed and generous rules possible.

I think all issues deserve a comment from someone in the community, no need for the bot to say hello.

If nobody have made a comment in 1 year time, then I think it is fair to say that nobody ever will make a comment.

nicolas-grekas · 2020-12-09T14:13:27Z

If nobody have made a comment in 1 year time, then I think it is fair to say that nobody ever will make a comment.

Does that happen in practice? Any example?

Nyholm · 2020-12-09T14:16:28Z

Actually yes. Sure it is sad that an issue has been open for long..

https://github.com/symfony/symfony/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-asc

15014
14068
19286

wouterj · 2020-12-09T14:59:09Z

More precisely, there are 57 issues without any comment that are open for 1+ year: https://github.com/symfony/symfony/issues?q=is%3Aissue+is%3Aopen+comments%3A0+updated%3A%3C2019-12-09

Nyholm · 2020-12-09T15:03:48Z

I think all issues deserve a comment from someone in the community, no need for the bot to say hello.

So currently there are 50+ issues that the community has "failed". There is another 250 issues with no activity in the past year.

These 300 issues will get a "bot comment" to encourage action when this PR is merged.

Nyholm · 2020-12-16T08:22:10Z

I added support for "Stale" label. So now one can easily find all issues that the bot is working with.

FYI @nicolas-grekas

weaverryan

Hi! Only a minor comment... mostly to help me deploy more safely. I'd like to deploy this some time during European time, while people are online. And after trying the --dry-run once, probably running the stale-issue command once manually so that the "big dump" of comments can happen in sync, so we can watch it (just in cases).

Thanks!

src/Service/StaleIssueCommentGenerator.php

weaverryan · 2020-12-18T00:27:38Z

src/Command/PingStaleIssuesCommand.php

+
+    protected function configure()
+    {
+        $this->addArgument('repository', InputArgument::REQUIRED, 'The full name to the repository, eg symfony/symfony-docs');


Sorry, but could we add a --dry-run option? And also write output in the command? For example:

Adding label and comment to issue #555 on symfony/symfony

And, of course, the --dry-run would make no changes. It would make me a bit more comfortable, as we could deploy, then run the command manually with --dry-run and make sure there are no surprises (like WAY too many issues being commented on, due to some bug).

Thanks!

The ´--dry-run´ option is added.

It will show you 100 issues on Symfony/symfony on the first run. We do not use pagination because it isn't really needed.

Here is a link to the search: https://github.com/symfony/symfony/issues?q=is%3Aissue+-linked%3Apr+-label%3A%22Keep+open%22+is%3Aopen+updated%3A%3C2019-12-18

weaverryan · 2020-12-18T00:29:12Z

Oh, also, could you update the PR title and description to reflect the current way it works?

nicolas-grekas · 2020-12-18T09:44:55Z

stale-issue command once manually so that the "big dump" of comments can happen in sync, so we can watch it (just in cases).

should we limit the number of notices the bot is going to generate at once?
eg 10 per batch max, one batch per day? (since I suppose this is going to run as a periodic task?)

Nyholm · 2020-12-18T09:45:58Z

Currently, there is a limit on 100 per batch. We run the script once per day.

Nyholm · 2020-12-18T14:09:45Z

Wohoo. Thank you for merging

Nyholm force-pushed the stale-issues branch 2 times, most recently from 8ad7054 to 597eca5 Compare November 8, 2020 17:12

Nyholm changed the title ~~Blocked: Close stale issues~~ Comment on stale issues and close them 2 weeks later Nov 8, 2020

Nyholm force-pushed the stale-issues branch 2 times, most recently from 4692965 to 7d25039 Compare November 8, 2020 17:35

Nyholm force-pushed the stale-issues branch from 7d25039 to cae3791 Compare November 14, 2020 10:34

wouterj reviewed Nov 14, 2020

View reviewed changes

src/Api/Issue/GithubIssueApi.php Outdated Show resolved Hide resolved

src/Service/StaleIssueCommentGenerator.php Outdated Show resolved Hide resolved

src/Service/StaleIssueCommentGenerator.php Outdated Show resolved Hide resolved

wouterj approved these changes Nov 16, 2020

View reviewed changes

Nyholm force-pushed the stale-issues branch from 94b79cc to 421e431 Compare November 18, 2020 07:58

Nyholm mentioned this pull request Nov 18, 2020

Adding return type hints #136

Merged

Nyholm force-pushed the stale-issues branch from 421e431 to 604aaad Compare November 18, 2020 18:39

Nyholm commented Nov 18, 2020

View reviewed changes

Nyholm force-pushed the stale-issues branch from 604aaad to 9eaeaf4 Compare November 21, 2020 13:04

nicolas-grekas approved these changes Dec 16, 2020

View reviewed changes

weaverryan reviewed Dec 18, 2020

View reviewed changes

Nyholm changed the title ~~Comment on stale issues and close them 2 weeks later~~ Comment on stale issues Dec 18, 2020

Comment on stale issues

9de3045

weaverryan force-pushed the stale-issues branch from c717a60 to 9de3045 Compare December 18, 2020 14:08

weaverryan merged commit 1240694 into symfony-tools:master Dec 18, 2020

Nyholm deleted the stale-issues branch December 18, 2020 14:09

Nyholm mentioned this pull request Dec 27, 2020

Information on issue message and call to action #147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comment on stale issues #109

Comment on stale issues #109

Nyholm commented Nov 7, 2020 •

edited

Loading

Nyholm commented Nov 7, 2020

wouterj commented Nov 7, 2020

Nyholm commented Nov 7, 2020

ogizanagi commented Nov 7, 2020

Nyholm commented Nov 7, 2020

Nyholm commented Nov 8, 2020

jderusse commented Nov 14, 2020 •

edited

Loading

wouterj commented Nov 14, 2020

jderusse commented Nov 14, 2020

Nyholm commented Nov 14, 2020

Nyholm commented Nov 14, 2020

wouterj left a comment •

edited

Loading

Nyholm commented Nov 15, 2020

Nyholm Nov 18, 2020

wouterj commented Nov 21, 2020

Nyholm commented Nov 21, 2020

nicolas-grekas commented Dec 9, 2020

Nyholm commented Dec 9, 2020

nicolas-grekas commented Dec 9, 2020

Nyholm commented Dec 9, 2020

wouterj commented Dec 9, 2020

Nyholm commented Dec 9, 2020

Nyholm commented Dec 16, 2020

weaverryan left a comment

weaverryan Dec 18, 2020

Nyholm Dec 18, 2020

weaverryan commented Dec 18, 2020

nicolas-grekas commented Dec 18, 2020

Nyholm commented Dec 18, 2020

Nyholm commented Dec 18, 2020

Comment on stale issues #109

Comment on stale issues #109

Conversation

Nyholm commented Nov 7, 2020 • edited Loading

Nyholm commented Nov 7, 2020

wouterj commented Nov 7, 2020

Nyholm commented Nov 7, 2020

ogizanagi commented Nov 7, 2020

Nyholm commented Nov 7, 2020

Nyholm commented Nov 8, 2020

jderusse commented Nov 14, 2020 • edited Loading

wouterj commented Nov 14, 2020

jderusse commented Nov 14, 2020

Nyholm commented Nov 14, 2020

Nyholm commented Nov 14, 2020

wouterj left a comment • edited Loading

Choose a reason for hiding this comment

Nyholm commented Nov 15, 2020

Nyholm Nov 18, 2020

Choose a reason for hiding this comment

wouterj commented Nov 21, 2020

Nyholm commented Nov 21, 2020

nicolas-grekas commented Dec 9, 2020

Nyholm commented Dec 9, 2020

nicolas-grekas commented Dec 9, 2020

Nyholm commented Dec 9, 2020

wouterj commented Dec 9, 2020

Nyholm commented Dec 9, 2020

Nyholm commented Dec 16, 2020

weaverryan left a comment

Choose a reason for hiding this comment

weaverryan Dec 18, 2020

Choose a reason for hiding this comment

Nyholm Dec 18, 2020

Choose a reason for hiding this comment

weaverryan commented Dec 18, 2020

nicolas-grekas commented Dec 18, 2020

Nyholm commented Dec 18, 2020

Nyholm commented Dec 18, 2020

Nyholm commented Nov 7, 2020 •

edited

Loading

jderusse commented Nov 14, 2020 •

edited

Loading

wouterj left a comment •

edited

Loading