I am somewhat suspicious that it thinks that was my first PR in my fork. So opened a couple more and it commented on those too. I note that in the action's own repo the dependabot PRs are getting flagged for the new contributor messageactions/first-interaction#363 (comment)

Something seems off to me.

The problem that this PR was trying to fix is now tracked atactions/first-interaction#365. The problem of every PR being identified as a new contribution is now tracked atactions/first-interaction#369. So I think it's better to wait and see what happens over there.

Replacement for#30633, since if you force push when the PR is closed, you can't re-open.

Off-topic: You can reopen a closed pull request by first force-pushing the expected commit hash, reopening and then force-pushing the wanted commit hash again:https://gist.github.com/robertpainsi/2c42c15f1ce6dab03a0675348edd4e2c.

I have disabled the workflow (via Github Web UI) in the mean time until the problems are addressed (As it is not doing what we intend and is giving CI failures where they are not warranted)