Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

generative-ai: Add anti pattern example#1679

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
corona10 wants to merge5 commits intopython:main
base:main
Choose a base branch
Loading
fromcorona10:ai-anti

Conversation

@corona10
Copy link
Member

@corona10corona10 commentedNov 1, 2025
edited by github-actionsbot
Loading

=============
- While AI-assisted tools such as autocompletion can enhance productivity, they sometimes rewrite entire code blocks instead of making small, focused edits.
This can make it more difficult to review changes and to fully understand both the original intent of the code and the rationale behind the new modifications.
Please keep maintaining consistency with the original code helps preserve clarity, traceability, and meaningful review.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
Please keep maintaining consistency with the original code helps preserve clarity, traceability, and meaningful review.
Maintaining consistency with the original code helps preserve clarity, traceability, and meaningful review.

We also might want to mention (and link to?) the idea that we want to avoid churn.

corona10 reacted with thumbs up emoji
those that are fully generated by AI. If a contributor repeatedly opens unproductive
issues or PRs, they may be blocked.

Anti-patterns
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Do we have a second anti-pattern to include in the list? Other bad things: hallucinations, incorrectly changing tests instead of fixing code, ...

corona10 reacted with thumbs up emoji
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

i think you covered the incorrectly changing tests instead of fixing one. Good call. I like having that in the list.

I don't personally see that so often anymore with Opus 4.1 or Sonnet 4.5, but older or lower end models tend to reward hack their way into that state more often - it's a thing we'll likely encounter coming from new contributors for a while as they can't be assumed to be using the latest and greatest.

This can make it more difficult to review changes and to fully understand both the original intent of the code and the rationale behind the new modifications.
Maintaining consistency with the original code helps preserve clarity, traceability, and meaningful reviews and also helps us avoid unnecessary code churn.
- Sometimes AI assisted tools make failing unit tests pass by altering or bypassing the tests rather than addressing the underlying problem in the code.
Such changes do not represent a real fix and should be avoided.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't understand what should be avoided:

  • avoid using AI-assisted tools?
  • avoid rewriting code blocks?
  • avoid using Ai-assisted tools that rewrite code blocks?
    What is the recommended approach instead if we are telling people not to do a certain thing?

I would suggest starting the paragraph with something like:
"We discourage the use of AI when ..... "

Then followed with the explanation of why.
Currently this section reads with the "why" first.

corona10 reacted with thumbs up emoji
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this sentence was only intended to be tied to the false testing / false success bullet it is part of. Rather than wording it as "should be avoided" the right thing for users to do is to request that authors have reviewed the work in detail for problems like this before proposing it as a PR.

"Such changes do not represent a real fix. Authors must review the work done by AI tooling in detail to ensure it actually makes sense before proposing it as a PR."


I could maybe suggest adding an AI use tip for this situation like"iterate with your AI agent on the quality and utility of the change and tests to understand if the issue is actually fixed" but I'd leave that out for now as topic is maybe larger and constantly evolving, beyond the scope of this PR.

I'm trying to focus on a proactive positive outcome based wording rather than a negative like "avoid" or "discourage".

The reason being, the statements in here don't reflect all AI tooling. This devguide doc is about the use of AI, so we should assume those who read it are interested in or already using AI and are here to learn how to do it in an appropriate and productive manner.

Statements like "we discourage use of AI when ..." are likely to be outdated very quickly and discourage things that won't be true for long. Thus my desire to focus this on what we do want and why it is important for project health and reviewer respect.


Anti-patterns
=============
- While AI-assisted tools such as autocompletion can enhance productivity, they sometimes rewrite entire code blocks instead of making small, focused edits.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'd actually tighten this phrasing up a bit, we don't need to explain what shape AI tools can take or what they can do (anyone reading this knows that).

I find this list of anti-patterns to not be AI specific. We already reject PRs coming from people who make larger than necessary changes that are difficult to review. So I'd suggest wording it as a "positive desired thing, not negative example" phrasing, starting something like this:

  • Make small focused edits, prefer not rewriting entire code blocks.
    This makes it easier to review changes and fully understand both the original intent of the code and the rationale behind the new modifications.
    ...

corona10 and jkaufeld-liine reacted with thumbs up emoji
This can make it more difficult to review changes and to fully understand both the original intent of the code and the rationale behind the new modifications.
Maintaining consistency with the original code helps preserve clarity, traceability, and meaningful reviews and also helps us avoid unnecessary code churn.
- Sometimes AI assisted tools make failing unit tests pass by altering or bypassing the tests rather than addressing the underlying problem in the code.
Such changes do not represent a real fix and should be avoided.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this sentence was only intended to be tied to the false testing / false success bullet it is part of. Rather than wording it as "should be avoided" the right thing for users to do is to request that authors have reviewed the work in detail for problems like this before proposing it as a PR.

"Such changes do not represent a real fix. Authors must review the work done by AI tooling in detail to ensure it actually makes sense before proposing it as a PR."


I could maybe suggest adding an AI use tip for this situation like"iterate with your AI agent on the quality and utility of the change and tests to understand if the issue is actually fixed" but I'd leave that out for now as topic is maybe larger and constantly evolving, beyond the scope of this PR.

I'm trying to focus on a proactive positive outcome based wording rather than a negative like "avoid" or "discourage".

The reason being, the statements in here don't reflect all AI tooling. This devguide doc is about the use of AI, so we should assume those who read it are interested in or already using AI and are here to learn how to do it in an appropriate and productive manner.

Statements like "we discourage use of AI when ..." are likely to be outdated very quickly and discourage things that won't be true for long. Thus my desire to focus this on what we do want and why it is important for project health and reviewer respect.

those that are fully generated by AI. If a contributor repeatedly opens unproductive
issues or PRs, they may be blocked.

Anti-patterns
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

i think you covered the incorrectly changing tests instead of fixing one. Good call. I like having that in the list.

I don't personally see that so often anymore with Opus 4.1 or Sonnet 4.5, but older or lower end models tend to reward hack their way into that state more often - it's a thing we'll likely encounter coming from new contributors for a while as they can't be assumed to be using the latest and greatest.

Copy link
Collaborator

@willingcwillingc left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

those that are fully generated by AI. If a contributor repeatedly opens unproductive
issues or PRs, they may be blocked.

Anti-patterns
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I would use a different title than antipatterns. I would like to move toward "Considerations or Keys for Success".

I would start by linking to the section about submitting a quality PR:

  • consider whether the change is necessary
  • small, focused changes
  • tests that exercise the change
  • style follows the existing code

AI coding tools will compose code. The contributor, not the AI tool, is responsible for the quality and validity of code submitted with a PR.

Take the time to consider the code changes suggested by an AI tool. Respect the reviewer's time by validating your PR before submission.

gpshead and corona10 reacted with thumbs up emojigpshead and corona10 reacted with heart emoji
@gpshead
Copy link
Member

It would be interesting to add a link to:https://arxiv.org/pdf/2507.09089

That "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity" paper was unfortunately out of date by the time it was published - which I don't expect all readers recognize. It requires understanding that the models used were already multiple generations behind and obsolete. If we're going to link to that, we'd need to explain that caveat up front. But it's also a developer productivity focused paper and doesn't really align with the topic of this devguide section. People should use what tooling is productive for them, but when it comes to OSS dev the thing I believe we all want is for our own time as recipients of people's reports/issues/PRs/etc to be well spent and respected.

I like your idea of a "section about submitting a quality PR" for that reason. That is more what I believe we actually care about.

and possiblyhttps://www.fast.ai/posts/2025-10-30-build-to-last.html

reality check: is my conscious bias against linking to things mentioning Mojo from official Python channels worthwhile here?
(bias: they have, to my knowledge - how true is this? - not been participating in the Python community while claiming to be connected to it?)

+1 Regardless of biases, I like its broader theme from Chris of "understand the system that must be built maintained long term" concept. Key for CPython and any big OSS maintenance.

willingc reacted with thumbs up emoji

@willingc
Copy link
Collaborator

@gpshead I think we can skip the links. I wrote the message before the one where I suggested putting AI use in a positive light where the contributor is responsible for the validity of the PR. I think there are two goals:

  • guiding contributor to success
  • reduce slop PRs
gpshead and corona10 reacted with thumbs up emoji

@nedbat
Copy link
Member

I like the idea of staying away from any opinion about AI on either side of the debate, and stay focused on the contributions.

To that end, I would change some of the opening text:

Generative AI has evolved rapidly over the past decade and will continue in the future.
Using generative AI and large language models (LLMs) can be helpful tools for contributors. Their overuse can also be problematic, such as generation of incorrect code, inaccurate documentation, and unneeded code churn.
Discretion, good judgement, and critical thinkingmust be used when opening issues and pull requests.

to:

Generative AI has evolved rapidly and can be impressive. As with using any tool, the resulting contribution is the responsibility of the contributor. We value good code, concise accurate documentation, and avoiding unneeded code churn. Discretion, good judgment, and critical thinking are the foundation of all good contributions, regardless of the tools used in their creation.

willingc, gpshead, and efimov-mikhail reacted with heart emoji

@willingc
Copy link
Collaborator

I love the suggested opening@nedbat.

gpshead and corona10 reacted with thumbs up emoji

@corona10
Copy link
MemberAuthor

corona10 commentedNov 5, 2025
edited
Loading

A lot of opinions!, I will address reviews by this weekend!

efimov-mikhail reacted with thumbs up emoji

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@nedbatnedbatnedbat left review comments

@gpsheadgpsheadgpshead left review comments

@willingcwillingcwillingc left review comments

@MariattaMariattaMariatta left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@corona10@gpshead@willingc@nedbat@Mariatta

[8]ページ先頭

©2009-2025 Movatter.jp