FYI: Changelog checks fails because of:actions/setup-python#886 .

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.85%. Comparing base(3f1f7b8) to head(3f62568).
Report is 2 commits behind head on main.

@@           Coverage Diff           @@##             main    #9721   +/-   ##=======================================  Coverage   95.84%   95.85%           =======================================  Files         174      174             Lines       18865    18868    +3     =======================================+ Hits        18082    18085    +3  Misses        783      783

🤖 According to the primer, this change hasno effect on the checked open source code. 🤖🎉

This comment was generated for commit3f62568

I don't really understand why we would provide another method to manipulatePYTHONPATH when users have multiple ways to do so.

I understand that we want to make this easier for users, but as far as I understand it setting upPYTHONPATH should be done before you call into your Python executable. Theinit-hook is the earliest thing that gets executed so that is probably the best place to get this behaviour? What was your reason for not documenting the current way to do this but add another option?

By the way, some of the PRs you link are not really related to this. For example,#5644 is just an issue with the project structure and not really related to any of the issues you mention in this PR.

By the way, some of the PRs you link are not really related to this. For example,#5644 is just an issue with the project structure and not really related to any of the issues you mention in this PR.

Actually,#5644 provides a very prominent - because it's pylint itself! - example for the benefit of my suggested changes. As far as I can see, and please correct me, if I'm wrong, as soon as pylint's code is moved to thesrc folder (as suggested in#5644),pylint's own pylintrcwill have to do something about the import situation if you donot want to see import errors when editing files intests. Currently, this would be theinit-hook workaround as you mentioned.

I understand that we want to make this easier for users, but as far as I understand it setting upPYTHONPATH should be done before you call into your Python executable. Theinit-hook is the earliest thing that gets executed so that is probably the best place to get this behaviour? What was your reason for not documenting the current way to do this but add another option?

Yes,init-hook might be one of the first places to hook up into. However, I consider this workaround to be ratherhacky and ugly. Compare, e.g., solutions of other parties likepytest,mypy,isort orsetuptools which all provide a nice and clean way with a specific argument/configuration option to tackle thisstandard project layout. (Nowadays, that is; seelink from #5644 for older alternatives.)
That said, my reason to add another option is mainly to provide a nice, clean and similarly-looking option in pylint, too. It will get easier for (new) users, in fact, easy enough that you almost don't need to consult the documentation/help output and could easily guesstimate the required option in yourpyproject.toml.

I don't really understand why we would provide another method to manipulatePYTHONPATH when users have multiple ways to do so.

For pylint, I am only aware of--source-roots to manipulatePYTHONPATH but it serves a different purpose because it is related to implicit namespace packages and does not help with theinit-hook vs. suggested option.
You are of course right that youcould setPYTHONPATH externally, e.g.,PYTHONPATH=./src pylint. But I see two problems with that. Firstly, you will need toconvince your editor/IDE of that which is again error-prone, not out-of-the-box and you have to do that for every editor you use (moreover, might be even required on a per-project basis). Secondly, you might very well get into troublefinding and specifying the right location which depends on your current working directory, the file location, the project root and from where exactly it will be called.
Thus I don't consider the external option to be a viable approach.

Actually,#5644 provides a very prominent - because it's pylint itself! - example for the benefit of my suggested changes.

I'm not sure if this is correct. Wouldn't this be solved by doing apip install -e . before callingpylint? In that setup, the tests directory won't be able to import frompylint unless it is installed. Changing thePYTHONPATH to act as if it can seems incorrect.

Compare, e.g., solutions of other parties

I think themypy example is exactly what frightens me from adding this. The amount of configuration and gotcha's on different platforms seems like a headache. For example, why use globs and not regular expressions like we do for other configuration options.
What I personally like about theinit-hook is that we don't need to provide any "support". We can point users to common patterns/commands that might satisfy their needs, but there is no reason to look into our code to see whether we automatically expand home directories (to take the example frommypy). It's just Python code.

For pylint, I am only aware of--source-roots to manipulatePYTHONPATH but it serves a different purpose because it is related to implicit namespace packages and does not help with theinit-hook vs. suggested option. You are of course right that youcould setPYTHONPATH externally, e.g.,PYTHONPATH=./src pylint. But I see two problems with that. Firstly, you will need toconvince your editor/IDE of that which is again error-prone, not out-of-the-box and you have to do that for every editor you use (moreover, might be even required on a per-project basis). Secondly, you might very well get into troublefinding and specifying the right location which depends on your current working directory, the file location, the project root and from where exactly it will be called. Thus I don't consider the external option to be a viable approach.

I understand your point, and can see how it is far from optimal. On the other hand, the same argument as I provided above applies here. We can tap into and refer to existing documentation for settingPYTHONPATH for other tools and IDEs. Making our own custom solution increases the scope for bugs and edge cases we didn't consider.

I want to stress that I really do understand where you are coming from and see this as a nice feature for most users. However, from the perspective of a maintainers it seems to invite more edge cases and bugs to our tool without providing any features that the currently available features don't already provide. Perhaps I missed this so let me ask explicitly: is there anything that this solution allows you to do that you can't using the latest version ofpylint? If there is then that might convince me to be more open to this change!

#9967 is alternative way of solving this problem. Instead of falling back to legacy__init__.py based walking up the directories when source-roots are specified, it changes the fallback behavior for--source-roots to return the--source-roots list if the linting files don't overlap with any of the--source-roots. This will simplify the behavior of--source-roots and opens it up to more usecases beyond just implicit namespace packages.

This PR needs take over because because it has been open 8 weeks with no activity.