Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

allow string imports outside of packages#736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
carloshorn wants to merge1 commit intoets-labs:master
base:master
Choose a base branch
Loading
fromcarloshorn:fix-string-import

Conversation

carloshorn
Copy link

Problem description

Currently, string imports are not possible outside of packages.

Minimal reproducible example

The following code snippet executed from a script or Jupyter notebook cell raises anAttributeError: 'NoneType' object has no attribute '__package__'

fromdependency_injectorimportprovidersdeque_factory=providers.Factory('collections.deque')

The trace back reveals that the exception is thrown insrc/dependency_injector/providers.pyx in dependency_injector.providers._resolve_calling_package_name().

Proposal for solution

In case the module independency_injector.providers._resolve_calling_package_name isNone, the package should also beNone. The subsequent call toimportlib.import_module will handle the situation properly.

@nightblure
Copy link

IMO this will not be a popular use case, using strings instead of objects is a rather exotic and rare practice..

@@ -5034,7 +5034,7 @@ def _resolve_calling_module():

def _resolve_calling_package_name():
module = _resolve_calling_module()
return module.__package__
returngetattr(module, "__package__", None)
Copy link

@nightblurenightblureNov 13, 2024
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It seems like it would be a good idea to write tests or add test cases for clients using this_resolve_calling_package_name function

nightblure

This comment was marked as duplicate.

allow string imports outside of packages
@carloshorn
Copy link
Author

IMO this will not be a popular use case, using strings instead of objects is a rather exotic and rare practice..

At that time, I was writing an image processing application, where the user could provide a list of filters, e.g. something like a Gaussian filter with given sigma and window size, within a config file. However, I got the feature request if I could also allow filters from other libraries, e.g. scipy, skimage and openCV... or a custom filter...
So I thought, let the user select whatever they want by providing the import string and the configuration inside the config file. Then I saw that the factory provider can handle string imports,https://python-dependency-injector.ets-labs.org/providers/factory.html#string-imports. So I thought "No problem!"... However, my first attempt resulted in the above error.

Generally, I agree that this is an exotic and rare practice. Looking at it today reviewing my own PR, I would even argue that providing a string import to another library is unsafe, so it would require either a safe alternative like thesafe_load in yaml, or an explicitly unsafe factory provider that allows this...

IMHO, you may abandon the string imports from the factory provider and add an explicit string import provider or delegate the import to the users providing some examples in the documentation. This may shrink the interface to a single way of doing things. However, I can just imagine the difficulties of maintaining such a popular library and dealing with breaking changes. BTW, thanks for providing and maintaining this library!

I could implement the string import provider that allows unsafe imports if you agree.

What do you think?

nightblure reacted with thumbs up emoji

@nightblure
Copy link

IMO this will not be a popular use case, using strings instead of objects is a rather exotic and rare practice..

At that time, I was writing an image processing application, where the user could provide a list of filters, e.g. something like a Gaussian filter with given sigma and window size, within a config file. However, I got the feature request if I could also allow filters from other libraries, e.g. scipy, skimage and openCV... or a custom filter... So I thought, let the user select whatever they want by providing the import string and the configuration inside the config file. Then I saw that the factory provider can handle string imports,https://python-dependency-injector.ets-labs.org/providers/factory.html#string-imports. So I thought "No problem!"... However, my first attempt resulted in the above error.

Generally, I agree that this is an exotic and rare practice. Looking at it today reviewing my own PR, I would even argue that providing a string import to another library is unsafe, so it would require either a safe alternative like thesafe_load in yaml, or an explicitly unsafe factory provider that allows this...

IMHO, you may abandon the string imports from the factory provider and add an explicit string import provider or delegate the import to the users providing some examples in the documentation. This may shrink the interface to a single way of doing things. However, I can just imagine the difficulties of maintaining such a popular library and dealing with breaking changes. BTW, thanks for providing and maintaining this library!

I could implement the string import provider that allows unsafe imports if you agree.

What do you think?

I'm just a user of this package, but I made my own analogue of the injector and understand how much it works inside. User experience is important, and I think that the provider should also be specified explicitly, and not as a string. And explicit is better than implicit!

but if we refer to specific arguments, I have two arguments in favor of not working with strings:

  1. less maintainable code (the best code is the one that is not written, especially in this specific case);
  2. imagine that you want to use more than one DI container. And in this case you will not get by with a string, because you will need to specify a specific container and its provider:
classContainer1(DeclarativeContainer): ...classContainer2(DeclarativeContainer): ...@injectdefdo_smth(dep=Provide["provider"]): ...# string definitely won’t help here, because there is more than one container

@carloshorn
Copy link
Author

[...] I think that the provider should also be specified explicitly, and not as a string. And [...]

but if we refer to specific arguments, I have two arguments in favor of not working with strings:

  1. less maintainable code (the best code is the one that is not written, especially in this specific case);
  2. imagine that you want to use more than one DI container. And in this case you will not get by with a string, because you will need to specify a specific container and its provider:
classContainer1(DeclarativeContainer): ...classContainer2(DeclarativeContainer): ...@injectdefdo_smth(dep=Provide["provider"]): ...# string definitely won’t help here, because there is more than one container

Even though they are related, I think we are talking about different string imports. While I am talking about the first argument ofproviders.Factory, seehttps://python-dependency-injector.ets-labs.org/providers/factory.html#string-imports, you are talking about string identifiers during wiring, seehttps://python-dependency-injector.ets-labs.org/wiring.html#string-identifiers.

Please note that I am not generally against string imports, see my use-case above.
One motivation for the string identifiers feature is given in the documentation: "You could also use string identifiers to avoid a dependency on a container:". However, this is out of scope regarding this PR.
The motivation for the string imports in factory providers is not explicitly given, but I can imagine the case where you want to avoid expensive imports that are not used in every instance of your application, e.g. various supported back-ends which live in different sub-modules where you select only one.

An addition for the feature of this PR would be anunsafe attribute for theproviders.Factory which would allow the imports from modules outside of the package. The default would beFalse and it should be possible to set it per instance. I will think further about it and update this PR.

@carloshorn
Copy link
Author

I just realized, that the above error message is specific to working in an interactive mode, e.g. Jupyter notebooks.

Copy link
Contributor

@ZipFileZipFile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The change is indeed interesting, but as@nightblure mentioned, tests for this specific edge case are needed. Something as simple aseval("_resolve_calling_package_name()") with bunch of asserts should suffice, since eval/exec do not produce any module info.

Note:_resolve_calling_module and_resolve_calling_package_name works as intended here because this is a Cython code, hence fetching top element from the stack returns thecaller python frame. These functions won't work if implemented in pure python. We have similar functions in containers module, they should probably be un-cpdef-ed.

@@ -5034,7 +5034,7 @@ def _resolve_calling_module():

def _resolve_calling_package_name():
module = _resolve_calling_module()
return module.__package__
returngetattr(module, "__package__", None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

To be inline withdocumented behavior, lets check thatmodule isNone here, instead of relying on behavior ofgetattr.

I.e.:

Suggested change
returngetattr(module,"__package__",None)
ifmoduleisNone:
returnNone
return module.__package__

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ZipFileZipFileZipFile requested changes

@nightblurenightblurenightblure requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@carloshorn@nightblure@ZipFile

[8]ページ先頭

©2009-2025 Movatter.jp