Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Implement pyright support via dataclass_transforms#796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
hynek merged 19 commits intopython-attrs:mainfromasford:dataclass_transforms
May 5, 2021
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
19 commits
Select commitHold shift + click to select a range
52d769c
Add __dataclass_transform__ decorator.
asfordApr 27, 2021
1891373
Add doc notes for pyright dataclass_transform support.
asfordApr 27, 2021
4f3787c
Fix docs build error.
asfordApr 28, 2021
bba9d7a
Expand docs on dataclass_transform
asfordApr 29, 2021
70e4286
Add changelog
asfordApr 29, 2021
41b4149
Fix docs build
asfordApr 29, 2021
8abde8a
Fix lint
asfordApr 29, 2021
2a4f8df
Add note on __dataclass_transform__ in .pyi
asfordApr 29, 2021
822722b
Add baseline test of pyright support via tox
asfordApr 29, 2021
55a0675
Add pyright tests to tox run configuration
asfordApr 29, 2021
7c21153
Fix test errors, enable in tox.
asfordApr 29, 2021
e8a4c0d
Fixup lint
asfordApr 29, 2021
6d10e30
Move pyright to py39
asfordApr 30, 2021
5fa3b2d
Add test docstring.
asfordApr 30, 2021
59714cf
Fixup docs.
asfordApr 30, 2021
68e86bf
Merge branch 'main' into dataclass_transforms
hynekMay 1, 2021
693e71b
Merge branch 'main' into dataclass_transforms
hynekMay 3, 2021
e63087b
Merge branch 'main' into dataclass_transforms
hynekMay 5, 2021
7876c40
Merge branch 'main' into dataclass_transforms
hynekMay 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletionschangelog.d/796.change.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
``attrs`` has added provisional support for static typing in ``pyright`` version 1.1.135 via the `dataclass_transforms specification <https://github.com/microsoft/pyright/blob/1.1.135/specs/dataclass_transforms.md>`.
Both the ``pyright`` specification and ``attrs`` implementation may change in future versions of both projects.

Your constructive feedback is welcome in both `attrs#795 <https://github.com/python-attrs/attrs/issues/795>`_ and `pyright#1782 <https://github.com/microsoft/pyright/discussions/1782>`_.
37 changes: 36 additions & 1 deletiondocs/extending.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -46,7 +46,10 @@ An example for that is the package `environ-config <https://github.com/hynek/env

Another common use case is to overwrite ``attrs``'s defaults.

Unfortunately, this currently `confuses <https://github.com/python/mypy/issues/5406>`_ mypy's ``attrs`` plugin.
Mypy
************

Unfortunately, decorator wrapping currently `confuses <https://github.com/python/mypy/issues/5406>`_ mypy's ``attrs`` plugin.
At the moment, the best workaround is to hold your nose, write a fake mypy plugin, and mutate a bunch of global variables::

from mypy.plugin import Plugin
Expand DownExpand Up@@ -86,6 +89,34 @@ Then tell mypy about your plugin using your project's ``mypy.ini``:
Please note that it is currently *impossible* to let mypy know that you've changed defaults like *eq* or *order*.
You can only use this trick to tell mypy that a class is actually an ``attrs`` class.

Pyright
*************

Generic decorator wrapping is supported in ``pyright`` via the provisional dataclass_transform_ specification.

For a custom wrapping of the form::

def custom_define(f):
return attr.define(f)

This is implemented via a ``__dataclass_transform__`` type decorator in the custom extension's ``.pyi`` of the form::

def __dataclass_transform__(
*,
eq_default: bool = True,
order_default: bool = False,
kw_only_default: bool = False,
field_descriptors: Tuple[Union[type, Callable[..., Any]], ...] = (()),
) -> Callable[[_T], _T]: ...

@__dataclass_transform__(field_descriptors=(attr.attrib, attr.field))
def custom_define(f): ...

.. note::

``dataclass_transform`` is supported provisionally as of ``pyright`` 1.1.135.

Both the ``pyright`` dataclass_transform_ specification and ``attrs`` implementation may changed in future versions.

Types
-----
Expand DownExpand Up@@ -272,3 +303,7 @@ It has the signature
{'dt': '2020-05-04T13:37:00'}
>>> json.dumps(data)
'{"dt": "2020-05-04T13:37:00"}'

*****

.. _dataclass_transform: https://github.com/microsoft/pyright/blob/1.1.135/specs/dataclass_transforms.md
37 changes: 33 additions & 4 deletionsdocs/types.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,11 @@ Also, starting in Python 3.10 (:pep:`526`) **all** annotations will be string li
When this happens, ``attrs`` will simply put these string literals into the ``type`` attributes.
If you need to resolve these to real types, you can call `attr.resolve_types` which will update the attribute in place.

In practice though, types show their biggest usefulness in combination with tools like mypy_orpytype_ that both have dedicated support for ``attrs`` classes.
In practice though, types show their biggest usefulness in combination with tools like mypy_, pytype_orpyright_ that have dedicated support for ``attrs`` classes.

The addition of static types is certainly one of the most exciting features in the Python ecosystem and helps you writing *correct* and *verified self-documenting* code.

If you don't know where to start, Carl Meyer gave a great talk on `Type-checked Python in the Real World <https://www.youtube.com/watch?v=pMgmKJyWKn8>`_ at PyCon US 2018 that will help you to get started in no time.

mypy
----
Expand All@@ -69,12 +72,38 @@ To mypy, this code is equivalent to the one above:
a_number = attr.ib(default=42) # type: int
list_of_numbers = attr.ib(factory=list, type=typing.List[int])

*****

The addition of static types is certainly one of the most exciting features in the Python ecosystem and helps you writing *correct* and *verified self-documenting* code.
pyright
-------

If you don't know where to start, Carl Meyer gave a great talk on `Type-checked Python in the Real World <https://www.youtube.com/watch?v=pMgmKJyWKn8>`_ at PyCon US 2018 that will help you to get started in no time.
``attrs`` provides support for pyright_ though the dataclass_transform_ specification.
This provides static type inference for a subset of ``attrs`` equivalent to standard-library ``dataclasses``,
and requires explicit type annotations using the :ref:`next-gen` or ``@attr.s(auto_attribs=True)`` API.

Given the following definition, ``pyright`` will generate static type signatures for ``SomeClass`` attribute access, ``__init__``, ``__eq__``, and comparison methods::

@attr.define
class SomeClass(object):
a_number: int = 42
list_of_numbers: typing.List[int] = attr.field(factory=list)

.. note::

``dataclass_transform``-based types are supported provisionally as of ``pyright`` 1.1.135 and ``attrs`` 21.1.
Both the ``pyright`` dataclass_transform_ specification and ``attrs`` implementation may changed in future versions.

The ``pyright`` inferred types are a subset of those supported by ``mypy``, including:

- The generated ``__init__`` signature only includes the attribute type annotations,
and does not include attribute ``converter`` types.

- The ``attr.frozen`` decorator is not typed with frozen attributes, which are properly typed via ``attr.define(frozen=True)``.

Your constructive feedback is welcome in both `attrs#795 <https://github.com/python-attrs/attrs/issues/795>`_ and `pyright#1782 <https://github.com/microsoft/pyright/discussions/1782>`_.

*****

.. _mypy: http://mypy-lang.org
.. _pytype: https://google.github.io/pytype/
.. _pyright: https://github.com/microsoft/pyright
.. _dataclass_transform: https://github.com/microsoft/pyright/blob/1.1.135/specs/dataclass_transforms.md
19 changes: 19 additions & 0 deletionssrc/attr/__init__.pyi
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -86,6 +86,21 @@ else:
takes_self: bool = ...,
) -> _T: ...

# Static type inference support via __dataclass_transform__ implemented as per:
# https://github.com/microsoft/pyright/blob/1.1.135/specs/dataclass_transforms.md
# This annotation must be applied to all overloads of "define" and "attrs"
#
# NOTE: This is a typing construct and does not exist at runtime. Extensions
# wrapping attrs decorators should declare a separate __dataclass_transform__
# signature in the extension module using the specification linked above to
# provide pyright support.
def __dataclass_transform__(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

There should probably be an implementation of this somewhere right? Otherwise this won't be useful outside of attrs.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is a little strange, the decorator is just being used as a marker forpyright. The upstream spec specifies that the__dataclass_transform__ decorator is defined on a per-project basiseither in the a.py or.pyi source file, essentially as a compatibility layer until this is moved intotyping_extensions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hmm. I'm a little concerned (ok not too much but it definitely feels weird) about having adef in the pyi file that doesn't exist in the .py file. It means that someone could try to use this function in their own code but mypy wouldn't warn them that it doesn't really exist. It would fail at runtime. Are we sure we don't want to put an implementation in _funcs.py or something?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

🤔 I've done a little test of this. Under the 1.1.135pyright implementation if we:

  • Define a__dataclass_transform__ decorator in__init__.pyi file.
  • Define a__dataclass_transform__ method in__funcs.py and import in__init__.py.

Then a custom annotation of the form has types inferred properly.

importattr@attr.__dataclass_transform__(fields=(attr.field,attr.attrib))defcustom_define(cls):returnattr.define(cls)@custom_defineclassCustom:a:int

However, given that this is a very early and provisional specification and implementation inpyright, adding a "surprisingly functional" shim inattrs introduces a potentially risky coupling. We've a choice between:

  1. An immediate and loud runtime failure if you "try to use" the attrs-internal__dataclass_transform__.pyi definition, that won't be caught static type checkers.
  2. A surprisingly functional form that "works" (a no-op a runtime, currently works at typingtime), but relies on unspecified upstream behavior.

Given how bleeding-edge this feature is I have a weakly held preference that we opt for 2 for this release to ensure that folks don't implictly rely on theimport attr.__dataclass_transform__ behavior and instead wait until the spec bakes enough for thedataclass_transform to show up intyping_extensions.

Of course, happy to reconsider.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Since this is only for pyright and doesn't currently get mypy to do the right thing (i.e. it doesn't tell mypy that this is an attrs class creator) perhaps it's best if we leave this only inside the pyi file. I might add a comment on here that says that this method doesn't actually exist at runtime.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Roger, comment added.

*,
eq_default: bool = True,
order_default: bool = False,
kw_only_default: bool = False,
field_descriptors: Tuple[Union[type, Callable[..., Any]], ...] = (()),
) -> Callable[[_T], _T]: ...

class Attribute(Generic[_T]):
name: str
Expand DownExpand Up@@ -276,6 +291,7 @@ def field(
on_setattr: Optional[_OnSetAttrArgType] = ...,
) -> Any: ...
@overload
@__dataclass_transform__(order_default=True, field_descriptors=(attrib, field))
def attrs(
maybe_cls: _C,
these: Optional[Dict[str, Any]] = ...,
Expand All@@ -301,6 +317,7 @@ def attrs(
field_transformer: Optional[_FieldTransformer] = ...,
) -> _C: ...
@overload
@__dataclass_transform__(order_default=True, field_descriptors=(attrib, field))
def attrs(
maybe_cls: None = ...,
these: Optional[Dict[str, Any]] = ...,
Expand All@@ -326,6 +343,7 @@ def attrs(
field_transformer: Optional[_FieldTransformer] = ...,
) -> Callable[[_C], _C]: ...
@overload
@__dataclass_transform__(field_descriptors=(attrib, field))
def define(
maybe_cls: _C,
*,
Expand All@@ -349,6 +367,7 @@ def define(
field_transformer: Optional[_FieldTransformer] = ...,
) -> _C: ...
@overload
@__dataclass_transform__(field_descriptors=(attrib, field))
def define(
maybe_cls: None = ...,
*,
Expand Down
43 changes: 43 additions & 0 deletionstests/dataclass_transform_example.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
import attr


@attr.define()
class Define:
a: str
b: int


reveal_type(Define.__init__) # noqa


@attr.define()
class DefineConverter:
# mypy plugin adapts the "int" method signature, pyright does not
with_converter: int = attr.field(converter=int)


reveal_type(DefineConverter.__init__) # noqa


# mypy plugin supports attr.frozen, pyright does not
@attr.frozen()
class Frozen:
a: str


d = Frozen("a")
d.a = "new"

reveal_type(d.a) # noqa


# but pyright supports attr.define(frozen)
@attr.define(frozen=True)
class FrozenDefine:
a: str


d2 = FrozenDefine("a")
d2.a = "new"

reveal_type(d2.a) # noqa
69 changes: 69 additions & 0 deletionstests/test_pyright.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
import json
import os.path
import shutil
import subprocess
import sys

import pytest

import attr


if sys.version_info < (3, 6):
_found_pyright = False
else:
_found_pyright = shutil.which("pyright")


@attr.s(frozen=True)
class PyrightDiagnostic(object):
severity = attr.ib()
message = attr.ib()


@pytest.mark.skipif(not _found_pyright, reason="Requires pyright.")
def test_pyright_baseline():
"""The __dataclass_transform__ decorator allows pyright to determine
attrs decorated class types.
"""

test_file = os.path.dirname(__file__) + "/dataclass_transform_example.py"

pyright = subprocess.run(
["pyright", "--outputjson", str(test_file)], capture_output=True
)
pyright_result = json.loads(pyright.stdout)

diagnostics = set(
PyrightDiagnostic(d["severity"], d["message"])
for d in pyright_result["generalDiagnostics"]
)

# Expected diagnostics as per pyright 1.1.135
expected_diagnostics = {
PyrightDiagnostic(
severity="information",
message='Type of "Define.__init__" is'
' "(self: Define, a: str, b: int) -> None"',
),
PyrightDiagnostic(
severity="information",
message='Type of "DefineConverter.__init__" is '
'"(self: DefineConverter, with_converter: int) -> None"',
),
PyrightDiagnostic(
severity="information",
message='Type of "d.a" is "Literal[\'new\']"',
),
PyrightDiagnostic(
severity="error",
message='Cannot assign member "a" for type '
'"FrozenDefine"\n\xa0\xa0"FrozenDefine" is frozen',
),
PyrightDiagnostic(
severity="information",
message='Type of "d2.a" is "Literal[\'new\']"',
),
}

assert diagnostics == expected_diagnostics
18 changes: 16 additions & 2 deletionstox.ini
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -15,14 +15,14 @@ python =
3.6: py36
3.7: py37, docs
3.8: py38, lint, manifest, typing, changelog
3.9: py39
3.9: py39, pyright
3.10: py310
pypy2: pypy2
pypy3: pypy3


[tox]
envlist = typing,lint,py27,py35,py36,py37,py38,py39,py310,pypy,pypy3,manifest,docs,pypi-description,changelog,coverage-report
envlist = typing,lint,py27,py35,py36,py37,py38,py39,py310,pypy,pypy3,pyright,manifest,docs,pypi-description,changelog,coverage-report
isolated_build = True


Expand DownExpand Up@@ -121,3 +121,17 @@ deps = mypy>=0.800
commands =
mypy src/attr/__init__.pyi src/attr/_version_info.pyi src/attr/converters.pyi src/attr/exceptions.pyi src/attr/filters.pyi src/attr/setters.pyi src/attr/validators.pyi
mypy tests/typing_example.py


[testenv:pyright]
# Install and configure node and pyright
# This *could* be folded into a custom install_command
# Use nodeenv to configure node in the running tox virtual environment
# Seeing errors using "nodeenv -p"
# Use npm install -g to install "globally" into the virtual environment
basepython = python3.9
deps = nodeenv
commands =
nodeenv --prebuilt --node=lts --force {envdir}
npm install -g --no-package-lock --no-save pyright@1.1.135
pytest tests/test_pyright.py -vv

[8]ページ先頭

©2009-2025 Movatter.jp