Sitelet https://web.archive.org/web/20210513061041/https://github.com/python/mypy/issues/5374
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

abstract dataclass inheritance gives Only concrete class can be given where "Type[Abstract]" is expected #5374

Open
ojii opened this issue Jul 19, 2018 · 22 comments

Comments

@ojii
Copy link

@ojii ojii commented Jul 19, 2018

mypy 0.620, --strict.

Sample code:

import abc
from dataclasses import dataclass


@dataclass  # error: Only concrete class can be given where "Type[Abstract]" is expected
class Abstract(metaclass=abc.ABCMeta):
    a: str

    @abc.abstractmethod
    def c(self) -> None:
        pass


@dataclass
class Concrete(Abstract):
    b: str

    def c(self) -> None:
        pass


instance = Concrete(a='hello', b='world')

Note that this error only occurs if at least one abstract method is defined on the abstract class. Removing the abstract method c (or making it non-abstract) makes the error go away.

@gvanrossum
Copy link
Member

@gvanrossum gvanrossum commented Jul 19, 2018

@ojii
Copy link
Author

@ojii ojii commented Jul 19, 2018

Removing the metaclass still triggers it. Minimal code to get that error:

import abc
from dataclasses import dataclass

@dataclass
class Abstract:
    @abc.abstractmethod
    def c(self) -> None:
        pass
@ilevkivskyi
Copy link
Collaborator

@ilevkivskyi ilevkivskyi commented Jul 19, 2018

The underlying cause is this:

from typing import Type, TypeVar
from abc import abstractmethod

T = TypeVar('T')

class C:
    @abstractmethod
    def f(self) -> None: ...

def fun(x: Type[T]) -> Type[T]: ...

fun(C)

A possible simple solution is to type dataclass in typeshed as roughly (S) -> S with S = TypeVar('S', bound=Type[Any]). Another option is to consider relaxing the restriction that mypy has for Type[T] with abstract classes, but I am not sure where to draw the line.

@antdking
Copy link

@antdking antdking commented Nov 6, 2018

Lifting this constraint would be great for fetching implementations to interfaces (abstracts/protocols).

T_co = TypeVar('T_co', covariant=True)

def register(interface: Type[T_co], implementation: Type[T_co]) -> Type[T_co]: ...
def inject(interface: Type[T_co]) -> T_co: ...
@ilevkivskyi
Copy link
Collaborator

@ilevkivskyi ilevkivskyi commented Nov 7, 2018

Just a bit of context why this is prohibited: mypy always allows instantiation of something typed as Type[...], in particular of function arguments, for example:

from typing import Type, TypeVar
from abc import abstractmethod

T = TypeVar('T')

class C:
    @abstractmethod
    def f(self) -> None: ...

def fun(x: Type[T]) -> T: ...
    return x()  # mypy allows this, since it is a very common pattern

fun(C)  # this fails at runtime, and it would be good to catch it statically,
        # what we do currently

I am still not sure what is the best way to proceed here. Potentially, we can only prohibit this if original function argument (or its upper bound) is abstract (so function actually expects implementations to be passed). This will be a bit unsafe, but more practical, as all examples provided so far would pass.

@JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Nov 8, 2018

Another option is to consider relaxing the restriction that mypy has for Type[T] with abstract classes, but I am not sure where to draw the line.

I'm leaning towards allowing abstract classes to be used without restriction. Type[x] is already unsafe since __init__ can be overriden with a different signature, and I expect that the extra unsafety from allowing abstract classes would be marginal in practice. We just need to document the potential issues clearly.

@ilevkivskyi
Copy link
Collaborator

@ilevkivskyi ilevkivskyi commented Nov 8, 2018

I'm leaning towards allowing abstract classes to be used without restriction. [...] We just need to document the potential issues clearly.

This will be false negative, so probably OK.

@ZeeD
Copy link

@ZeeD ZeeD commented Mar 23, 2019

Sorry to bother, but is there some ETA on this issue?

@gvanrossum
Copy link
Member

@gvanrossum gvanrossum commented Mar 23, 2019

There is no concrete work planned for this issue yet, we are focusing on high-priority issues.

@nickwilliams-eventbrite
Copy link

@nickwilliams-eventbrite nickwilliams-eventbrite commented Oct 1, 2019

I'm having a similar error and I don't know if it's the same problem or a different, related problem (or perhaps I'm doing something wrong), related to abstract classes and Attrs.

I have this abstract class: https://github.com/eventbrite/pymetrics/blob/ab608978/pymetrics/recorders/base.py#L38-L39

Then I have this constant:

_DEFAULT_METRICS_RECORDER = NoOpMetricsRecorder()  # type: MetricsRecorder

And this Attrs class:

@attr.s
@six.add_metaclass(abc.ABCMeta)
class RedisTransportCore(object):
    ...
    metrics = attr.ib(
        default=_DEFAULT_METRICS_RECORDER,
        validator=attr.validators.instance_of(MetricsRecorder),
    )  # type: MetricsRecorder

On the line with attr.validators.instance_of, I'm getting this error:

error: Only concrete class can be given where "Type[MetricsRecorder]" is expected

What's interesting is that this only happens with today's released Attrs 19.2.0 and last week's released MyPy 0.730. If I downgrade to Attrs 19.1.0 and keep the latest MyPy, the error goes away. If I downgrade to MyPy 0.720 and keep the latest Attrs, the error goes away.

I'm able to get around the problem by adding # type: ignore at the end of the line with attr.validators.instance_of, so I'm good for now, but I would like to know if I need to file a separate issue or if this is related or if this is a bug with Attrs.

@aldanor
Copy link

@aldanor aldanor commented Oct 28, 2019

This is really sad, given that dataclasses are being pushed as a go-to class container in 3.7+, having abstract dataclasses is not such a rare occasion. But mypy in the current state basically prohibits using it.

Given that this issue is 1.5 years old, wonder if there's any known workarounds?

@JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Oct 28, 2019

Marking as high priority, since there seems to be quite a lot of interest in getting this fixed.

@ToBeReplaced
Copy link

@ToBeReplaced ToBeReplaced commented Dec 23, 2019

Depending on your needs, for a temporary workaround, you can do the following, which type checks with --strict.

import abc
import dataclasses


class AbstractClass(abc.ABC):
    """An abstract class."""

    @abc.abstractmethod
    def method(self) -> str:
        """Do something."""


@dataclasses.dataclass
class DataClassMixin:
    """A dataclass mixin."""

    spam: str


class AbstractDataClass(DataClassMixin, AbstractClass):
    """An abstract data class."""


@dataclasses.dataclass
class ConcreteDataClass(AbstractDataClass):
    """A concrete data class."""

    eggs: str

    def method(self) -> str:
        """Do something."""
        return "{} tastes worse than {}".format(self.spam, self.eggs)


print(ConcreteDataClass("spam", "eggs").method())
# -> "spam tastes worse than eggs"
@twhaples
Copy link

@twhaples twhaples commented Feb 4, 2020 •

Similar to @cybojenix, we would be very happy if mypy could support service locator patterns similar to the following (the code for which works, but does not pass mypy type checks for want of a concrete class.)

from typing import Any, Dict, TypeVar, Type
from typing_extensions import Protocol

ServiceType = TypeVar("ServiceType")

registrations: Dict[Any, Any] = {}
def register(t: Type[ServiceType], impl: ServiceType) -> None:
    registrations[t] = impl

def locate(t: Type[ServiceType]) -> ServiceType:
    return registrations[t]


class ThingService(Protocol):
    def serve(str) -> str:
        pass

class ThingImpl(ThingService):
    def serve(str) -> str:
        return "fish"


register(ThingService, ThingImpl())
thing: ThingService = locate(ThingService)
print(thing.serve())
@schuderer
Copy link

@schuderer schuderer commented Mar 8, 2020

Interestingly, the given rationale of not allowing abstract classes ("mypy always allows instantiation of something typed as Type[...], in particular of function arguments"), does not seem to match its implementation, as classes based on abc.ABC cannot be instantiated, but code using only this actually pass the mypy check, whereas classes with @abstractmethods can be instantiated, but classes using those trigger the false positives discussed in this issue.

To elaborate (and give another example to possibly test a fix against later), the following code uses only @abstractmethods and triggers the false positive:

from abc import ABC, abstractmethod
from typing import List, Type, TypeVar

AB = TypeVar("AB", "A", "B")  # Could be A or B (Union)

class A:
    @abstractmethod
    def snork(self):
        pass
class B:
    @abstractmethod
    def snork(self):
        pass

class oneA(A):
    pass
class anotherA(A):
    pass
class someB(B):
    pass

def make_objects(of_type: Type[AB]) -> List[AB]:
    return [cls() for cls in of_type.__subclasses__()]

if __name__ == "__main__":
    print(make_objects(A))
    print(make_objects(B))

Removing the two @abstractmethod decorators and changing class A: and class B: to class A(ABC): and class B(ABC): actually passes mypy's check without errors.

@jdlourenco
Copy link

@jdlourenco jdlourenco commented May 6, 2020

Running into the same issue.

@chadlagore
Copy link

@chadlagore chadlagore commented Jun 28, 2020 •

Similar to @ToBeReplaced, workaround I've been using:

@dataclass
class _AbstractDataclass:
    id: int

class AbstractDataclass(_AbstractDataclass, abc.ABC):
    @abc.abstractmethod
    def c(self) -> None:
        pass
legeana added a commit to legeana/config that referenced this issue Jul 5, 2020
@fabioz
Copy link

@fabioz fabioz commented Aug 19, 2020

Another use-case which also doesn't work due to this limitation is the adapter pattern using typing Protocols.

i.e.:

from typing import Protocol, TypeVar, Type

T = TypeVar('T')

class IFoo(Protocol):

    def foo(self) -> str:
        pass

class Adaptable(object):

    def get_adapter(self, class_: Type[T]) -> T:
        ...

def some_call(adaptable: Adaptable) -> None:
    foo = adaptable.get_adapter(IFoo)
jmanuel1 added a commit to jmanuel1/concat that referenced this issue Oct 25, 2020
Mypy complains about abstract dataclassses. See
python/mypy#5374.
tandemdude pushed a commit to tandemdude/hikari-lightbulb that referenced this issue Nov 26, 2020
Found the following issue that needs me to change some implementation logic
in Hikari to make this work properly.

```
            self._context.bot.unsubscribe(hikari.ReactionAddEvent, self._p...
                                          ^

lightbulb/utils/nav.py:136:39: error: Only concrete class can be given where
"Type[ReactionAddEvent]" is expected  [misc]
```

Seems to be linked to python/mypy#5374
@mkielar
Copy link

@mkielar mkielar commented Jan 26, 2021 •

This is over 2 years old, and 0.800 just started doing a better job at finding these cases which resulted with a wall of errors in my project. Thanks, mypy. :(

@JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented Jan 26, 2021

@mkielar could you give concrete examples of code that started erroring? Perhaps we can easily fix the regression.

@mkielar
Copy link

@mkielar mkielar commented Jan 26, 2021

@JelleZijlstra

My production code cut to be smaller:

import dataclasses as dc
from abc import ABCMeta, abstractmethod


@dc.dataclass(frozen=True)
class CanaryConfiguration(metaclass=ABCMeta):
    @abstractmethod
    def is_fast_track(self) -> bool:
        """
        """


@dc.dataclass(frozen=True)
class CanaryConfigurationEcs(CanaryConfiguration):

    def is_fast_track(self) -> bool:
        return True


@dc.dataclass(frozen=True)
class Canary:
    configuration: CanaryConfiguration


def test_me() -> bool:
    c: Canary = Canary(CanaryConfigurationEcs())
    return c.configuration.is_fast_track()

Console output:

→ mypy --version
mypy 0.800
→ mypy test.py
test.py:5: error: Only concrete class can be given where "Type[CanaryConfiguration]" is expected
Found 1 error in 1 file (checked 1 source file)

In actual code it additionally complains that the return statement in test_me() returns Any instead of bool but somehow if I cut the business logic I can no longer reproduce it.

vlasovskikh added a commit to pauleveritt/andrey_abstract that referenced this issue Feb 3, 2021
 The error about "Only concrete class can be given where "Type[View]" is expected" is a false positive in Mypy. They should fix it on their side. See also python/mypy#5374
@lundybernard
Copy link

@lundybernard lundybernard commented Mar 16, 2021

Adding one more (hopefully) valid use case

from dataclasses import dataclass, fields, is_dataclass
from typing import Protocol, Optional, cast

@dataclass
class MyConfig:
    v1: str
    v2: 'Optional[str]' = None

@dataclass
class GlobalConfig:
    MyConfig: MyConfig

class SourceList(Protocol):
    def get(self, key: str) -> str:
        pass

class ConfigProtocol(Protocol):  # this is one option, but just being able to type check dataclass would be fine
    __dataclass_fields__: dict

class Configuration:
    def __init__(
        self,
        source_list: SourceList,
        config_class: ConfigProtocol = GlobalConfig
    ):
        self._config_sources = source_list

        for f in fields(self._config_class):
            if is_dataclass(f.type):
                setattr(
                    self,
                    f.name,
                    Configuration(
                        source_list=source_list,
                        config_class=cast(ConfigProtocol, f.type)
                    )
                )

    def __getattr__(self, name):
        if value := self._config_sources.get(name):
            return value
        raise AttributeError()

mypy output:

> mypy --version
mypy 0.800
> mypy test.py 
test.py:24: error: Incompatible default for argument "config_class" (default has type "Type[GlobalConfig]", argument has type "ConfigProtocol")
Found 1 error in 1 file (checked 1 source file)

Define dataclasses for config objects in modules and sub-modules, which can be instantiated and used directly, or as shown here, used as templates to build a more complex config class.

nicoddemus added a commit to nicoddemus/oop-ext that referenced this issue Mar 19, 2021
Also fixes a problem in AssertImplements due to python/mypy#5374.
nicoddemus added a commit to nicoddemus/oop-ext that referenced this issue Mar 19, 2021
Also fixes a problem in AssertImplements due to python/mypy#5374.
nicoddemus added a commit to nicoddemus/oop-ext that referenced this issue Mar 19, 2021
Also fixes a problem in AssertImplements due to python/mypy#5374.
nicoddemus added a commit to nicoddemus/oop-ext that referenced this issue Mar 19, 2021
Also fixes a problem in AssertImplements due to python/mypy#5374.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet