Sitelet https://github.com/python/cpython/issues/93820
Skip to content

AttributeError when copying flags in the re module. #93820

Description

@pschanely

Bug report

In CPython 3.11, I've noticed that an attempt to copy (combined) regular expression flags fails with an attribute error; like so:

>>> import copy, re
>>> copy.copy(re.A | re.I)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lib/python3.11/copy.py", line 102, in copy
    return _reconstruct(x, None, *rv)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "lib/python3.11/copy.py", line 265, in _reconstruct
    y = func(*args)
        ^^^^^^^^^^^
  File "lib/python3.11/enum.py", line 862, in __getattr__
    raise AttributeError(name) from None
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: ASCII|IGNORECASE

Your environment

  • CPython versions tested on: 3.11.0b3
  • Operating system and architecture: macOS 10.15.7

Background

This behavior was detected from a CrossHair test failure here, in which we realize possibly symbolic arguments prior to regex compilation. (it's an esoteric use case, and I expect not very high priority)

Activity

  1. JelleZijlstra commented on Jun 14, 2022

    @JelleZijlstra
    Member

    Thanks for reporting! Confirmed that copy.copy(re.A | re.I) fails on 3.11 but works on 3.10. Marking as a release blocker since this is a regression.

    cc @ethanfurman for enums

  2. JelleZijlstra commented on Jun 14, 2022

    @JelleZijlstra
    Member

    Also repros with enum.IntFlag directly:

    $ ./python.exe
    Python 3.11.0b3+ (heads/3.11:7aa4038a6e, Jun 14 2022, 09:35:10) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import copy, enum
    >>> class F(enum.IntFlag):
    ...     a = 2
    ...     b = 4
    ... 
    >>> copy.copy(F.a | F.b)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/jelle/py/cpython/Lib/copy.py", line 102, in copy
        return _reconstruct(x, None, *rv)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/jelle/py/cpython/Lib/copy.py", line 265, in _reconstruct
        y = func(*args)
            ^^^^^^^^^^^
      File "/Users/jelle/py/cpython/Lib/enum.py", line 862, in __getattr__
        raise AttributeError(name) from None
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AttributeError: a|b
    
  3. dignissimus commented on Jun 15, 2022

    @dignissimus
    Contributor

    Also occurs on 3.12

  4. tiran commented on Jun 15, 2022

    @tiran
    Member

    In 3.11 the __reduce__ and __reduce_ex__ methods returns different values:

    # 3.10
    >>> import re
    >>> flags = re.I | re.A
    >>> flags.__reduce__()
    (<function _reconstructor at 0x7f58bb775ea0>, (<enum 'RegexFlag'>, <class 'int'>, 258), {'_name_': None, '_value_': 258})
    >>> flags.__reduce_ex__(1)
    (<enum 'RegexFlag'>, (258,))
    >>> flags._name_ is None
    True
    >>> flags.name is None
    True
    # 3.11
    >>> import re
    >>> flags = re.I | re.A
    >>> flags.__reduce__()
    (<function _reconstructor at 0x7fd0b8d4f920>, (<flag 'RegexFlag'>, <class 'int'>, 258), {'_value_': 258, '_name_': 'ASCII|IGNORECASE'})
    >>> flags.__reduce_ex__(1)
    (<built-in function getattr>, (<flag 'RegexFlag'>, 'ASCII|IGNORECASE'))
    >>> flags._name_
    'ASCII|IGNORECASE'
    >>> flags.name
    'ASCII|IGNORECASE'
  5. tiran commented on Jun 15, 2022

    @tiran
    Member

    I guess the regression was introduced by GH-26658 and commit 62f1d2b .

    A quick fix would be to reduce flags by value instead of by name.

    class Flag(Enum, boundary=STRICT):
        def __reduce_ex__(self, proto):
            return self.__class__, (self._value_, )
    
  6. dignissimus commented on Jun 15, 2022

    @dignissimus
    Contributor

    Running the example in 0507303 so works fine, so commit 62f1d2b introduced the issue

  7. dignissimus commented on Jun 15, 2022

    @dignissimus
    Contributor

    Oh, it looks like we came across it at the same time 😂

  8. tiran commented on Jun 15, 2022

    @tiran
    Member

    PR #93876 is a potential fix. Waiting for @ethanfurman to confirm.

  9. added 2 commits that reference this issue on Jun 16, 2022
  10. serhiy-storchaka commented on Jun 16, 2022

    @serhiy-storchaka
    Member

    Ideally, re.I | re.A should be reconstructed as operator.or_(re.ASCII, re.IGNORECASE).

  11. tiran commented on Jun 16, 2022

    @tiran
    Member

    It gets a bit tricky when the flags have a residual member, e.g. re.A | re.I | 0x10000. IntFlag supports additional values that are not covered by an enum member.

  12. added 2 commits that reference this issue on Jun 16, 2022
  13. added 4 commits that reference this issue on Jun 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions