MutableSet isn't change aware on bitwise operators

Migrated issue, originally created by Daniel Kolsoi (@TheDan64)

Python sets support common set operations as bitwise operators such as &=, |=, ^=, -=. The MutableSet class seems to fully support these operations as full methods (.intersection, .union, ...) but not bitwise operators.

This is unexpected since the bitwise operators do in fact modify the underlying set(due to inheriting from set), but since the .changed() method is not called by MutableSet, sqlalchemy is never notified that the data has been modified. This leads to the possibility of having a mutable set column which is modified locally but does not get updated on commit or flush.

For example,

from sqlalchemy.ext.mutable import MutableSet
from sqlalchemy.types import TypeDecorator, UnicodeText

class _JSONEncodedSet(TypeDecorator):
    impl = UnicodeText

    def process_bind_param(self, value, dialect):
        if value is not None:
            value = json.dumps(list(value))
        return value

    def process_result_value(self, value, dialect):
        if value is not None:
            value = set(json.loads(value))
        return value

JSONEncodedSet = MutableSet.as_mutable(_JSONEncodedSet)

from sqlalchemy.ext.declarative import declarative_base

BaseModel = declarative_base()

class MyModel(BaseModel):
    json_set = Column(JSONEncodedSet)

from sqlalchemy.orm import scoped_session, sessionmaker
from zope.sqlalchemy import ZopeTransactionExtension

session = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))

session.add(MyModel(json_set={1, 2, 3}))
session.flush()

model = session.query(MyModel).first()
model.json_set &= {1, 3}

session.flush()

del model

model = session.query(MyModel).first()

assert model.json_set == {1, 3}  # AssertionError

Similarly, MutableList doesn't seem to support += [1]

Tested in SqlAlchemy 1.1.3, Sqlite but doesn't seem to be DB specific.

Changes by Daniel Kolsoi (@TheDan64):

edited description

Changes by Daniel Kolsoi (@TheDan64):

edited description

Michael Bayer (@zzzeek) wrote:

a simple pull request would be helpful here.

Changes by Michael Bayer (@zzzeek):

added labels: ext

Changes by Michael Bayer (@zzzeek):

set milestone to "1.1.x"

Changes by Michael Bayer (@zzzeek):

changed milestone from "1.1.x" to "1.2"

Michael Bayer (@zzzeek) wrote:

https://gerrit.sqlalchemy.org/421

Michael Bayer (@zzzeek) wrote:

Implement in-place mutation operators for MutableSet, MutableList

Implemented in-place mutation operators __ior__, __iand__,
__ixor__ and __isub__ for :class:.mutable.MutableSet
and __iadd__ for :class:.mutable.MutableList so that change
events are fired off when these mutator methods are used to alter the
collection.

Change-Id: Ib357a96d3b06c5deb6b53eb304a8b9f1dc9e9ede
Fixes: #3853

→ e8ad398

Changes by Michael Bayer (@zzzeek):

changed status to closed

Navigation Menu

Repository navigation

MutableSet isn't change aware on bitwise operators #3853

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

MutableSet isn't change aware on bitwise operators #3853

Description

Activity

sqlalchemy-bot commented on Nov 17, 2016

sqlalchemy-bot commented on Nov 17, 2016

sqlalchemy-bot commented on Nov 17, 2016

sqlalchemy-bot commented on Nov 17, 2016

sqlalchemy-bot commented on Nov 17, 2016

sqlalchemy-bot commented on Mar 31, 2017

sqlalchemy-bot commented on Jun 5, 2017

sqlalchemy-bot commented on Jun 5, 2017

sqlalchemy-bot commented on Jun 5, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions