Sitelet https://web.archive.org/web/20200820055818/https://github.com/pandas-dev/pandas/issues/35611
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Make top-level Pandas functions serializable #35611

Closed
mrocklin opened this issue Aug 7, 2020 · 5 comments
Closed

ENH: Make top-level Pandas functions serializable #35611

mrocklin opened this issue Aug 7, 2020 · 5 comments

Comments

@mrocklin
Copy link
Contributor

@mrocklin mrocklin commented Aug 7, 2020

import pandas as pd
import pickle
pickle.dumps(pd.read_csv)
AttributeError: Can't pickle local object '_make_parser_function.<locals>.parser_f'
@mroeschke
Copy link
Member

@mroeschke mroeschke commented Aug 8, 2020

Looks like read_csv is serializable in 1.1.0, as well as all callable object in the top level namespace

In [20]: import pickle

In [21]: import pandas as pd

In [22]: pd.__version__
Out[22]: '1.1.0'

In [23]: pickle.dumps(pd.read_csv)
Out[23]: b'\x80\x03cpandas.io.parsers\nread_csv\nq\x00.'

In [24]: for name, obj in pd.__dict__.items():
    ...:     if callable(obj):
    ...:         try:
    ...:             pickle.dumps(obj)
    ...:         except:
    ...:             print(f'{name}' is not pickleable)
    ...:

In [25]:
@mroeschke
Copy link
Member

@mroeschke mroeschke commented Aug 8, 2020

#34976 probably made read_csv serializable.

I suppose we could add unit tests to ensure all these objects are serializable

@ikedaosushi
Copy link
Contributor

@ikedaosushi ikedaosushi commented Aug 9, 2020

May I take it?

@mrocklin
Copy link
Contributor Author

@mrocklin mrocklin commented Aug 9, 2020

Oh that's great news. Thank you @mroeschke for checking, and my apologies for testing against an older version.

@ikedaosushi I think that you do not need permission to submit a pull request. I do not know Pandas' development policies, but I think that it is ok for you to contribute.

@mroeschke
Copy link
Member

@mroeschke mroeschke commented Aug 9, 2020

No worries @mrocklin

Yes, @ikedaosushi, happy to have you take this issue.

@simonjayhawkins simonjayhawkins added this to the Contributions Welcome milestone Aug 11, 2020
ikedaosushi added a commit to ikedaosushi/pandas that referenced this issue Aug 12, 2020
@jreback jreback modified the milestones: Contributions Welcome, 1.2 Aug 19, 2020
jreback pushed a commit that referenced this issue Aug 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants
You can’t perform that action at this time.