Are pandas dataframes supported as function arguments in a @cached decorated function?
I tried to simplify this example with a smaller dataframe but @cached does seem to behave as one would expect for smaller dataframes.
However, when I tried the minimal code below with the attached data I ran into a problem where the two clearly different dataframes are being interpreted as identical in the @cached decorated function. Thus, df2 doesn't make it through which_df but instead gets the value from the cache since it assumes df2 is equals to df1 (and it is not!)
This is the test to replicate. Please use the attached data get the unexpected behavior explained in this issue
import pandas as pd
from memoization import cached
@cached()
def which_df(df):
# print("got inside function")
return df.name
df1 = pd.read_pickle('memoization_test.pkl')
df1.name = "This is DF No. 1"
df2 = df1.interpolate()
df2.name = "This is DF No. 2"
df1.equals(df2) # ==> False, since they are not identical
print(which_df(df1) + ', and it should be DF No. 1')
print(which_df(df2) + ', BUT it should be DF No. 2')
memoization_test.zip
Are pandas dataframes supported as function arguments in a @cached decorated function?
I tried to simplify this example with a smaller dataframe but @cached does seem to behave as one would expect for smaller dataframes.
However, when I tried the minimal code below with the attached data I ran into a problem where the two clearly different dataframes are being interpreted as identical in the @cached decorated function. Thus, df2 doesn't make it through
which_dfbut instead gets the value from the cache since it assumes df2 is equals to df1 (and it is not!)This is the test to replicate. Please use the attached data get the unexpected behavior explained in this issue
memoization_test.zip