Rework frozen modules and directly deserialize to CodeObject<Literal> #4608
Conversation
59488d9
to
d9dd7ce
Compare
|
|
||
| let output = quote! { | ||
| #crate_name::CodeObject::from_bytes(#bytes) | ||
| .expect("Deserializing CodeObject failed") | ||
| #crate_name::frozen_lib::FrozenCodeObject { bytes: &#bytes[..] } |
There was a problem hiding this comment.
In my opinion, the name py_compile doesn't imply its result will be compressed but getting code object directly.
Renaming or making a new macro?
There was a problem hiding this comment.
Perhaps, yeah, but because vm.ctx.new_code() still accepts it, it won't actually make a difference for users.
There was a problem hiding this comment.
Users will expanse unexpected compression cost after this change.
I expect vm.compile and py_compile! are doing similar things because they use the same term.
one of my suggestion is, renaming current py_freeze to py_freeze_module and renaming py_compile to py_freeze.
There was a problem hiding this comment.
They already were - the old CodeObject::from_bytes also did lz4 decompression, and py_compile!() just generated CodeObject::from_bytes(b"..."). I don't think compile returning a frozen code object is necessarily unintuitive (they're expecting it to compile it at build time and embed it in the binary; what's that if not freezing?) and again, since it's still accepted in the same places as before it doesn't actually make much of a difference to the user
| pub code: CodeObject<ConstantData>, | ||
| pub package: bool, | ||
| } | ||
|
|
||
| pub mod frozen_lib { |
There was a problem hiding this comment.
this is going bigger. maybe frozen_lib.rs later
Now frozen modules are stored as lz4-compressed in static memory until they are imported, which should help with memory usage in a freeze-stdlib environment (and the size increase from less efficient(?) compression due to potentially less dict-sharing(?? i have no clue how compression algorithms actually work) should hopefully not be too bad)