Sitelet https://web.archive.org/web/20260101070627/https://github.com/github/codeql/pull/6091
Skip to content

Conversation

@tausbn
Copy link
Contributor

@tausbn tausbn commented Jun 16, 2021

According to the official documentation, the purpose of __main__.py
files is that their presence in a package (say, foo) means one can
execute the package directly using python -m foo (which will run the
aforementioned foo/__main__.py file).

In principle this means that adding if __name__ == "__main__" in these
files is superfluous, as they are only intended to be executed (and not
imported by some other file).

However, in practice people often do include the above construct.
Here are some instances of this on LGTM.com:
https://lgtm.com/query/7521266095072095777/

In particular, 10 out of 33 files in cpython have this construct.

This causes some confusion in our module naming, as we usually see the
presence of __name__ == "__main__" as an indication that a file may
be run directly (and hence with "absolute import" semantics). However,
when run with python -m, the interpreter uses the usual package
semantics, and this leads to modules getting multiple names.

For this reason, I think it makes sense to simply exclude __main__.py
files from consideration. Note that if there is a #! line mentioning
the Python interpreter, then they will still be included as entry
points.


This is a minor change, and so I don't think this requires a change note.

According to the official documentation, the purpose of `__main__.py`
files is that their presence in a package (say, `foo`) means one can
execute the package directly using `python -m foo` (which will run the
aforementioned `foo/__main__.py` file).

In principle this means that adding `if __name__ == "__main__"` in these
files is superfluous, as they are only intended to be executed (and not
imported by some other file).

However, in practice people often _do_ include the above construct.
Here are some instances of this on LGTM.com:
https://lgtm.com/query/7521266095072095777/

In particular, 10 out of 33 files in `cpython` have this construct.

This causes some confusion in our module naming, as we usually see the
presence of `__name__ == "__main__"` as an indication that a file may
be run directly (and hence with "absolute import" semantics). However,
when run with `python -m`, the interpreter uses the usual package
semantics, and this leads to modules getting multiple names.

For this reason, I think it makes sense to simply exclude `__main__.py`
files from consideration. Note that if there is a `#!` line mentioning
the Python interpreter, then they will still be included as entry
points.
@RasmusWL
Copy link
Member

As we discussed, __main__.py can be used in zip-files to make the zip executable by Python. I would assume that in those cases, imports are absolute, just like with normal python scripts. I think the use of executable zip-files is so low, that it's probably a better tradeoff to make __main__.py files in Python packages work 👍

Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
@tausbn
Copy link
Contributor Author

tausbn commented Jun 21, 2021

As we discussed, __main__.py can be used in zip-files to make the zip executable by Python. I would assume that in those cases, imports are absolute, just like with normal python scripts. I think the use of executable zip-files is so low, that it's probably a better tradeoff to make __main__.py files in Python packages work

Yeah, I also tested the zip-file situation, and indeed that uses absolute imports. I agree that this is likely a very rare use-case, and so probably okay to not cover (as I'm not sure how we would even detect this setup). 👍

Copy link
Member

@RasmusWL RasmusWL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I'm just realizing that now we handle python -m foo.bar better if the structure is foo/bar/__main__.py, but (still) not very good if the structure is foo/bar.py (where bar.py has a if __name__ == "__main__": if-statement).

So definitely a step in the right direction, but we might have more work to do in this area 😥

@RasmusWL RasmusWL merged commit 5db6270 into github:main Jun 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants