Sitelet https://github.com/python/cpython/issues/84694
Skip to content

[subinterpreters] Add --experimental-isolated-subinterpreters build option #84694

Closed
Closed
[subinterpreters] Add --experimental-isolated-subinterpreters build option#84694
Labels
3.10 (EOL)end of lifebuildThe build process and cross-buildtopic-subinterpreters

Description

@vstinner
vstinner
opened on May 5, 2020
BPO 40514
Nosy @vstinner, @ericsnowcurrently, @corona10
PRs
  • bpo-40514: Add --with-experimental-isolated-subinterpreters #19926
  • [3.9] bpo-40514: Remove --with-experimental-isolated-subinterpreters in 3.9 #20228
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2020-10-31.22:59:17.920>
created_at = <Date 2020-05-05.13:13:51.880>
labels = ['expert-subinterpreters', 'build', '3.10']
title = '[subinterpreters] Add --experimental-isolated-subinterpreters build option'
updated_at = <Date 2020-10-31.22:59:17.920>
user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2020-10-31.22:59:17.920>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-10-31.22:59:17.920>
closer = 'vstinner'
components = ['Build', 'Subinterpreters']
creation = <Date 2020-05-05.13:13:51.880>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40514
keywords = ['patch']
message_count = 6.0
messages = ['368141', '368156', '368159', '368172', '369401', '370745']
nosy_count = 3.0
nosy_names = ['vstinner', 'eric.snow', 'corona10']
pr_nums = ['19926', '20228']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue40514'
versions = ['Python 3.10']

    Linked PRs

    Activity

    1. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      Modifying Python internals to have one GIL per interpreter (bpo-40512) is a large project which requires to modify many small things (again, see bpo-40512).

      I propose to add a temporary build --experimental-isolated-subinterpreters build option to configure for developers who want to hack on isolated subinterpreters. The intent is to speedup bpo-40512 development by making some practice compromises, to have more time to properly design the real fixes.

      For example, tuple, dict and frame have free lists which are shared by subinterpreters. A practical solution is to simply disable them at build time to avoid the need to have per-interpreter free lists.

      Another example is pymalloc which is shared by all subinterpreters and rely on the unique global interpreter lock (GIL) to protect its internal states. A practical solution is to disable it and force the usage of libc malloc() function instead.

      Some compromosises cannot be the default since they have a significant negative impact on performances. So I propose to add a temporary build option until all these small issues will be fixed.

    2. vstinner
      added
      3.9 (EOL)end of life
      buildThe build process and cross-build
      on May 5, 2020

    3. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      New changeset c5fa364 by Victor Stinner in branch 'master':
      bpo-40514: Add --with-experimental-isolated-subinterpreters (GH-19926)
      c5fa364

    4. vstinner commented on May 5, 2020

      @vstinner
      vstinner
      on May 5, 2020
      MemberAuthor

      --with-experimental-isolated-subinterpreters option name is very long on purpose: to advertize that you must not use it, unless you fully understand its purpose :-)

      I didn't document the build option in Misc/SpecialBuilds.txt or Doc/whatsnew/3.9.rst on purpose: you should not use this special build :-) Basically, this option is only for Eric Snow and me :-D

    5. ericsnowcurrently commented on May 5, 2020

      @ericsnowcurrently
      ericsnowcurrently
      on May 5, 2020
      Member

      It would probably make sense to remove the build option in the 3.9 release. We can leave it in master, but remove it in the 3.9 branch once it has been created.

    6. vstinner
      added
      topic-subinterpreters
      on May 15, 2020
    7. vstinner
      changed the title [-]Add --experimental-isolated-subinterpreters build option[/-] [+][subinterpreters] Add --experimental-isolated-subinterpreters build option[/+] on May 15, 2020
    8. vstinner
      added
      topic-subinterpreters
      on May 15, 2020
    9. vstinner
      changed the title [-]Add --experimental-isolated-subinterpreters build option[/-] [+][subinterpreters] Add --experimental-isolated-subinterpreters build option[/+] on May 15, 2020

    10. vstinner commented on May 19, 2020

      @vstinner
      vstinner
      on May 19, 2020
      MemberAuthor

      New changeset 9512ad7 by Victor Stinner in branch '3.9':
      [3.9] bpo-40514: Remove --with-experimental-isolated-subinterpreters in 3.9 (GH-20228)
      9512ad7

    11. vstinner commented on Jun 5, 2020

      @vstinner
      vstinner
      on Jun 5, 2020
      MemberAuthor

      I started to *remove* a few"#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS":

      • commit 88ec919: Make list free list per-interpreter
      • commit 3744ed2: Make frame free list per-interpreter
      • PR 20645 (under review): Make dict free lists per-interpreter
    12. vstinner
      added
      3.10 (EOL)end of life
      on Jun 5, 2020

    13. 4 remaining items

    14. vstinner
      closed this as completedon Oct 31, 2020
    15. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    16. kumaraditya303 commented on May 25, 2022

      @kumaraditya303
      kumaraditya303
      on May 25, 2022
      Contributor

      Reopened for #93185

    17. kumaraditya303
      reopened this on May 25, 2022

    18. erlend-aasland commented on May 25, 2022

      @erlend-aasland
      erlend-aasland
      on May 25, 2022
      Contributor

      Reopened for #93185

      Why do you need to reopen it? The title says "add --experimental-isolated-subinterpreters build option". If you are to remove it, it can be done in the new issue, no?

    19. kumaraditya303 commented on May 25, 2022

      @kumaraditya303
      kumaraditya303
      on May 25, 2022
      Contributor

      I reopened it as #93185 uses this issue in title, if that PR will use a new issue, this can be closed.

    20. erlend-aasland commented on May 25, 2022

      @erlend-aasland
      erlend-aasland
      on May 25, 2022
      Contributor

      Use #93185 for the removal; there is no need for this to be reopened.

    21. erlend-aasland
      closed this as completedon May 25, 2022
    22. ericsnowcurrently
      added 2 commits that reference this issue on May 27, 2022

      bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)

      caa279d

      Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (cherry-picked GH-pythongh…

      f9a33b0
    23. miss-islington
      added a commit that references this issue on May 28, 2022

      [3.11] bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)…

      3637425
    24. CAM-Gerlach
      added 3 commits that reference this issue on Oct 16, 2022

      Add subinterpreters removal from issue pythongh-84694 PRs pythongh-96499

      c6d321b

      Add subinterpreters removal from issue pythongh-84694 PRs pythongh-96499

      6c536da

      Add subinterpreters removal from issue pythongh-84694 PRs pythongh-93185

      1790ddd
    25. miss-islington
      added a commit that references this issue on Oct 17, 2022

      gh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (GH-98345)

      5fe0431
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.10 (EOL)end of lifebuildThe build process and cross-buildtopic-subinterpreters

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions