Repository navigation
Change shutil.rmtree and os.walk to support very deep hierarchies #89727
Description
Activity
AlexanderPatrakov commented
on Oct 21, 2021 AlexanderPatrakovmannequinMannequinAuthorIt is possible to create deep directory hierarchies that cannot be removed via shutil.rmtree or walked via os.walk, because these functions exceed the interpreter recursion limit. This may have security implications for web services (e.g. various webdisks) that have to clean up user-created mess or walk through it.
[aep@aep-haswell ~]$ mkdir /tmp/badstuff [aep@aep-haswell ~]$ cd /tmp/badstuff [aep@aep-haswell badstuff]$ for x in `seq 2048` ; do mkdir $x ; cd $x ; done [aep@aep-haswell 103]$ cd [aep@aep-haswell ~]$ python Python 3.9.7 (default, Oct 10 2021, 15:13:22) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import shutil >>> shutil.rmtree('/tmp/badstuff') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.9/shutil.py", line 726, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/usr/lib/python3.9/shutil.py", line 663, in _rmtree_safe_fd _rmtree_safe_fd(dirfd, fullname, onerror) File "/usr/lib/python3.9/shutil.py", line 663, in _rmtree_safe_fd _rmtree_safe_fd(dirfd, fullname, onerror) File "/usr/lib/python3.9/shutil.py", line 663, in _rmtree_safe_fd _rmtree_safe_fd(dirfd, fullname, onerror) [Previous line repeated 992 more times] File "/usr/lib/python3.9/shutil.py", line 642, in _rmtree_safe_fd fullname = os.path.join(path, entry.name) File "/usr/lib/python3.9/posixpath.py", line 77, in join sep = _get_sep(a) File "/usr/lib/python3.9/posixpath.py", line 42, in _get_sep if isinstance(path, bytes): RecursionError: maximum recursion depth exceeded while calling a Python object >>> import os >>> list(os.walk('/tmp/badstuff')) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.9/os.py", line 418, in _walk yield from _walk(new_path, topdown, onerror, followlinks) File "/usr/lib/python3.9/os.py", line 418, in _walk yield from _walk(new_path, topdown, onerror, followlinks) File "/usr/lib/python3.9/os.py", line 418, in _walk yield from _walk(new_path, topdown, onerror, followlinks) [Previous line repeated 993 more times] File "/usr/lib/python3.9/os.py", line 412, in _walk new_path = join(top, dirname) File "/usr/lib/python3.9/posixpath.py", line 77, in join sep = _get_sep(a) File "/usr/lib/python3.9/posixpath.py", line 42, in _get_sep if isinstance(path, bytes): RecursionError: maximum recursion depth exceeded while calling a Python object >>>
This also affects
pathlib.Path.walk()Reacted by Stanislav Zmiev- added a commit that references this issue
on Nov 30, 2022 - added a commit that references this issue
on Dec 15, 2022 101 remaining items
- added 5 commits that reference this issue
on Jun 1, 2024 - added 4 commits that reference this issue
on Jul 11, 2024 - added 4 commits that reference this issue
on Jul 17, 2024
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
shutil.rmtree()recursion error on deep trees. #103164pathlib.Path.fwalk()method #103566shutil.rmtree()recursion error on deep trees #119634os.fwalk()recursion error on deep trees #119638shutil.rmtree()recursion error on deep trees (GH-119634) #119748shutil.rmtree()recursion error on deep trees (GH-119634) #119749os.fwalk()recursion error on deep trees (GH-119638) #119764os.fwalk()recursion error on deep trees (GH-119638) #119765os.fwalk()generator finalization. #119766os.fwalk()generator finalization. (GH-119766) #119767os.fwalk()generator finalization. (GH-119766) #119768shutil.rmtree()recursion error on deep trees #119808shutil.rmtree()recursion error on deep trees (GH-119808) #119918shutil.rmtree()recursion error on deep trees (GH-119808) #119919