
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2015-12-14 10:30 bySamson Lee, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| fwalk-silently-skips-dirs-when-error-occurs.patch | Samson Lee,2015-12-14 10:30 | review | ||
| fwalk-silently-skips-dirs-when-error-occurs-with-tests.patch | serhiy.storchaka,2015-12-18 18:32 | review | ||
| Messages (4) | |||
|---|---|---|---|
| msg256377 -(view) | Author: Samson Lee (Samson Lee) | Date: 2015-12-14 10:30 | |
I noticed os.fwalk() produced different results as os.walk(). It turned outthat os.fwalk() skips all remaining directories when OSError occurs (e.g. dueto PermissionError).To reproduce the bug, first create a test directory structure: $ mkdir 1; touch 1/a $ mkdir 2; touch 2/b $ mkdir 3; touch 3/c $ mkdir 4; touch 4/cAt this stage, everything is okay, both os.fwalk() os.walk() produce the sameresults: >>> import os >>> for root, dirs, files in os.walk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d'] >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d']To introduce an error, force a PermissionError on one of the directories: $ sudo chown root:root 2 $ sudo chmod 700 2Now, the os.fwalk() results are different (trust me, os.walk() is still okay): >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a']So it seems that os.fwalk skips remaining directories after an error occurs.The cause of the problem is in this part of os.py: for name in dirs: try: orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks) dirfd = open(name, O_RDONLY, dir_fd=topfd) except OSError as err: if onerror is not None: onerror(err) breakTo fix it, simply replace break with continue. Patch attached.Cheers | |||
| msg256699 -(view) | Author: Serhiy Storchaka (serhiy.storchaka)*![]() | Date: 2015-12-18 18:32 | |
LGTM. Here is a patch with tests. | |||
| msg256871 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-12-22 22:09 | |
New changeset767262c149ca by Serhiy Storchaka in branch '3.5':Issue#25860: os.fwalk() no longer skips remaining directories when error occurs.https://hg.python.org/cpython/rev/767262c149caNew changeseta85675dabb8f by Serhiy Storchaka in branch 'default':Issue#25860: os.fwalk() no longer skips remaining directories when error occurs.https://hg.python.org/cpython/rev/a85675dabb8f | |||
| msg256876 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2015-12-22 22:38 | |
New changeset7995a81236b6 by Serhiy Storchaka in branch '3.5':Issue#25860: Fixed test failure caused by inconsistency of os.walk() andhttps://hg.python.org/cpython/rev/7995a81236b6New changesetdcf9e9ae5393 by Serhiy Storchaka in branch 'default':Issue#25860: Fixed test failure caused by inconsistency of os.walk() andhttps://hg.python.org/cpython/rev/dcf9e9ae5393 | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:24 | admin | set | github: 70047 |
| 2015-12-23 09:31:10 | serhiy.storchaka | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
| 2015-12-22 22:38:13 | python-dev | set | messages: +msg256876 |
| 2015-12-22 22:09:25 | python-dev | set | nosy: +python-dev messages: +msg256871 |
| 2015-12-18 18:32:27 | serhiy.storchaka | set | files: +fwalk-silently-skips-dirs-when-error-occurs-with-tests.patch assignee:serhiy.storchaka versions: + Python 3.6, - Python 3.4 nosy: +serhiy.storchaka messages: +msg256699 stage: patch review |
| 2015-12-14 10:57:36 | Samson Lee | set | nosy: +vstinner,benhoyt |
| 2015-12-14 10:41:58 | Samson Lee | set | nosy: +neologix,hynek |
| 2015-12-14 10:30:55 | Samson Lee | create | |