Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-102828: add onexc arg to shutil.rmtree. Deprecate onerror.#102829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
iritkatriel merged 4 commits intopython:mainfromiritkatriel:rmtree
Mar 19, 2023

Conversation

iritkatriel
Copy link
Member

@iritkatrieliritkatriel commentedMar 19, 2023
edited
Loading

onexc expects an exception instead of exc_info. This is part of the larger effort to move on from exc_info.

Fixes#102828.

@iritkatrieliritkatriel added the stdlibPython modules in the Lib dir labelMar 19, 2023
@giampaolo
Copy link
Contributor

giampaolo commentedMar 19, 2023
edited
Loading

I like the idea in principle, but existing code usingonerror will stop working, becauseonerror arg is accepted but silently ignored. I think a retro-compatible solution to this could be to callonexc ANDonerror for a certain period of time.

I'm not sure what's the current policy re. deprecations, but I suppose there can be a transitional time (2 releases? 3?) during which usingonerror arg results inDeprecationWarning and then it's finally removed. Something like (not tested):

try:    ...exceptOSErroraserr:ifonerrorisnotNone:warnings.warn("'onerror' argument is deprecated and will be removed in XXX. Use 'onexc' argument instead.",DeprecationWarning)onerror(..., ...,sys.exc_info())ifonexcisnotNone:onerror(..., ...,err)

@iritkatriel
Copy link
MemberAuthor

existing code usingonerror will stop working

It will still work, because onerror gets wrapped by onexc in Lib/shutil.py:708-714.

Note that I didn't remove any tests, so all previous functionality is still there.

@giampaolo
Copy link
Contributor

Oh you're right sorry. I misread your patch.

iritkatriel reacted with thumbs up emoji

@bedevere-bot
Copy link

When you're done making the requested changes, leave the comment:I have made the requested changes; please review again.

@iritkatriel
Copy link
MemberAuthor

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@bedevere-bot
Copy link

Thanks for making the requested changes!

@giampaolo: please review the changes made to this pull request.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotwasm32-wasi 3.x has failed when building commitd51a6dc.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1046/builds/1597) and take a look at the build logs.
  4. Check if the failure is related to this commit (d51a6dc) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1046/builds/1597

Failed tests:

  • test_shutil

Failed subtests:

  • test_both_onerror_and_onexc - test.test_shutil.TestRmTree.test_both_onerror_and_onexc

Summary of the results of the build (if available):

== Tests result: FAILURE ==

325 tests OK.

10 slowest tests:

  • test_compile: 5 min 40 sec
  • test_math: 1 min 13 sec
  • test_tokenize: 48.5 sec
  • test_lib2to3: 40.6 sec
  • test_unparse: 33.8 sec
  • test_capi: 24.1 sec
  • test_hashlib: 24.1 sec
  • test_unicodedata: 18.5 sec
  • test_statistics: 13.9 sec
  • test_pickle: 13.7 sec

1 test failed:
test_shutil

107 tests skipped:
test__xxinterpchannels test__xxsubinterpreters test_asyncgen
test_asyncio test_bz2 test_clinic test_cmd_line
test_concurrent_futures test_contextlib_async test_ctypes
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_doctest
test_docxmlrpc test_dtrace test_embed test_epoll test_faulthandler
test_fcntl test_file_eintr test_fork1 test_ftplib test_gdb
test_grp test_gzip test_httplib test_httpservers test_idle
test_imaplib test_interpreters test_ioctl test_kqueue
test_launcher test_lzma test_mailbox test_mmap test_msilib
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_main_handling test_multiprocessing_spawn
test_nis test_openpty test_ossaudiodev test_pdb test_peg_generator
test_perf_profiler test_pipes test_poll test_poplib test_pty
test_pwd test_queue test_readline test_regrtest test_repl
test_resource test_select test_selectors test_smtplib test_smtpnet
test_socket test_socketserver test_spwd test_sqlite3 test_ssl
test_stable_abi_ctypes test_startfile test_subprocess
test_sys_settrace test_syslog test_tcl test_telnetlib test_thread
test_threadedtempfile test_threading test_threading_local test_tix
test_tkinter test_tools test_ttk test_ttk_textonly test_turtle
test_urllib test_urllib2 test_urllib2_localnet test_urllib2net
test_urllib_response test_urllibnet test_venv test_wait3
test_wait4 test_webbrowser test_winconsoleio test_winreg
test_winsound test_wmi test_wsgiref test_xmlrpc test_xmlrpc_net
test_xxlimited test_zipfile64 test_zipimport_support test_zlib
test_zoneinfo
0:08:09 load avg: 1.99
0:08:09 load avg: 1.99 Re-running failed tests is not supported with --python host runner option.

Total duration: 8 min 9 sec

Click to see traceback logs
Traceback (most recent call last):  File"/Lib/test/test_shutil.py", line532, intest_both_onerror_and_onexcself.assertTrue(onexc_called)AssertionError:False is not trueTraceback (most recent call last):  File"/Lib/shutil.py", line762, inrmtreereturn _rmtree_unsafe(path, onexc)^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"/Lib/shutil.py", line583, in_rmtree_unsafe    onexc(os.scandir, path, err)  File"/Lib/shutil.py", line580, in_rmtree_unsafewith os.scandir(path)as scandir_it:^^^^^^^^^^^^^^^^FileNotFoundError:[Errno 44] No such file or directory: '@test_42_tmpæ'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 Debian root 3.x has failed when building commitd51a6dc.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/4352) and take a look at the build logs.
  4. Check if the failure is related to this commit (d51a6dc) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/4352

Failed tests:

  • test_shutil

Failed subtests:

  • test_both_onerror_and_onexc - test.test_shutil.TestRmTree.test_both_onerror_and_onexc

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

411 tests OK.

10 slowest tests:

  • test_tools: 10 min 20 sec
  • test_math: 5 min 20 sec
  • test_multiprocessing_spawn: 3 min 57 sec
  • test_compile: 3 min 33 sec
  • test_concurrent_futures: 2 min 56 sec
  • test_asyncio: 2 min 40 sec
  • test_tokenize: 2 min 6 sec
  • test_unparse: 2 min 6 sec
  • test_lib2to3: 1 min 52 sec
  • test_capi: 1 min 52 sec

1 test failed:
test_shutil

21 tests skipped:
test_dbm_ndbm test_devpoll test_gdb test_idle test_ioctl
test_kqueue test_launcher test_msilib test_peg_generator
test_startfile test_tcl test_tix test_tkinter test_ttk
test_ttk_textonly test_turtle test_winconsoleio test_winreg
test_winsound test_wmi test_zipfile64

1 re-run test:
test_shutil

Total duration: 39 min 34 sec

Click to see traceback logs
Traceback (most recent call last):  File"/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line725, inrmtree    onexc(os.lstat, path, err)  File"/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line723, inrmtree    orig_st= os.lstat(path,dir_fd=dir_fd)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^FileNotFoundError:[Errno 2] No such file or directory: '@test_3079655_tmpæ'Traceback (most recent call last):  File"/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line725, inrmtree    onexc(os.lstat, path, err)  File"/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line723, inrmtree    orig_st= os.lstat(path,dir_fd=dir_fd)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^FileNotFoundError:[Errno 2] No such file or directory: '@test_3084007_tmpæ'Traceback (most recent call last):  File"/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_shutil.py", line532, intest_both_onerror_and_onexcself.assertTrue(onexc_called)AssertionError:False is not true

@giampaolo
Copy link
Contributor

@iritkatriel looks like this broke some build bots.

Traceback (most recent call last):  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line 725, in rmtree    onexc(os.lstat, path, err)  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line 723, in rmtree    orig_st = os.lstat(path, dir_fd=dir_fd)              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^FileNotFoundError: [Errno 2] No such file or directory: '@test_3079655_tmpæ'Traceback (most recent call last):  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line 725, in rmtree    onexc(os.lstat, path, err)  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/shutil.py", line 723, in rmtree    orig_st = os.lstat(path, dir_fd=dir_fd)              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^FileNotFoundError: [Errno 2] No such file or directory: '@test_3084007_tmpæ'Traceback (most recent call last):  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_shutil.py", line 532, in test_both_onerror_and_onexc    self.assertTrue(onexc_called)AssertionError: False is not true
iritkatriel reacted with eyes emoji

@iritkatriel
Copy link
MemberAuthor

I see what I did - I copied part of this test from another test, but I didn't copy the skip instructions that make it not run where it doesn't work. Will fix.

@terryjreedy
Copy link
Member

Ignore my experiment with the 1st buildbot report.

@barneygale
Copy link
Contributor

barneygale commentedApr 3, 2023
edited
Loading

Hey, why doesonexc accept three arguments (function, path, exc) rather than just the exception?

If it accepted only one argument, then:

  1. It would match theonerror argument toos.walk() andos.fwalk()
  2. Users could still get the file path that failed viaexc.filename
  3. Users could still print the exception/traceback to display exactly what failed

The only thing lost would be thefunction value, which seems like an unreasonably detailed insight into the internals ofrmtree(). In fact,rmtree() already bends the truth about the function that actually failed - for exampleos.path.islink is given if we fail a junction check on Windows. And no internal code looks at the value offunction.

@iritkatriel
Copy link
MemberAuthor

Hey, why doesonexc accept three arguments (function, path, exc) rather than just the exception?

Because that's what onerror did. This PR was just about replacing exc_info by exc. Feel free to suggest additional changes in a separate issue.

@iritkatrieliritkatriel deleted the rmtree branchApril 3, 2023 17:40
@barneygale
Copy link
Contributor

Righto, thanks. I've logged#103218.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@giampaologiampaologiampaolo approved these changes

Assignees
No one assigned
Labels
stdlibPython modules in the Lib dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

add option for error callback of to shutil.rmtree to accept exception rather than exc_info
5 participants
@iritkatriel@giampaolo@bedevere-bot@terryjreedy@barneygale

[8]ページ先頭

©2009-2025 Movatter.jp