Movatterモバイル変換


[0]ホーム

URL:


homepage

Issue31380

This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title:test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS
Type:behaviorStage:resolved
Components:macOS, TestsVersions:Python 3.7, Python 3.6
process
Status:closedResolution:fixed
Dependencies:Superseder:
Assigned To:Nosy List: howarthjw, mattbillenstein, ned.deily, ronaldoussoren, serhiy.storchaka, vstinner
Priority:normalKeywords:patch

Created on2017-09-07 09:58 byhowarthjw, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.

Pull Requests
URLStatusLinkedEdit
PR 4720mergedned.deily,2017-12-05 04:25
PR 4721mergedpython-dev,2017-12-05 04:43
Messages (9)
msg301580 -(view)Author: Jack Howarth (howarthjw)Date: 2017-09-07 09:58
The Python 3.x test suite produces a new regression on macOS 10.13 under the new APFS filesystem when executing the test_undecodable_filename() test fromLib/test/test_httpservers.py. The error appears as...======================================================================ERROR: test_undecodable_filename(test.test_httpservers.SimpleHTTPServerTestCase)----------------------------------------------------------------------Traceback (most recent call last):  File "/Users/howarth/Python-3.6.2/Lib/test/support/__init__.py",line 601, in wrapper    return func(*args, **kw)  File "/Users/howarth/Python-3.6.2/Lib/test/test_httpservers.py",line 380, in test_undecodable_filename    with open(os.path.join(self.tempdir, filename), 'wb') as f:OSError: [Errno 92] Illegal byte sequence:'/var/folders/7g/1x2rsy3j40n1pydq931hzlkm0000gn/T/tmpomp1r36b/@test_58317_tmp\udce7w\udcf0.txt'----------------------------------------------------------------------Ran 59 tests in 3.207sSeehttps://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/FAQ/FAQ.html"""APFS accepts only valid UTF-8 encoded filenames for creation, and preserves both case and normalization of the filename on disk in all variants..Some differences between how APFS and HFS+ handle filenames include the following:...        • APFS doesn’t allow files to be created with filenames that contain unassigned codepoints in the Unicode 9.0 standard, whereas HFS+ does.
msg306006 -(view)Author: Matt Billenstein (mattbillenstein)Date: 2017-11-10 06:20
Reflected on the High Sierra buildbot now:http://buildbot.python.org/all/#/builders/14/builds/162
msg307106 -(view)Author: Ned Deily (ned.deily)*(Python committer)Date: 2017-11-28 02:41
More background: APFS became available in macOS 10.12 although not used by default.  But with the current macOS 10.13 (High Sierra), root file systems on certain devices, such as SSDs, are automatically migrated from HFS+ to APFS so are much more likely to be seen starting in 10.13 and, in general, there is no easy way of knowing what type of FS a particular file resides on.# TMPDIR on APFS volume$ ./bin/python3.7 -m test test_httpserversRun tests sequentially0:00:00 load avg: 1.43 [1/1] test_httpserverstest test_httpservers failed -- Traceback (most recent call last):  File ".../lib/python3.7/test/support/__init__.py", line 598, in wrapper    return func(*args, **kw)  File ".../lib/python3.7/test/test_httpservers.py", line 395, in test_undecodable_filename    with open(os.path.join(self.tempdir, filename), 'wb') as f:OSError: [Errno 92] Illegal byte sequence: '/var/folders/sn/0m4rnbyj2z1byjs68sz838300000gn/T/tmpqeg89k76/@test_64005_tmp\udce7w\udcf0.txt'test_httpservers failed1 test failed:    test_httpserversTotal duration: 3 secTests result: FAILURE# TMPDIR redirected to HFS+ volume$ TMPDIR=/Volumes/euterpe/a ./bin/python3.7 -m test test_httpserversRun tests sequentially0:00:00 load avg: 1.28 [1/1] test_httpservers1 test OK.Total duration: 3 secTests result: SUCCESS@vstinner, as the expert in this area, what do you suggest as the best way to deal with this?  In support.__init__.py?  Or test_httpservers itself?
msg307109 -(view)Author: Serhiy Storchaka (serhiy.storchaka)*(Python committer)Date: 2017-11-28 07:38
I suggest to catch an error and skip the test if OSError with the specific errno has been raised.
msg307114 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2017-11-28 09:54
> @vstinner, as the expert in this area, what do you suggest as the best way to deal with this?  In support.__init__.py?  Or test_httpservers itself?It's tricky when a filename is valid or not depending on the path, depending on the filesystem. The tempfile module checks if O_TMPFILE is supported: same issue, it depends on the path, and the user can configure the path. So we check each time if O_TMPFILE works or not.
msg307116 -(view)Author: Ronald Oussoren (ronaldoussoren)*(Python committer)Date: 2017-11-28 10:25
Another option is to skip this test unconditionally on macOS when using APFS, the test is already skipped on win32 and tests a generic codepath where only testing on Linux should be fine.BTW. It is fairly easy to detect if a path is on APFS:```import subprocess  def is_apfs(path):    lines = subprocess.check_output(['df', path]).decode('utf-8').splitlines()    mountpoint = lines[1].split(None, 1)[0]    lines = subprocess.check_output(['mount']).decode('utf-8').splitlines()    for ln in lines:        path = ln.split(None, 1)[0]        if path == mountpoint:            return '(apfs' in ln    return False```
msg307641 -(view)Author: Ned Deily (ned.deily)*(Python committer)Date: 2017-12-05 04:42
New changesetb3edde8dd44c878e9f039a2165d00ff075157d4b by Ned Deily in branch 'master':bpo-31380: Skip test_httpservers test_undecodable_file on macOS. (#4720)https://github.com/python/cpython/commit/b3edde8dd44c878e9f039a2165d00ff075157d4b
msg307643 -(view)Author: Ned Deily (ned.deily)*(Python committer)Date: 2017-12-05 05:03
New changesetd9cadc5f597e5966132c9249f6c7ff0ed1eba0cb by Ned Deily (Miss Islington (bot)) in branch '3.6':[3.6]bpo-31380: Skip test_httpservers test_undecodable_file on macOS. (GH-4720) (#4721)https://github.com/python/cpython/commit/d9cadc5f597e5966132c9249f6c7ff0ed1eba0cb
msg307644 -(view)Author: Ned Deily (ned.deily)*(Python committer)Date: 2017-12-05 05:09
I've chosen to unconditionally skip test_undecodable_filename on macOS; it was already skipped for older macOS systems and I think it would be a mistake to test for a specific error code that could change in later releases.  As Ronald points out, testing on Linux should be sufficient.  Thanks everyone for your input.  Fixed in 3.6.4 and 3.7.0.
History
DateUserActionArgs
2022-04-11 14:58:52adminsetgithub: 75561
2017-12-05 05:09:03ned.deilysetstatus: open -> closed
resolution: fixed
messages: +msg307644

stage: patch review -> resolved
2017-12-05 05:03:31ned.deilysetmessages: +msg307643
2017-12-05 04:43:12python-devsetpull_requests: +pull_request4630
2017-12-05 04:42:04ned.deilysetmessages: +msg307641
2017-12-05 04:25:25ned.deilysetkeywords: +patch
stage: patch review
pull_requests: +pull_request4629
2017-11-28 10:25:36ronaldoussorensetmessages: +msg307116
2017-11-28 09:54:15vstinnersetmessages: +msg307114
2017-11-28 07:38:06serhiy.storchakasetnosy: +serhiy.storchaka
messages: +msg307109
2017-11-28 02:41:30ned.deilysetmessages: +msg307106
2017-11-17 14:51:49vstinnersetnosy: +vstinner
2017-11-17 14:51:44vstinnerlinkissue32061 superseder
2017-11-10 09:46:45serhiy.storchakasettype: crash -> behavior
components: + Tests
versions: - Python 3.3, Python 3.4, Python 3.5
2017-11-10 09:45:27serhiy.storchakalinkissue32000 superseder
2017-11-10 06:20:37mattbillensteinsetnosy: +mattbillenstein
messages: +msg306006
2017-09-07 09:58:29howarthjwcreate
Supported byThe Python Software Foundation,
Powered byRoundup
Copyright © 1990-2022,Python Software Foundation
Legal Statements

[8]ページ先頭

©2009-2026 Movatter.jp