➜
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2013-12-02 19:36 bygergely.erdelyi, last changed2022-04-11 14:57 byadmin. This issue is nowclosed.
| Pull Requests | |||
|---|---|---|---|
| URL | Status | Linked | Edit |
| PR 14081 | merged | ZackerySpytz,2019-06-14 14:33 | |
| PR 14087 | merged | miss-islington,2019-06-14 15:55 | |
| PR 14088 | merged | miss-islington,2019-06-14 15:55 | |
| Messages (9) | |||
|---|---|---|---|
| msg205045 -(view) | Author: Gergely Erdélyi (gergely.erdelyi) | Date: 2013-12-02 19:36 | |
create_unicode_buffer() fails on Windows if the initializer string contains unicode code points outside of the Basic Multilingual Plane and an explicit length is not specified.The problem appears to be rooted in the fact that, sincePEP 393, len() returns the number of code points, which does not always correspond to the number of 16-bit wchar words needed for the encoding on Windows. Because of that, the preallocated c_wchar buffer will be too short for the UTF-16 string.The following small snippet demonstrates the problem:from ctypes import create_unicode_bufferb = create_unicode_buffer("\U00028318\U00028319")print(b) File "c:\Python33\lib\ctypes\__init__.py", line 294, in create_unicode_buffer buf.value = initValueError: string too long | |||
| msg228405 -(view) | Author: Mark Lawrence (BreamoreBoy)* | Date: 2014-10-03 22:43 | |
I can confirm that this problem still exists so can someone take a look please, thanks. | |||
| msg228424 -(view) | Author: Eryk Sun (eryksun)* | Date: 2014-10-04 02:24 | |
When sizeof(c_wchar) == 2, it can just count the number of non-BMP ordinals in the string. Another approach would be to use size = pythonapi.PyUnicode_AsWideChar(init, None, 0), but then the whole function may as well be implemented in the _ctypes extension module. | |||
| msg330583 -(view) | Author: Leonard de Ruijter (Leonard de Ruijter) | Date: 2018-11-28 09:18 | |
I'm still able to reproduce this issue with ctypes under Python 3.7.0 | |||
| msg345596 -(view) | Author: Zackery Spytz (ZackerySpytz)* | Date: 2019-06-14 14:36 | |
I have created a pull request for this issue. Please take a look. | |||
| msg345601 -(view) | Author: STINNER Victor (vstinner)* | Date: 2019-06-14 15:54 | |
New changeset9765efcb39fc03d5b1abec3924388974470a8bd5 by Victor Stinner (Zackery Spytz) in branch 'master':bpo-19865: ctypes.create_unicode_buffer() supports non-BMP strings on Windows (GH-14081)https://github.com/python/cpython/commit/9765efcb39fc03d5b1abec3924388974470a8bd5 | |||
| msg345609 -(view) | Author: miss-islington (miss-islington) | Date: 2019-06-14 16:30 | |
New changeset0b592d513b073cd3a4ba7632907c25b8282f15ce by Miss Islington (bot) in branch '3.7':bpo-19865: ctypes.create_unicode_buffer() supports non-BMP strings on Windows (GH-14081)https://github.com/python/cpython/commit/0b592d513b073cd3a4ba7632907c25b8282f15ce | |||
| msg345610 -(view) | Author: miss-islington (miss-islington) | Date: 2019-06-14 16:43 | |
New changesetb0f6fa8d7d4c6d8263094124df9ef9cf816bbed6 by Miss Islington (bot) in branch '3.8':bpo-19865: ctypes.create_unicode_buffer() supports non-BMP strings on Windows (GH-14081)https://github.com/python/cpython/commit/b0f6fa8d7d4c6d8263094124df9ef9cf816bbed6 | |||
| msg345611 -(view) | Author: STINNER Victor (vstinner)* | Date: 2019-06-14 16:54 | |
Thanks Zackery Spytz for the fix. Thanks Gergely Erdélyi for the bug report! Sorry for the long delay. | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:57:54 | admin | set | github: 64064 |
| 2019-07-10 05:13:07 | eryksun | link | issue37536 superseder |
| 2019-06-14 16:54:21 | vstinner | set | status: open -> closed resolution: fixed messages: +msg345611 stage: patch review -> resolved |
| 2019-06-14 16:43:26 | miss-islington | set | messages: +msg345610 |
| 2019-06-14 16:30:30 | miss-islington | set | nosy: +miss-islington messages: +msg345609 |
| 2019-06-14 15:55:09 | miss-islington | set | pull_requests: +pull_request13944 |
| 2019-06-14 15:55:01 | miss-islington | set | pull_requests: +pull_request13943 |
| 2019-06-14 15:54:04 | vstinner | set | messages: +msg345601 |
| 2019-06-14 14:36:53 | ZackerySpytz | set | nosy: +ZackerySpytz messages: +msg345596 versions: + Python 3.9, - Python 3.4, Python 3.5, Python 3.6 |
| 2019-06-14 14:33:31 | ZackerySpytz | set | keywords: +patch stage: patch review pull_requests: +pull_request13940 |
| 2018-11-28 13:20:06 | josh.r | set | keywords: -3.2regression |
| 2018-11-28 13:18:55 | josh.r | set | keywords: +3.2regression versions: + Python 3.6, Python 3.7, Python 3.8 |
| 2018-11-28 12:01:58 | BreamoreBoy | set | nosy: -BreamoreBoy |
| 2018-11-28 09:18:48 | Leonard de Ruijter | set | nosy: +Leonard de Ruijter messages: +msg330583 |
| 2014-10-04 02:24:52 | eryksun | set | nosy: +eryksun messages: +msg228424 |
| 2014-10-03 22:43:19 | BreamoreBoy | set | nosy: +BreamoreBoy messages: +msg228405 versions: + Python 3.5, - Python 3.3 |
| 2013-12-02 22:37:50 | pitrou | set | nosy: +amaury.forgeotdarc,belopolsky,vstinner,meador.inge |
| 2013-12-02 20:14:41 | serhiy.storchaka | set | type: crash -> behavior |
| 2013-12-02 19:36:45 | gergely.erdelyi | create | |