Example:vstinner@apu$ ./pythonPython 3.8.0a0 (heads/master-dirty:bcd3a1a18d, Jun 23 2018, 10:31:03) [GCC 8.1.1 20180502 (Red Hat 8.1.1-1)] on linuxType "help", "copyright", "credits" or "license" for more information.>>> import locale>>> locale.str(2.5)'2.5'>>> '{:n}'.format(2.5)'2.5'>>> locale.setlocale(locale.LC_ALL, '')'fr_FR.UTF-8'>>> locale.str(2.5)'2,5'>>> '{:n}'.format(2.5)python:Objects/unicodeobject.c:474: _PyUnicode_CheckConsistency: Assertion `maxchar < 128' failed.Aborted (core dumped)Another example:vstinner@apu$ ./pythonPython 3.8.0a0 (heads/master-dirty:bcd3a1a18d, Jun 23 2018, 10:31:03) >>> import locale; locale.setlocale(locale.LC_ALL, '')'fr_FR.UTF-8'>>> (2.5).__format__('n')python:Objects/unicodeobject.c:474: _PyUnicode_CheckConsistency: Assertion `maxchar < 128' failed.Aborted (core dumped)Result of my system Python 3.6 of Fedora 28:vstinner@apu$ python3Python 3.6.5 (default, Mar 29 2018, 18:20:46) [GCC 8.0.1 20180317 (Red Hat 8.0.1-0.19)] on linuxType "help", "copyright", "credits" or "license" for more information.>>> import locale>>> locale.str(2.5)'2.5'>>> '{:n}'.format(2.5)'2.5'>>> locale.setlocale(locale.LC_ALL, '')'fr_FR.UTF-8'>>> locale.str(2.5)'2,5'>>> '{:n}'.format(2.5)'°,5'>>> '{:n}'.format(3.5)'°,5'>>> '{:n}'.format(33.5)'°\x18,5'>>> '{:n}'.format(333.5)'°\x186,5'

Aha, the problem occurs when the thousands separator code point is greater than 255.On my Fedora 28 (glibc 2.27), it's U+202f:vstinner@apu$ ./pythonPython 3.8.0a0 (heads/master-dirty:492572715a, Jun 28 2018, 00:18:54) >>> import locale>>> locale.setlocale(locale.LC_ALL, '')'fr_FR.UTF-8'>>> locale.localeconv()['thousands_sep']'\u202f'The bug is in _PyUnicode_InsertThousandsGrouping(): if thousands_sep kind is different than unicode kind, "data = _PyUnicode_AsKind(unicode, thousands_sep_kind);" is used, but later this memory is released. So the function writes into a temporary buffer which is then released. It doesn't work...It seems like I introduced the regression 6 years ago inbpo-13706:commit90f50d4df9e21093f006427fd7ed11a0d704f792Author: Victor Stinner <victor.stinner@haypocalc.com>Date:   Fri Feb 24 01:44:47 2012 +0100    Issue#13706: Fix format(float, "n") for locale with non-ASCII decimal point (e.g. ps_aF)

> The bug is in _PyUnicode_InsertThousandsGrouping()This function should take a _PyUnicodeWriter as input, not a PyUnicodeObject.

Minimum reproducer, on Fedora 29:import localelocale.setlocale(locale.LC_ALL, 'fr_FR.UTF-8')print(ascii('{:n}'.format(1.5)))Current result:'H,5'Output withPR 10623:'1,5'

My private test suite for locales:https://github.com/vstinner/misc/blob/master/python/test_all_locales.pySince each platform uses a different locale database, it's hard to write reliable portable and future-proof tests :-(My tests only work on specific Windows, macOS, FreeBSD and Linux versions.

New changeset59423e3ddd736387cef8f7632c71954c1859bed0 by Victor Stinner in branch 'master':bpo-33954: Fix _PyUnicode_InsertThousandsGrouping() (GH-10623)https://github.com/python/cpython/commit/59423e3ddd736387cef8f7632c71954c1859bed0

New changeset6f5fa1b4be735159e964906ab608dc467476e47c by Victor Stinner in branch '3.7':bpo-33954: Fix _PyUnicode_InsertThousandsGrouping() (GH-10623) (GH-10718)https://github.com/python/cpython/commit/6f5fa1b4be735159e964906ab608dc467476e47c

New changesetfc4a44b0c3a69390eca4680d89c2ae5fe967f882 by Victor Stinner in branch '3.6':bpo-33954: Fix _PyUnicode_InsertThousandsGrouping() (GH-10623) (GH-10718) (GH-10720)https://github.com/python/cpython/commit/fc4a44b0c3a69390eca4680d89c2ae5fe967f882

Python 3.6, 3.7 and master have been fixed.

Objects/unicodeobject.c: In function ‘_PyUnicode_FastFill’:Objects/unicodeobject.c:10126:24: warning: passing argument 2 of ‘unicode_fill’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]     unicode_fill(kind, data, fill_char, start, length);                        ^~~~Objects/unicodeobject.c:224:1: note: expected ‘void *’ but argument is of type ‘const void *’ unicode_fill(enum PyUnicode_Kind kind, void *data, Py_UCS4 value, ^~~~~~~~~~~~In file included from /usr/include/wchar.h:850:0,                 from ./Include/unicodeobject.h:97,                 from ./Include/Python.h:87,                 from ./Modules/getpath.c:3:

New changeset163403a63e9272fcd14707e344122c2e3c5e0244 by Victor Stinner in branch 'master':bpo-33954: Fix compiler warning in _PyUnicode_FastFill() (GH-10737)https://github.com/python/cpython/commit/163403a63e9272fcd14707e344122c2e3c5e0244

New changeset7f9fb0f34555641722be5468b7fbd53dd3c0f1e2 by Victor Stinner in branch '3.7':bpo-33954: Rewrite FILL() macro of unicodeobject.c (GH-10738)https://github.com/python/cpython/commit/7f9fb0f34555641722be5468b7fbd53dd3c0f1e2

New changeset54fa83e0a3f3b077763cb50705d7a7dbe4a40a4a by Victor Stinner in branch '3.6':bpo-33954: Rewrite FILL() macro of unicodeobject.c (GH-10740)https://github.com/python/cpython/commit/54fa83e0a3f3b077763cb50705d7a7dbe4a40a4a

Serhiy Storchaka:"""Objects/unicodeobject.c: In function ‘_PyUnicode_FastFill’:Objects/unicodeobject.c:10126:24: warning: passing argument 2 of ‘unicode_fill’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]"""Yeah, I was aware of the warning. I had a local branch but I started to make many unrelated changes and so I lost track of the issue :-)The warning should now be fixed. Thanks for the report.Note: it was a real bug which exists since at least Python 3.6 ;-)