- Notifications
You must be signed in to change notification settings - Fork13.3k
Resolve Windows path problems#8860
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation
Strange problem creating files with file paths with spaces.Added try/except catch on main to commit print buffer for context of traceback.
Continue using "utf-8" to read or write "C/C++" source filess.Tested on Windows 10 (en-US) with Arduino IDE 2.0.3 Under anaccount with a diacritic charcter in the user ID path.Needs testing on Japanese Windows
mcspr commentedFeb 21, 2023
>>>locale.getdefaultlocale()<stdin>:1:DeprecationWarning:Usesetlocale(),getencoding()andgetlocale()instead('en_US','UTF-8') For sizes .py, I thought POpen univeral_newlines should already default to locale setting by wrapping input with TextIOWrapper. I guess we do need to source some fixed locale / encoding value by external means, but what are values for the rest of stdlib functions? Why it breaks on buildopts we understand, but where does the size .py issue come from? What are default locale & encoding values for your environment from sys / locale? Some external references, since I am pretty sure both manage to work ok withmost installations. PIP change was a pretty recent one, though https://github.com/platformio/platformio-core/blob/ec5bf1b5e78cf6baa180e8efb51db4e9a30e5546/platformio/compat.py, hover 'get_filesystem_encoding' and 'get_locale_encoding' and see 'References' |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
| ifnotos.path.exists(include_fqfn): | ||
| # If file is missing, we need an place holder | ||
| withopen(include_fqfn,'w',encoding="utf-8"): | ||
| withopen(include_fqfn,'w',encoding=default_encoding): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
missed one odd encoding=
| withopen(include_fqfn,'w',encoding=default_encoding): | |
| withopen(include_fqfn,'w'): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What you are suggesting is I didn't need any of thisget_encoding() logic - tangent I went on. In the absence of the-X utf8, I just needed to selectively remove all the, encoding="utf-8" options where I needed the system encoding to dominate.
Since merged - do I push an update or need a new PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I just needed to selectively remove all the , encoding="utf-8" options where I needed the system encoding to dominate.
nothing is written to file, so no need for encoding= to possibly confuse future reader(s)
python won't re-encode stuff, this is just the thing for read / write through text io wrapper
in absence of -Xutf-8, text io selects something default aka depends on version
https://github.com/python/cpython/blob/cbd192b6ff137aec25913cc80b2bf7bf1ef3755b/Modules/_io/textio.c#L1102-L1141 (our bundled version)
https://github.com/python/cpython/blob/ddb65c47b1066fec8ce7a674ebb88b167f417d84/Modules/_io/textio.c#L1126-L1132 (3.11 on any recent installation)
utf-8 output is ok, reading cli / writing stuff for external tools should have system encoding
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nothing is written to file, so no need for encoding= to possibly confuse future reader(s)
Just for clarification when creatingbuild.opt, we are readingutf-8 from dot-h files and writing with system encoding tobuild.opt otherwise I can get a file not found for a-include ... if the file path contains diacritics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
also nit:
diacritics
any time we have characters from codepage different from ansii e.g.C:\Users\Максім in cp1251
just my guess that GCC probably does not use win32_wfopen and widechar, so we have to use exact codepage bytes for filepath representation so system exec (that also does not encode / decode anything in args) for GCC gets proper paths
path and text have different code sections dealing with them, just to denote the difference
(I am gonna go readPython PEP 529,https://vstinner.github.io/python36-utf8-windows.html series and cpythonunicodeobject.c)
Moved up os.makedirs for ./core/ directory.Strange problem creating files with file paths with spaces.Strange that "create file" would work when the path did not contain spaces and the last folder of the path hadn't been created.Added try/except on main to commit print buffer on traceback for context.Additional issues with diacritics and locale character encoding for shell vs source code.build.opt is written with the same encoding as the shell; however, the data read from the Sketch.ino.global.h is UTF-8.Tested on Windows 10 (en-US) with Arduino IDE 2.0.3 Under anaccount with a diacritic character in the user ID path.Needs testing on Japanese Windows
Uh oh!
There was an error while loading.Please reload this page.
Moved up
os.makedirsfor./core/directory.Strange problem creating files with file paths with spaces.
Strange that "create file" would work when the path did not contain spaces and the last folder of the path hadn't been created.
Added try/except on main to commit print buffer on traceback for context.
Additional issues with diacritics and locale character encoding for shell vs source code.
build.optis written with the same encoding as the shell; however, the data read from theSketch.ino.global.hisUTF-8.shouldresolve#8856 (comment)