Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Resolve Windows path problems#8860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
mcspr merged 8 commits intoesp8266:masterfrommhightower83:pr-mkbuildopt-path
Feb 22, 2023

Conversation

@mhightower83
Copy link
Contributor

@mhightower83mhightower83 commentedFeb 17, 2023
edited
Loading

Moved upos.makedirs for./core/ directory.
Strange problem creating files with file paths with spaces.
Strange that "create file" would work when the path did not contain spaces and the last folder of the path hadn't been created.

Added try/except on main to commit print buffer on traceback for context.

Additional issues with diacritics and locale character encoding for shell vs source code.
build.opt is written with the same encoding as the shell; however, the data read from theSketch.ino.global.h isUTF-8.

shouldresolve#8856 (comment)

Strange problem creating files with file paths with spaces.Added try/except catch on main to commit print buffer for context of traceback.
Continue using "utf-8" to read or write "C/C++" source filess.Tested on Windows 10 (en-US) with Arduino IDE 2.0.3 Under anaccount with a diacritic charcter in the user ID path.Needs testing on Japanese Windows
@mhightower83mhightower83 changed the titleResolve Windows paths with spaces issueResolve Windows path problemsFeb 21, 2023
@mcspr
Copy link
Collaborator

>>>locale.getdefaultlocale()<stdin>:1:DeprecationWarning:Usesetlocale(),getencoding()andgetlocale()instead('en_US','UTF-8')

For sizes .py, I thought POpen univeral_newlines should already default to locale setting by wrapping input with TextIOWrapper.
https://docs.python.org/3/library/io.html#text-encoding

I guess we do need to source some fixed locale / encoding value by external means, but what are values for the rest of stdlib functions? Why it breaks on buildopts we understand, but where does the size .py issue come from? What are default locale & encoding values for your environment from sys / locale?


Some external references, since I am pretty sure both manage to work ok withmost installations. PIP change was a pretty recent one, though

https://github.com/platformio/platformio-core/blob/ec5bf1b5e78cf6baa180e8efb51db4e9a30e5546/platformio/compat.py, hover 'get_filesystem_encoding' and 'get_locale_encoding' and see 'References'
https://github.com/pypa/pip/search?q=sys.getfilesystemencoding
https://github.com/pypa/pip/search?q=locale.getpreferredencoding

@mcsprmcspr added this to the3.1.2 milestoneFeb 21, 2023
mhightower83and others added3 commitsFebruary 22, 2023 07:25
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
Co-authored-by: Max Prokhorov <prokhorov.max@outlook.com>
@mcsprmcspr merged commit71a51b1 intoesp8266:masterFeb 22, 2023
ifnotos.path.exists(include_fqfn):
# If file is missing, we need an place holder
withopen(include_fqfn,'w',encoding="utf-8"):
withopen(include_fqfn,'w',encoding=default_encoding):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

missed one odd encoding=

Suggested change
withopen(include_fqfn,'w',encoding=default_encoding):
withopen(include_fqfn,'w'):

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

What you are suggesting is I didn't need any of thisget_encoding() logic - tangent I went on. In the absence of the-X utf8, I just needed to selectively remove all the, encoding="utf-8" options where I needed the system encoding to dominate.

Since merged - do I push an update or need a new PR?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I just needed to selectively remove all the , encoding="utf-8" options where I needed the system encoding to dominate.

nothing is written to file, so no need for encoding= to possibly confuse future reader(s)
python won't re-encode stuff, this is just the thing for read / write through text io wrapper

in absence of -Xutf-8, text io selects something default aka depends on version
https://github.com/python/cpython/blob/cbd192b6ff137aec25913cc80b2bf7bf1ef3755b/Modules/_io/textio.c#L1102-L1141 (our bundled version)
https://github.com/python/cpython/blob/ddb65c47b1066fec8ce7a674ebb88b167f417d84/Modules/_io/textio.c#L1126-L1132 (3.11 on any recent installation)

utf-8 output is ok, reading cli / writing stuff for external tools should have system encoding

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

nothing is written to file, so no need for encoding= to possibly confuse future reader(s)

Just for clarification when creatingbuild.opt, we are readingutf-8 from dot-h files and writing with system encoding tobuild.opt otherwise I can get a file not found for a-include ... if the file path contains diacritics.

Copy link
Collaborator

@mcsprmcsprFeb 22, 2023
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

also nit:

diacritics

any time we have characters from codepage different from ansii e.g.C:\Users\Максім in cp1251
just my guess that GCC probably does not use win32_wfopen and widechar, so we have to use exact codepage bytes for filepath representation so system exec (that also does not encode / decode anything in args) for GCC gets proper paths

path and text have different code sections dealing with them, just to denote the difference

(I am gonna go readPython PEP 529,https://vstinner.github.io/python36-utf8-windows.html series and cpythonunicodeobject.c)

hasenradball pushed a commit to hasenradball/Arduino that referenced this pull requestNov 18, 2024
Moved up os.makedirs for ./core/ directory.Strange problem creating files with file paths with spaces.Strange that "create file" would work when the path did not contain spaces and the last folder of the path hadn't been created.Added try/except on main to commit print buffer on traceback for context.Additional issues with diacritics and locale character encoding for shell vs source code.build.opt is written with the same encoding as the shell; however, the data read from the Sketch.ino.global.h is UTF-8.Tested on Windows 10 (en-US) with Arduino IDE 2.0.3 Under anaccount with a diacritic character in the user ID path.Needs testing on Japanese Windows
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@mcsprmcsprmcspr left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

3.1.2

Development

Successfully merging this pull request may close these issues.

Boards Manager upgrade ESP8266 Core 3.0.2 to 3.1.0/3.1.1 fails to compile - error missing CommonHFile.h

2 participants

@mhightower83@mcspr

[8]ページ先頭

©2009-2025 Movatter.jp