Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
gh-102255: Improve build support on xbox#102256
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Eryk Sun <eryksun@gmail.com>
zooba commentedMar 6, 2023
This looks like it's ready, or very close.@maxbachmann,@eryksun is there anything left to do here? I don't mind it being done in future PRs if they're necessary. |
maxbachmann commentedMar 6, 2023
I am done with everything I planned for this PR. As suggested by@eryksun I plan to extend the |
zooba commentedMar 6, 2023
Give that a new issue as well. We really ought to add the platform to |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Co-authored-by: Eryk Sun <eryksun@gmail.com>
Uh oh!
There was an error while loading.Please reload this page.
Python/fileutils.c Outdated
| size_tfile_len=relfile ?wcslen(relfile) :0; | ||
| /* path is at max dirname + filename + backslash + \0 */ | ||
| size_tnew_len=dir_len+file_len+2; | ||
| if (bufsize >=MAXPATHLEN||new_len>bufsize) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The value ofbufsize isn't limited. The result length may be limited. IfPATHCCH_ALLOW_LONG_PATHS isn't set inflags, andnew_len >= MAX_PATH (notMAXPATHLEN), the returned error should beHRESULT_FROM_WIN32(ERROR_FILENAME_EXCED_RANGE).
Else ifnew_len is greater thanbufsize, the returned error should beHRESULT_FROM_WIN32(ERROR_INSUFFICIENT_BUFFER).
PATHCCH_ALLOW_LONG_PATHS also determines whether to convert to or from an extended path, depending on whether the current process supports long paths (Python 3.6+ does). But it's okay to omit that behavior forGAMES.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Oh yeah, we shouldn't have references to MAXPATHLEN in here.
I'm not so concerned about error codes. They're only going to be reported to users, and really we ought to be getting the buffer right ourselves before calling.
I think it's fine for this implementation to assumePATHCCH_ALLOW_LONG_PATHS. Don't worry about mimicking every possible behaviour of the original.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
| if (bufsize >=MAXPATHLEN||new_len>bufsize) { | |
| if (new_len>bufsize) { |
Don't worry about mimicking every possible behaviour of the original.
I agree, if getting the correct behavior is excessively complicated with no tangible benefit. What about my other comment on this function? The wayPathCchStripToRoot() is used, whenrelfile is a rooted path, leads to incorrect and inconsistent behavior. If I know I'm combining a rooted path, I may use a smaller buffer because I know that most ofdirname won't be copied -- just the drive part. The current implementation requires using a buffer that's big enough to holddirname,relname and a joining backslash. OTOH, ifrelname has a drive or UNC share and thus replacesdirname, then the length of the passed-indirname is ignored.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Firstly, the suggested change is good.
I'm not particularly worried about the case whererelname would shorten the overall drive. It seems something youcould take advantage of, but generally we don't control either half of the paths that will come in here, which means the caller is going to have to assume worst case of plain-old concatenation.
Maybe just add a comment like/* assuming worst case result length, so our caller should as well */.
The only reasonable alternative (without complicating everything) would involve modifying the result buffer before potentially returning an error, which I think is worse. I'm pretty sure the original API uses temporary buffers internally, but there's no real need to go there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes I would assume the implementation to use something likePathAllocCombine internally
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If we really wanted to I could come up with an implementation that behaves closer toPathCchCombineEx. I was simply not sure it was worth the complexity (I kind of hope that at some point microsoft makes these APIs available on the xbox, since there does not appear to be any inherent reason to provide them on every platform except the xbox)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The only reasonable alternative (without complicating everything) would involve modifying the result buffer before potentially returning an error, which I think is worse. I'm pretty sure the original API uses temporary buffers internally, but there's no real need to go there.
It would calculatedir_len based onPathCchSkipRoot() ifrelfile begins with a backslash. Then it's the same initial test based onnew_len = dir_len + file_len + 2, requiringnew_len < bufsize. Copyingdirname would now usewcsncpy(buffer, dirname, dir_len), andPathCchStripToRoot() would not have to be implemented. So you get better behavior AND it's simpler overall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hold off on any more changes on this function for a day or two - I'm having some interesting conversations with colleagues right now. Might have a more suitable approach (or at least more information about it).
Co-authored-by: Eryk Sun <eryksun@gmail.com>
zooba commentedMar 9, 2023
Okay, thanks for holding. What I'm hearing from my colleagues at Microsoft is that the Until the GDK update arrives, itseems like we can probably use code like what we used to have to load the API dynamically (from @maxbachmann Would you be able to give this approach a try and see if it works? Unfortunately, I don't think there's any way to detect the version of the games SDK involved, so we'd just have to keep this until we assume everyone's on the fixed update. If it helps (and I suspect it won't), it also ought to be okay to temporarily define the PARTITION constants needed when including |
maxbachmann commentedMar 9, 2023
Yes I did read in the developer forums, that there is already an implementation as well. Great to hear that they are likely going to get exposed at some point.
just gave this a quick test and it works both on the Xbox One and Xbox Scarlett.
I do not think we need to take Windows 7 into consideration here:
|
zooba commentedMar 9, 2023
!buildbot .Windows. |
bedevere-bot commentedMar 9, 2023
zooba commentedMar 9, 2023
There we go. I expect there'll be no buildbot issues, but worth double checking (there are some unrelated stack issues, so may still fail, but we can ignore those). Other than that, I'm happy to merge this whenever.@eryksun I'll wait for a positive signal from you. |
| externPyObject*PyInit_gc(void); | ||
| externPyObject*PyInit_nt(void); | ||
| externPyObject*PyInit__signal(void); | ||
| #if defined(MS_WINDOWS_DESKTOP)|| defined(MS_WINDOWS_SYSTEM)|| defined(MS_WINDOWS_GAMES) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@zooba, since the check for building winreg and mmap occurs in several places (here, in "PC/config.c", and the respective modules), maybeHAVE_WINDOWS_REG_API andHAVE_WINDOWS_MMAP_API should be defined in "PC/pyconfig.h". And to match that, maybe renameHAVE_WINDOWS_CONSOLE_IO toHAVE_WINDOWS_CONSOLE_API. That's a better name anyway, sinceGenerateConsoleCtrlEvent() is a console API, but not one that's particularly related to I/O.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The mmap API should lose the definitions fairly soon, we just spun that work out to another issue.
For winreg I wouldn't worry aboutpyconfig.h, but the definition could be calculated at the top of the file. Those aren't C API functions, so they can be detected byAttributeError.
None of these feel important enough to block on, but I wouldn't oppose doing them later. Let's call this PR done and let Max focus on the other changes that have been lined up
zooba commentedMar 9, 2023
Thanks@maxbachmann! This was a huge effort, but I think it's worth it. Don't shy away from adjusting things we decided here if it seems important - nothing we changed should count as a public API change, so we can revert or modify if needed. |
Uh oh!
There was an error while loading.Please reload this page.