Problem

When building MicroPython projects with a large number of source files, the qstr generation step fails with shell errors like:

/bin/sh: Argument list too long

This occurs because themakeqstrdefs.py script receives all source file paths as command-line arguments, which can exceed the shell'sARG_MAX limit. This issue was first encountered when including large user C modules like LVGL, which adds 1200+ files to the qstr scanning list.

TheARG_MAX limit varies significantly between systems:

• Typical Linux systems: 128KB - 2MB depending on kernel version
• This explains why builds succeed on some machines but fail on others with identical source code
• The same project may build successfully on a system with a newer kernel but fail on older systems with lower limits

The issue manifests during the qstr preprocessing step inpy/mkrules.mk when generating$(HEADER_BUILD)/qstr.i.last.

Solution

This PR implements support for response files (argument files) in the qstr generation toolchain. Response files allow passing the large file list via a file instead of on the command line, completely avoiding ARG_MAX limitations for the problematic argument.

Changes

1. py/mkrules.mk: Modified the qstr generation rule to write only the source file list to a temporary file (qstr_sources.txt). All other arguments (CPP, flags, dependencies) remain as command-line arguments.

2. py/mkrules.cmake: Modified to write the semicolon-separated source list to a file at configure time. This avoids passing the huge list as command-line arguments to any build-time process.

3. py/makeqstrdefs.py: Enhanced the argument parser to detect and read response files when an argument starts with@. The parser now handles both newline-separated format (from Make builds) and semicolon-separated format (from CMake builds).

Implementation Details

• Response files are indicated by prefixing the filename with@ (e.g.,sources @qstr_sources.txt)
• Only the large source file list is moved to a response file; other arguments stay on the command line
• The format is simple: one file path per line
• Maintains backward compatibility with existing command-line invocations
• Minimal, targeted code changes focused on the specific problem (file list too long)

Testing

This fix has been tested with:

• Standard MicroPython ports (mimxrt, unix, stm32) using Make builds
• ESP32 and RP2 ports using CMake builds
• LVGL-based projects with 1200+ source files where the previous implementation would fail
• Existing projects to verify backward compatibility

Related Issues

This addresses build failures in projects with extensive use of external libraries, particularly those using LVGL bindings or other large codebases that push against shell argument limits.