Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

py/emitglue: Reduce RAM footprint of frozen code and SYS_SETTRACE feature#18346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
yoctopuce wants to merge3 commits intomicropython:master
base:master
Choose a base branch
Loading
fromyoctopuce:frozen_mp_obj_fun_bc

Conversation

@yoctopuce
Copy link
Contributor

@yoctopuceyoctopuce commentedOct 30, 2025
edited
Loading

Summary

My MicroPython port has relatively large flash storage, but not much RAM (only 100 KB is available for MicroPython heap). Therefore I rely on frozen modules to save RAM. My port includes SYS_SETTRACE to allow the end user to debug their own code using Visual Studio Code.

Examining my runtime heap map in detail, I realised that frozen functions use a significant amount of RAM: 32 bytes per function or method. This equated to 20 KB of RAM when importing my main library.

Further investigation revealed a significant waste of ROM and RAM when enabling SYS_SETTRACE. This pull request attempts to address this issue:

  • the first commit reduces the footprint of themp_raw_code_t structure, which uses unnecessary ROM space in frozen code, but also uses RAM for dynamically loaded functions. Depending on the port options, this may also reduce the code footprint for builds not using SYS_SETTRACE.
  • the second commit solves the problem ofmp_obj_fun_bc_t objects using two GC units instead of one when SYS_SETTRACE is enabled. This requires the use of a different structure for each kind of function object, instead of just two types as it is currently the case, so some assumptions about the usage of function objects had to be made more explicit.
  • the third commit goes one step further by adding support for frozenmp_obj_fun_bc_t objects, not using any RAM. This improvement may benefit to any port with frozen modules, even when SYS_SETTRACE is not enabled. The price to pay for this is of course a bit of ROM, and the use of an indirect pointer to load the module globals dictionary.

edit on 2025-10-31: this third commit has been updated based on the alternative implementation that I had considered in my first submit, i.e. frozing themp_module_context_t object as well, with just the pointer to the module-specific globals dictionary in RAM. This results in simpler code so I think this is preferable. The first version is still available on my repository for reference if needed.

Testing

The code has been extensively tested on my port, executing both frozen and runtime code.

I started from a build with 185 KB of rodata for frozen modules. Loading my main frozen library used 36KB RAM (for defining classes involving about 600 methods). After the optimizations, I ended up with 162 KB of rodata for frozen modules, and only 17KB RAM used after loading my main library.

So the total savings in my case are 20 KB flash, 20 KB RAM.

I have enabled the new features for some configurations of CI tests to be sure they are covered.

Trade-offs and Alternatives

There are other ways to somehow reduce the RAM footprint of frozen modules, such as lazy loading (as done in asyncio for instance). I have gone that way first, but ultimately:

  • there was still way to much RAM wasted once I had loaded the libraries that I needed
  • the lazy loading solution tends to defeat type checking algorithms used by IDE
  • the lazy loading solution breaks if the user uses a star import
  • cross references between multiple classes is really tricky to implement properly with the lazy loading method
    So ultimately, freezingmp_obj_fun_bc_t objects was a much cleaner solution.

When a port defines MICROPY_PY_SYS_SETTRACE, there is currently alarge overhead in memory usage due to the extra pointers requiredto access code information for tracing. This overhead exists evenwhen tracing is not active. It impacts both RAM and frozen codein flash memory.This commit reduces this overhead by:- removing unused fields in the profiling data structure- optimizing the size of mp_raw_code_t, which is a temporary  structure for normal use but is persistent when SYS_SETTRACE  is enabled (and also impacts frozen code size).As a side effect, this commit also reduce the code footprint offrozen functions when MICROPY_PERSISTENT_CODE_SAVE is enabled,even for builds without SYS_SETTRACE enabled.Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
@codecov
Copy link

codecovbot commentedOct 30, 2025
edited
Loading

Codecov Report

❌ Patch coverage is97.64151% with5 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.40%. Comparing base (673524f) to head (6849d21).
⚠️ Report is 12 commits behind head on master.

Files with missing linesPatch %Lines
py/objcode.c70.00%3 Missing⚠️
py/frozenmod.c88.88%1 Missing⚠️
py/persistentcode.c0.00%1 Missing⚠️
Additional details and impacted files
@@            Coverage Diff             @@##           master   #18346      +/-   ##==========================================+ Coverage   98.38%   98.40%   +0.02%     ==========================================  Files         171      171                Lines       22297    22348      +51     ==========================================+ Hits        21936    21992      +56     + Misses        361      356       -5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report?Share it here.

🚀 New features to boost your workflow:
  • ❄️Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link

github-actionsbot commentedOct 30, 2025
edited
Loading

Code size report:

Reference:  stm32/usb: Add VBUS sensing configuration for TinyUSB on F4/F7. [27b7bf3]Comparison: tools/mpy-tool: Allow frozen function objects in frozen modules. [merge of 6849d21]  mpy-cross:   +16 +0.004%    bare-arm:   +16 +0.028% minimal x86:   -13 -0.007%    unix x64:   +72 +0.008% standard      stm32:   +80 +0.020% PYBV10     mimxrt:   +80 +0.021% TEENSY40        rp2:   +56 +0.006% RPI_PICO_W       samd:   +72 +0.026% ADAFRUIT_ITSYBITSY_M4_EXPRESS  qemu rv32:   +50 +0.011% VIRT_RV32

When a port defines MICROPY_PY_SYS_SETTRACE, there is currently alarge overhead in memory usage due to the mp_obj_fun_bc_t objecttaking 2 GC units per function instead of 1. This can be avoidedby removing redundant information from mp_obj_fun_bc_t when thefunction kind is BYTECODE and SYS_SETTRACE is enabled.This requires however to make the storage underlying mp_obj_fun_*objects different for bytecode and native/viper function objects.There is currently a grey zone with regard to the exact underlyingdata structure for function objects: inline_asm functions havetheir own mp_obj_fun_asm_t structure, but native and viper functionscurrently use the mp_obj_fun_bc_t structure, although they arenot really bytecode. This commit make them explicitly differentto increase the code readability, and clarify assumptions in code.Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
Currently, loading a frozen module with many functions andmethods results in substantial RAM usage due to the creationfunction objects (one GC block is created for each functionor method defined in the module).This commit provides a way to freeze the bytecode functionobjects in ROM, along with the raw_code objects. In order toachieve this, the module context structure must also be frozen,to allow frozen bytecode functions to reference it. Forfrozen module contexts, the module-specific globals dict istherefore referenced using an indirect pointer in RAM.Signed-off-by: Yoctopuce dev <dev@yoctopuce.com>
@dpgeorgedpgeorge added the py-coreRelates to py/ directory in source labelNov 3, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

py-coreRelates to py/ directory in source

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yoctopuce@dpgeorge

[8]ページ先頭

©2009-2025 Movatter.jp